\section{Classifying spaces: the Grassmann model}\label{grassmannmodel}
We will now shift our focus somewhat and talk about classifying spaces for
principal bundles and for vector bundles. We will do this in two ways: the
first will be via the Grassmann model and the second via simplicial methods.
\begin{lemma}\label{embedding}
    Over a compact Hausdorff space, any $n$-plane bundle embeds in a trivial
    bundle.
\end{lemma}
\begin{proof}
    Let $\cU$ be a trivializing open cover of the base $B$; since $B$ is
    compact, we may assume that $\cU$ is finite with $k$ elements. There is no
    issue with numerability, so there is a subordinate partition of unity $\phi_i$.
    Consider an $n$-plane bundle $E\to B$. By trivialization, there is a
    fiberwise isomorphism $p^{-1}(U_i)\xrightarrow{f_i}\RR^n$ where the $U_i\in
    \cU$. A map to a trivial bundle is the same thing as a bundle map in the
    following diagram:
    \begin{equation*}
	\xymatrix{
	    E\ar[r]\ar[d] & \RR^N\ar[d]\\
	    B\ar[r] & \ast
	    }
    \end{equation*}
    We therefore define $E\to (\RR^n)^k$ via
    $$e\mapsto (\phi_i(p(e))f_i(e))_{i=1,\cdots,k}.$$
    This is a fiberwise linear embedding, generally called a ``Gauss map''.
    Indeed, observe that this map has no kernel on every fiber, so it is an
    embedding.
\end{proof}
The trivial bundle has a metric on it, so choosing the orthogonal complement of
the embedding of Lemma \ref{embedding}, we obtain:
\begin{corollary}
    Over a compact Hausdorff space, any $n$-plane bundle has a complement (i.e.
    a $\xi^\perp$ such that $\xi\oplus\xi^\perp$ is trivial).
\end{corollary}
Another way to say this is that if $B$ is a compact Hausdorff space with an
$n$-plane bundle $\xi$, there is a map $f:X\to\Gra_n(\RR^{kn})$; this is
exactly the Gauss map. It has the property that taking the pullback
$f^\ast\gamma^n$ of the tautologous bundle over $\Gra_n(\RR^{kn})$ gives back
$\xi$.

In general, we do not have control over the number $k$. There is an easy fix to
this problem: consider the tautologous bundle $\gamma^n$ over
$\Gra_n(\RR^\infty)$, defined as the union of $\Gra_n(\RR^m)$ and given the
limit topology. This is a CW-complex of finite type (i.e. finitely many cells
in each dimension). Note that $\Gra_n(\RR^m)$ are not the $m$-skeleta of
$\Gra_n(\RR^\infty)$!

The space $\Gra_n(\RR^\infty)$ is ``more universal'':
\begin{lemma}\label{universal}
    Any (numerable) $n$-plane bundle is pulled back from $\gamma^n\downarrow
    \Gra_n(\RR^\infty)$ via the Gauss map.
\end{lemma}
Lemma \ref{universal} is a little bit tricky, since the covering can be wildly
uncountable; but this is remedied by the following bit of point-set topology.
\begin{lemma}\label{sublemma}
    Let $\cU$ be a numerable cover of $X$. Then there's another numerable cover
    $\cU^\prime$ such that:
    \begin{enumerate}
	\item the number of open sets in $\cU^\prime$ is countable, and
	\item each element of $\cU^\prime$ is a disjoint union of elements of
	    $\cU$.
    \end{enumerate}
\end{lemma}
If $\cU$ is a trivializing cover, then $\cU^\prime$ is also a trivializing
cover.
\begin{proof}
    See \cite[Proposition 3.5.4]{husemoller}.
\end{proof}
It is now an exercise to deduce Lemma \ref{universal}. The main result of this
section is the following.
\begin{theorem}
    The map $[X,\Gra_n(\RR^\infty)]\to \Vect_n(X)$ defined by $[f]\mapsto
    [f^\ast\gamma^n]$ is bijective, where $[f]$ is the homotopy class of $f$
    and $[f^\ast\gamma^n]$ is the isomorphism class of the bundle
    $f^\ast\gamma^n$.
\end{theorem}
This is why $\Gra_n(\RR^\infty)$ is also called the \emph{classifying space}
for $n$-plane bundles. The Grassmannian provides a very explicit geometric
description for the classifying space of $n$-plane bundles. There is a more
abstract way to produce a classifying space for principal $G$-bundles, which
we will describe in the next section; the Grassmannian is the special case when
$G = \GL_n(\RR)$.
\begin{proof}
    We have already shown surjectivity, so it remains to prove injectivity.
    Suppose $f_0,f_1:X\to \Gra_n(\RR^\infty)$ such that $f_0^\ast\gamma^n$ and
    $f_1^\ast\gamma^n$ are isomorphic over $X$. We need to construct a homotopy
    $f_0\simeq f_1$. For ease of notation, let us identify $f_0^\ast\gamma^n$
    and $f_1^\ast\gamma_n$ with each other; call it $\xi:E\downarrow X$.

    The maps $f_i$ are the same thing as Gauss maps $g_i:E\to\RR^\infty$, i.e.,
    maps which are fiberwise linear embeddings. The homotopy $f_0\simeq f_1$ is
    created by saying that we have a homotopy from $g_0$ to $g_1$ through Gauss
    maps, i.e., through other fiberwise linear embeddings.

    In fact, we will prove a much stronger statement: \emph{any two} Gauss maps
    $g_0,g_1:E\to \RR^\infty$ are homotopic through Gauss maps. This is very far from
    true if I didn't have a $\RR^\infty$ on the RHS there.

    Let us attempt (and fail!) to construct an affine homotopy between $g_0$
    and $g_1$. Consider the map $tg_0 + (1-t)g_1$ for $0\leq t\leq 1$. In order
    for these maps to define a homotopy via Gauss maps, we need the following
    statement to be true: for all $t$, if $tg_0(v) + (1-t)g_1(v) =
    0\in\RR^\infty$, then $v=0$.  In other words, we need $tg_0+(1-t)g_1$ to be
    injective. Of course, this is not guaranteed from the injectivity of $g_0$
    and $g_1$!

    Instead, we will construct a composite of affine homotopies between $g_0$
    and $g_1$ using the fact that $\RR^\infty$ is an infinite-dimensional
    Euclidean space. Consider the following two linear isometries:
    \begin{equation*}
	\xymatrix{
	    & \RR^\infty = \langle
	    e_0,e_1,\cdots\rangle\ar[dl]^\alpha_{e_i\mapsto
	    e_{2i}}\ar[dr]_\beta^{e_i\mapsto e_{2i+1}} & \\
	    \RR^\infty & & \RR^\infty
	    }
    \end{equation*}
    Then, we have four Gauss maps: $g_0$, $\alpha\circ g_0$, $\beta\circ g_1$,
    and $g_1$. There are affine homotopies through Gauss maps:
    $$g_0\simeq \alpha \circ g_0\simeq \beta\circ g_1\simeq g_1.$$
    We will only show that there is an affine homotopy through Gauss maps $g_0
    \simeq \alpha \circ g_0$; the others are left as an exercise. Let $t$ and
    $v$ be such that $tg_0(v) + (1-t)\alpha g_0(v) = 0$. Since $g_0$ and
    $\alpha g_0$ are Gauss maps, we may suppose that $0<t<1$. Since $\alpha
    g_0(v)_i$ has only even coordinates, it follows by definition of the map
    $\alpha$ that $g_0(v)$ only had nonzero coordinates only in dimensions
    congruent to $0$ mod $4$. Repeating this argument proves the desired
    result.
\end{proof}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage[legalpaper, portrait, margin=1in]{geometry}

\title{2016 ISI Calculus Problem}

\author{Shreenabh Agrawal}

\date{\today}

\begin{document}

\maketitle

\section{Question}
Let $f$ be a differentiable function such that $f(f((x))=x \text { for all } x \in[0,1]$ Suppose $f(0)=1 .$ Determine the value of $\int\limits_{0}^{1}(x-f(x))^{2016} \: d x$
\section{Solution}
$$f(f(x))=x \implies
  f(x)=f^{-1}(x)$$
The graph of this is symmetric about $y=x$,
$$\therefore f(0) = 1 \implies f(1) = 0$$
Now, let us solve the integral (call it (i))
$${I}=\int\limits_{0}^{1}(x-f(x))^{2016}\:  d x$$
Taking substitution,
$$\begin{aligned}
f(x) &=t \\
x &=f^{-1}(t)=f(t) \\
d x &=f^{\prime}(t) \: d t
\end{aligned}$$
Plugging this in (call this (ii)),
$$\begin{aligned}
\therefore \quad I &=\int\limits_{1}^{0}(f(t)-t)^{2016} f^{\prime}(t)\: d t \\
&=-\int\limits_{0}^{1}(t-f(t))^{2016} f^{\prime}(t) \:d t
\end{aligned}$$
Now adding (i) and (ii),
$$2 I=\int\limits_{0}^{1}(x-f(x))^{2016}\left(1-f^{\prime}(x)\right) \: d x$$
Taking another substitution,
$$\begin{aligned}
(x - f(x)) &= u\\
(1 - f^{\prime}(x)) \: dx &= \: du
\end{aligned}
$$
Finally,
$$\begin{aligned}
\therefore \quad 2 I  & =  \int\limits_{-1}^{1}  u^{2016}\: d u \\
 2 I &=  2 \cdot\left[\frac{u^{2017}}{2017}\right]_{0}^{1}
\end{aligned}$$
Thus, the value of our original integral is:
$$\boxed{I = \frac{1}{2017}}$$
\end{document}

Calculus/Creative Math Solving's Problems/2016 ISI Calculus Problem.tex


\chapter{Model Selection}
\label{chp:subsel}

In the previous chapters the chosen number of predictors in the models have not been discussed very thoroughly. The p-value was used for selecting the best predictor in Lab 4.6.2 in section \ref{sec:lab462}. The question though remains how many predictors should be included when creating a model. This chapter discusses three different methods for selecting a subset of the predictors for a model.

\section{Best Subset Selection}
Best subset selection is a method for choosing the best possible subset of predictors. In this approach a model is fitted for each possible combination of the predictors.

The algorithm for best subset selection is explained in Algorithm \ref{algo:bestsubset}. First the null models are created in step 1. In line 2 it iterates from 1 predictor and up to the number of available predictors. In line 3 all possible models with k predictors are fitted and in line 4 and 5 the best model with k predictors is chosen. When all iterations are done the single best model is chosen.

% Insert the algorithm
\begin{algorithm}
	\caption{Best subset selection}
	\label{algo:bestsubset}
	\begin{algorithmic}[1]
		\State Let $M_0$ denote null model, containing no predictors. Which means that $M_0$ contains the sample mean for each observation.
		\For {each $k$ from $1$ to $length(p)$}
		\State Fit all the different models with a combination of k predictors.
		\State \parbox[t]{\dimexpr\linewidth-\algorithmicindent}{Pick the best model with k predictors. Best can be defined as smallest RSS or largest $R^2$}
		\State $M_k = best\_model$
		\EndFor
		\State Choose a single best models from all the best models in $M_0,...M_k$. The best model is chosen based on $C_p$, $BIC$ or adjusted $R^2$ if the whole data set is used for training the models.
	\end{algorithmic}
\end{algorithm}

\subsection{Lab 6.5.1 - Best Subset Selection}
In this exercise a best subset selection is performed on a data set called \emph{Hitters}. The data set is used for predicting the salary of baseball players. Before performing best subset selection the data is split into predictors and the response variable.

A function for choosing the best subset is then defined in listing \ref{lst:best_subset_selection}. In line 3 it iterates through all combinations of the predictors with k number of predictors. A function \emph{process\_subset} is called which returns the subset with calculated model, RSS, $R^2$, BIC and $C_p$. In line 4 all the different combinations are appended to the \emph{models} array. In line 5 the best model is picked based on lowest RSS and then returned.

\begin{lstlisting}[language=Python, label=lst:best_subset_selection, caption=Function for choosing best subset]
def best_subset_selection(k):
	models = []
	for combo in combinations(X.columns, k):
		models.append(process_subset(combo))
	return models[np.argmin([r['RSS'] for r in models])]
\end{lstlisting}

The function in listing \ref{lst:process_subset} is called in listing \ref{lst:best_subset_selection} in line 4. It fits a linear regression model and computes RSS, $R^2$, $C_p$ and BIC for the model.

\begin{lstlisting}[language=Python, label=lst:process_subset, caption=Function for computing metrics for subset]
def process_subset(feature_set):
	d = len(feature_set)
	n = X.shape[0]
	model = lm.LinearRegression().fit(X[[i for i in feature_set]], Y)
	Y_hat = model.predict(X[list(feature_set)])

	rss = metricsUtil.RSS(Y, Y_hat)
	rsquared = metrics.r2_score(Y, Y_hat)
	cp = metricsUtil.Cp(rss, d, Y_hat, n)
	bic = metricsUtil.Bic(n, rss, d, Y_hat)

	return {"model":model, "features": feature_set,
			"RSS":rss, "rsquared": rsquared, "bic": bic, "cp":cp }
\end{lstlisting}

For calculating RSS, $R^2$, $C_p$ and BIC a python module called metricsUtil was implemented. The module has defined functions for calculating these values. After these functions are defined the best subset selection is computed. In listing \ref{lst:best_subset9} a best subset selection is performed for the 19 possible predictors.

\begin{lstlisting}[language=Python, label=lst:best_subset9, caption=Calculating the best subset from 1 to 19 predictors]
for i in xrange(1,20):
	subsets.loc[i] = best_subset_selection(i)
\end{lstlisting}

Plotting RSS, $R^2$, $C_p$ and BIC for the best models from 1-19 predictors gives the plotted graphs shown in figure \ref{fig:best_plottet}. It shows how RSS and $R^2$ respectively monotonically increases and decreases. This is caused by using the same data for both training and testing. When doing this Cp, BIC or adjusted $R^2$ can be used for estimating the optimal amount of predictorss. When calculating BIC and $C_p$ a penalty is added to the calculation. For adjusted $R^2$ the calculation takes into account that only small increases in RSS are often caused by adding a noise variable. Noise variables are added, which consequently leads to a decrease of $R^2$ \citep[pp. 598]{propandstats} \citep[pp. 210-213]{ISLR}.

\myFigure{subset_all_graph.PNG}{Plotted RSS, $R^2$, $C_p$ and BIC for best subset selection from 1-19 predictors}{fig:best_plottet}{1}
\FloatBarrier

BIC indicates that the best model contains 6 predictors. The best model with 6 predictors include: \emph{AtBat, Hits, Walks, CRBI, Division} and \emph{PutOuts}. $C_p$ indicates that the best model contains 10 predictors, this also makes sense as BIC generally places a heavier penalty \citep[pp. 212]{ISLR}. The best 10 predictors for a model included: \emph{AtBat, Hits, Walks, CAtBat, CRuns, CRBI, CWalks, PutOuts, Assists} and \emph{Division}. This also shows how the set of 6 predictors is not just a subset of the 10 predictors. This is because every combination is tried for every number of predictors. A drawback of the algorithm is therefore that it is computationally expensive. The models that must be computed for 19 predictors to find the best amount of predictors is calculated in equation (5.1) to 524,288.

\begin{equation}
2^{19} = 524,288
\end{equation}

The following section presents two different algorithms that can be used for selecting predictors with fewer computations than just described for best subset selection.

\section{Forward and Backward Stepwise Selection}
Forward and backward stepwise selection are two similar algorithms for stepwise selection of a subset. They are both more computationally efficient than the best subset selection.

The forward stepwise selection algorithm adds predictors to the model one by one. When computing the k+1'th model, the predictors chosen for the k'th model defines the first k predictors and then one new predictor is added. This means that the k'th best model will always be a subset of the k+n'th best model. The predictor that improves most upon the model is always the one added.

Backward stepwise selection works very similar to forward stepwise selection. However, the model instead starts by containing all predictors and at each step a predictor is removed. The predictor that is removed is chosen by which gives the best model. This means that instead of fitting 524,288 models with best subset selection, forward and backward stepwise selection only has to fit 191 models as calculated in equation (5.2). This is a significant difference in models that must be fitted.  This was also obvious when executing the exercises for best subset selection compared to forward and backward stepwise selection.

\begin{equation}
1+ \sum_{k=0}^{18}(19-k) = 191
\end{equation}

\subsection{Lab 6.5.2 - Forward and Backward Stepwise Selection}
The exercise for 6.5.3 is very similar to 6.5.2 and a function for forward and backward stepwise selection was implemented and used instead of \emph{best\_subset\_selection} implemented in listing \ref{lst:best_subset_selection}. The rest of the code could be reused. In listing \ref{lst:forwardselection} the implemented function for forward stepwise selection is shown. First the remaining predictors to choose from are extracted. \emph{X.columns} is all the predictors while \emph{already\_chosen\_predictors} is passed as a parameter. It then runs a for loop iterating through all the remaining predictors and adding them to the model. The best model is then selected based on RSS.

\begin{lstlisting}[language=Python, label=lst:forwardselection, caption=Function for forward stepwise selection]
def forward_stepwise_selection(already_chosen_predictors):
	remaining_predictors = [p for p in X.columns if p not in
		already_chosen_predictors]

	models = []
	for p in remaining_predictors:
		models.append(process_subset(already_chosen_predictors+[p]))
	return models[np.argmin([r['RSS'] for r in models])]
\end{lstlisting}

Similarly a function for backward stepwise selection was implemented. Comparing the models for best subset selection, forward and backward stepwise selection showed that some models were identical while others differed. For example the models with predictors from 1 to 6 for both best subset selection and forward stepwise selection were identical while the model with 7 predictors differed. This is not the case with backward stepwise selection which differs in all these. Actually the best model with 7 predictors differs in all three algorithms. The following 7 predictors were chosen:

\begin{itemize}
\item Forward stepwise: \emph{CRBI, Hits, PutOuts, Division, AtBat, Walks} and \emph{CWalks}
\item Best subset: \emph{Hits, Walks, CAtBat, CHits, ChMRun, PutOuts} \emph{Division}
\item Backward stepwise: \emph{AtBat, Hits, Walks, CRuns, CWalks, PutOuts} and \emph{Division}
\end{itemize}

\section{Validation Set Approach and Cross-Validation}
In the previous exercises $C_p$, AIC, BIC and adjusted $R^2$ was used for picking the right amount of predictors. This was needed because the models were trained on the complete data set and so the test error would provide an underestimate of the true error.

In this exercise the validation set approach and cross-validation are used. Therefore instead of using $C_p$, AIC, BIC and adjusted $R^2$ the test error can be directly calculated.
\subsection{Lab 6.5.3 - Choosing Among Models Using The Validation Set Approach and Cross-Validation}

Initially the validation set approach will be used in combination with best subset selection. This is shown in listing \ref{lst:validationsetapproach}. First the data is split into a training and testing set. When this is done the best subset selection is run from 1 to 19 predictors. A small modification of the functions were needed so that the training and test sets were used appropriately.

\begin{lstlisting}[language=Python, label=lst:validationsetapproach, caption=Validation set approach and best subset selection for choosing predictors]
train, test = train_test_split(data, test_size = 0.5)

.......

for i in xrange(1, 20):
	subsets.loc[i] = best_subset_selection(i, X_train, Y_train,
	X_test, Y_test)
\end{lstlisting}

The mean squared error for different numbers of predictors is shown in table \ref{table:mse_validation}. By using this validation set approach and best subset selection the optimal number of predictors is computed to 9. The optimal number of predictors is based on the model with lowest mean squared error. It is important to note that a different split of train and test data can give a different number of optimal predictors.

The validation set approach splits the data equally in two parts. This does not leave much data for training and the test error is therefore very likely an overestimate for the model fit on the entire data. Another approach is k-fold cross-validation. In this exercise a 10-fold cross-validation is executed on the data set in combination with best subset selection.

The implementation of 10-fold cross-validation is shown in listing \ref{lst:crossvalidation}. First a 10-fold random split of the data is done. Then a for loop iterates through the folds and gives a train and test index for each iteration. These indexes are used to create a training and test set for each fold. For each fold a best subset selection is performed from 1 to 19 predictors. After performing the best subset selection 10 times, one for each fold, then a mean of the MSE is calculated for all the folds.

\begin{lstlisting}[language=Python, label=lst:crossvalidation, caption=Cross-validation and best subset selection for choosing predictors]
kf = KFold(n_splits=10, shuffle=True)
kf.get_n_splits(X)
j = 0
mse = np.zeros((kf.n_splits, 19))
for train_index, test_index in kf.split(X):
    X_train = X.iloc[train_index]
    Y_train = Y.iloc[train_index]
    X_test = X.iloc[test_index]
    Y_test = Y.iloc[test_index]
    for i in xrange(1, 20):
		subset = best_subset_selection(i, X_train, Y_train, X_test, Y_test)
        mse[j,i-1] = subset["mse"]
    j = j+1
\end{lstlisting}
\FloatBarrier

The results for cross-validation and best subset selection is shown in table \ref{table:mse_cross}. Using this approach the best model is achieved using a model with 10 predictors. Compared to the best subset selection, this method is even more computationally expensive because it runs the best subset selection one time for each fold.

\begin{table}
	\centering
	\begin{subtable}[t]{2in}
		\centering
		\begin{tabular}{ p{2.5cm} p{1.5cm}  }
			\textbf{Predictors} & \textbf{MSE} \\
			\hline
			\vspace{0.1px}
			7 & \vspace{0.1px} 90,133  \\\hline
			\vspace{0.03em}
			8 & \vspace{0.03em} 89,914  \\\hline
			\vspace{0.03em}
			\textbf{9} & \vspace{0.03em} \textbf{89,782}  \\\hline
			\vspace{0.03em}
			10 &\vspace{0.03em} 90,015  \\\hline
			\vspace{0.03em}
			11 & \vspace{0.03em} 90,192  \\\hline
			\vspace{0.03em}
			12 & \vspace{0.03em} 90,363  \\\hline
		\end{tabular}
		\caption{Validation set approach}\label{table:mse_validation}
	\end{subtable}
	\quad
	\begin{subtable}[t]{2in}
		\centering
	\begin{tabular}{ p{2.5cm} p{1.5cm}  }
		\textbf{Predictors} & \textbf{MSE} \\
		\hline
		\vspace{0.03em}
		7 & \vspace{0.03em} 78,360  \\\hline
		\vspace{0.03em}
		8 & \vspace{0.03em} 77,971  \\\hline
		\vspace{0.03em}
		9 & \vspace{0.03em} 77,944  \\\hline
		\vspace{0.03em}
		\textbf{10} & \vspace{0.03em} \textbf{77,895} \\\hline
		\vspace{0.03em}
		11 & \vspace{0.03em} 78,352  \\\hline
		\vspace{0.03em}
		12 & \vspace{0.03em} 78,795 \\\hline
	\end{tabular}
		\caption{Cross-validation}\label{table:mse_cross}
	\end{subtable}
	\caption{Calculated mean squared error from 7 to 12 predictors.}\label{table:mse}
\end{table}

When choosing among the different techniques for subset selection it is often a matter of how much computation power is available.  The best subset selection is a lot more expensive than forward and backward stepwise selection. Combining best subset selection with cross-validation also makes the process even more computationally expensive. If many predictors are available, then best subset selection might simply be too expensive, even for very fast modern computers. If more than 40 predictors are available, then best subset selection is rarely feasible and methods such as forward and backward stepwise selection could be used instead  \citep[pp. 207]{ISLR}.

An important aspect of subset selection is that the best model might be more complex than a simpler model performing approximately as good. When choosing subset of predictors complexity vs simplicity should also be included in the final choice of predictors


\chapter{\uppercase{Extensions and Applications of MUD Points and Skewness} \label{chapter:vector-valued}}
In Chapter~\ref{chapter:geometry}, we introduced the notion of skewness and showed examples of how it impacts the accuracy of approximating SIP solutions with finite sampling.
Here, we demonstrate how an awareness of skewness allows us to a priori define a QoI that will\---on average\---better resolve $\paramref$ by providing information in mutually distinct directions in $\pspace$.
We begin by revisiting the example in Section \ref{subsec:pde-example} involving the Poisson problem and uncertain Neumann boundary condition $g$.
Recall that $\qoi_\text{2D}$ is able to better resolve $\paramref$ (in this case, $\paramref$ is defined by a finite-dimensional representation of an exact $g$), than $\qoi_\text{1D}$.
The map $\qoi_\text{2D}$ is presented as an alternative option for aggregating the same $100$ measurements and using them to construct a more informative 2D map.
In this chapter, we present a more detailed study of the construction of this map, and perform a case-study in designing a problem where the data-constructed QoI map is more informative.

\input{extensions/mud_vector_map.tex}
\FloatBarrier

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\FloatBarrier
\input{extensions/mud_pde_geometry.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\input{extensions/mud_pde_5d.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\input{extensions/mud_pde_sequence.tex}
\FloatBarrier
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Conclusions}

We have shown that when posing and solving SIPs, we are motivated to choose data-constructed QoI maps which have (a) as many components as possible (up to the dimension of $\pspace$), and (b) whose components exhibit high-degrees of mutually geometrically distinct information (i.e., low average skewness).
There is a dramatic reduction in the sensitivity to the measurement noise when the same hundred measurements are used to construct a map with lower skewness.
In the next chapter, we discuss some other considerations for constructing SIPs, including measurement precision and placement.
We show how different experimental setups can lead to parameter estimates which exhibit different levels of accuracy and precision.
We also introduce a novel method for leveraging the results shown here in a sequential manner to gain possible computational efficiency.


\section{Introduction}
Given a graph $G$ and $k$ agents, each of which is associated with an initial vertex and a goal vertex of $G$, \emph{Multi-Agent Pathfinding (MAPF)} is the problem of finding $k$ conflict-free paths which connect the initial vertex with the goal vertex of each agent.

There are a number of applications of MAPF, which range from industrial applications, in which the increase in automation may promote the need for dozens---perhaps hundreds---of robots navigating in indoor environments (e.g., warehouses), to aviation, underground mining, and multi-agent videogames \cite{WangB08}.

Solving MAPF optimally is NP-complete \cite{YuL13,MaK17}. When viewed as a standard AI search problem, it is straightforward to notice that the branching factor of MAPF is exponential on the number of agents since at each moment in time each agent can perform a number of actions, relatively fixed. It is not surprising, then, that building algorithms that scale reasonably well with the number of agents has been challenging.

For its most simple version, that is, MAPF over 4-connected grids, a number of approaches have been proposed, but two classes of solvers are the most relevant for the research we report here. A first class, is search-based solvers (e.g., \nbcite{Standley10}), which use heuristic search as the main component.
A state-of-the-art search-based solver is Conflict-Based Search (CBS) \cite{SharonSFS12,FelnerLB00KK18,LiFB0K19}, which uses A* at its core. A second class, is compilation-based solvers; for example which translate MAPF to Satisfiability Testing (SAT) (e.g., \nbcite{SurynekFSB16}; \nbcite{BartakZSBS17}; \nbcite{BartakS19}), and Answer-Set Programming \cite{ErdemKOS13,GebserOOS18}.

When seeking for an optimal solution for MAPF, different objective functions can be considered. Under \emph{sum-of-costs}, the most popular variant of MAPF, the objective is to minimize the moves agents perform before stopping at the goal. % This is variant is used in state-of-the-art solvers like CBS.

In this paper, we continue to explore the potential of ASP solvers for MAPF, and propose the first compilation that solves MAPF optimally under the sum-of-costs assumption. A second contribution consists of proposing the first compilation from MAPF to ASP that grows linearly with the number of agents, unlike existing compilations to ASP that are quadratic on the number of agents. In addition, we propose an optimization which uses information drawn from a search algorithm that is run as a preprocessing step to make the encoding more compact.

In our empirical evaluation, we evaluate our approach on synthetic square grids and warehouse grids with an increasing number of agents. We compare against MDD-SAT~\cite{SurynekFSB16}, a representative of the state of the art in SAT-based MAPF, and iCBS-h~\cite{FelnerLB00KK18}, a representative of the state-of-the-art in search-based MAPF. We observe that our approach outperforms both MDD-SAT and iCBS-h when congestion is high. Specifically, our approach has a greater coverage as the number of agents increase. We conclude that Answer-Set Programming is a viable approach to solving MAPF problems.


\documentclass[11pt]{article}

\usepackage{amsfonts}
\usepackage{fancyvrb}
\usepackage{url}
\usepackage{graphicx}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{dsfont}

\setlength{\oddsidemargin}{0in}
\setlength{\evensidemargin}{0in}
\setlength{\textwidth}{6.5in}
\setlength{\topmargin}{0in}
\setlength{\headsep}{0.5in}
\setlength{\textheight}{8.5in}
\setcounter{page}{1}
%\pagestyle{empty}
%\hbadness=10000

\begin{document}
\huge
\noindent
{Discrete Optimization Assignment:}
\vspace{0.25cm}

\noindent
{\bf Facility Location}
\normalsize

\section{Problem Statement}

In this assignment you will design an algorithm to solve a problem faced by distribution companies, {\em The Facility Location Problem}. A distribution company uses bulk storage facilities to provide goods to many different customers.  The goal of this problem is to determine which facilities will be the most cost effective for serving the customers.  The complexity of the problem comes from the fact that each facility has different costs and storage capabilities.\footnote{Hint: The facility location problem is closely related to the warehouse location problem discussed in the lectures.}

%\begin{figure}[h]
%        \centering
%        \begin{subfigure}[b]{8.0cm}%{0.3\textwidth}
%                \centering
%                \includegraphics[width=8cm]{figures/coloring_1.pdf}
%                \caption{A graph.}
%                \label{fig:graph:input}
%        \end{subfigure}%
%        ~ %add desired spacing between images, e. g. ~, \quad, \qquad etc.
%          %(or a blank line to force the subfigure onto a new line)
%        \begin{subfigure}[b]{8.0cm}
%                \centering
%                \includegraphics[width=8cm]{figures/coloring_2.pdf}
%                \caption{A three-coloring of the graph.}
%                \label{fig:graph:coloring}
%        \end{subfigure}
%        \caption{A Graph Coloring Example}\label{fig:graph}
%\end{figure}

\section{Assignment}

Write an algorithm to solve the facility location problem.  The problem is mathematically formulated in the following way:  there are $N = 0 \ldots n-1$ facilities to choose from and $M = n \ldots n+m-1$ customers that need to be served.  Each facility, ${\it f} \in N$ has a setup cost $s_{\it f}$ and a capacity $cap_{\it f}$.  Each customer, $c \in M$, has a demand $d_c$.  Both the facilities and customers are located in a Euclidian space, $\langle x_i,y_i \rangle \; i \in N \cup M$.  The cost to deliver goods to a particular customer $c$ from a facility ${\it f}$ is the Euclidean distance between two locations, ${\it dist}({\it f},c)$.\footnote{${\it dist}(i,j) = \sqrt{(x_{i} - x_{j})^2 + (y_{i} - y_{j})^2}$}  Lastly, all customers must be served by exactly 1 facility.  Let $a_f$ be a set variable denoting the customers assigned to facility $f$.  Then the facility location problem is formalized as the following optimization problem:
$$
\begin{array}{ll}
\mbox{minimize:} & \displaystyle \sum_{f \in N} \left( (|a_{\it f}| > 0) s_{\it f} + \sum_{c \in a_{\it f}} {\it dist}({\it f},c) \right) \\
\mbox{subject to:} & \\
     & \displaystyle \sum_{c \in a_{\it f}}  d_c \leq cap_{\it f} \;\;\; ({\it f} \in N)\\
     & \displaystyle \sum_{{\it f} \in N}  (c \in a_{\it f}) = 1 \;\;\; (c \in M)
\end{array}
$$

\clearpage
\section{Data Format Specification}

The input consists of $|N| + |M| + 1$ lines.  The first line contains two numbers, $|N|$ followed by $|M|$.
The first line is followed by $|N|$ lines, where each line encodes the facility's setup cost $s_{\it f}$, capacity $cap_{\it f}$, and the location $x_{\it f}, y_{\it f}$.  The remaining $|M|$ lines capture the customer information, where each line encodes the customer's demand, $d_c$, and location $x_c, y_c$.

\vspace{0.2cm}
\noindent
Input Format
\vspace{-0.2cm}
\begin{Verbatim}[frame=single]
|N| |M|
s_0 cap_0 x_0 y_0
s_1 cap_1 x_1 y_1
...
s_|N|-1 cap_|N|-1 x_|N|-1 y_|N|-1
d_|N| x_|N| y_|N|
d_|N|+1 x_|N|+1 y_|N|+1
...
d_|N|+|M|-1  x_|N|+|M|-1 y_|N|+|M|-1
\end{Verbatim}
%
The output has two lines.  The first line contains two values: $obj$ and $opt$.  $obj$ is the cost of the customer to facility assignment (i.e. the objective value) as a real number.  $opt$ should be $1$ if your algorithm proved optimality and $0$ otherwise.  The next line is a list of $|M|$ values in $N$ -- this is the mapping of customers to facilities.

\vspace{0.2cm}
\noindent
Output Format
\vspace{-0.2cm}
\begin{Verbatim}[frame=single]
obj opt
c_0 c_1 c_2 ... c_|M|-1
\end{Verbatim}
%
%It is essential that the value order in the solution output matches the value order of the input.  Otherwise the grader will misinterpret the output.

\clearpage
\paragraph{Examples}
%\vspace{0.1cm}
%\noindent
%(based on Figure \ref{fig:graph})

\vspace{0.2cm}
\noindent
Input Example
\vspace{-0.2cm}
\begin{Verbatim}[frame=single]
 3 4
 100 100 1065.0 1065.0
 100 100 1062.0 1062.0
 100 500 0.0 0.0
 50 1397.0 1397.0
 50 1398.0 1398.0
 75 1399.0 1399.0
 75 586.0 586.0
\end{Verbatim}

\vspace{0.2cm}
\noindent
Output Example
\vspace{-0.2cm}
\begin{Verbatim}[frame=single]
2550.013 0
1 1 0 2
\end{Verbatim}
%
This output represents the assignment of customers to facilities, $a_0 = \{2\}, a_1 = \{0,1\}, a_2 = \{3\}$.  That is, customers 0 and 1 are assigned to facility 1, customer 2 is assigned to facility 0, and customers 3 is assigned to facility 2.

\section{Instructions}

\input{instructions.tex}

%We use \texttt{stdout} for output.
%Output to other stream will be ignored (you may want to send runtime information to \texttt{stderr}). Your submission will be tested on a department linux machine. If your algorithm is a standalone program, please name it \texttt{nr},
%otherwise, please specify the compilation procedure,
%it is appreciated if you also provide a script that follows the above format to run the program.

\paragraph{Resources}
You will find several  facility location problem instances in the \texttt{data} directory provided with the handout.

%An example output file, \texttt{blabla.out}, is also provided.

%\section{Remarks}

\input{handin.tex}

\input{grading.tex}

\input{collaboration.tex}

%\paragraph{Questions} Please contact the class GTA Carleton (cjc@cs.brown.edu).

\input{warnings.tex}

%\paragraph{Hint}
%The optimal value for  \texttt{data/gc\_1000\_5} is near $85$.

\input{techReqs.tex}

\end{document}


\section{Case Study: Rössler System}

The well known Rössler system is given by the equations
\[
	\begin{array}{rcl}
		\dot x &=& -y - z \\
		\dot y &=& x + a \cdot y \\
		\dot z &=& b + z \cdot (x - c) \\
	\end{array}
\]
with parameters $a$, $b$, and $c$. For this example, $a$ and $b$
are fixed to $0.1$, while we are varying $c$.

The system has several periodic solutions for each value of $c$ with different
periodicities, though only one is stable at a time. We are interested in the origin and
branching of those solutions and, thus, drawing a bifurcation diagram using the map
\[
	\mathbf y \mapsto \{ \|\mathbf y(t)\|  \ | \ t \in [0, 2\pi), \ \mathbf y_1(t) = 0, \ \mathbf y'_1(t) > 0 \} \ .
\]
Here, $f$ denotes a single periodic solution. Note, that we use only the approximation
given by Galerkin's method.

The exploration of the bifurcations with the given tools follows roughly these steps:
\begin{enumerate}
	\item As a starting point, search for a value of $c$ such that the system has a
		stable, periodic solution and use the method described in
		\autoref{sec:initial} to find it. 4000 iterations in the transient part with step
		size of $0.01$, 120 generated intersections with the $(x=0)$-plane and tests for
		at most $30$ periods were sufficient for all initial solutions of the Rössler
		system. We started at $c=4$ with $64$ samples.
	\item Trace out the branch just by following the newly found solution in both directions
		using the predictor-corrector continuation method (\autoref{sec:cont}). The
		parameters $\kappa=0.4$, $\delta=3.0$, and $\alpha=10.0$ are a decent choice for
		the whole diagram as a good tradeoff between performance and accuracy.
	\item It is possible to detect a pitchfork bifurcation by doubling up the periodicity
		artificially. Using the described predictor-corrector method, the value
		of $c$ will eventually converge to the bifurcation point, i.e., a point with a
		singular tangent. Since the method is now stuck, reaching the doubly-periodic
		solution can be done by solving a slightly perturbed problem for a short time.
		A short perturbation of strength $0.2$ was enough to reach the doubly-periodic
		solution. Also, it might be necessary to increase the sample size due to increased
		complexity in the descending branch.
	\item Remove the perturbation and continue tracing out the bifurcation, i.e. start
		over in 1 or 2.
\end{enumerate}

\begin{figure}[!ht]
	\centering
	\subfloat[][]{\includegraphics[width=.4\textwidth]{img/roessler1a.pdf}}\quad
	\subfloat[][]{\includegraphics[width=.4\textwidth]{img/roessler1b.pdf}}\\
	\subfloat[][]{\includegraphics[width=.4\textwidth]{img/roessler1c.pdf}}\quad
	\subfloat[][]{\includegraphics[width=.4\textwidth]{img/roessler1d.pdf}}
	\caption{Tracing out the branch of solutions beginning with the solution for
	$c=4$ (a). The bifurcation point in $c \approx 5.376$ is avoided by using a permutation (b),
	and the two-periodic solutions can be traced out after removing the inaccurate values
	from before (c). Eventually, we reach (d), a widely traced out bifurcation diagram.
	Note the solutions with periodicity three and descendants, that do not originate in
	the periodicity-one branch.}
	\label{fig:reosslerseq}
\end{figure}

See figure \ref{fig:reosslerseq} for a visualization of this process, and figures \ref{fig:roesslerfull}, \ref{fig:roesslercut} for a depiction of the results.

%\begin{figure}[!ht]
	%\centering
	%\includegraphics[width=1\textwidth]{img/roessler2a.pdf}
	%\caption{The bifurcation diagram of the Rössler system for varying $c$. The
		%even-periodic solutions are colored red, the odd-periodic ones green.}
%\end{figure}
%
%
%\begin{figure}[!ht]
	%\centering
	%\includegraphics[width=1\textwidth]{img/roessler2b.pdf}
	%\caption{All found solutions for $c=\dots$ plotted into one diagram.
	%Observe how the odd-periodic and even-periodic solutions dodge since they do not
	%originate from each other.}
%\end{figure}

\newgeometry{top=0cm}

\begin{figure}
\centering\makebox[0pt]{\rotatebox{90}{\includegraphics{img/roessleroverview.png}}}
\caption{An overview of the Rössler bifurcation diagram.}
\label{fig:roesslerfull}
\end{figure}

\begin{figure}
\centering\makebox[0pt]{\rotatebox{0}{\includegraphics{img/roesslercut20.png}}}
\caption{The periodic orbits for $c=20$, together with an underlying trajectory obained through forward intergration (grey) in phase-space. The colors of the periodic solutions match the bifurcation diagram.}
\label{fig:roesslercut}
\end{figure}

\restoregeometry


% !TeX root = ./corona_contact_tracing.tex
% chktex-file 46
% !TeX spellcheck = en-GB
% !TeX encoding = utf8

\subsection{Probabilistic Health Prediction}

We model partially infected populations as a graph, where each individual (interchangeably called agent) is a node. Edges of this graph model contacts of two agents. We use graph convolution, as explained below, to update our assumptions on the current health state of all individuals.

The definition of a graph convolution~\cite{Kipf2017SemiSupervisedCW} for here is

\begin{equation}
	\label{eq:graph_convolution}
	h_{v_i}^{(l+1)} = \sum_{j\in A(i)} h_{v_j}^{(l)}
\end{equation}

with feature vector $h$ of node $i$ from iteration $(l)$ to $(l+1)$. This formulation is equivalent to the matrix formulation $h^{(l+1)} = A h^{(l)}$ with $A$ as adjacency matrix as shown in \cref{sec:consistency}. We consider here adjacency matrices without diagonal elements.

\begin{itemize}
    \item Each agent $i$ is modeled by $D$ features $h_{v_i} \in \mathbb{R}^D$. Therefore, the feature matrix, $h^{(l)}$, consists of all agents' features at time $(l)$ and is thereby of dimension $N\times D$ where there are $N$ agents in the population and each agent is described by $D$ features. A three dimensional feature space is used, $D=3$, modeling the three possible health states. The unit vectors of this space are interpreted as following:
	\begin{itemize}
		\item $\vec{e}_0$: susceptible state
		\item $\vec{e}_1$: infected state
		\item $\vec{e}_2$: recovered state
	\end{itemize}
	A uniform distribution over these possible states expresses complete uncertainty of the health state of an agent.
\end{itemize}

To accurately model the disease, there is graph contribution and an individual health contribution.

\subsubsection{Explanation of the graph contribution term}
The graph contribution models how infected agents spread the disease through contacts with susceptible agents.

\begin{equation}
    h_{v_i, m}^{(l+1)}
	=
	\underbrace{
		\sum_k \textcolor{red}{\frac{\hat{A}_{v_i, k}^{(l)}}{\sum_j \hat{A}_{v_i, j}^{(l)}}} h_{k, m}^{(l)} \textcolor{blue}{\delta_{m, e_I}}
	}_{\text{Graph}}
\end{equation}

\begin{itemize}
    \item A sum over all agents' features $h_{k,m}^{(l)}$ is weighted by the normalised infection-adjusted graph connections as shown in red.
    \item The Kronecker delta, as shown in blue, ensures that only the I feature is added as this is the only one that matters during social contacts between agents.
    \item The infection-adjusted adjacency matrix $\hat{A}$ is constructed from $A$ and $I$ which are the regular continuous adjacency matrix and the infection matrix, respectively. These three quantities are explained in the following:
    \begin{itemize}
        \item The adjacency matrix $A$ is time dependent, $A^{(l)}$, and inferred from data. In our use case, $A_{ij} = \frac{1}{dist(v_i, v_j)+\epsilon}$, hence $A_{ij}$ is large when persons $i$ and $j$ have been in contact. $\epsilon$ serves as regularization for small distances.
        \item The infection matrix is constructed as
        \begin{equation}
            I =
            \begin{pmatrix}
                0     &  0  & 0 \\
                \beta &  0  & \alpha \\
                0     &  0  & 0
            \end{pmatrix}
            =
            (I_{ij})_{i,j}
        \end{equation}
        with $i$ as the index of the host state and $j$ is the index of the contact person state. The states that we consider here are ordered as follows: susceptible, infected, recovered. $\beta$ denotes the probability of infection  after contact (also known as attack rate). $\alpha$ models the probability of being reinfected, which we assume to be zero ($\alpha=0$) based upon current medical \todo{cite!}research.
        \item $\hat{A}$, with $\hat{A}_{ij}\in [0, 1]$, is the adjacency matrix that takes the infection interactions into account and is computed as follows
        \begin{equation}
            \hat{A}_{ij} = A_{ij}\cdot \frac{ h_{v_1}^T I h_{v_2} + h_{v_2}^T I h_{v_1} }{\beta}.
        \end{equation}
        The weighted scalar product of the health states of agents $i$ and $j$ is used to evaluate whether the edge is relevant for the infection dynamics. Only when an infected person and a susceptible have contact, the edge $A_{ij}$ should be considered, otherwise it should be dropped.	The sum in the denominator comes from the fact that both, agent $i$ and $j$, can act as host during a contact. The division by $\beta$ normalises the factor to one to ensure $\hat{A}_{ij} \in [0, 1]$. Since $I$ is not symmetric, $p_a$ is a proper normalization because the sum is in $\{0, p_a\}$. Note that the fraction has the desired properties for pure $S$-, $I$- and $R$-persons.
    \end{itemize}
\end{itemize}

\subsubsection{Temporal}
% \item Explanation of the temporal term:
The transition of a persons' health state $h_{v_i}^{(l)}$ is determined by the following assumptions:
\begin{itemize}
	\item A susceptible person always stays susceptible
	\item An infected person has a probability $\gamma$, called recovery rate, to recover. The remaining probability $1-\gamma$ denotes that the person stays sick.
	\item A recovered person could have a probability to be re-infected, but we assume this to be zero. Thus a recovered person always stays recovered.
\end{itemize}
Thus the temporal transition matrix $T$ is:
\begin{equation}
	T =
	\begin{pmatrix}
		1 &     0    & 0      \\
		0 & 1-\gamma & \gamma \\
		0 &     0    & 1      \\
	\end{pmatrix}
\end{equation}
The temporal update rule based on the health status thus becomes:
\begin{equation}
	H^{(l+1)} = H^{(l)} T
\end{equation}

\subsubsection{Joint Model}

Our main propagation rule is based on the definition of a graph convolution shown in Eq.~\eqref{eq:graph_convolution} and reads as follows in component notation
\begin{equation}
	h_{v_i, m}^{(l+1)}
	=
	\underbrace{
		\sum_k \textcolor{red}{\frac{\hat{A}_{v_i, k}^{(l)}}{\sum_j \hat{A}_{v_i, j}^{(l)}}} h_{k, m}^{(l)} \textcolor{blue}{\delta_{m, e_I}}
	}_{\text{Graph}}
	+
	\underbrace{
		{(h_{v_i}^{(l)}\cdot T)}_m
	}_{\text{Temporal}}
\end{equation}
with $m$ being the index of the health state.

This propagation rule is based on the following aspects:

\begin{itemize}
	\item The propagation consists of two parts, first the graph contribution and second the temporal contribution. While the former captures the dynamics of infections based on the social contacts between agents, the former ensures that an infected agent heals over time and becomes resistant against the Corona virus.
\end{itemize}


\section{Differentiation}
Until further notice, we will just study differentiation of a function $f:I\to\mathbb R$ where $I$ is an interval.
\subsection{The Derivative}
\begin{definition}
    If $a\in I$ and we have
    $$\lim_{x\to a}\frac{f(x)-f(a)}{x-a}$$
    exists and equals $c\in\mathbb R$, we say $f$ is differentiable at $a$ and the derivative is $f^\prime(a)=c$.
    Moreover, if $f$ is differentiable at all $a\in I$, we say $f$ is differentiable.
\end{definition}
Geometrically $f^\prime(a)$ is the slope of the tangent line of $f$ at $a$.\\
Another (formal) way to put this is to observe that the definition is essentially $\forall \epsilon>0,\exists\delta>0$,
$$|x-a|<\delta\implies\left|\frac{f(x)-f(a)}{x-a}-c\right|<\epsilon\implies |f(x)-f(a)-c(x-a)|<\epsilon|x-a|$$
Another way we can write is
$$f^\prime(a)=\lim_{h\to 0}\frac{f(a+h)-f(a)}{h}$$
where $h\in I-a$.
\begin{example}
    1. The constant function is differentiable everywhere and has zero derivative everywhere.\\
    2. The identity function is differentiable everywhere and has derivative $1$.\\
    3. $f:(0,\infty)\to\mathbb R$ by $x\mapsto 1/x$ is differentiable and has derivative $f^\prime(a)=-a^{-2}$.\\
    4. (non-example) The function $x\mapsto |x|$ is not differentiable at $0$.
\end{example}
\begin{lemma}
    Suppose $f$ is differentiable at $a$ with derivative $f^\prime(a)$ iff
    $$f(a+h)=f(a)+f^\prime(a)h+\alpha(h)h$$
    where $\alpha(0)=0$ and $\alpha$ is continuous at $0$.
\end{lemma}
Note that given this formula, then $\alpha(h)$ is completely deterined by $f$ and $f^\prime(a)$.
\begin{proof}
    Obvious.
\end{proof}
\begin{corollary}
    $f$ is differentiable at $a$ implies $f$ is continuous at $a$.
\end{corollary}
\begin{proof}
    Follows directly.
\end{proof}
\begin{proposition}
    If $f,g:I\to\mathbb R$ are differentiable at some $a\in I$, so are $f+g,fg$ with derivatives $f^\prime(a)+g^\prime(a),f^\prime(a)g(a)+f(a)g^\prime(a)$ respectively.
\end{proposition}
\begin{proof}
    Trivial.
\end{proof}
\begin{theorem}[Chain Rule]
    Suppose $f:I_1\to I_2$ is differentiable at $a\in I_1$ and $g:I_2\to\mathbb R$ is differentiable at $f(a)$, then $(g\circ f)^\prime(a)=g^\prime(f(a))f^\prime(a)$.
\end{theorem}
\begin{proof}
    Let $b=f(a)$, then by our conditions,
    $$f(a+h)=f(a)+f^\prime(a)h+\alpha(h)h,g(b+k)=g(b)+g^\prime(b)k+\beta(k)k$$
    And $\alpha(0)=\beta(0)=0$ and they are both continuous at $0$.
    Take $k=f(a+h)-f(a)=f^\prime(a)h+\alpha(h)h$, so
    \begin{align*}
        (g\circ f)(a)&=g(b+k)\\
        &=g(b)+g^\prime(b)k+\beta(k)k\\
        &=g(b)+g^\prime(b)(f^\prime(a)h+\alpha(h)h)+\beta(f^\prime(a)h+\alpha(h)h)(f^\prime(a)h+\alpha(h)h)\\
        &=(g\circ f)(a)+g^\prime(f(a))f^\prime(a)h+\phi(h)h
    \end{align*}
    Where $\phi(h)=g^\prime(b)\alpha(h)+\beta(f^\prime(a)h+\alpha(h)h)(f^\prime(a)+\alpha(h))$ which is $0$ at $0$ and is continuous at $0$.
    The chain rule follows.
\end{proof}
\begin{corollary}
    Suppose $f,g:I\to\mathbb R$ are differentiable at $a$ and $g(a)\neq 0$, then $f/g$ is differentiable at $a$ and $(f/g)^\prime=(f^\prime g-fg^\prime)/g^2$.
\end{corollary}
\begin{proof}
    Immediate from the chain rule.
\end{proof}
\begin{corollary}
    The polynomial $p(x)=a_0+a_1x+\cdots+a_nx^n$ is differentiable at all $x\in\mathbb R$, and if $p,q$ are polynomials, $p/q$ is differentiable at $a$ when $q(a)\neq 0$.
\end{corollary}
\begin{proof}
    Trivial.
\end{proof}
\subsection{Mean Value Theorem}
Suppose we have some $f:I\to\mathbb R$.
\begin{definition}
    We say some $c\in I$ is a global maximum of $f$ if $\forall x\in I,f(x)\le f(c)$ and a global minimum if $\forall x\in I,f(x)\ge f(c)$.
\end{definition}
We know that if $I$ is a closed bounded interval, then $f$ has global maxima and minima.
\begin{definition}
    Some $c\in I$ is called a local maximum of $f$ if $\exists\epsilon>0,\forall x\in (c-\epsilon,c+\epsilon)\cap I,f(x)\le f(c)$.
    Similarly it is a local minimum if $\exists\epsilon>0,\forall x\in (c-\epsilon,c+\epsilon)\cap I,f(x)\ge f(c)$.
\end{definition}
\begin{definition}
    $c\in I$ is called an interior point of $I$ if $\exists \epsilon>0,(c-\epsilon,c+\epsilon)\subset I$.
\end{definition}
\begin{proposition}
    Suppose $f:I\to\mathbb R$ and $c$ is an interior point of $I$ where $f$ attains local maximum or local minimum at $c$ and $f$ is differentiable at $c$, then $f^\prime(c)=0$.
\end{proposition}
\begin{proof}
    Suffice to prove the local maximum case.
    Since $c$ is an interior point of $I$, we can choose $\epsilon'>0,(c-\epsilon',c+\epsilon')\subset I$.
    As $c$ is a local maximum, then there is some $\epsilon$ such that $0<\epsilon\le\epsilon'$ and that $\forall x\in (c-\epsilon,c+\epsilon),f(x)\le f(c)$.
    We can easily find sequences $(x_n^-)\in (c-\epsilon,c),(x_n^+)\in (c,c+\epsilon)$ both converging to $c$.
    Let $g:(c-\epsilon,c+\epsilon)\setminus\{c\}\to\mathbb R$ defined by $g(x)=(f(x)-f(c))/(x-c)$, then
    \begin{align*}
        0&\le\lim_{n\to\infty}\frac{f(x_n^-)-f(c)}{x_n^--c}\\
        &=\lim_{x\to c^-}g(x)=\lim_{x\to c}g(x)=\lim_{x\to c^+}g(x)\\
        &=\lim_{n\to\infty}\frac{f(x_n^+)-f(c)}{x_n^+-c} \le 0
    \end{align*}
    Hence $f^\prime(c)=\lim_{x\to c}g(x)=0$.
\end{proof}
\begin{theorem}[Rolle's Theorem]
    Suppose $f:[a,b]\to\mathbb R$ is continuous and is differentiable in $(a,b)$ and $f(a)=f(b)=0$, then there is some $c\in (a,b)$, $f^\prime(c)=0$.
\end{theorem}
\begin{proof}
    It is obvious if $f\equiv 0$, otherwise WLOG $f$ attains some positive value.
    Then let $c$ be a global maximum of $f$, then $c$ is a local maximum and an interior point of $f$ (since $f$ is zero on endpoints but we have assumed that $f$ attains some positive value), then $c\in (a,b)$ and we have $f^\prime(c)=0$ by the preceding proposition.
\end{proof}
\begin{theorem}[Mean Value Theorem]
    If $f:[a,b]\to\mathbb R$ is continuous on $[a,b]$ and differentiable on $(a,b)$, then there is some $c\in (a,b)$ with $f^\prime(c)=(f(b)-f(a))/(b-a)$.
\end{theorem}
\begin{proof}
    Let
    $$p(x)=\frac{1}{b-a}((x-a)f(b)-(x-b)f(a))$$
    which although looks dreadful is just a linear polynomial.
    Note that $p(a)=f(a),p(b)=f(b)$, so it is just a line joining the endpoints.
    Then $g(x)=f(x)-p(x)$ is continuous on $[a,b]$ and differentiable on $(a,b)$, so by Rolle's Theorem, there is some $c\in (a,b)$ sucb that
    $$0=g^\prime(c)=f^\prime(c)-\frac{f(b)-f(a)}{b-a}\implies f^\prime(c)=\frac{f(b)-f(a)}{b-a}$$
    As desired.
\end{proof}
\begin{proposition}
    Suppose $f:[a,b]\to\mathbb R$ is continuous and differentiable in $(a,b)$.\\
    1. $f^\prime\equiv 0\iff f$ is constant.\\
    2. $f^\prime\ge 0\iff f$ is increasing.\\
    3. $f^\prime>0\implies f$ is strictly increasing.
\end{proposition}
\begin{proof}
    Immediate from Mean Value Theorem.
\end{proof}
Note that the converse of the third statement is false in general.
E.g, $f(x)=x^3$.\\
Note that all these depends on the least upper bound property since we used the Mean Value Theorem which depends on Rolle's Theorem which depends on maximum value theorem which depends on Bolzano-Weierstrass Theorem which depends on the Monotone Sequence Theorem which depends on the least-upper-bound property.
\subsubsection{Inverse Function Theorem}
\begin{lemma}
    If $f:[a,b]\to\mathbb R$ satisfies $f^\prime(x)>0$ for all $x\in [a,b]$, then $f:[a,b]\to[f(a),f(b)]$ is a bijection.
\end{lemma}
\begin{proof}
    Injectivity is obvious by Mean Value Theorem.
    For surjectivity we can just exploit Intermediate Value Theorem.
\end{proof}
\begin{lemma}
    Let $I$ be an interval.
    Suppose $f:I\to f(I)$ is a continuous bijection, then $f^{-1}:f(I)\to I$ is also continuous.
\end{lemma}
\begin{proof}
    Just check definition.
\end{proof}
\begin{theorem}[Inverse Function Theorem]
    Suppose $I\subset R$ is an interval and $f^\prime(x)>0$ for any $x\in I$, then the continuous inverse (exists by the preceding lemmas) $f^{-1}:f(I)\to I$ is differentiable and $(f^{-1})^\prime(y)=1/f^\prime(f^{-1}(y))$.
\end{theorem}
\begin{example}
    1. $f:x\mapsto x^n$ on $(0,\infty)$ satisfies the conditions, so its inverse $f^{-1}:y\mapsto y^{1/n}$ has $(f^{-1})^\prime(y)=(1/n)y^{1/n-1}$.\\
    2. Consider $\tan:(-\pi/2,\pi/2)\to\mathbb R$, it shall be invertible with differentiable inverse $\tan^{-1}$, then we can calculate, by the formula, that $(\tan^{-1})(y)=1/(y^2+1)$
\end{example}
\begin{proof}[Proof assuming differentiability]
    Assuming $f^{-1}$ is differentiable, then we know that $f(f^{-1}(y))=y$, then by the chain rule, we get $f^\prime(f^{-1}(y))(f^{-1})^\prime(y)=1$, rearrange to give the formula.
\end{proof}
\begin{proof}[Actual proof]
    Fix $b\in f(I)$ and let $a=f^{-1}(b)$.
    Suppose $(y_n)$ is a sequence in $f(I)\setminus \{b\}$ with $y_n\to b$, and let $x_n=f^{-1}(y_n)$.
    We know that $f$ is differentiable hence continuous, so $f^{-1}$ is continuous by the preceding lemma.
    Therefore $x_n=f^{-1}(y_n)\to f^{-1}(b)=a$.
    Since we know that
    $$\lim_{x\to a}\frac{f(x)-f(a)}{x-a}=f^\prime(a)$$
    We have
    $$\frac{f(x_n)-f(a)}{x_n-a}\to f^\prime(a)$$
    By hypothesis, $f^\prime(a)\neq 0$, so
    $$\frac{f^{-1}(y_n)-f^{-1}(b)}{y_n-b}=\frac{x_n-a}{f(x_n)-f(a)}\to\frac{1}{f^\prime(a)}=\frac{1}{f^\prime(f^{-1}(b))}$$
    which is true for every sequence $y_n\to b$, so the theorem is proved.
\end{proof}
\subsubsection{L'H\^opital's Rule}
\begin{proposition}[L'H\^opital's Rule]
    Suppose $f,g:I\to\mathbb R$ are differentiable with $f(a)=g(a)=0$, and $\exists r>0, g(x)\neq 0\neq g^\prime(x)$ for $0<|x-a|<r$.
    Then if $\lim_{x\to a}f^\prime(a)/g^\prime(a)$ exists and equals $k$, then
    $$\lim_{x\to a}\frac{f(x)}{g(x)}=k$$
\end{proposition}
\begin{proof}
    Suppose $x\in I$ and $x>a$.
    Consider $h(t)=f(t)g(x)-f(x)g(t),t\in [a,x]$.
    Since $f,g$ are differentiable and we have $h(a)=h(a)=0$, so by Rolle's Theorem, there is some $c\in (a,x)$ with $f^\prime(c)g(x)-f(x)g^\prime(c)=h^\prime(c)=0$.
    Therefore
    $$\frac{f^\prime(c)}{g^\prime(c)}=\frac{f(x)}{g(x)}$$
    Same for $x<a$.\\
    Given $\epsilon>0$, choose $\delta>0$ such that
    $$0<|c-a|<\delta\implies\left|\frac{f^\prime(c)}{g^\prime(c)}-k\right|<\epsilon$$
    So when $0<|x-a|<\delta$, one can find $c$ such that $0<|c-a|<|x-a|<\delta$ and $f^\prime(c)/g^\prime(c)=f(x)/g(x)$, which means that
    $$\left|\frac{f(x)}{g(x)}-k\right|=\left|\frac{f^\prime(c)}{g^\prime(c)}-k\right|<\epsilon$$
    which is as we wanted.
\end{proof}
\subsection{Higher Derivatives and Taylor Series}
\begin{definition}
    We say (inductively) that $f:I\to\mathbb R$ is $k$ times differentiable with $k^{th}$ derivative $f^{(k)}$ if $f$ is $(k-1)$ times differentiable with differentiable $(k-1)^{th}$ derivative $f^{(k-1)}$ and $f^{(k-1)})^\prime=f^{(k)}$.
\end{definition}
\begin{example}
    A polynomial is $k$ times differentiable for any $k\in\mathbb N$.
\end{example}
\begin{definition}
    We say $f$ is $C^k$ if $f$ is $k$ times differentiable and $f^{(k)}$ is continuous.
\end{definition}
\begin{example}[Non-example]
    Consider
    $$f(x)=\begin{cases}
        x^2\sin(1/x)\text{, for $x\neq 0$}\\
        0\text{, for $x=0$}
    \end{cases}$$
    So one can verify that $f$ is differentiable but the derivative, namely,
    $$f^\prime(x)=\begin{cases}
        2x\sin(1/x)-\cos(1/x)\text{, for $x\neq 0$}\\
        0\text{, for $x=0$}
    \end{cases}$$
    is not continuous at $0$.
\end{example}
Now, given $f:I\to\mathbb R$, we want a degree $k$ polynomial that best approximates $f$.
The answer depends on the meaning of ``best''.
We can either calculate the total squared difference (we have not defined it and will not care about it), but on the other hand, the theory of differentiation hints that
$$f(x)=f(a)+f^\prime(a)(x-a)=P_1(x)$$
is the best linear approximation of $f$ around $a$.
Why ``around $a$''?
say for example we want to approximate $\sin x$ over a large interval, then the best approximation intuitively might be a line with slope very close to $0$, which is obviously not the tangent at, e.g., $0$.
And we are only interested in the best approximation close to $a$.\\
Now back to our best linear approximation.
The reason we say it is the ``best'' in the sense that $f(x)-P_1(h)=\alpha(h)h$ with $\alpha(0)=0$ and $\alpha$ is continuous at $0$.
We can also easily know that such a $P_1$ is unique so as for such an $\alpha$ to exist by working out their coefficients by substitution and differentiating (with appropriate comments on differentiability, of course).
More generally, for a sufficiently smooth $f$, we can try to approximate $f(a+h)=P_k(a+h)+h^k\alpha(h)$ where $\alpha(0)=0$ and $\alpha$ is continuous at $0$ and $P_k$ is a polynomial.
Using the same trick, such an $P_k$ is unique.
In particular, we obtain $P_k^{(r)}=f^{(r)}$ for any $r\le k$.
\begin{lemma}
    If $a,c_0,c_1,\ldots,c_k\in\mathbb R$, then there is a unique degree $k$ polynomial $P$ with $P^{(i)}(a)=c_i$.
\end{lemma}
\begin{proof}
    Write out $P(x)=a_0+a_1(x-a)+\cdots+a_k(x-a)^k$ and differentiating gives $i!a_i=c_i=P^{(i)}(a)$, so the polynomial is uniquely determined.
\end{proof}
\begin{definition}
    Given $f:I\to\mathbb R$ which is $k$ times differentiable and $a$ is an interior point of $I$, then the $k^{th}$ Taylor polynomial of $f$ centered at $a$ is
    $$P_k(x)=\sum_{i=0}^k\frac{f^{(i)}(a)}{i!}(x-a)^i$$
    which is chosen such that $P_k^{(i)}(a)=f^{(i)}(a)$.
\end{definition}
\begin{remark}
    $P_k(x)$ is the unique polynomial with degree at most $k$ having the property $P_k^{(i)}(a)=f^{(i)}(a)$ by the preceding lemma.
\end{remark}
\begin{theorem}[Taylor's Theorem with Remainder]
    Suppose $f:I\to\mathbb R$ is $(k+1)$ times differentiable, and $[a,x]\subset I$, then
    $$f(x)=\left(\sum_{i=0}^k\frac{f^{(i)}(a)}{i!}(x-a)^i\right)+\frac{f^{(k+1)}(c)}{(k+1)!}(x-a)^{k+1}$$
    for some $c\in (a,x)$.
\end{theorem}
For $k=0$, this reduces to the mean value theorem.
\begin{proof}
    Fix $x>a$ and define
    $$g(t)=f(t)+\sum_{i=1}^k\frac{f^{(i)}(t)}{i!}(x-t)^i+\alpha\frac{(x-t)^{k+1}}{(k+1)!}$$
    where $\alpha$ is chosen such that $g(a)=f(x)$ (which exists as a solution to a nondegenerate linear equation).
    Our goal is to find $c\in (a,x)$ such that $f^{(k+1)}(c)=\alpha$.
    We have, by definition,
    $$f(x)=g(a)=f(a)+\sum_{i=1}^k\frac{f^{(i)}(a)}{i!}(x-a)^i+\alpha\frac{(x-a)^{k+1}}{(k+1)!}$$
    Now $f$ is $k+1$ times differentiable so $g$ is differentiable and $g(a)=f(x)=g(x)$, so by Rolle's Theorem, there is some $c\in (a,x)$ such that $g^\prime(c)=0$.
    But we have
    $$g^\prime(t)=f^\prime(t)+\sum_{i=1}^k\left(\frac{f^{(i+1)}(t)}{i!}(x-t)^i-\frac{f^{(i)}(t)}{(i-1)!}(x-t)^{i-1}\right)-\alpha\frac{(x-t)^k}{k!}$$
    The first part of this expression is telescoping, so we can simplify it to
    $$g^\prime(t)=(f^{(k+1)}(t)-\alpha)\frac{(x-t)^k}{k!}$$
    Now plugging in $t=c$ shows $f^{(k+1)}(c)=\alpha$, which completes the proof.
\end{proof}
If $f$ is infinitely differentiable, one can then form the Taylor series
$$\sum_{i=0}^\infty\frac{f^{(i)}(a)}{i!}(x-a)^i$$
as the formal limit of the Taylor polynomilas $P_k(x)$, which sometimes does converge and equals $f$.
If we understand the derivatives of $f$ well-enough and that those derivatives do behave nicely, then it is sometimes possible to use Taylor's Theorem to prove that
$$p(x)=\lim_{n\to\infty}P_n(x)=f(x)$$
\begin{example}
    Take $f(x)=\cos x$ (which, albeit hasn't been properly defined, we shall assume standard facts about), then the Taylor series about $0$ is
    $$1-\frac{x^2}{2!}+\frac{x^4}{4!}-\frac{x^6}{6!}+\cdots=\sum_{n=0}^\infty(-1)^n\frac{x^{2n}}{(2n)!}$$
    The series always converges by the ratio test.
    Must the series converges to $\cos x$ then?
    The $(2k)^{th}$ partial sum of the series is exactly $P_{2k}(x)$ of $\cos$ about $0$, then there is $c\in (0,x)$ with
    $$|f(x)-P_{2k}(x)|=\left|\frac{f^{2k+1}(c)}{(2k+1)!}x^{2k+1}\right|\le\frac{|x|^{2k+1}}{(2k+1)!}\to 0$$
    as $k\to\infty$, so the series does converge to $\cos x$.
\end{example}
\begin{definition}
    We say $f:I\to\mathbb R$ is $C^\infty$ (or smooth) if $f$ is $C^k$ for all $k\in\mathbb N$.
\end{definition}
So for a smooth $f:I\to\mathbb R$, one can form the Taylor series (formally)
$$\sum_{n=0}^\infty\frac{f^{(n)}(a)}{n!}(x-a)^n$$
But even if the series converges for all $x\in I$, it may not converge to $f(x)$ for $x\neq a$.
Philosophically, one cannot fully expect that the behaviour of $f$ near $a$ control its global behaviour.
\footnote{For complex-valued functions, however, the local behaviour of a nice enough function $f$ can be controlled by its local properties.}
Indeed, for $f_1,f_2:\mathbb R\to\mathbb R$ smooth and $a<b$, it is possible to construct some smooth $f:\mathbb R\to\mathbb R$ such that $f|_{<a}=f_1|_{<a}$ and $f|_{>b}=f_2|_{>b}$.
This will be proven in example sheet.
\subsection{Complex Differentiation}
\begin{definition}
    If $c\in\mathbb C,r\in[0,\infty)$, we define the open ball of radius $r$ centered at $c$ by $B_r(c)=\{z\in\mathbb C:|z-c|<r\}$.
\end{definition}
\begin{definition}
    A subset $\Omega\subset\mathbb C$ is open if $\forall c\in\Omega,\exists\epsilon>0,B_\epsilon(c)\subset\Omega$.
\end{definition}
\begin{example}
    1. $\mathbb C,\varnothing$ are open.\\
    2. Open balls are open.\\
    3. (non-example) $\{z\in\mathbb C:|z-a|\le r\}$ is not open, but its complement is.
\end{example}
\begin{definition}
    Suppose $\Omega\in\mathbb C$ is open and $c\in\Omega$.
    For a function $f:\Omega\to\mathbb C$, we say $f$ is complex differentiable at $c$ if
    $$\lim_{z\to c}\frac{f(z)-f(c)}{z-c}$$
    exists.
    In case it does exist, we write it as $f^\prime(c)$.
\end{definition}
All the rules in real differentiation still applies, but we don't really have mean value theorem.
We can write $f(x+iy)=u(x,y)+iv(x,y)$ where $u,v:\tilde{\Omega}\to\mathbb R$ where $\tilde{\Omega}=\{(x,y)\in\mathbb R^2:x+iy\in\Omega\}$.
\begin{definition}
    Say $\tilde{\Omega}$ is open if $\Omega$ is.
    Then the partial derivatives of $F:\tilde{\Omega}\to\mathbb R$ at $(a,b)\in\tilde{\Omega}$ are
    \begin{align*}
        \left.\frac{\partial F}{\partial x}\right|_{(a,b)}&=\lim_{h\to 0}\frac{F(a+h,b)-F(a,b)}{h}\\
        \left.\frac{\partial F}{\partial y}\right|_{(a,b)}&=\lim_{h\to 0}\frac{F(a,b+h)-F(a,b)}{h}
    \end{align*}
    Provided that they exist.
\end{definition}
Alternatively we can define $g_b=F(x,b)$ and so
$$\left.\frac{\partial F}{\partial x}\right|_{(a,b)}=g^\prime_b(a)$$
Similar for the other component.
\begin{proposition}
    If $f:\Omega\to\mathbb C$ is complex differentiable at $c=a+ib$ with derivative $f^\prime(c)=\alpha+i\beta$, then the partial derivatives of $u,v$ at $(a,b)$ exist and satisfy
    \begin{align*}
        \left.\frac{\partial u}{\partial x}\right|_{(a,b}&= \left.\frac{\partial v}{\partial y}\right|_{(a,b)}=\alpha\\
        -\left.\frac{\partial u}{\partial y}\right|_{(a,b}&=\left.\frac{\partial v}{\partial x}\right|_{(a,b)}=\beta
    \end{align*}
\end{proposition}
\begin{proof}
    For any sequence $h_n\to 0$ in $\mathbb R\setminus\{0\}$, then it is also a sequence in $\mathbb C\setminus\{0\}$, so since $f$ is differentiable at $c$,
    \begin{align*}
        \alpha+i\beta&=f^\prime(c)\\
        &=\lim_{h\to 0}\frac{f(c+h)-f(c)}{h}\\
        &=\lim_{n\to\infty}\frac{u(a+h_n,b)-u(a,b)}{h_n}+i\lim_{n\to\infty}\frac{v(a+h_n,b)-v(a,b)}{h_n}
    \end{align*}
    So $\partial u/\partial x$ and $\partial v/\partial x$ exist and are $\alpha,\beta$ respectively.
    We can get the other half of the equations by replacing $h_n$ by $ih_n$ and casting exactly the same argument.
\end{proof}
\begin{remark}
    If $f:\Omega\to\mathbb C$ is complex differentiable, then (assuming $C^1$ partial derivatives and symmetry of partial derivatives)
    $$\frac{\partial^2u}{\partial x^2}+\frac{\partial^2u}{\partial y^2}=0$$
    which is the Laplace Equation.
\end{remark}
\begin{proposition}
    Suppose $F:B_r((a,b))\to\mathbb R$ has partial derivatives with
    $$\frac{\partial F}{\partial x}=\frac{\partial F}{\partial y}=0$$
    then $F$ is constant.
\end{proposition}
\begin{proof}
    For $(a',b')\in B_r((a,b))$, we consider the functions $f_b(x)=F(x,b)$ and $f_a(x)=F(a',x)$ on the reals.
    Then they are differentiable and using mean value theorem on $f_b$ wrt $a,a'$ shows $f(a,b)=f(a',b)$, and using it again on $f_a$ wrt $b,b'$ shows $f(a',b)=f(a',b')$, so $f(a,b)=f(a',b')$.
\end{proof}
\begin{corollary}
    If $f:B_r(c)\to\mathbb C$ has derivative zero, then $f$ is constant.
\end{corollary}
\begin{proof}
    Immediate.
\end{proof}
\begin{remark}
    Complex differentiable functions $f:\mathbb C\to\mathbb C$ behave very nicely.
    Firstly, if $f$ is complex differentiable, so is $f^\prime$, consequently $f$ is $C^\infty$.
    Even better, its Taylor series always converges to itself.
    Secondly, if $f$ is bounded then $f$ is constant.
\end{remark}


\section{Notation}

A Markov decision process (MDP) $\mdp$ consists of a tuple $(\mathcal{S}, \mathcal{A}, P, R, \gamma, s_0)$, where $\mathcal{S}$ is the state space, $\mathcal{A}$ is the action space, $P$ is the transition function mapping $\mathcal{S} \times \mathcal{A}$ to distributions on $\mathcal{S}$, $R$ is the scalar reward function on $\mathcal{S} \times \mathcal{A}$, $\gamma$ is the discount factor, and $s_0$ is the starting state.
Note that a single start state can be converted to a start distribution by letting $P(s_0, \cdot)$ be independent of the action taken.
% This work focuses on the episodic setting, where after some number of steps $T$
An \emph{agent} interacts with an MDP by producing a policy $\pi$ at each timestep and observing the transitions it visits.

The \emph{value} of a state $s$ for a policy $\pi$ is the (discounted) sum of future rewards obtained by running that policy starting from the state $s$:
\begin{align}
    V^{\pi}(s) = \E_{\substack{a_t \sim \pi(\cdot \mid s_t) \\ s_{t+1} \sim P(s_t, a_t)}} \sum_{t=0}^\infty \gamma^t R(s_t, a_t) ~\big|~ s_0=s
\end{align}
Similarly the value of a state-action pair $(s, a)$ is written as
\begin{align}
    Q^{\pi}(s, a) = \E_{\substack{a_t \sim \pi(\cdot \mid s_t) \\ s_{t+1} \sim P(s_t, a_t)}} \sum_{t=0}^\infty \gamma^t R(s_t, a_t) ~\big|~ s_0=s, a_0 = a
\end{align}
Any MDP admits a deterministic optimal policy $\pi^*$ with corresponding value functions $V^*$ and $Q^*$ such that $V^*(s) \ge V^\pi(s)$ for all $s$ and $\pi$ \citep{sutton2018reinforcement}.

\section{Discounting and Resets}

When $\gamma < 1$ we say that the setting is \emph{discounted}, and for $\gamma = 1$ we say that it is \emph{undiscounted}.
An environment may have a time limit $T$ such that after every $T$ steps, the state is reset to $s_0$.
In this case we call it \emph{episodic}.
To satisfy the Markov property, episodic environments should include the current timestep $t$ in the state.

For episodic environments, it is natural to define a policy's quality in terms of the total reward earned (on average) in a single episode, with discounted environments preferring rewards obtained earlier in the episode.
For non-episodic environments comparisons between policies are less clear-cut; see Section 2 of \citet{strehl2008analysis} for a discussion.

Common practice in deep reinforcement learning is to train agents with discounting and resets, but not include the timestep in the state observations and simply ignore the transition from $s_T$ to $s_0$.
Policies are typically evaluated by the total undiscounted reward in an episode, despite the conflict with the training setup.
In most cases the rewards are truncated to reflect the limited episode duration \citep{schulman2017proximal,Fujimoto2018AddressingFA,haarnoja2018soft}.
Since the agent is unable to tell when an episode will end, this effectively introduces noise into value prediction targets, and this noise varies by state depending on how often the agent has ended an episode on that state.
In other cases value targets may be bootstrapped from the state $s_T$ as if the environment were not episodic.
This has two issues: (1) it introduces bias by treating an estimate of $V(s_T)$ as the true value, when in some states and environments this estimate may never be updated; and (2) by pretending the environment has no resets, it introduces a mismatch between the training and test objectives.\endnote{This mismatch is decreased if the divergence between the distributions $P(s_T, \pi(\cdot \mid s_T))$ and $s_0$ is smaller. Like most problems in RL, it also becomes smaller if smaller discount factors are used. However in general there are actions which would provide more short-term reward, and thus perform better toward the end of an episode, than the infinite-horizon optimal actions.}
However, it does not introduce noise.

More fundamentally, practically all discounted policy gradient algorithms drop the discounting term from the state distribution.
\citet{Nota2020IsTP} show that this results in following a direction which is not the gradient of any function, and which is not guaranteed to converge to a good solution with respect to the discounted or undiscounted objectives.
While this is deeply worrying, these methods frequently work well in practice.
This may be due to their usage with overparameterized neural network models, which are largely invariant to a reweighting of the data \citep{byrd2018effect,brandfonbrener2021offlinecontextual}.

% This thesis focuses on the episodic setting.
% Following common practice in deep reinforcement learning, agents are trained with algorithms which use $\gamma < 1$, but evaluated according to the total reward earned in an episode.

\section{Defining Sample Complexity}

We say that a policy $\pi$ is $\varepsilon$-optimal if $V^*(s_0) \le V^\pi(s_0) + \varepsilon$.
Define $\bm{\pi} = A(\mdp, i)$ to be the policy obtained by running the agent (i.e. algorithm) $A$ in the environment $\mdp$ for $i$ timesteps, being careful to note that the policy $\bm{\pi}$ is itself a random variable due to the randomness in the experience collected in those $i$ steps.
Let the \emph{sample complexity} of learning a $\varepsilon$-optimal policy on $\mdp$ with $A$ be the expected number of steps (indexed as $i$) such that
\begin{align}
    V^{\pi_i}(s_0) < V^*(s_0) - \varepsilon, && \text{where } \pi_i = A(\mdp, i).
\end{align}
This definition is related to those proposed by \citet{Fiechter1994EfficientRL} and \citet{strehl2008analysis} for PAC learning.
Sometimes it is also useful to consider the "anytime" performance of an algorithm $A$ on an MDP $\mdp$.
An algorithm $A_1$ would dominate $A_2$ if $\forall i$, $V^{A_1(\mdp, i)}(s_0) \ge V^{A_2(\mdp, i)}(s_0)$.

\section{Online and Deployment Settings} \label{sec:regret-deployment}

While the overall MDP framework is largely shared in the community, several different objectives for a learning agent are commonly studied.
The online and offline settings are perhaps the most studied in the theory community, but the "learn-and-deploy" setting has the most relevance for present applications of RL.

\paragraph{The online setting.}
% \side{talk about sample complexity}
Here an RL agent learns by continually interacting with the environment with the goal of maximizing the total reward earned across all time.
This gives rise to the explore-exploit tradeoff when acting: at each moment, the agent may choose to take an action which is uninformative but leads to greater short-term reward, or one which will yield more information at the cost of lower reward.
The objective for this setting is to minimize the rate of accumulation of \emph{regret}, which measures the difference between the total reward obtained by an optimal policy and the agent:
\begin{align}
    L(A, \mdp, T) = \E \left[ \sum_{i=1}^T R(s^*_i, a^*_i) - R(s_i, a_i) \right].
\end{align}
This setting is appropriate when an agent is being trained "on the job," where mistakes early in learning have just as much deleterious effect as those made later.
In this context, sample efficiency controls the total amount of regret the agent will accumulate.

\paragraph{The learn-and-deploy setting.}
% \side{talk about sample complexity}
This setting consists of distinct learning and deployment phases.
In the learning phase, the agent is not required to perform well and may collect whatever data is most informative.
In the deployment phase, the policy is fixed and should be as close to optimal as possible.
Note that the policy produced at the end of the training procedure need bear no resemblance to those used to collect data.
For episodic environments, with a training period consisting of $N$ steps we can write the this objective as
\begin{align}
    L(A, \mdp, N) = V^*(s_0) - V^{\pi_N}(s_0), && \text{where } \pi_N = A(\mdp, N)
\end{align}
Historically this setting has not been much discussed, though it is analagous to the task of best-arm identification in bandits \citep{Russo2016SimpleBA,Kaufmann2016OnTC}.
It is also related to the iterative technique of fitted Q iteration \citep{Ernst2005TreeBasedBM,Riedmiller2005NeuralFQ}.

Crucially, this setting is the one used in practice in nearly every application of reinforcement learning.
RL algorithms are not yet safe and reliable enough to allow them to update a policy on the fly, especially with a physical system which may be damaged or cause injury.
Furthermore, most works studying RL implicitly provide results in this setting by evaluating according to a different policy than the one used for training, for example showing learning curves with a deterministic policy \citep{Mnih2015HumanlevelCT,Lillicrap2016ContinuousCW,Fujimoto2018AddressingFA,haarnoja2018soft}.
Comparisons of final, large-data performance similarly reflect this setting \citep{Silver2016MasteringTG,Vinyals2019GrandmasterLI,openai2019dota,OpenAI2019SolvingRC}.

In this setting, sample efficiency reflects the amount of data the agent needs to collect during training time before it can perform near-optimally at deployment time.

\paragraph{The offline setting.}
% \side{talk about sample complexity}
Also known as the \emph{batch} setting, this consists only of pure policy optimization given a fixed dataset of environment interactions collected by an extrinsic behavior policy.
After learning from this data in whatever way it sees fit, an algorithm produces a fixed policy with the objective of earning as much reward as possible.
Though described as a reinforcement learning setting, it does not include any actual reinforcement as the agent never learns from its own interactions with the environment.
However, this makes the offline RL setting uniquely valuable for isolating how much can be learned from particular data.
This setting is also appealing as it would in principle allow an agent to be trained in a risk-free way by using data collected from a safe policy, and for free (in terms of samples) if data from one experiment can be repurposed as training data for another.\endnote{While there are doubtless some settings where this cross-task data reuse is possible, it has quite clear limits. For a policy to be trained to solve task B using data collected from task A, the policy used for task A would have had to actually also solve task B. It could have been done piecewise rather than in a single good trajectory, but unless the tasks are extremely similar it is vanishingly unlikely. Perhaps a more practical application would be to start with a safe but poor policy for solving a task, then incrementally collect new data, refine the policy using offline RL, validate that the new policy is also safe, and then collect data once more.}
For this setting, sample efficiency must be redefined to include the behavior policy under which the data was collected.
It then reflects the minimum amount of data collected under that behavior policy required for a particular algorithm to recover the optimal policy.\endnote{Note that this quantity of data should be expected to be extraordinarily large for most behavior policies.}

% \side{break into \textbackslash paragraphs}
% \side{write math definitions}
% \side{write sample complexity definitions}
% \begin{itemize}
%     \item The \emph{online} or \emph{regret} setting.

%     \item The \emph{learn-and-deploy} setting.
%     \side{Need a better name for learn-and-deploy}
%     This setting consists of distinct learning and deployment phases. In the learning phase, the agent is not required to perform well and may collect whatever data is most informative. In the deployment phase, the policy is fixed and should be as close to optimal as possible. This setting schematically breaks down into pure data collection followed by pure policy optimization, though in practice policy optimization is an essential tool during exploration.
%     \item The \emph{offline} or \emph{batch} setting.
%     This consists only of pure policy optimization given a fixed dataset of environment interactions collected by an extrinsic behavior policy. After learning from this data in whatever way it sees fit, an algorithm produces a fixed policy with the objective of earning as much reward as possible. Though described as a reinforcement learning setting, it does not include any actual reinforcement as the agent never learns from its own interactions with the environment. However, this makes the offline RL setting uniquely valuable for isolating how much can be learned from particular data.
% \end{itemize}

% \side{Do I want a section on simulated versus physical envs?}
% \section{Simulated and Physical Environments}

\printendnotes


\documentclass{article}

\usepackage{blindtext}
\usepackage{multicol}
\usepackage{caption}
\usepackage{bm}
\usepackage{tikz}
\usepackage{pgfplots}
\usepackage{hyperref}
\hypersetup{
    colorlinks=true,
    linkcolor=blue,
    filecolor=magenta,
    urlcolor=cyan,
}
\usepackage{geometry}
\geometry{
	a4paper,
	noheadfoot=true,
	left=1.0in,
	right=1.0in,
	top=1.0in,
	bottom=1.0in
}

% url package
\usepackage{hyperref}
\usepackage{subcaption}

% Titling and Author
\title{Latex Tikz Examples, Annotate on Graph}
\author{\href{https://fanwangecon.github.io/}{Fan Wang}\thanks{https://fanwangecon.github.io, repository: \href{https://fanwangecon.github.io/Tex4Econ/}{Tex4Econ}}}
\date{\today}

\begin{document}

\maketitle

\section{Annotate Text in Figure}

\subsection{Slope and Intercept Annotate}

Draw axis below, and a line, middle of page, and annotate the slope of the intercept with arrows. Annotate with box without color.

\begin{verbatim}
\begin{center}
\begin{tikzpicture}
\draw (0,2) -- (0.25*\textwidth,2);
\draw (0,1) -- (0.5*\textwidth,1);
\end{tikzpicture}
\end{center}
\end{verbatim}
\bigskip
\begin{center}
\begin{tikzpicture}
\draw[->] (-1,0) -- (6,0) node[right] {$N_m$};
\draw[->] (0,-1) -- (0,6) node[above] {$N_{m^{\prime}}$};

% A. Clip area so that line below is only drawn inside finite box
\clip (-3,-1) rectangle (6,6);

% B. Draw a line with intercept and slope
\draw[line width=0.50mm,domain=-1:6,smooth,variable=\x, blue] plot ({\x},{1+\x*1.25});

% C. Draw Line pointing to Intercept of line
\draw[->, line width=0.75mm] (-1.7,2.2) -- (-0.1,1.1);

% D. Draw a transparent text box that wraps text
\node[text width=3cm] at (-1.4,3.25) {\large{$\bm{\alpha_i}$ (this is the position of the \textbf{x-intercept}), wrap text.}};

\end{tikzpicture}
\end{center}

\subsection{Anote Slope, X intercept and Y intercept}

Draw a line, then point to its slope, x and y intercepts.

\begin{enumerate}
  \item Define the slope and y-intercept: $a_i, b_i$
  \item Define and calculate the x-intercept: $c_i = -\frac{a_i}{b_i}$
  \item Define the text that should appear for each element: $a_{desca}$
  \item Drawing lines pointing to $a$, $b$ or $c$ points:
  \begin{itemize}
    \item direction: $d_i \in \left\{NE, SE, SW, NW\right\}$
    \item rotation: $0 <= e_i <= 90$, which direction points to text
    \item dist one: $f_i > 0 $, straight distance from origin
    \item dist two: $g_i > 0 $, distance of line
    \item point start: $h^o_i, h^o_i$, starting point based on the four pieces of information
    \item point end: $h^d_i, h^d_i$, ending point based on the four pieces of information
  \end{itemize}
\end{enumerate}

So end up drawing using $a$, $b$ for line, $h^o$ for one point $h*d$ line small segment.

\begin{verbatim}
\end{verbatim}
\bigskip
\begin{center}
\begin{tikzpicture}
\draw[->] (-1,0) -- (6,0) node[right] {$N_m$};
\draw[->] (0,-1) -- (0,6) node[above] {$N_{m^{\prime}}$};

% A. Clip area so that line below is only drawn inside finite box
\clip (-3,-1) rectangle (6,6);

% B. Draw a line with intercept and slope
\draw[line width=0.50mm,domain=-1:6,smooth,variable=\x, blue] plot ({\x},{1+\x*1.25});

% C. Draw Line pointing to Intercept of line
\draw[->, line width=0.75mm] (-1.7,2.2) -- (-0.1,1.1);

% D. Draw a transparent text box that wraps text
\node[text width=3cm] at (-1.4,3.25) {\large{$\bm{\alpha_i}$ (this is the position of the \textbf{x-intercept}), wrap text.}};

\end{tikzpicture}
\end{center}

\end{document}


\section{Current Operator in Landau Levels}

Now consider about the current density operator for $N$th Landau level. Since we have already found the extact solution for our time depenedent Hamiltonian and we have identify them as Floquet states with quesi energies. From these solutions we can identify the \textit{Floquet modes} as given in Eq. \eqref{3.15} and using quantum numbers we can represent those states as follows
\begin{equation} \label{6.1}
  \ket{\phi_{\alpha}} = \ket{\phi_{n,m}} \equiv \ket{n,k_x} \quad
  \text{where} \quad
  k_x = m \frac{2\pi}{L_x}
\end{equation}
Using above complete set of eigenstates of Floquet Hamiltonian we can represent the single particle current operator's matrix element as
\begin{equation} \label{6.2}
  \qty(\mb{j})_{nm,n'm'} = \mel{n,k_x}{\;\hat{\mb{j}}\;}{n',k'_x}
\end{equation}
where particle current operator for this system will be
\begin{equation} \label{6.3}
  \hat{\mb{j}} = \frac{1}{m} \qty(\hat{\mb{P}} - e\qty[\mb{A}_s + \mb{A}_d(t)]).
\end{equation}
Let's consider the transverse conductivity in $x$ direction and we can identify that $x$ directional current operator as
\begin{equation} \label{6.4}
  \hat{j}_x = \frac{1}{m} \qty(-i\hbar\pdv{x} + eBy).
\end{equation}
Now we can calculate the matrix elements of $x$ directional current operator's matrix in Floquet mode basis as
\begin{equation} \label{6.5}
  \qty({j}_x)_{nm,n'm'} = \mel{n,k_x}{\;\hat{j}_x\;}{n',k'_x} =
  \mel{n,k_x}{\;\frac{1}{m} \qty(-i\hbar\pdv{x} + eBy)\;}{n',k'_x}
\end{equation}
and we can evaluate these using Floquet modes derived in Eq.\eqref{3.15} as follows
\begin{equation} \label{6.6}
  \begin{aligned}
    \qty({j}_x)_{nm,n'm'} & =
    \int dx \int dy \;
    \frac{1}{\sqrt{L_x}} \chi_{n}\big(y - y_0 - \zeta(t)\big)
    \exp(-ik_x x) \\
    & \times
    \frac{1}{m} \qty(-i\hbar\pdv{x} + eBy)
    \frac{1}{\sqrt{L_x}} \chi_{n'}\big(y - y_0 - \zeta(t)\big)
    \exp(i k'_x x)
  \end{aligned}
\end{equation}
and this can be simplified as
\begin{equation} \label{6.7}
  \begin{aligned}
    \qty({j}_x)_{nm,n'm'} & =
    \frac{1}{{mL_x}}
    \int dx \exp(-i(k_x-k'_x) x)
    \int dy \;
     \chi_{n}\big(y - y_0 - \zeta(t)\big) \\
    & \times
    \qty(\hbar k'_x + eBy)
    \chi_{n'}\big(y - y_0 - \zeta(t)\big)
  \end{aligned}
\end{equation}
and
\begin{equation} \label{6.8}
  \begin{aligned}
    \qty({j}_x)_{nm,n'm'} & =
    \frac{1}{{m}}
    \delta_{k_x,k'_x}
    \int dy \;
    \qty(\hbar k'_x + eBy)
     \chi_{n}\big(y - y_0 - \zeta(t)\big)
    \chi_{n'}\big(y - y_0 - \zeta(t)\big).
  \end{aligned}
\end{equation}
Now let $y - y_0 - \zeta(t) = \bar{y}$ and we will get
\begin{equation} \label{6.9}
  \begin{aligned}
    \qty({j}_x)_{nm,n'm'} & =
    \frac{1}{{m}}
    \delta_{k_x,k'_x}
    \int d\bar{y} \;
    \qty(\hbar k'_x + eB\bar{y} + eBy_0 + eB\zeta(t))
     \chi_{n}(\bar{y})
    \chi_{n'}(\bar{y}).
  \end{aligned}
\end{equation}
using definition of $y_0$ given in Eq. \eqref{1.11} this will be modiofied to
\begin{equation} \label{6.10}
  \begin{aligned}
    \qty({j}_x)_{nm,n'm'} & =
    \frac{1}{{m}}
    \delta_{k_x,k'_x}
    \int d\bar{y} \;
    \qty(\hbar k'_x + eB\bar{y} -\hbar k'_x + eB\zeta(t))
     \chi_{n}(\bar{y})
    \chi_{n'}(\bar{y})
  \end{aligned}
\end{equation}
and using integral indentities of Gauss-Hermite functions
\begin{equation} \label{6.11}
    \int d{y} \;
    \chi_{n}({y})
    \chi_{n'}({y}) =
    \delta_{n',n}
\end{equation}
\begin{equation} \label{6.12}
    \int dy \;
    y
    \chi_{n}({y})
    \chi_{n'}({y}) =
    \qty(\sqrt{\frac{n+1}{2}} \delta_{n',n+1} + \sqrt{\frac{n}{2}}
    \delta_{n',n-1})
\end{equation}
this becomes
\begin{equation} \label{6.13}
  \begin{aligned}
    \qty({j}_x)_{nm,n'm'} & =
    \frac{1}{{m}}
    \delta_{k_x,k'_x}
    eB
    \qty[
    \qty(\sqrt{\frac{n+1}{2}} \delta_{n',n+1} + \sqrt{\frac{n}{2}}
    \delta_{n',n-1})
    + \zeta(t) \delta_{n',n}
    ]
  \end{aligned}
\end{equation}
\textcolor{red}{Due to complexity we can only consider the constant contribution and we allows only the one-cycle averaged current flow} and then we can derive the $s=0$ components of the Fourier series as
\begin{equation} \label{6.14}
  \begin{aligned}
    \qty({j}^x_{s=0})_{nm,n'm'} & =
    \frac{1}{T} \int_0^T dt \;
    \frac{1}{{m}}
    \delta_{k_x,k'_x}
    eB
    \qty[
    \qty(\sqrt{\frac{n+1}{2}} \delta_{n',n+1} + \sqrt{\frac{n}{2}}
    \delta_{n',n-1})
    + \frac{eE}{m(\omega_0^2 - \omega^2)}\sin(\omega t) \delta_{n',n}
    ]
  \end{aligned}
\end{equation}
and this can be evaluate and get
\begin{equation} \label{6.15}
  \begin{aligned}
    \qty({j}^x_{s=0})_{nm,n'm'} & =
    \frac{eB}{{m}}
    \delta_{k_x,k'_x}
    \qty(\sqrt{\frac{n+1}{2}} \delta_{n',n+1} + \sqrt{\frac{n}{2}}
    \delta_{n',n-1})
  \end{aligned}
\end{equation}

\noindent
For electric current operator we can introduce the electron's charge and effective mass
\begin{equation} \label{6.16}
  \begin{aligned}
    \qty({j}^x_{s=0})_{nm,n'm'} & =
    \frac{e^2B}{{m_e}}
    \delta_{k_x,k'_x}
    \qty(\sqrt{\frac{n+1}{2}} \delta_{n',n+1} + \sqrt{\frac{n}{2}}
    \delta_{n',n-1})
  \end{aligned}
\end{equation}
\hfill$\blacksquare$

\noindent
Next we can consider the transverse conductivity in $y$ direction and we can identify that $y$ directional current operator as
\begin{equation} \label{6.17}
  \hat{j}_y = \frac{1}{m} \qty(-i\hbar\pdv{y} - \frac{eE}{\omega}\cos(\omega t)).
\end{equation}
Now we can calculate the matrix elements of $y$ directional current operator's matrix in Floquet mode basis as
\begin{equation} \label{6.18}
  \qty({j}_y)_{nm,n'm'} = \mel{n,k_x}{\;\hat{j}_y\;}{n',k'_x} =
  \mel{n,k_x}{\;\frac{-1}{m} \qty(i\hbar\pdv{y} + \frac{eE}{\omega}\cos(\omega t))\;}{n',k'_x}
\end{equation}
and we can evaluate these using Floquet modes derived in Eq.\eqref{3.15} as follows
\begin{equation} \label{6.19}
  \begin{aligned}
    \qty({j}_y)_{nm,n'm'} & =
    \int dx \int dy \;
    \frac{1}{\sqrt{L_x}} \chi_{n}\big(y - y_0 - \zeta(t)\big)
    \exp(-ik_x x) \\
    & \times
    \frac{-1}{m} \qty(i\hbar\pdv{y} + \frac{eE}{\omega}\cos(\omega t))
    \frac{1}{\sqrt{L_x}} \chi_{n'}\big(y - y_0 - \zeta(t)\big)
    \exp(i k'_x x)
  \end{aligned}
\end{equation}
then introducing new variable
\begin{equation} \label{6.20}
  y - y_0 - \zeta(t) = \bar{y} \quad \longrightarrow \quad dy = d{\bar{y}}
\end{equation}
and follwing identity [*Ref: Appendix A Hermite fucntions]
\begin{equation} \label{6.21}
  \pdv{\chi_n(y)}{y} = -\sqrt{\frac{n+1}{2}} \chi_{n+1}(y) +
  \sqrt{\frac{n}{2}} \chi_{n-1}(y)
\end{equation}
above expression can be simplified as previous case (let $y=\bar{y}$)
\begin{equation} \label{6.22}
  \begin{aligned}
    \qty({j}_y)_{nm,n'm'} & =
    \frac{-1}{{m}}
    \delta_{k_x,k'_x}
    \int dy \;
     \chi_{n}(y)
    \qty[-i\hbar \qty(
    \sqrt{\frac{n'+1}{2}} \chi_{n'+1}(y) - \sqrt{\frac{n'}{2}} \chi_{n'-1}(y)
    ) + \frac{eE}{\omega}\cos(\omega t)
    \chi_{n'}(y)].
  \end{aligned}
\end{equation}
Then considering Gauss-Hermite integral indentities we can derive that
\begin{equation} \label{6.23}
  \begin{aligned}
    \qty({j}_y)_{nm,n'm'} & =
    \frac{1}{{m}}
    \delta_{k_x,k'_x}
    \qty[i\hbar \qty(
    \sqrt{\frac{n'+1}{2}} \delta_{n,n'+1}
    - \sqrt{\frac{n'}{2}} \delta_{n,n'-1}
    ) - \frac{eE}{\omega}\cos(\omega t) \delta_{n,n'}]
  \end{aligned}
\end{equation}
\textcolor{red}{Due to complexity we can only consider the constant contribution and we allows only the one-cycle averaged current flow} and then we can derive the $s=0$ components of the Fourier series as
\begin{equation} \label{6.24}
  \begin{aligned}
    \qty({j}^y_{s=0})_{nm,n'm'} & =
    \frac{1}{T} \int_0^T dt \;
    \frac{1}{{m}}
    \delta_{k_x,k'_x}
    \qty[i\hbar \qty(
    \sqrt{\frac{n'+1}{2}} \delta_{n,n'+1}
    - \sqrt{\frac{n'}{2}} \delta_{n,n'-1}
    ) - \frac{eE}{\omega}\cos(\omega t) \delta_{n,n'}]
  \end{aligned}
\end{equation}
and this can be evaluate and get
\begin{equation} \label{6.25}
  \begin{aligned}
    \qty({j}^y_{s=0})_{nm,n'm'} & =
    \frac{i\hbar}{{m}}
    \delta_{k_x,k'_x}
    \qty[
    \sqrt{\frac{n'+1}{2}} \delta_{n,n'+1}
    - \sqrt{\frac{n'}{2}} \delta_{n,n'-1}
    ]
  \end{aligned}
\end{equation}
and this can be re-write as
\begin{equation} \label{6.26}
  \begin{aligned}
    \qty({j}^y_{s=0})_{nm,n'm'} & =
    \frac{i\hbar}{{m}}
    \delta_{k_x,k'_x}
    \qty[
    \sqrt{\frac{n}{2}} \delta_{n',n-1}
    - \sqrt{\frac{n+1}{2}} \delta_{n',n+1}
    ]
  \end{aligned}
\end{equation}

\noindent
For electric current operator we can introduce the electron's charge and effective mass
\begin{equation} \label{6.27}
  \begin{aligned}
    \qty({j}^y_{s=0})_{nm,n'm'} & =
    \frac{ie\hbar}{{m}}
    \delta_{k_x,k'_x}
    \qty[
    \sqrt{\frac{n}{2}} \delta_{n',n-1}
    - \sqrt{\frac{n+1}{2}} \delta_{n',n+1}
    ]
  \end{aligned}
\end{equation}
\hfill$\blacksquare$

xx