%             %
%%           %%
%%% CHAPTER %%%
%%           %%
%             %

\chapter{Mathematics -- Basics}

	\label{cha::maths:basics}

Mathematics writing is one of the most advantage of \LaTeX{} compared to common text editors.
On a first approach, it looks like a programming language but it is in fact quite intuitive.

\section{Packages for mathematics}

	The first package which was extremely useful in mathematics writing was \emph{amsmath}.
	Since then, it has been upgraded by the \emph{mathtools} package that I recommend to use.
	Hence the following line in the preamble of the document:
\begin{lstlisting}[language={[LaTeX]TeX}]
\usepackage{mathtools}
\end{lstlisting}

	Other interesting packages are:
	\begin{itemize}
		\item \emph{cases} which provides the \texttt{numcases} command to number all lines of a system of equations,
		\item \emph{systeme} which provides command to format a system of equations for better readility, and
		\item \emph{physics} which provides many commands to facilitate the writing of \enquote{complex} equations including derivatives and partial derivatives.
	\end{itemize}
	However, \emph{systeme} is quite new at the time this document is being written and so it could lack of maturity.

\section{Writing equations}

	Usually, when a writer wants to put mathematics in a document, it takes the form of an equation.
	Writing an equation is simply done thanks to the \texttt{equation} environment.
	Maxwell's equations will be used as examples: so, the
\begin{lstlisting}[language={[LaTeX]TeX}]
\begin{equation}
	\vec{\nabla}\cdot\vec{B}=0.
	\label{eq::Maxwell:no_magnetic_monopole}
\end{equation}
\end{lstlisting}
	\LaTeX{} code generates
	\begin{equation}
		\vec{\nabla}\cdot\vec{B}=0.
		\label{eq::Maxwell:no_magnetic_monopole}
	\end{equation}

	Here is the power of \LaTeX{}: automatic numbering of equations.
	In \emph{book}-like documents, equations are numbered within chapters by default, i.e., (1.x) in chapter~1, (2.x) in chapter~2 and so on.
	Moreover, equations numbers (a.k.a. tags) are placed on the right side.
	Of course, this layout can be modified.

	\subsection{Unnumbered equations}

		Automatic numbering can be avoided by using the starred version of the previous environment: \texttt{equation*}.
		For instance,
\begin{lstlisting}[language={[LaTeX]TeX}]
\begin{equation*}
	\vec{\nabla}\times\vec{E}
		=-\frac{\partial\vec{B}}{\partial t}.
\end{equation*}
\end{lstlisting}
		produces
		\begin{equation*}
			\vec{\nabla}\times\vec{E}
				=-\frac{\partial\vec{B}}{\partial t}.
		\end{equation*}
		Nevertheless, it is generally recommended to number all equations in scientific documents for easier reference.

		Shorter forms of the unnumbered version are offered by the package: the \textbackslash{}[ \ldots \textbackslash{}] wrapper or the double \$\$ symbol.
		Please note that the latter is plain \TeX{}, which means that it should not be used with \LaTeX{} because it is not robust.
		Though, the reason for which \$\$ is presented in this document is that it is overly used on the Internet \footnote{And, as we all know, if it is on the Internet, it must be true\ldots}.

		Here follow the corresponding examples:
\begin{lstlisting}[language={[LaTeX]TeX}]
$$\iint_{\Sigma_f} \vec{B} \cdot \mathrm{d}\vec{S},$$
\end{lstlisting}
		is the code corresponding to
		$$ \iint_{\Sigma_f} \vec{B} \cdot \mathrm{d}\vec{S}, $$
		while
\begin{lstlisting}[language={[LaTeX]TeX}]
\[ \oint_{C} \vec{E} \cdot \mathrm{d}\vec{l}
	= - \iint_{S} \frac{\partial\vec{B}}{\partial t}
	  \cdot \mathrm{d}\vec{S}. \]
\end{lstlisting}
		creates
		\[ \oint_{C} \vec{E} \cdot \mathrm{d}\vec{l}
			= - \iint_{S} \frac{\partial\vec{B}}{\partial t}
			  \cdot \mathrm{d}\vec{S}. \]

		However, I do not recommend to use short forms.
		On the one hand, the \texttt{equation*} environment highlights the mathematics when looking in the \LaTeX{} code.
		On the other hand, if the author changes his mind and wants to number the equation, he must simply remove the \texttt{*} character.

	\subsection{Inline equations}

		Inline equations are equations written in the text.
		It can be useful in some circumstances, such as the description of a variable.
		For instance, I could specify that $\vec{B}$ in \cref{eq::Maxwell:no_magnetic_monopole} is the magnetic field.
		To do so, the equation is surrounded by single \$ signs:
\begin{lstlisting}[language={[LaTeX]TeX}]
$\vec{B}$
\end{lstlisting}

	\subsection{Text-mode VS math-mode}

		Inline equations underlines a fundamental behaviour of \LaTeX{}: the difference between \emph{math-mode} and \emph{text-mode}.
		As the names explain, text-mode is the regular mode of \LaTeX{} as the main part of the document is usually text, while math-mode is set in specific environments intended for mathematics.

		Compare
		\begin{center}
			regular behaviour (text-mode),
			$text in mathematical environment (math-mode)$.
		\end{center}
		that is produced by
\begin{lstlisting}[language={[LaTeX]TeX}]
regular behaviour (text-mode),
$text in mathematical environment (math-mode)$.
\end{lstlisting}

		Math-mode has several effects:
		\begin{itemize}
			\item a math font is used instead of the text font,
			\item the default font family is slanted while it is normal roman font in text,
			\item it manages white spaces in a different way than the ongoing typographical rules
				\begin{itemize}
					\item any white space is automatically removed,
					\item white space is added around mathematical operators,
				\end{itemize}
			\item commands available in math-mode only will not generate errors.
		\end{itemize}

	\subsection{Consequences of text- and math-mode: display style}

		Sometimes, inline equations may introduce unpleasant distortion in the text, especially with \enquote{big} symbols.
		As an example, let us use express the acceleration as the derivative of the speed:
		$a=\frac{\mathrm{d}v}{\mathrm{d}t}$.
		It can be seen that the fraction symbol has been compacted to fit with the line space.
		It happens when math-mode is used inside a block text-mode, like it is the case with inline equations.

		It is possible to prevent the fraction from being reshaped by forcing the \emph{displayed math-mode}.
		To do so, the writer must use the \texttt{displaystyle} command, which exists in shorter forms for common mathematical symbols such as fractions.
		Expressing again the acceleration:
		$a=\dfrac{\mathrm{d}v}{\mathrm{d}t}$, or, equivalently,
		$\displaystyle a=\frac{\mathrm{d}v}{\mathrm{d}t}$.
		It can be seen that the space line is increased above and below the line including the equation, creating a somewhat uncomfortable text arrangement.

		The problem presented here above is related to the following \LaTeX{} codes:
\begin{lstlisting}[language={[LaTeX]TeX}]
$a=\frac{\mathrm{d}v}{\mathrm{d}t}$
$a=\dfrac{\mathrm{d}v}{\mathrm{d}t}$
$\displaystyle a=\frac{\mathrm{d}v}{\mathrm{d}t}$
\end{lstlisting}
		It will appear every time a \enquote{big} symbol (fraction, integral, sum, etc.) is used.
		However, it never occurs within mathematical environments such as \texttt{equation} because in that case the maths are \emph{displayed}, i.e., horizontally centred and wrapped with additional vertical space in order to be highlighted and readable.

	\subsection{General recommendations for equations}

		Recommendations:
		\begin{itemize}
			\item use numbered equation only (with the \texttt{equation} environment),
			\item try to avoid inline equations except
			\begin{itemize}
				\item to describe variables or operators,
				\item for very small, less important and/or very well-known formulae which do not contain \enquote{big} symbols (e.g., integral, fraction).
			\end{itemize}
		\end{itemize}

\section{Writing groups of equations}

	Several commands and environments allow to group equations.
	The most-used are presented here after.
	For a complete presentation, please refer to the \emph{amsmath} and \emph{mathtools} packages documentation.

	\subsection{Group of equations}

		The first tool which allow to group equations is the \texttt{gather} environment.
		Inside the environment, a double backslash (\textbackslash\textbackslash) indicates the end of an equation and the beginning of a new one.
		Consequently, a new line is produced and another equation can be written.
		Pay attention: no double backslash must be put after the last equation.
		Otherwise, an additional space is added at the end of the group.

		Unless the starred version (\texttt{gather*}) is used, all equations are numbered.
		To prevent one line from being numbered, the \texttt{\textbackslash{}notag} or the \texttt{\textbackslash{}nonumber} command can be used.

		It is also possible to write text between equations while still being in the \texttt{gather} environment.
		This is done with the \texttt{intertext} command, or \texttt{shortintertext} which removes extra vertical space.
		It is specifically useful for a mathematical development.
		Within these commands, \LaTeX{} is in text-mode, while it is in math-mode within the rest of the \texttt{gather} environment.

		As an example, the
\begin{lstlisting}[language={[LaTeX]TeX}]
\begin{gather}
	\vec{\nabla} \times \vec{B}
		= \mu_0 \vec{\jmath}
		+ \varepsilon_0 \mu_0 \frac{\partial \vec{E}}{\partial t},
		\\
	\intertext{which can be written in the integral form by
		applying the Green theorem}
	\oint_{C} \vec{B} \cdot \mathrm{d} \vec{l}
		= \mu_0 \iint_S \vec{\jmath} \cdot \mathrm{d} \vec{S}
		+ \varepsilon_0 \mu_0 \iint_S \frac{\partial \vec{E}}{
			\partial t}	\cdot \mathrm{d} \vec{S}
\end{gather}
\end{lstlisting}
		code will generate
		\begin{gather}
			\vec{\nabla} \times \vec{B}
				= \mu_0 \vec{\jmath}
				+ \varepsilon_0 \mu_0 \frac{\partial \vec{E}}{\partial t},
				\\
			\intertext{which can be written in the integral form by applying the Green theorem}
			\oint_{C} \vec{B} \cdot \mathrm{d} \vec{l}
				= \mu_0 \iint_S \vec{\jmath} \cdot \mathrm{d} \vec{S}
				+ \varepsilon_0 \mu_0 \iint_S \frac{\partial \vec{E}}{\partial t} \cdot \mathrm{d} \vec{S}
		\end{gather}
		(see the use of \texttt{\textbackslash{}intertext}).

	\subsection{Group of aligned equations}

		The second tool allowing to group equations is the \texttt{align} environment.
		It does the same as the \texttt{gather} environment but also allows to align the equations.
		The alignment is performed thanks to the ampersand (\&) symbol.
		All other commands and symbols performs the same as in the \texttt{gather} environment.

		For instance,
\begin{lstlisting}[language={[LaTeX]TeX}]
\begin{align}
	\vec{B} &= \vec{\nabla} \times \vec{A} \\
	\vec{E} &= -\vec{\nabla} V - \frac{\partial \vec{A}}{\partial t}.
\end{align}
\end{lstlisting}
		creates
		\begin{align}
			\vec{B} &= \vec{\nabla} \times \vec{A} \\
			\vec{E} &= -\vec{\nabla} V - \frac{\partial \vec{A}}{\partial t}.
		\end{align}

	\subsection{General recommendations for groups of equations}

		Recommendations:
		\begin{itemize}
			\item use numbered equations only (unstarred versions) with possible exceptions for
			\begin{itemize}
				\item numerical computations,
				\item proofs (theorems, formulae, etc.),
			\end{itemize}
			\item do not abuse of intertext because it makes the \LaTeX{} code less readable.
		\end{itemize}

\section{Common symbols for mathematics}

	Mathematics would not be mathematics without any symbols.
	Common ones are:
	\begin{itemize}
		\item arithmetic operators such as $+$ and $-$,
		\item comparison operators like $>$ and $<$.
	\end{itemize}

	Less intuitive but still useful symbols are:
	\begin{itemize}
		\item the multiplication operator $\cdot$ generated by the \texttt{cdot} command,
		\item the fraction symbol $\frac{n}{d}$ created by the \texttt{frac} command,
		\item the integral $\int$ which is produced by the \texttt{int} command, and
		\item any text symbol that is usually generated by a command having the same name (e.g., $\sin$ with the \texttt{sin} command or $\exp$ with \texttt{exp}).
	\end{itemize}

	There are many other symbols and being exhaustive is not the purpose of this mini-guide.
	If the reader is looking for specific symbols, he may refer to:
	\begin{itemize}
		\item his \LaTeX{} editor, which generally provides a list of shortcuts and buttons to generate the correct commands,
		\item a \href{https://fr.sharelatex.com/learn/List_of_Greek_letters_and_math_symbols}{quick review} of symbols from ShareLaTeX,
		\item a \href{http://www.rpi.edu/dept/arc/training/latex/LaTeX_symbols.pdf}{big list} of symbols native from \TeX{} and coming from different packages.
	\end{itemize}

% *** End of chapter ***


\chapter{Contraction and Index Gymnastics}
\emph{Index gymnastics} is the process by which we try and keep track of indices of tensors when written without their basis.
Suppose we had a tensor $A \in \tps{T}^3_2(V)$.
Formally, we would write this $(3,2)$ tensor as
\[ \tensor{A}{^{\alpha\beta\gamma}_{\mu\nu}} \vec{e}_\alpha \otimes \vec{e}_\beta \otimes \vec{e}_\gamma \otimes \vec{e}^\mu \otimes \vec{e}^\nu : V^* \times V^* \times V^* \times V \times V \to \mathbb{F}, \]
which is kind of ridiculous to carry around\footnote{Remember, what's written here is the \emph{shortened} version using Einstein summation. The truly complete version would also have 5 summation signs at the beginning.}, especially when the component $\tensor{A}{^{\alpha\beta\gamma}_{\mu\nu}}$ contains all the information we need.

As we've written it, $A$ is a mapping from $\tps{T}^3_2$ to the scalar field $\mathbb{F}$; however, we've just shown that we can also `trick' this tensor into producing other objects if we give it a different number of arguments than it is expecting.
For example, if we were to give $A$ a single covector $\Lambda$ as an argument, we've suddenly created a map which takes two covectors and two vectors to $\mathbb{F}$, so
\[ A(\Lambda,\cdot,\cdot\mid\cdot,\cdot) \in \tps{T}^2_2, \]
where $\mid$ is just a divider to visually separate the covector arguments from the vector arguments.
Notice that this isn't the only way we can make $A$ a $\tps{T}^2_2$ tensor; we can use $\Lambda$ as any of the covector arguments expected, and create an entirely new rank $(2,2)$ tensor, like
\[ A(\cdot,\Lambda,\cdot\mid\cdot,\cdot) \in \tps{T}^2_2. \]

\section{The Universal Property}
Suppose we have a multilinear map $Z$ defined as
\[ B : \underbrace{V \times V \times \cdots}_{\text{$p$ times}}\underbrace{V^* \times V^* \times \cdots}_{\text{$q$ times}} \to U, \]
where $U$ is some vector space.
The \emph{Universal Property} says that if such a map exists, then we are guaranteed to have a multilinear map
\[ \beta : \underbrace{V \otimes V \otimes \cdots}_{\text{$p$ times}}\underbrace{V^* \otimes V^* \otimes \cdots}_{\text{$q$ times}} \to U, \]
and that these will be the same map, meaning that their results will always be the same.
Graphically, the universal property can be written as
\begin{center}
\begin{tikzcd}
V \times V^* \arrow[rr] \arrow[rrdd, "B"'] &  & V \otimes V^* \arrow[dd, "\beta", dashed] \\
 &  &  \\
 &  & U
\end{tikzcd}
\end{center}
for a simplified example with only a single vector-dual space product.

\begin{proof}[Proof of the Universal Property]
We will prove it holds for a simple case, and by linearity it will hold for all more general cases.
Suppose we have a vector space $V$ over a scalar field $\mathbb{F}$.
Let $\{\vec{e}_\mu\}$ be the basis for $V$ and $\{\vec{e}^\nu\}$ be a basis for its dual space, $V^*$.
Let $B$ be a map defined as
\[ B : V \times V^* \to U, \]
where $U$ is some vector space.
It must be the case that $B(\vec{e}_\mu, \vec{e}^\nu) \in U$.
Let $\beta$ then be the map
\[ \beta : \tps{T}^1_1 \to U \]
defined in such a way that $\beta(\vec{e}_\mu \otimes \vec{e}^\nu) = B(\vec{e}_\mu, \vec{e}^\nu)$.
Since we've defined how $\beta$ operates on the basis $\vec{e}_\mu \otimes \vec{e}^\nu$, we can represent any member of $V \otimes V^*$ as a linear combination of that basis, and since $B$ and $\beta$ are linear, we can write the result of any argument for $\beta$ in this way.
\end{proof}

\section{Contraction of Bases}
To see why we care at all about the Universal Property, consider the notion of a map we defined at the very beginning of this lesson,
\begin{align*}
    \langle \cdot, \cdot \rangle : V \times V^* \to \mathbb{F}.
\end{align*}
We know this map exists because the creation of $V$ \emph{automatically} implies the creation of the dual space $V^*$, so we must have the concept of multilinear mapping for any vector space.
By the Universal Property, we know that there must automatically be a function $\beta$ such that
\begin{align*}
    \beta &: \tps{T}^1_1 (V) \to \mathbb{F} \\
      &: V \otimes V^* \to \mathbb{F}.
\end{align*}
While this example might seem trivial, we can use $\langle \cdot, \cdot \rangle$ to build more complicated mappings.
Suppose we have some mapping
\begin{align*}
    B &: V^p \times V^{*q} \to \tps{T}^{p-1}_{q-1} \\
      &: \underbrace{V \times V \times \cdots}_{\text{$p$ times}}\underbrace{V^* \times V^* \times \cdots}_{\text{$q$ times}} \to \tps{T}^{p-1}_{q-1}.
\end{align*}
By the Universal Property, we must be able to find some equivalent function $\beta$ such that
\[ \beta : \tps{T}^p_q \to \tps{T}^{p-1}_{q-1}. \]
This is what is known as a \emph{contraction}; we can jump between tensor product spaces by lowering both indices by 1.
Obviously for this to occur, both $p$ and $q$ must be greater than 0.
One way we can do this is by taking a vector-covector pair from $\tps{T}^p_q$ and applying the linear map to them; that is,
\[ B^i_j : \vec{e}_1 \times \cdots \vec{e}_i \times \cdots \vec{e}_p \times \vec{e}^1 \times \cdots \vec{e}^j \times \cdots \vec{e}^q \mapsto \langle \vec{e}^j, \vec{e}_i \rangle \vec{e}_{\alpha \not= i} \otimes \vec{e}^{\gamma \not= j}. \]
This map $B^i_j$ rips out the $i$\textsuperscript{th} basis vector and $j$\textsuperscript{th} basis covector from $\tps{T}^p_q$, applies them to one another, and multiplies the result by the tensor product of all remaining basis vectors.
Since we defined this map $B$ to act on the Cartesian product of the vector spaces and dual spaces, we must be able to find our equivalent function $\beta^i_j$ which does the same thing, but maps from the tensor product space, so
\[ \beta^i_j : \tps{T}^p_q \to \tps{T}^{p-1}_{q-1} : \vec{e}_1 \otimes \cdots \vec{e}_p \otimes \vec{e}^1 \otimes \cdots \vec{e}^q \mapsto \langle \vec{e}^j, \vec{e}_i \rangle \vec{e}_{\alpha \not=1}\otimes \vec{e}^{\gamma \not= j}. \]
Note that because we can vary $1 \leq i \leq p$ and $1 \leq j \leq q$, this isn't the only possible mapping we could have created using this method.
In fact, there will always be $pq$ possible mappings.
We refer to the specific contraction as $\beta^i_j$ or the $(i,j)$ contraction.
If we want to contract more than once, we can simply apply this mapping more than once, so a jump from $\tps{T}^p_q$ to $\tps{T}^{p-\alpha}_{q-\alpha}$ can be achieved by some mapping
\[ \gamma = \underbrace{\beta^{k}_{l} \circ \cdots \circ \beta^i_j}_{\text{$\alpha$ times}} \]
applied to the original tensor product space.

\section{Contraction of Generalized Tensors}
Let's consider an arbitrary tensor
\[
T = \tensor{T}{^{i_1\cdots i_\alpha \cdots i_p}_{j_1 \cdots j_\beta \cdots j_q}} \vec{e}_{i_1} \otimes \cdots \vec{e}_{i_\alpha} \cdots \otimes \vec{e}_{i_p} \otimes \vec{e}^{j_1} \otimes \cdots \vec{e}^{j_\beta} \cdots \otimes \vec{e}^{j_q}.
\]
We want to find the map
\begin{align*}
C^{i_\alpha}_{j_\beta} &: \tps{T}^p_q \to \tps{T}^{p-1}_{q-1} \\
                       &: T \mapsto \tensor{T}{^{i_1 \cdots i_p}_{j_1 \cdots j_q}} \langle \vec{e}^{j_\beta}, \vec{e}_{i_\alpha} \rangle \vec{e}_{i \not= \alpha } \otimes \vec{e}^{j \not= \beta} \\
                       &: T \mapsto \tensor{T}{^{i_1\cdots i_\alpha \cdots i_p}_{j_1\cdots i_\alpha \cdots j_q}} \vec{e}_{i \not= \alpha } \otimes \vec{e}^{j \not= \beta}.
\end{align*}
Notice that we end up with an $i_\alpha$ in both the superscript and subscript indices of $T$.
Recall this happens because $\langle \vec{e}^{j_\beta}, \vec{e}_{i_\alpha} \rangle = \delta^{j_\beta}_{i_\alpha}$.
As we will see, this causes that index to be dropped in the resultant tensor component.

\section{Symmetric and Antisymmetric Tensors}
A tensor is \emph{symmetric} if we can rearrange the vector arguments, and rearrange the covector arguments (but not switch the two).
For example, a tensor $T$ is symmetric if
\[ \tensor{T}{^{\alpha\beta}_{\gamma\delta}} = \tensor{T}{^{\beta\alpha}_{\delta\gamma}}. \]
A tensor is \emph{antisymmetric} if switching any pair of arguments switches the sign of the tensor, so
\[ T_{\alpha\beta\gamma} = -T_{\beta\alpha\gamma} = T_{\beta\gamma\alpha}. \]
If we refer to any pair of arguments $\alpha\beta$ of a tensor $T$, we can find its \emph{symmetric part} and \emph{antisymmetric part} by the following:
\begin{align*}
    T_{(\alpha\beta)\gamma} &= \frac{1}{2}\qty(T_{\alpha\beta\gamma} + T_{\beta\alpha\gamma}) \tag{Symmetric part} \\
    T_{[\alpha\beta]\gamma} &= \frac{1}{2}\qty(T_{\alpha\beta\gamma} - T_{\beta\alpha\gamma}). \tag{Antisymmetric part}
\end{align*}
This means that a tensor is the sum of its symmetric and antisymmetric parts, so $T_{\alpha\beta\gamma} = T_{(\alpha\beta)\gamma} + T_{[\alpha\beta]\gamma}$.

\section{Index Gymnastics}
As you can see, it is very cumbersome to carry around the basis vectors in each calculation.
Instead of writing everything out, we can keep track of all the necessary information by just shuffling the indices of the tensor component.
The simplest form of index gymnastics is raising or lowering a single index of a tensor.
For example, if we have a contravariant tensor $A^\mu$ (written fully $A^\mu \vec{e}_\mu \in V$), we can create an associated covariant tensor $A_\nu$ by lowering the index.
In fact, we've already done this with the metric tensor; consider Equation~\eqref{eqn:metric-index-lowering}, where we saw that
\[ g_{\mu\nu} \vec{e}^\mu \otimes \vec{e}^\nu \qty(A^\alpha \vec{e}_\alpha) \mapsto A^\mu g_{\mu\nu} \vec{e}^\nu \in V^*. \]
If we drop the basis vectors from this equation and look only at what is happening to the components, we get the equivalent
\[ g_{\mu\nu}\qty(A^\alpha \vec{e}_\alpha) \mapsto A^\mu g_{\mu\nu}, \]
which we often simply abbreviate as $A_\nu$; this means that the metric tensor can be used to lower the index of a tensor via $g_{\mu\nu}A^\mu \mapsto A^\nu$.
This is an example of how contraction reduces the indices when they are the same: because this tensor component as $\mu$ as both an upper and lower index, it gets summed over, and we are left with a component with only $\nu$ as a lower index.
Note that it may be more proper to write this as $g_{\nu\mu}A^\mu$, so that the summed-over indices are adjacent; we are allowed to do this because the metric tensor (in most derivations) is symmetric.

\section{Removing Bases}
We've already been doing a bit of index gymnastics when we refer to a tensor with multiple vector spaces in its tensor product space by only a single component with multiple indices.
Consider this simple example of a $\tps{T}^2_0$ tensor,
\[ T^{\mu\nu} = A^\mu \vec{e}_\mu \otimes B^\nu \vec{e}_\nu. \]
Perhpas more properly, we should write this tensor as $A^\mu B^\nu$, but because these are both simply field elements, we can simplify them into a single component $T^{\mu\nu}$.
Notice, however, that when we keep the basis vectors around, we can change the order of the components without changing the tensor, because of the commutativity of scalar multiplication.
\[ A^\mu B^\nu \vec{e}_\mu \otimes \vec{e}_\nu = B^\nu A^\mu \vec{e}^\mu \otimes \vec{e}^\nu. \]
While we swap the components, we keep the basis vectors in place, because $\vec{e}^\mu \otimes \vec{e}^\nu \not= \vec{e}^\nu \otimes \vec{e}^\mu$.
This means that when we drop the basis vectors, we have to remember that the order of the indices is now important, because although $A^\mu B^\nu = B^\nu A^\mu$, $T^{\mu\nu} \not= T^{\nu\mu}$, as the latter implies a different ordering of the basis vectors.
Consider another example, from a tensor with a single vector and covector basis.
Say we had a tensor
\[ A^\mu B_\nu \vec{e}_\mu \otimes \vec{e}^\nu. \]
If we combine the components in the appropriate way, we get a $\tps{T}^1_1$ tensor $\tensor{T}{^\mu_\nu}$, but if we combine them in the wrong order, we get $\tensor{T}{_\nu^\mu}$, which isn't technically even a tensor.

\section{Contraction \& Index Gymnastics}
Recall that when we learned how to contract tensors, we saw that we ended up with a scenario where an upper and lower index were equal, causing that index to be removed and dropping the tensor from a $(p,q)$ tensor to a $(p-1,q-1)$ tensor.
Without worrying about the basis vectors, we can write this much more succinctly as
\[ \tensor{T}{^{\alpha\beta\gamma}_{\lambda\mu\nu}} \mapsto \tensor{T}{^{\alpha\beta\gamma}_{\alpha\mu\nu}} \mapsto \tensor{M}{^{\beta\gamma}_{\mu\nu}}. \]

\section{The Metric Tensor as a Metric}
We we learn about the inner product, its often in conjunction with the idea of the magnitude of a vector; in linear algebra, we learn that $\vec{a}\cdot\vec{a} = \norm{a}^2$.
Even the name \emph{metric tensor} implies that we can use it to get a notion of measure.
To show how this works in tensor form, lets consider the metric tensor $g_{\mu\nu}$ acting on two copies of an arbitrary vector:
\begin{align*}
    g_{\mu\nu}\qty(A^\alpha \vec{e}_\alpha, A^\alpha \vec{e}_\alpha) &= A^\alpha A^\alpha g_{\mu\nu} \langle \vec{e}^\mu, \vec{e}_\alpha \rangle \langle \vec{e}^\nu, \vec{e}_\alpha \rangle \\
    &= A^\alpha A^\alpha g_{\alpha\alpha} \\
    &= A^\alpha A_\alpha. \tag{via Index Contraction}
\end{align*}
This is where we get the notion that $\vec{a}\cdot\vec{a} = \norm{a}^2$ in the world of tensors.


\section{Rates of Change \& Tangent Lines}
You might already be familiar from physics with the idea of an average rate of change over some interval (often a time interval in physics).
It is simply the amount of change that occurred in the interval, divided by the length of the interval.
This is exactly the same idea as the slope of a line being "rise over run"
\begin{equation*}
	\overline{\Delta f_{a,b}} = \frac{f(b)-f(a)}{b-a}.
\end{equation*}
This is also known as the ``secant slope'', which gets its name from secant lines on circles.

\begin{example}
	Find the average rate of change of $x^2-1$ over the interval $[1,4]$.
\end{example}
\begin{answer}
	Applying the formula,
	\begin{equation*}
		\overline{\Delta f_{1,4}} = \frac{f(4)-f(1)}{4-1} = \frac{15-0}{3} = 5.
	\end{equation*}
\end{answer}

As we decrease the size of the interval, the secant line becomes closer and closer to a tangent line.
In the limit, as the size of the interval approaches 0, we get a tangent line, representing an instantaneous rate of change.
\begin{equation*}
	\Delta f_a = \lim_{h \to 0}{\frac{f(a+h)-f(a)}{h}}.\footnote{You may recognize this from pre-calculus as the ``difference quotient''.}
\end{equation*}

\begin{figure}[H]
	\label{sectant_tangent_line}
	\centering
	\includegraphics[width = 0.5\textwidth]{./derivatives/secant_tangent_line.png}
	\caption{\hyperref{}{}{}{Secant and Tangent Line}}
\end{figure}

\begin{example}
	Find the equation of the tangent line to $f(x)=x^2-4x$ at $x=1$.
\end{example}
\begin{answer}
	First, we need to find the instantaneous rate of change of $f$ at $x=1$.
	We do this by evaluating the limit.
	\begin{align*}
		\Delta f_{1} &= \lim_{h \to 0}{\frac{f(1+h)-f(1)}{h}} \\
		&= \lim_{h \to 0}{\frac{(1+h)^2-3(1+h) - (1)^2 + 3(1)}{h}} \\
		&= \lim_{h \to 0}{\frac{h^2 + 2h + 1 - 3 - 3h - 1 + 3}{h}} \\
		&= \lim_{h \to 0}{\frac{h^2 - h}{h}} \\
		&= \lim_{h \to 0}{h - 1} \\
		&= -1.
	\end{align*}

	Now that we have the slope of the tangent line, we can write the equation of the line in point-slope form.
	We can also rearrange to standard for is needed.
	\begin{align*}
		y - f(1) &= -1(x - 1) \\
		y + 2 &= -x + 1 \\
		y &= -x - 1.
	\end{align*}
\end{answer}


\subsection{Rotations}

Any matrix of the form $\begin{bmatrix}a&-b\\b&a\end{bmatrix}$ where $a^2+b^2=1$.
Thus, $\theta=\tan^{-1}(\frac{b}{a})$, or by any other trigonometric relation.

\subsection{Scaling and dilation}

Horizontal scaling affects the $x$-component:

\[
    T \begin{bmatrix}x\\ y\end{bmatrix}
    =
    \begin{bmatrix}kx\\ y\end{bmatrix}
    \implies
    A=
    \begin{bmatrix}k&0\\ 0&1\end{bmatrix}
\]

Vertical scaling affects the $y$-component:

\[
    T \begin{bmatrix}x\\ y\end{bmatrix}
    =
    \begin{bmatrix}x\\ ky\end{bmatrix}
    \implies
    A=
    \begin{bmatrix}1&0\\ 0&k\end{bmatrix}
\]

Dilation is scaling by $k$ for both $x$ and $y$:

\[
    T \begin{bmatrix}x\\ y\end{bmatrix}
    =
    \begin{bmatrix}kx\\ ky\end{bmatrix}
    \implies
    A=
    \begin{bmatrix}k&0\\ 0&k\end{bmatrix}
\]

\subsection{Normalizing a vector}

Can make any vector into a unit vector parallel to the original:

\[\boxed{\tb{u}=\frac{\tb{v}}{||\tb{v}||}}\]

The magnitude of a unit vector is always 1 ($||\tb{u}||=1$).

\subsection{Projections}

$x$-axis:

\[
      T \begin{bmatrix}x\\ y\end{bmatrix}=
      \begin{bmatrix}x\\ 0\end{bmatrix}
\]

$y$-axis:

\[
  T \begin{bmatrix}x\\ y\end{bmatrix}
  = \begin{bmatrix}0\\ y\end{bmatrix}
\]

Mathematically:

\[||\mathrm{proj}_l(\tb{v})||=||\tb{v}||\cos\theta\]

The dot product:

\[\boxed{\tb{a}\cdot \tb{b}=||\tb{a}||\tb{b}||\cos\theta}\]

Unit vector $\tb{u}$ given by the following because the line $l$ can be represented by $\begin{bmatrix}
    1\\m
\end{bmatrix}$:

\[\tb{u}=\frac{1}{\sqrt{1+m^2}}\begin{bmatrix}
    1\\m
\end{bmatrix}\]

The projection matrix, onto a line of slope $m$:

\[
    \mathrm{proj}_l(\tb{v})=(\tb{u}\cdot \tb{v})\tb{u} =
    \begin{bmatrix}\frac{v_1+v_2m}{1+m^2}\\\frac{v_1m+v_2m^2}{1+m^2} \end{bmatrix}
\]

Thus,

\[A=
    \begin{bmatrix}\frac{1}{1+m^2}&\frac{m}{1+m^2}\\\frac{m}{1+m^2}&\frac{m^2}{1+m^2}\end{bmatrix}
\]

General projection matrix given $a^2+b^2=1$:

\[\boxed{
    A= \begin{bmatrix}a^2&ab\\ ab&b^2\end{bmatrix}
}\]

\subsection{Reflections}

Given by:

\[\mathrm{refl}_l(\tb{v})=2\mathrm{proj}_l(\tb{v})-\tb{v}\]

Has the matrix $A$:

\[A= \begin{bmatrix}\frac{1-m^2}{1+m^2}&\frac{2m}{1+m^2}\\\frac{2m}{1+m^2}&\frac{m^2-1}{1+m^2}\end{bmatrix}\]

If $a^2+b^2=1$:

\[\boxed{
    A= \begin{bmatrix}a&b\\ b&-a\end{bmatrix}
}\]

\subsection{Shear}

Horizontal:

\[A= \begin{bmatrix}1&k\\ 0&1\end{bmatrix}\]

Vertical:

\[A= \begin{bmatrix}1&0\\ k&1\end{bmatrix}\]


\section{Recap}
\begin{align}
    \rho &= \frac{e^{-\beta H}}{Z}
\end{align}
where
\begin{align}
    Z &= \Tr e^{-\beta H}
\end{align}
and for any operator $\mathcal{O}$,
\begin{align}
    \bar{\mathcal{O}} &= \Tr(\rho\mathcal{O})
\end{align}
and if it's a continuous system,
\begin{align}
    \rho(q, p) &=
    \frac{e^{-\beta H(q, p)}}{Z}
\end{align}
and the partition function is
\begin{align}
    Z &=
    \frac{\int d^Nq d^Np e^{-\beta H(q, p)}}{h^N n!}
\end{align}

\begin{example}[Harmonic oscillator]
    The partition function is
    \begin{align}
        Z &=
        \sum_{n=0}^{\infty} \bra{n} e^{-\beta H} \ket{n}\\
        &=
        \sum_{n=0}^{\infty} e^{-\beta\hbar\omega\left( n + \frac{1}{2}
        \right)}\\
        &=
        e^{-\beta \hbar\omega/2}
        \sum_{n=0}^{\infty} \left( e^{-\beta \hbar \omega} \right)^n\\
        &=
        \frac{1}{e^{\beta \hbar\omega/2} + e^{-\beta\hbar\omega/2}}\\
        &=
        \frac{1}{2}
        \frac{1}{\sinh\left(\beta\hbar\omega/2 \right)}
    \end{align}
    So then the average energy is
    \begin{align}
        \bar{E} &= \Tr(\rho H)\\
        &= \frac{1}{Z} \sum_{n=0}^{\infty}
        \bra{n} e^{-\beta H} H \ket{n}\\
        &=
        \frac{1}{Z} \sum_{n=0}^{\infty}
        E_n e^{-\beta E_n}\\
        &= \frac{1}{Z}\left( -\frac{\partial}{\partial\beta} \right) Z\\
        &=
        -\frac{\partial}{\partial\beta}\ln Z\\
        &=
        \frac{\partial}{\partial\beta}
        \ln\left(
            Z \sinh(\beta\hbar\omega/2)
        \right)\\
        &=
        \frac{\cosh(\beta\hbar\omega/2)}{\sinh(\beta\hbar\omega/2)}
        \frac{\hbar\omega}{2}
    \end{align}
\end{example}

In the high temperature limit $k_B T \ge \hbar\omega$,
this expression approaches
\begin{align}
    \bar{E} \to k_B T
\end{align}
At high energy,
it doesn't matter what the energy is,
the energy is just going to be given by the thermal energy.
You might know this result from somewhere right?
It's closely related to the equipartition theorem.

In the low-temperature limit,
\begin{align}
    \bar{E} &=
    \hbar\omega
    \frac{1 + e^{-\beta\hbar\omega}}{1 + e^{-\beta\hbar\omega}}\\
    &\approx
    \frac{\hbar\omega}{2}\left(
        1 + 2e^{-\beta\hbar\omega} + \cdots
    \right)
\end{align}
so no matter how low temperature you get,
you can't get lower than $\hbar\omega/2$,
which is the \emph{zero point energy}.
The excitations are exponentially small.

This is how quantum mechanics was invented by the way.
If you have a cavity,
each mode behaves like a harmonic oscillator.
The only thing with different wave lengths is that the frequency changes.
Every mode of the EM rays inside the black body is going to contribute some
energy $k_B T$,
but there's an infinite number of modes,
because I can make them smaller,
so this $\bar{E} = k_B T$ must be wrong.

The correct QM calculation is that no matter how high $T$ is,
when $\omega$ is large enough,
it stays very close to the ground state,
exponentially so.
No matter what the temperature is,
for the high energy modes,
they stay in the ground state,
and they shut off,
and that's how QM saves it.

There's so much physics to talk about.
In a certain way,
we're going to derive this result over and over again,
and get all amazing results.

For example,
get a metal piece reasonably cold.
By metal,
I mean a non-conducting crystal,
so a salt,
a rock.
The goal of this class is that you can run this calculation in your head
immediately.
We're going to derive this result,
hand it to your previous self and tell me how are you going to derive it.
Here we're still understanding the formalism.

How let's go back to the classical harmonic oscillator
and the classical calculator..

\begin{example}[Classical harmonic oscillator]
    Let's say we have a simple harmonic oscillator
    \begin{align}
        H(x, p) &=
        \frac{p^2}{2m} + \frac{m\omega^2}{2}x^2
    \end{align}
    and the partition function should be an integral
    \begin{align}
        Z &=
        \int_{-\infty}^{\infty}\frac{dx\,dp}{h}
        e^{-\beta\left(
            \frac{p^2}{2m} + \frac{m\omega^2}{2}x^2
        \right)}
    \end{align}
    which is a Gaussian integral.
\end{example}
My intent for today was to give you homework was to show you how to do a
Gaussian integral,
but you have a midterm,
so it's next week.
My opinion is that every integral should be done by computer,
but this one is worth known.

Suppose you are stuck on an island.
The only way out is to build a steam engine out of coconuts and bananas.
To build a steam engine,
you need to know thermodynamics,
and so you need to calculate partition functions,
so you need to know how to calculate Gaussian integrals.

Remember this one
\begin{align}
    Z = \frac{1}{h}
    \int_{-\infty}^{\infty} dx\,
    e^{\frac{-\beta m\omega^2}{2} x^2}
    \int_{-\infty}^{\infty} dp\,
    e^{\frac{-\beta p^2}{2}}
\end{align}
Look at the first integral. By dimension analysis,
there's only one thing it could depend on,
because $x$ is of units lengths,
and the argument of the exponential is dimensionless,
so $1/\sqrt{\beta m \omega^2}$ has units of length.
\begin{align}
    \int_{-\infty}^{\infty} dx\,
    e^{\frac{-\beta\hbar\omega^2}{2} x^2}
    \propto
    \frac{1}{\sqrt{\beta m \omega}}
\end{align}
up to some dimensionless constant, and it happens to be
\begin{align}
    \int_{-\infty}^{\infty} dx\,
    e^{\frac{-\beta\hbar\omega^2}{2} x^2}
    &=
    \sqrt{\frac{2\pi k_B T}{\omega^2}}
\end{align}
and
\begin{align}
    \int_{-\infty}^{\infty} dp\,
    e^{\frac{-\beta p^2}{2}}
    &=
    \sqrt{2\pi m k_B T}
\end{align}
so
\begin{align}
    Z &= \frac{k_B T}{\hbar \omega}.
\end{align}
Relax about the $\hbar$, it's just a constant that cancels out when
you calculate expectations!

So if you're stuck on an island and your life depends it,
you'll know.

One thing you'll learn in this course is how to take derivatives with logs.
\begin{align}
    \bar{E} &=
    \frac{1}{Z} \int dx\,dp\,
    e^{-\beta H} H\\
    &=
    \frac{1}{Z}\left( \frac{-\partial}{\partial\beta} \right)Z\\
    &=
    -\frac{\partial}{\partial \beta}Z\\
    &= -\frac{\partial}{\partial\beta}\ln Z\\
    &=
    \frac{\partial}{\partial\beta} \ln(\beta\hbar\omega)\\
    &=
    \frac{1}{\beta}\\
    &= k_B T
\end{align}
which is not surprising as you did the quantum calculation
and showed this is the high energy limit.

That's why there's a delivery of liquid nitrogen and once in a blue moon liquid
helium,
because people want to do experiments at low energy.

In the old days, the notation $\langle E\rangle$ was common,
and Dirac decided to break it up into $\bra{\psi}E\ket{\psi}$.

Recall how to compute the variance.
\begin{align}
    \overline{(E - \overline{E})^2}
    &=
    \overline{E^2} - \overline{E}^2\\
    &=
    \frac{1}{Z} \int dx\,dp\,
    e^{-\beta H} H^2
    -\left(
        \int dx\, dp
        \frac{e^{-\beta H}}{Z} H
    \right)^2\\
    &= \frac{\partial^2}{\partial\beta^2} \ln Z\\
    &= \left( k_B T \right)^2
\end{align}
Now if you recall,
\begin{align}
    \frac{\partial}{\partial\beta} \ln Z
    &=
    \frac{1}{Z} \frac{\partial}{\partial\beta} Z\\
    \frac{\partial^2}{\partial\beta^2} \ln Z &=
    - \frac{1}{Z^2}\left( \frac{\partial^2 Z}{\partial \beta^2} \right)
\end{align}
Student: Stop!
You want me to copy how to do second derivatives of a log?

Anyway,
so I'm going to consider the spread of energy,
the noise to signal ratio,
\begin{align}
    \frac{\sqrt{\overline{(E - \overbar{E})^2}}}{\bar{E}} = 1
\end{align}
which is not good,
because there's so much fluctuation.
But it's no good doing statistical mechanics on just one single harmonic
oscillator,
we need many more.

Let me do $N$ harmonic oscillators.
Instead of having just 1 $x$ and one $p$,
I'm going to have $N$ $x$s and $N$ $p$s.
So Hamiltonian is
\begin{align}
    H(x, p) &=
    \frac{p_1^2}{2m}
    + \frac{m\omega^2}{2} x_1^2
    \frac{p_2^2}{2m}
    + \frac{m\omega^2}{2} x_2^2
    + \cdots
\end{align}
and so we get a product
\begin{align}
    Z &=
    \prod_{i=1}^{N}\int_{-\infty}^{\infty}\frac{dx_i\,dp_i}{h}
    e^{-\beta\left(
        \frac{p_i^2}{2m} + \frac{m\omega^2}{2}x_i^2
    \right)}
\end{align}
And if you follow the steps,
you get
\begin{align}
    \overline{(E - \overline{E})^2} = N(k_B T)^2
\end{align}
and the noise to signal ratio drops to
\begin{align}
    \frac{\sqrt{\overline{(E - \overbar{E})^2}}}{\bar{E}} =
    \frac{\sqrt{N} k_B T}{N k_B T}\\
    = \frac{1}{\sqrt{N}}
\end{align}
so if you have $10^{23}$ particles,
the fluctuation in total energy is very very small,
and that's why statistical mechanics works.


\subsection{Hidden layers}

In the perceptron we have input vector \(x\), and output:

\(a=a(wx)\)

We can augment the perceptron by adding a hidden layer.

Now the output on the activation function is an input to a second layer. By using different weights, we can create a second vector of inputs to the second layer.

\subsubsection{The parameters of a feed forward model}

\(\Theta^{j}\) is a matrix of weights for mapping layer \(j\) to \(j+1\). So we have \(\Theta^1\) and \(\Theta^2\).

If we have \(s\) units in the hidden layer, \(n\) features and \(k\) classes:

\begin{itemize}
\item The dimension of \(\Theta^1\) is \((n+1) \times s\)
\item The dimension of \(\Theta^2\) is \((s+1) \times k\)
\end{itemize}

These include the offsets for each layer.

\subsubsection{The activation function of a multi-layer perceptron}

For a perceptron we had \(a=f(wx)\). Now we have:

\(a_i^j=f(a_{j-1}\Theta_{j-1})\)

We refer to the value of a node as \(a_i^{j}\), the activation of unit \(i\) in layer \(j\).

\subsubsection{Initialising parameters}

We start by randomly initialisng the value of each \(\theta \).

We do this to prevent each neuron from moving in sync.


\subsection{Fubini's Theorem \& Domain Regions}
\begin{theorem}[Fubini's Theorem]
	The order of integration on a domain where the variables of integration $\left(x, y, \text{etc.}\right)$ vary independently doesn't matter.
\end{theorem}

\noindent
For example, let's find the volume under $f(x,y) = 9-x^2-y^2$, $(x,y) \in [0,1] \times [1,2]$.\\
We will do so in two ways to show that they are equivalent: one with $x$ first and then $y$ and another with $y$ first and then $x$.\\
\begin{center}
	\begin{tabular}{c|c}
		$V = \int_{0}^{1}{\int_{1}^{2}{9 - x^2 - y^2\mathrm{d}y}\mathrm{d}x}$ & $V =\int_{2}^{1}{\int_{0}^{1}{9 - x^2 - y^2\mathrm{d}x}\mathrm{d}y}$ \\
		$= \int_{0}^{1}{\left[9y - x^2y - \frac{y^2}{3}\right]_{1}^{2}\mathrm{d}x}$ & $= \int_{1}^{2}{\left[9x - \frac{x^3}{3} - xy^2\right]_{0}^{1}\mathrm{d}y}$ \\
		$= \int_{0}^{1}{\frac{20}{3} - x^2\mathrm{d}x}$ & $= \int_{1}^{2}{\frac{26}{3} - y^2\mathrm{d}y}$ \\
		$= \left[\frac{20}{3}x - \frac{x^3}{3}\right]_{0}^{1}$ & $= \left[\frac{26}{3}y - \frac{y^3}{3}\right]_{1}^{2}$ \\
		$= \frac{19}{3}$ & $= \frac{19}{3}$ \\
	\end{tabular}
\end{center}

\noindent
Let's look at a case where $x$ and $y$ are not independent. specifically, where the bounds on $y$ are a function of $x$.

[INSERT IMAGE]

\noindent
This is called a Type I Region. Formally, a Type I Region is a domain $D = \left\{ (x,y) \mid a \leq x \leq b, g(x) \leq y \leq h(x) \right\}$.\\

\begin{theorem}[Fubini's Theorem for Type I Regions]
	Let $D$ be a Type I Region in $\mathbb{R}^2$.\\
	\begin{equation*}
		\iint\limits_{D}{f(x,y)\mathrm{d}A} = \int_{a}^{b}{\int_{g(x)}^{h(x)}{f(x,y)\mathrm{d}y}\mathrm{d}x}
	\end{equation*}
\end{theorem}

\noindent
It's also possible for y to have constant bounds and the bound for $x$ to be a function of $y$. this is a Type II Region. Formally, a Type II Region is a domain $D = \left\{(x,y) \mid g(y) \leq x \leq h(y), a \leq y \leq b\right\}$.

\begin{theorem}[Fubini's Theorem for Type II Regions]
	Let $D$ be a Type II Region in $\mathbb{R}^2$.
	\begin{equation*}
		\iint\limits_{D}{f(x,y)\mathrm{d}A} = \int_{a}^{b}{\int_{g(y)}^{h(y)}{f(x,y)\mathrm{d}x}\mathrm{d}y}
	\end{equation*}
\end{theorem}

\noindent
Sometimes, a region can be describes as both Type I and Type II. You should pick whichever description is most convenient.\\

\noindent
The previous two theorems can be summarized as dependent variables need to be integrated before the variables that they depend on. This core idea extends into higher dimensions where classifying regions becomes tedious and not very helpful.\\

\noindent
One can split larger, harder to describe domains into smaller domains. Let $D_1 \cup D_2 = D$.

[INSERT IMAGE]

\begin{equation*}
	\iint\limits_{D}{f(x,y)\mathrm{d}A} = \iint\limits_{D_1}{f(x,y)\mathrm{d}A} + \iint\limits_{D_2}{f(x,y)\mathrm{d}A} - \iint\limits_{D_1 \cap D_2}{f(x,y)\mathrm{d}A}
\end{equation*}


\section{Definition and equality}

\begin{outcome}
  \begin{enumerate}
  \item Identify the dimension and entries of a matrix.
  \item Check equality of matrices.
  \end{enumerate}
\end{outcome}

We have solved systems of equations by writing them in terms of an
augmented matrix and then doing row operations. It turns out that
matrices are important not only for systems of equations but also for
many other purposes.

\begin{definition}{Matrix}{matrix}
  A \textbf{matrix}%
  \index{matrix} is a rectangular array of numbers
  \begin{equation*}
    A = \begin{mymatrix}{cccc}
      a_{11} & a_{12} & \cdots & a_{1n} \\
      a_{21} & a_{22} & \cdots & a_{2n} \\
      \vdots & \vdots & \ddots & \vdots \\
      a_{m1} & a_{m2} & \cdots & a_{mn} \\
    \end{mymatrix},
  \end{equation*}
  where the $a_{ij}$ are scalars, called the \textbf{entries}%
  \index{matrix!entry of}%
  \index{entry of a matrix} or \textbf{components}%
  \index{matrix!component of}%
  \index{component!of a matrix} of $A$.  The \textbf{size}%
  \index{matrix!size of}%
  \index{size of a matrix} or \textbf{dimension}%
  \index{matrix!dimension of}%
  \index{dimension!of a matrix} of a matrix is defined as $m\times n$,
  where $m$ is the number of rows and $n$ is the number of columns.
\end{definition}

For example, here is a $3\times 4$-matrix (pronounced ``three-by-four
matrix''):
\begin{equation*}
  \begin{mymatrix}{rrrr}
    1 & 2 & 3 & 4 \\
    5 & 2 & 8 & 7 \\
    6 & -9 & 1 & 2
  \end{mymatrix}.
\end{equation*}
This is a $3\times 4$-matrix because there are three rows and four
columns. When specifying the size of a matrix, we always list the
number of rows before the number of columns.

Entries of the matrix are identified according to their position. The
\textbf{$(i,j)$-entry}%
\index{matrix!ij-entry@$(i,j)$-entry}%
\index{ij-entry of a matrix@$(i,j)$-entry of a matrix} of a matrix is
the entry in the $i\th$ row and $j\th$ column, and is often denoted
$a_{ij}$. For example, in the above matrix, the $(2,3)$-entry is the
entry in the second row and the third column, and is equal to $8$. We
sometimes use $A=\mat{a_{ij}}$ as a short-hand notation for the entire
$m\times n$-matrix whose $(i,j)$-entry is equal to $a_{ij}$ for all
$i=1,\ldots m$ and $j=1,\ldots,n$.

There are various operations which are done on matrices of appropriate
sizes. Matrices can be added and subtracted, multiplied by a scalar,
and multiplied by other matrices. We will never divide a matrix by
another matrix, but we will see later how matrix inverses play a
similar role.

\begin{definition}{Equality of matrices}{matrix-equality}
  Two matrices are \textbf{equal}%
  \index{matrix!equality}%
  \index{equality!of matrices} if they have the same size and the same
  corresponding entries. More precisely, if $A=\mat{a_{ij}}$ and
  $B=\mat{b_{ij}}$ are two $m\times n$-matrices, then $A=B$ means that
  $a_{ij}=b_{ij}$ for all $i=1,\ldots m$ and $j=1,\ldots,n$.
\end{definition}

For example,
\begin{equation*}
  \begin{mymatrix}{rr}
    0 & 0 \\
    0 & 0 \\
    0 & 0
  \end{mymatrix} \neq \begin{mymatrix}{rr}
    0 & 0 \\
    0 & 0
  \end{mymatrix}
\end{equation*}
because they are different sizes.
Also,
\begin{equation*}
  \begin{mymatrix}{rr}
    0 & 1 \\
    3 & 2
  \end{mymatrix} \neq \begin{mymatrix}{rr}
    1 & 0 \\
    2 & 3
  \end{mymatrix}
\end{equation*}
because, although they are the same size, their corresponding entries are not identical.

There are special names for matrices of certain dimensions: some
matrices are called square matrices, columns vectors, or row vectors.

\begin{definition}{Square matrix}{square-matrix}
  A matrix of size $n\times n$ is called a \textbf{square matrix}%
  \index{matrix!square}%
  \index{square matrix}.  In other words, $A$ is a square matrix if it
  has the same number of rows and columns.
\end{definition}

\begin{definition}{Column vectors and row vectors}{row-and-column-vectors}
  A matrix of size $n\times 1$ is called a \textbf{column vector}%
  \index{vector!column vector}%
  \index{column vector}. A matrix of size $1\times n$ is called a
  \textbf{row vector}%
  \index{vector!row vector}%
  \index{row vector}.  Here is an example of a column vector $X$ and a
  row vector $Y$:
  \begin{equation*}
    X=\begin{mymatrix}{c}
      x_1 \\
      \vdots \\
      x_n
    \end{mymatrix},
    \quad
    Y = \begin{mymatrix}{ccc}
      y_1 & \cdots & y_n
    \end{mymatrix}.
  \end{equation*}
\end{definition}

We have already encountered column vectors in
Chapter~\ref{cha:vectors-rn}.  When we use the term \textbf{vector}
without further qualification, we always mean a column vector. Also
recall from Definition~\ref{def:column-vector} that the set of
$n$-dimensional column vectors is called $\R^n$.


\section{Count Regressions}

Count data is common, and often incorrectly analyzed using ordinary least squares. In general, least squares is only applicable when the error on the data, or a transformation of the data, has normal distribution with uniform variance. Count data tends to defy that expectation, especially for low counts. Count data also tends to be heteroskedastic: higher counts tend to fluctuate more widely.

\begin{itemize}
\item The data is strictly positive. OLS models can predict negative values.
\item Especially for low counts, the distribution is highly skewed
\item The data is discrete, not continuous.
\item The fluctuation on the mean is not constant. The data is heteroskedastic.
\end{itemize}

The definite resource for modeling count data is \citeasnoun{hilbe2014modeling}.

\subsection{Poisson Regression Models}
Poisson models tend to be the first pass at modeling count data, perhaps analogous to how ordinary least squares (OLS) regression is the first pass at modeling continuous data. Poisson models distinguish themselves, in general, by assuming that the fluctuation of the data about the mean has Poisson distribution.

\subsubsection{``Ordinary" Poisson Regression}
The term Poisson regression tends to be used interchangeably with a specific type of model, which is a generalized linear model with Poisson noise and log link. A concise reference for those models is chapter 4 in \citeasnoun{rodriguez2007generalized}.

Specifically, the likelihood of observing a value $y$ is assumed to follow a Poisson distribution:

\begin{equation}
p(y|\mu) = \frac{e^{-\mu} \mu^{y}}{y!}
\end{equation}

Where the only parameter, $\mu$, is both the mean and the variance of the Poisson distribution. The model acquires additional structure when $\mu$ is assumed to be a function of some explanatory variables, $x$, i.e. $\mu = f(x)$. The canonical Poisson regression uses a log-linear relationship between the coefficients and the mean, i.e. $\mu = e^{\mathbf{\beta \cdot x}}$. The result is a generalized linear model with Poisson error and link log.

The assumptions of this "ordinary Poisson regression" (my idea to call it that) are:
\begin{itemize}
\item The error has Poisson distribution.
\item The data is strictly positive.
\item The data has discrete distribution (though the generalization to continuous numbers is pretty trivial)
\item The data is i.i.d., meaning that the observed count events are results of independent trials. (Example: the number of kids in a family is unaffected by the number of kids the neighbors have)
\end{itemize}

And, very importantly:
\begin{itemize}
\item It is a log-linear model! The relationship between the dependent variable ($Y$) and the independent variables is log linear. I.e. $ln(Y)$ is a linear function of the coefficients.
\item The distribution of the data is heteroskedastic so that the mean equals the variance. The Poisson distribution only has one parameter and (i.e. $\mu = \sigma$)!
\end{itemize}

In the model, the likelihood of observing a value $y$ is assumed to follow a Poisson distribution:

\begin{equation}
p(y|\mu) = \frac{e^{-\mu} \mu^{y}}{y!}
\end{equation}

Where the only parameter, $\mu$, is both the mean and the variance of the Poisson distribution. The model acquires additional structure when $\mu$ is assumed to be a function of some explanatory variables, $x$, i.e. $\mu = f(x)$. The canonical Poisson regression uses a log-linear relationship between the coefficients and the mean, i.e. $\mu = e^{\mathbf{\beta \cdot x}}$. The result is a generalized linear model with Poisson error and link log.

\begin{equation}
p(y|\mathbf{x};\mathbf{\beta}) = \frac{e^{-\exp(\mathbf{\beta\cdot x})} (\exp(\mathbf{\beta\cdot x}))^{y}}{y!}
\end{equation}

\begin{figure}
\centering
    \includegraphics[width=\textwidth]{poissonmodels02.png}
    \caption{Left: Ordinary Poisson Regression on data that satisfies the assumptions of the model. For small rates, the model is approximately linear, but beyond that it shows exponential growth because of the log-link. Right: Residuals showing heteroskedasticity with variance increasing with the mean. Dividing the residuals by the mean yields constant variance.}
    \label{fig:ordinarypoisson}
\end{figure}

\subsubsection{Poisson Noise \& Central Limit Theorem}

For small count rates, the Poisson distribution is highly skewed and strictly positive. For large count rates, the Poisson distribution is essentially normal, except that variance and mean are locked.

If you are dealing with large counts, then the Poisson model still has the feature of being heteroskedastic.

\subparagraph{Advantages of using Poisson Noise}

\begin{itemize}
\item The Poisson Distribution is highly skewed for small rates, and strictly positive. For high enough count rates, this advantage disappears.
\item The Poisson Distribution is heteroskedastic
\end{itemize}

\subparagraph{Drawbacks of using Poisson Noise}

\begin{itemize}
\item The Poisson distribution only has a single parameter. The assumption that the mean and the variance are the same is very restrictive.
\end{itemize}

\begin{figure}
\centering
    \includegraphics[width=\textwidth]{poissonmodels01.png}
    \caption{Left: The Poisson Distribution for small rates is highly skewed. Right: The Poisson Distribution for intermediate rates looks very similar to a normal distribution. In both cases, the variance and the mean are the same value.}
    \label{fig:poissonandcentrallimit}
\end{figure}

\subsubsection{Poisson Noise Analogy to Least Squares Regression}
To anchor intuition in familiar territory, consider least squares regression with a log-linear relationship between endogenous and exogenous variables (that is, the model assumes a relationship of $log(y)=a + \mathbf{bx}$ that is beset with Gaussian noise). The familiar form for the model is:

\begin{equation}
\begin{array}{rl}
y &= a\exp{\mathbf{b \cdot x}} + \epsilon \\
&= \exp\mathbf{\beta \cdot x} + \epsilon
\end{array}
\end{equation}

Where the constant $a$ was absorbed into the coefficient vector $\mathbf{\beta}$ in the second line, and $\mathbf{x} \rightarrow [1,\mathbf{x}]$. $\epsilon$ is an error term that is assumed to have normal distribution with zero mean, i.e. $\epsilon \sim \mathscr{N}(0,\sigma)$. It's a bit unnatural, but this can be rewritten, absorbing the parameters into the random term:

\begin{equation}
y = 0 + \epsilon'
\end{equation}

With $\epsilon' \sim \mathscr{N}(\mu = \exp{\mathbf{\beta \cdot x}},\sigma)$. Now what if the fluctuations aren't normally distributed about the mean, but they are Poisson distributed about the mean? In that case, $\epsilon' \sim \mathrm{Poisson}(\mu = \exp{\mathbf{\beta \cdot x}})$.

\subsubsection{Multivariate Poisson Model}
For least squared regression, it's totally common to look at multivariate models with interesting codependence structure captured by a covariance matrix. In analogy to that, there is multivariate Poisson Regression. The math looks quite different.

\url{http://www2.stat-athens.aueb.gr/~karlis/multivariate%20Poisson%20models.pdf}

\subsubsection{Goodness of Fit}
The Poisson deviance is given by:

\begin{equation}
D = 2\sum\left\{ y_i \log\left( \frac{y_i}{\hat{\mu}_i} \right) - \left( y_i - \hat{\mu}_i \right) \right\}
\end{equation}

Here, $\hat{\mu}_i = e^{\mathbf{x}^T_i \hat{\mathbf{\beta}}}$ is the fitted mean of the $i$th data point, and $y_i$ is the observed count of the $i$th datapoint.

For large sample sizes, the deviance will be distributed approximately chi-squared with $n-p$ degrees of freedom, where $n$ is the number of data points and $p$ is the number of features. An alternative to the deviance is Pearson's chi-squared statistic.

\subsubsection{General $\mu = f(\mathbf{x})$}
The Poisson-ness only has to do with how the data fluctuates about the mean. How the mean is expected to depend on the explanatory variables is another question. In so far, general other functions, including highly-nonlinear machine learning models, can be fit under the assumption of Poisson noise.

XGBoost supports poisson loss. Empirically, it seems that Poisson loss performs worse in the regime where the model is underfitting, and slightly better in the regime where the model is overfitting. The difference is especially pronounced in the Poisson deviance and for sparse data.

\begin{figure}
\centering
    \includegraphics[width=\textwidth]{poissonmodels03.png}
    \caption{Left: Ordinary Poisson Regression on data that satisfies the assumptions of the model. For small rates, the model is approximately linear, but beyond that it shows exponential growth because of the log-link. Right: Residuals showing heteroskedasticity with variance increasing with the mean. Dividing the residuals by the mean yields constant variance.}
    \label{fig:ordinarypoisson}
\end{figure}


\section{Low Power Design}
\begin{itemize}
	\item $\alpha$ switching activity
	\item $C_L$ load capacity
	\item $f$ clock frequency
	\item $V_{dd}$ supply voltage
	\item $V_T$ threshold voltage ($V_T \ll V_{dd}$)
\end{itemize}
\begin{align*}
	P &\sim \alpha C_L V_{dd}^2 f \\
	\tau &\sim C_L \frac{V_{dd}}{\left(V_{dd} - V_{T}\right)^2} \\
	f_{max} &\sim V_{dd} \\
	E &\sim \alpha C_L V_{dd}^2 \underbrace{\left(\text{\#cycles}\right)}_{f \cdot t}
\end{align*}

\subsection{YDS Algorithm}
The YDS algorithm can be use to schadule tasks with dynamic voltage scaling such
that the engergy consuption is minimal.

We define
\begin{align*}
	V'([z, z']) &= \set{v_i \in V: z \leq a_i < d_i \leq z'} \\
	G([z, z']) &= \sum_{v_i in V'([z,z'])} \frac{C_i}{z' - z}
\end{align*}

Algorithm:
\begin{enumerate}
	\item Calculate intensities
	\item Schedule jobs in highest intensity interval using EDF, intensity as
		frequency
	\item Adjust arrival times and deadlines by excluding the interval, restart
\end{enumerate}

The online version continuously updates best schedule for all available tasks.
It has at most 27 times higher energy consumption.

\subsection{DPM}
With dpm, make sleep periods as long as possible, reduce the amount of on/off
transitions. Use YDS and round frequencies up.


\subsection{Fourier transforms}

\subsubsection{Fourier transform}

\(\hat f(\Xi )=\int_{-\infty}^{\infty }f(x)e^{-2\pi ix\Xi }dx\)

\subsubsection{Inverse Fourier transform}

\(f(x)=\int_{-\infty}^{\infty }\hat f(\Xi )e^{2\pi ix\Xi }d\Xi \)

\subsubsection{Fourier inversion theorem}


\documentclass[11pt]{amsart}
\usepackage{amsmath,amsfonts,amsthm,amssymb, amsaddr}

\makeatletter
\def\subsection{\@startsection{subsection}{3}%
  \z@{.5\linespacing\@plus.7\linespacing}{.1\linespacing}%
  {\normalfont\itshape}}
\makeatother

\title{Coordinate Systems}

\author{Joe Bentley}

\date{\today}

\begin{document}

\maketitle

\newpage

\section{Orthogonal Coordinate Systems}

When we are looking to construct a new coordinate system, we want to be able to translate between unit vectors and coordinates of both coordinate systems. In this section we can explore how we can construct orthogonal coordinate systems, which is, coordinate systems where the basis are always orthogonal.

First, suppose that the Cartesian coordinates $(x, y, z)$ of any point $P$ in space can be expressed as functions of three variables $(u_1, u_2, u_3)$ such that,

\begin{align*}
  \mathbf{r} =
  \begin{pmatrix}
    x \\
    y \\
    z \\
  \end{pmatrix} =
  \begin{pmatrix}
    x(u_1, u_2, u_3) \\
    y(u_1, u_2, u_3) \\
    z(u_1, u_2, u_3)
  \end{pmatrix}
\end{align*}

If each point $P$ corresponds to a \textit{unique} set of new coordinates $(u_1, u_2, u_3)$ then these are called curvilinear coordinates of $P$. That is, for the coordinates $(u_1, u_2, u_3)$ to be curvilinear, they must be the \textit{only} set of new coordinates that corresponds to point $P$.

We know from directional derivatives that $\frac{\partial\mathbf{r}}{\partial u_1}$, $\frac{\partial\mathbf{r}}{\partial u_2}$, $\frac{\partial\mathbf{r}}{\partial u_3}$ are vectors in the directions of increasing $u_1$, $u_2$, and $u_3$ respectively. These may not necessarily be unit vectors, as they might not have a length of unity, so we may write,

\begin{align*}
   \frac{\partial\mathbf{r}}{\partial u_1} = h_1 \mathbf{e_1} \qquad \frac{\partial\mathbf{r}}{\partial u_2} = h_2 \mathbf{e_2} \qquad \frac{\partial\mathbf{r}}{\partial u_3} = h_3 \mathbf{e_3}
\end{align*}

where $\mathbf{e_1}$, $\mathbf{e_2}$, $\mathbf{e_3}$ are unit vectors in the directions of increasing $u_1$, $u_2$, and $u_3$ respectively. The quantities $h_1$, $h_2$, and $h_3$ are \textit{scale factors} which scale the derivative down to unity, such that,

\begin{align*}
  \frac{1}{h_i} \left|\frac{\partial\mathbf{r}}{\partial u_i}\right| = 1
\end{align*}

and therefore we can calculate the scale factor $h_i$ by,

\begin{align*}
  \left|\frac{\partial\mathbf{r}}{\partial u_i}\right| = h_i
\end{align*}

If our unit vectors $\mathbf{e_1}$, $\mathbf{e_2}$, $\mathbf{e_3}$ are mutually orthogonal, then we have an orthogonal coordinate system.

One final point is the order the coordinates are written in. This is mathematically arbitrary, but we want to choose coordinates such that,

\begin{align*}
  \hat{\imath}\times\hat{\jmath}&=\hat{k}
\end{align*}

or in terms of our more general unit vectors,

\begin{align*}
  \mathbf{e_1}\times\mathbf{e_2}&=\mathbf{e_3}
\end{align*}

If the coordinates are our of order, such that for example $\mathbf{e_1}$ and $\mathbf{e_2}$ are switched so that the coordinates are written $(u_2, u_1, u_3)$, then the result of the cross product of the first two unit basis vectors will be negative instead of positive. We would prefer them to be positive as this feels more physically intuitive and natural.

In the next few examples, we will find the unit vectors of various coordinate systems, as well as proving their orthogonality.

\section{Cylindrical Polar Coordinates}

We have enough experience in cylindrical polar coordinates to know how to write the Cartesian coordinates $(x, y, z)$ in terms of the cylindrical coordinates $(\rho, \phi, z)$,

\begin{align*}
  \mathbf{r} =
  \begin{pmatrix}
    x \\
    y \\
    z
  \end{pmatrix} =
  \begin{pmatrix}
    \rho\cos\phi \\
    \rho\sin\phi \\
    z
  \end{pmatrix}
\end{align*}

How do we know to order the coordinates $(\rho, \phi, z)$? We don't, and we will check their order once finding the unit vectors.

First, we want to find the unit basis $\mathbf{e_{\rho}}$. We can do this by taking the derivative of the position vector with respect to the coordinate $\rho$. This will give us a possibly non-unit vector in the direction of changing $\rho$, which we then need to divide by the scale factor $h_{\rho}$ to give us the unit basis vector $\mathbf{e_{\rho}}$.

\begin{align*}
  \frac{\partial\mathbf{r}}{\partial\rho} =
  \begin{pmatrix}
    \cos\phi \\
    \sin\phi \\
    0
  \end{pmatrix}
\end{align*}

We can see here that the result is already a unit vector, as $\sin^2\phi + \cos^2\phi = 1$. This means that our scale factor $h_{\rho} = 1$, and our unit basis vector is given by,

\begin{align*}
  \mathbf{e_{\rho}} =
  \begin{pmatrix}
    \cos\phi \\
    \sin\phi \\
    0
  \end{pmatrix}
\end{align*}

Similarly, for the $\phi$ and $z$ coordinates,

\begin{align*}
  \frac{\partial\mathbf{r}}{\partial\phi} = \rho
  \begin{pmatrix}
    -\sin\phi \\
    \cos\phi \\
    0
  \end{pmatrix}
\end{align*}

This time we see that the result is not a unit vector, but by factoring out $\rho$, we get the scale factor multiplied by a unit basis vector, therefore we have,

\begin{align*}
  h_{\phi} = \rho
\end{align*}

and

\begin{align*}
  \mathbf{e_{\phi}} =
  \begin{pmatrix}
    -\sin\phi \\
    \cos\phi \\
    0
  \end{pmatrix}
\end{align*}

Finally we have the $z$ coordinate,

\begin{align*}
  \frac{\partial\mathbf{r}}{\partial z} =
  \begin{pmatrix}
    0 \\
    0 \\
    1
  \end{pmatrix} \to
  \mathbf{e_z} =
  \begin{pmatrix}
    0 \\
    0 \\
    1
  \end{pmatrix}
\end{align*}

Here the scale factor again is just $h_z = 1$.

Next we need to check whether these unit basis form an orthogonal coordinate system,

\begin{alignat*}{2}
  \mathbf{e_{\rho}}\cdot\mathbf{e_{\phi}}&=
  \begin{pmatrix}
    \cos\phi \\
    \sin\phi \\
    0
  \end{pmatrix}\cdot
  \begin{pmatrix}
    -\sin\phi \\
    \cos\phi \\
    0
  \end{pmatrix} &&= 0 \\
  \mathbf{e_{\rho}}\cdot\mathbf{e_z}&=
  \begin{pmatrix}
    \cos\phi \\
    \sin\phi \\
    0
  \end{pmatrix}\cdot
  \begin{pmatrix}
    0 \\
    0 \\
    1
  \end{pmatrix} &&= 0 \\
  \mathbf{e_{\phi}}\cdot\mathbf{e_z}&=
  \begin{pmatrix}
    -\sin\phi \\
    \cos\phi \\
    0
  \end{pmatrix}\cdot
  \begin{pmatrix}
    0 \\
    0 \\
    1
  \end{pmatrix} &&= 0
\end{alignat*}

Therefore we have that all the unit basis vectors are mutually orthogonal. Also to check that the order of the coordinates $(\rho, \phi, z)$ we calculate the cross product and see that $\mathbf{e_{\rho}}\times\mathbf{e_{\phi}}=\mathbf{e_z}$ although this will not be shown explicitly here as it is simply the cross product of two vectors.

\section{Spherical Polar Coordinates}

Again we will take the same approach as last time. This time we know we can write $(x, y, z)$ as $(r, \theta, \phi)$ by the relations,

\begin{align*}
  \mathbf{r} =
  \begin{pmatrix}
    r\sin\theta\cos\phi \\
    r\sin\theta\sin\phi \\
    r\cos\theta
  \end{pmatrix}
\end{align*}

So we first take the derivative of the position vector with respect to each coordinate to find the vectors in the direction of each increasing coordinate,

\begin{align*}
  \frac{\partial\mathbf{r}}{\partial r} =
  \begin{pmatrix}
    \sin\theta\cos\phi \\
    \sin\theta\sin\phi \\
    \cos\theta
  \end{pmatrix}
\end{align*}

By $\sin^2\phi + \cos^2\phi = 1$ this again is already a unit vector, so $h_r = 1$ and,

\begin{align*}
  \mathbf{e_r} =
  \begin{pmatrix}
    \sin\theta\cos\phi \\
    \sin\theta\sin\phi \\
    \cos\theta
  \end{pmatrix}
\end{align*}

and for the $\theta$ and $\phi$ coordinates,

\begin{align*}
  \frac{\partial\mathbf{r}}{\partial\theta} = r
  \begin{pmatrix}
    \cos\theta\cos\phi \\
    \cos\theta\sin\phi \\
    -\sin\theta
  \end{pmatrix}
\end{align*}

The scale factor is $h_{\theta} = r$ and,

\begin{align*}
  \mathbf{e_{\theta}} =
  \begin{pmatrix}
    \cos\theta\cos\phi \\
    \cos\theta\sin\phi \\
    -\sin\theta
  \end{pmatrix}
\end{align*}

For the last coordinate $\phi$,

\begin{align*}
  \frac{\partial\mathbf{r}}{\partial\phi} = r\sin\theta
  \begin{pmatrix}
    \cos\theta\cos\phi \\
    \cos\theta\sin\phi \\
    -\sin\theta
  \end{pmatrix}
\end{align*}

Therefore this time we have a scale factor of $h_{\phi} = r\sin\theta$ and,

\begin{align*}
  \mathbf{e_{\phi}} =
  \begin{pmatrix}
    -\sin\phi \\
    \cos\phi \\
    0
  \end{pmatrix}
\end{align*}

Again we check for orthogonality,

\begin{align*}
  \mathbf{e_r}\cdot\mathbf{e_{\theta}} &= 0 \\
  \mathbf{e_r}\cdot\mathbf{e_{\phi}} &= 0 \\
  \mathbf{e_{\phi}}\cdot\mathbf{e_{\theta}} &= 0
\end{align*}

And finally we check that $(r, \theta, \phi)$ is the correct order of the coordinates,

\begin{align*}
  \mathbf{e_r}\times\mathbf{e_{\theta}}=\mathbf{e_{\phi}}
\end{align*}

\section{Paraboidal Coordinates}

The paraboidal coordinate $(u, v, \phi)$ are related to the cartesian coordinates such that,

\begin{align*}
  \mathbf{r}=
  \begin{pmatrix}
    uv\cos\phi \\
    uv\sin\phi \\
    \frac{1}{2}\left(u_2 - v^2\right)
  \end{pmatrix}
\end{align*}

Again we take the derivatives,

\begin{align*}
  \frac{\partial\mathbf{r}}{\partial u} =
  \begin{pmatrix}
    v\cos\phi \\
    v\sin\phi \\
    u
  \end{pmatrix}
\end{align*}

This time, unlike before, finding the scale factor and thus the unit basis vector is not as easy as a simple factorisation to construct the unit vector. This time we must find the modulus of the derivative to give us the scale factor,

\begin{align*}
  h_u = \left|\frac{\partial\mathbf{r}}{\partial u}\right| = \sqrt{u_2 + v_2}
\end{align*}

so our unit vector is given by,

\begin{align*}
  \mathbf{e_u} = \frac{1}{\sqrt{u^2 + v^2}}
  \begin{pmatrix}
    v\cos\phi \\
    v\sin\phi \\
    u
  \end{pmatrix}
\end{align*}

and for coordinate $v$ we find,

\begin{align*}
  \frac{\partial\mathbf{r}}{\partial v} =
  \begin{pmatrix}
    u\cos\phi \\
    u\sin\phi \\
    -v
  \end{pmatrix}
\end{align*}

so just as before $h_v = \sqrt{u^2 + v^2}$ and,

\begin{align*}
  \mathbf{e_v} =
  \begin{pmatrix}
    u\cos\phi \\
    u\sin\phi \\
    -v
  \end{pmatrix}
\end{align*}

and for coordinate $\phi$,

\begin{align*}
  \frac{\partial\mathbf{r}}{\partial\rho} =
  \begin{pmatrix}
    -uv\sin\phi \\
    uv\cos\phi \\
    0
  \end{pmatrix}
\end{align*}

This time we can factorize so that $h_{\phi} = uv$ and,

\begin{align*}
  \mathbf{e_{\phi}} =
  \begin{pmatrix}
    -\sin\phi \\
    \cos\phi \\
    0
  \end{pmatrix}
\end{align*}

This time, to check for orthogonality, we will take the dot product of the derivatives of the unit vectors instead of the unit vectors themselves. We do this as a convenience to make the calculation easier. It is possible to do this because the derivative $\partial \mathbf{r} / \partial u_1$ is just the unit vector $\mathbf{e_1}$ multiplied by a constant, the scale factor. By computing the dot products we see that,

\begin{align*}
  \frac{\partial\mathbf{r}}{\partial u}\cdot\frac{\partial\mathbf{r}}{\partial v} &= 0 \\
  \frac{\partial\mathbf{r}}{\partial u}\cdot\frac{\partial\mathbf{r}}{\partial \phi} &= 0 \\
  \frac{\partial\mathbf{r}}{\partial v}\cdot\frac{\partial\mathbf{r}}{\partial \phi} &= 0
\end{align*}

and therefore,

\begin{align*}
  \mathbf{e_u}\cdot\mathbf{e_v} &= 0 \\
  \mathbf{e_u}\cdot\mathbf{e_{\phi}} &= 0 \\
  \mathbf{e_v}\cdot\mathbf{e_{\phi}} &= 0
\end{align*}

Therefore we have an orthogonal basis. The order of the coordinates is checked just as before and we find that $(u, v, \phi)$ is indeed the correct order, such that,

\begin{align*}
  \mathbf{e_u}\times\mathbf{e_v}=\mathbf{e_{\phi}}
\end{align*}

\section{The Gradient}

In this section we will discover how to take the gradient of a scalar function in an orthogonal coordinate system. First we note that the gradient will be a vector function acting on a scalar field $\Psi$ such that,

\begin{align*}
  \nabla\Psi = A_1 \mathbf{e_1} + A_2 \mathbf{e_2} + A_3 \mathbf{e_3}
\end{align*}

The difficulty arises since in an orthogonal coordinate system $\mathbf{e_1}$, $\mathbf{e_2}$, and $\mathbf{e_3}$ can vary in space, unlike the Cartesian coordinate system where $\hat{\imath}$, $\hat{\jmath}$, and $\hat{k}$ are constant.

The coefficients, $A_i$ can be extracted by using the orthogonality of the vectors $\mathbf{e_i}$. This allows us to use the dot product to extract each component of the gradient,

\begin{align*}
  A_i = \mathbf{e_i} \cdot \nabla\Psi = \frac{1}{h_i} \frac{\partial\mathbf{r}}{\partial u_i} \cdot \nabla\Psi
\end{align*}

Here we have used the discovery from the last section that the derivative in some direction $\partial\mathbf{r}/\partial u_i$ is just the unit vector $\mathbf{e_i}$ multiplied by some scale factor $h_i$. Expanding out this scalar product gives,

\begin{align*}
  \frac{\partial\mathbf{r}}{\partial u_i} &= \left(\frac{\partial x}{\partial u_i}, \frac{\partial y}{\partial u_i}, \frac{\partial z}{\partial u_i}\right) \cdot \left(\frac{\partial\Psi}{\partial x}, \frac{\partial\Psi}{\partial y}, \frac{\partial\Psi}{\partial z}\right) \\
                                          &= \frac{\partial\Psi}{\partial x} \frac{\partial x}{\partial u_i} + \frac{\partial\Psi}{\partial y} \frac{\partial y}{\partial u_i} + \frac{\partial\Psi}{\partial z} \frac{\partial z}{\partial u_i} \\
                                          &= \frac{\partial\Psi}{\partial u_i}
\end{align*}

In the last step we have just applied the chain rule in reverse, that is, that the derivative of $\Psi$ with respect to $u_i$ can be expressed as the derivatives of each Cartesian component. If you apply the chain rule to the final line it can be seen that this is what results. We now have an expression for an arbitrary coefficient $A_i$ of the gradient,

\begin{align*}
  A_i = \frac{1}{h_i} \frac{\partial\Psi}{\partial u_i}
\end{align*}

Therefore, by substituting these coefficients into our equation for the gradient,

\begin{align*}
  \nabla\Psi = \frac{1}{h_1}\frac{\partial\Psi}{\partial u_1} \mathbf{e_1} + \frac{1}{h_2}\frac{\partial\Psi}{\partial u_2} \mathbf{e_2} + \frac{1}{h_3}\frac{\partial\Psi}{\partial u_3} \mathbf{e_3}
\end{align*}

For example, in spherical polar coordinates this would be,

\begin{align*}
  \nabla\Psi = \frac{\partial\Psi}{\partial r} \mathbf{e_r} + \frac{1}{r}\frac{\partial\Psi}{\partial\theta}\mathbf{e_{\theta}} + \frac{1}{r\sin\theta}\frac{\partial\Psi}{\partial\Phi}\mathbf{e_{\Phi}}
\end{align*}

We now have an expression for the differential operator $\nabla$ in any orthogonal coordinate system,

\begin{align*}
  \nabla = \frac{\mathbf{e_1}}{h_1}\frac{\partial}{\partial u_1} + \frac{\mathbf{e_2}}{h_2}\frac{\partial}{\partial u_2} + \frac{\mathbf{e_3}}{h_3}\frac{\partial}{\partial u_3}
\end{align*}

When we evaluate $\nabla\cdot\mathbf{A}$ or $\nabla\times\mathbf{A}$, the derivative operators also apply to the unit vectors $\mathbf{e_i}$. This was fine in Cartesian coordinates because the unit vectors did not change in space, but now we have unit vectors which vary in the $x$, $y$, and $z$ directions. Therefore if we want to calculate the divergence or the curl we have to take a lot of derivatives, but this isn't necessary and it turns out that it is easier to apply Gauss' theorem and Stoke's theorem instead.

\section{Divergence in Orthogonal Coordinates: Integral Approach}

Consider an infinitesimal cuboid situated at $(u_1, u_2, u_3)$ with sides of length $h_1 du_1$, $h_2 du_2$, and $h3 du_3$. We want to calculate the surface integral over the surface of the cuboid,

\begin{align*}
  \oint_S \mathbf{A}\cdot d\mathbf{S}
\end{align*}

Our vector field $\mathbf{A}$ will be of the form,

\begin{align*}
  \mathbf{A} = A_1 \mathbf{e_1} + A_2 \mathbf{e_2} + A_3 \mathbf{e_3}
\end{align*}

First we will consider the flux through the surfaces parallel to $\mathbf{e_1}$,

\begin{align*}
  \int_{S_1} \mathbf{A}\cdot d\mathbf{S_1} &= \mathbf{A}\cdot d\mathbf{S_1} = \mathbf{A}\cdot h_2 du_2 h_2 du_3 \mathbf{e_1} = h_2 h_3 A_1 du_2 du_3 \\
  \int_{S_2} \mathbf{A}\cdot d\mathbf{S_2} &= \mathbf{A}\cdot d\mathbf{S_2} = -\mathbf{A}\cdot h_2 du_2 h_2 du_3 \mathbf{e_1} = -h_2 h_3 A_1 du_2 du_3
\end{align*}

The minus sign appears because $\mathbf{S_2}$ is in the opposite direction to $\mathbf{S_1}$. It may seem almost like when we sum them, they will disappear, but we have to remember that these coefficients will vary in different places in space; the $A_1$ in the first integral is evaluated at $(u_1 + du_1, u_2, u_3)$ whereas the $A_1$ in the second integral is evaluated at $(u_1, u_2, u_3)$. Now by summing these integrals,

\begin{align*}
  \int_{S_1 + S_2} \mathbf{A}\cdot d\mathbf{S} &= \left[h_2 h_3 A_1(u_1 + du_1) - h_2 h_3 A_1(u_1)\right] du_2 du_3 \\
                                               &= \left[\frac{\partial}{\partial u_1} (h_2 h_3 A_1) du_1\right] du_2 du_3
\end{align*}

In this last line we have used the definition of the derivative to see that the term in square brackets in the first line is just the derivative $\partial \dots / \partial u1$ multiplied by $du_1$.

When we sum all of the other pairs of surfaces we get similar results,

\begin{align*}
  \oint_S \mathbf{A}\cdot d\mathbf{S} = \left[\frac{\partial}{\partial u_1}(h_2h_3A_1) + \frac{\partial}{\partial u_2}(h_3h_1A_2) + \frac{\partial}{\partial u_3}(h_1h_2A_3)\right] du_1 du_2 du_3
\end{align*}

Gauss' theorem tells us that the divergence is just the flux per unit volume, so we define the volume of the cuboid as,

\begin{align*}
  \Delta V = h_1h_2h_3du_1du_2du_3
\end{align*}

and then by using that divergence is the flux divided by volume,

\begin{align*}
  \nabla\cdot\mathbf{A} = \frac{\oint_S \mathbf{A}\cdot d\mathbf{S}}{\Delta V} = \frac{1}{h_1h_2h_3} \left[\frac{\partial}{\partial u_1}(h_2h_3A_1) + \frac{\partial}{\partial u_2}(h_3h_1A_2) + \frac{\partial}{\partial u_3}(h_1h_2A_3)\right]
\end{align*}

Similarly this allows us to find an expression for the Laplacian of a scalar field, $\nabla^2 \Psi = \nabla \cdot (\nabla \Psi)$,

\begin{align*}
  \nabla\Psi = \frac{1}{h_1}\frac{\partial\Psi}{\partial u_1} \mathbf{e_1} + \frac{1}{h_2}\frac{\partial\Psi}{\partial u_2} \mathbf{e_2} + \frac{1}{h_3}\frac{\partial\Psi}{\partial u_3} \mathbf{e_3}
\end{align*}

and then substituting this into the divergence,

\begin{align*}
  \nabla^2\Psi = \frac{1}{h_1h_2h_3}\left[\frac{\partial}{\partial u_1}\left(\frac{h_2h_3}{h_1}\frac{\partial\Psi}{\partial u_1}\right) + \frac{\partial}{\partial u_2}\left(\frac{h_3h_1}{h_2}\frac{\partial\Psi}{\partial u_2}\right) + \frac{\partial}{\partial u_3}\left(\frac{h_1h_2}{h_3}\frac{\partial\Psi}{\partial u_3}\right)\right]
\end{align*}

\section{Using the Laplacian in Orthogonal Coordinate Systems}

\subsection*{Cylindrical Polar Coordinates}

In cylindrical polar coordinates, the coordinates $(u_1, u_2, u_3)$ are given by $(\rho, \phi, z)$. From earlier we have shown that the scale factors are given by $(h_1, h_2, h_3) = (1, \rho, 1)$. We can therefore calculate the Laplacian,

\begin{align*}
  \nabla^2\Psi&=\frac{1}{\rho}\left[\frac{\partial}{\partial\rho}\left(\rho\frac{\partial\Psi}{\partial\rho}\right) + \frac{\partial}{\partial\phi}\left(\frac{1}{\rho}\frac{\partial\Psi}{\partial\phi}\right) + \frac{\partial}{\partial z}\left(\rho\frac{\partial\Psi}{\partial z}\right)\right] \\
              &=\frac{1}{\rho}\frac{\partial}{\partial\rho}\left(\rho\frac{\partial\Psi}{\partial\rho}\right) + \frac{1}{\rho^2}\frac{\partial^2\Psi}{\partial\phi^2} + \frac{\partial^2\Psi}{\partial z^2} \\
              &=\frac{1}{\rho}\frac{\partial\Psi}{\partial\rho}+\frac{\partial^2\Psi}{\partial\rho^2}+\frac{1}{\rho^2}\frac{\partial^2\Psi}{\partial\phi^2} + \frac{\partial^2\Psi}{\partial z^2}
\end{align*}

In the last line we have expanded the $\partial/\partial\rho$ operator using the product rule, although we will not bother to expand in later examples as it is not needed.

\subsection*{Spherical Polar Coordinates}

In spherical polar coordinates, $(u_1, u_2, u_3) = (r, \theta, \phi)$, and the scale factors $(h_1, h_2, h_3) = (1, r, r\sin\theta)$. We will not write the working from now on, only the result, as the method is identical to the cylindrical polar case. The Laplacian is thus given by,

\begin{align*}
  \nabla^2\Psi=\frac{1}{r^2}\frac{\partial}{\partial r}\left(r^2\frac{\partial\Psi}{\partial r}\right)+\frac{1}{r^2\sin\theta}\frac{\partial}{\partial\theta}\left(\sin\theta\frac{\partial\Psi}{\partial\theta}\right)+\frac{1}{r^2\sin^2\theta}\frac{\partial^2\Psi}{\partial\phi^2}
\end{align*}

\subsection*{Paraboloidal Coordinates}

In paraboloidal coordinates, the coordinates are given by $(u_1, u_2, u_3) = (u, v, \phi)$, and the scale factors given by $(h_1, h_2, h_3) = (\sqrt{u^2 + v^2}, \sqrt{u^2 + v^2}, uv)$. We can see that in this case $h_1 = h_2$. The Laplacian is given by,

\begin{align*}
  \nabla^2\Psi=\frac{1}{(u^2+v^2)}\frac{1}{u}\frac{\partial}{\partial u}\left(u\frac{\partial\Psi}{\partial u}\right)+\frac{1}{(u^2+v^2)}\frac{1}{v}\frac{\partial}{\partial v}\left(v \frac{\partial\Psi}{\partial v}\right)+\frac{1}{(u^2 + v^2)}\frac{\partial^2\Psi}{\partial\phi^2}
\end{align*}

\section{Curl in Orthogonal Coordinate Systems: Integral Approach}

In this section we will find expression for the curl of a vector field in an orthogonal coordinate system. First we consider an infinitesimal rectangle in the $u_1u_2$ plane situated at $(u_1, u_2)$ such that this is the lower left corner or the rectangle. The rectangle has sides length $h_1du_1$ and $h_2du_2$. The corners are thus given such that the lower left is $(u_1, u_2)$, the lower right is $(u_1 + du_1, u_2)$, the upper left is $(u_1, u_2 + du_2)$, the upper right is $(u_1 + du_2, u_2 + du_2)$. We label the bottom side $1$, the right side $2$, the top side $3$, and the left side $4$.

The circulation of a vector field $\mathbf{A}$ around this loop is given by,

\begin{align*}
  \oint_C \mathbf{A}\cdot d\mathbf{r}=\mathbf{A}\cdot d\mathbf{r_1} + d\mathbf{r}=\mathbf{A}\cdot d\mathbf{r_2} + d\mathbf{r}=\mathbf{A}\cdot d\mathbf{r_3} + d\mathbf{r}=\mathbf{A}\cdot d\mathbf{r_4}
\end{align*}

The vectors, $d\mathbf{r_i}$ are given by,

\begin{alignat*}{2}
  d\mathbf{r_1}&=h_1du_1\mathbf{e_1}\qquad d\mathbf{r_2}&&=h_2du_2\mathbf{e_2} \\
  d\mathbf{r_1}&=-h_1du_1\mathbf{e_1}\qquad d\mathbf{r_4}&&=-h_2du_2\mathbf{e_2}
\end{alignat*}

Therefore the circulation over the entire loop is,

\begin{align*}
  \oint_C\mathbf{A}\cdot d\mathbf{r}&=h_1A_1du_1(u_1, u_2) + h_2A_2du_2(u_1+du_1, u_2)\\
                                    &- h_1A_1du_1(u_1, u_2+du_2) - h_2A_2du_2(u_1, u_2) \\
                                    &=\left[\frac{\partial}{\partial u_1}(h_2A_2)du_1\right]du_2 - \left[\frac{\partial}{\partial u_2}(h_1A_1)du_2\right]du_1 \\
                                    &=\left[\frac{\partial}{\partial u_1}(h_2A_2) - \frac{\partial}{\partial u_2}(h_1A_1)\right]du_1 du_2
\end{align*}

Again here we have used the definition of the derivative. Stokes' theorem tells us that the $u_3$ component of the curl is thus given by,

\begin{align*}
  {\left[\nabla\times\mathbf{A}\right]}_3 &= \frac{\oint_C\mathbf{A}\cdot d\mathbf{r}}{\text{Area}} = \frac{\oint_C\mathbf{A}\cdot d\mathbf{r}}{h_1h_2du_1du_2} \\
&=\left[\frac{\partial}{\partial u_1}(h_2A_2)du_1\right]du_2 - \left[\frac{\partial}{\partial u_2}(h_1A_1)du_2\right]du_1 \\
&=\left[\frac{\partial}{\partial u_1}(h_2A_2) - \frac{\partial}{\partial u_2}(h_1A_1)\right]du_1 du_3
\end{align*}

The curl of the vector field is therefore,

\begin{align*}
  \nabla\times\mathbf{A} &= \frac{1}{h_2h_3}\left[\frac{\partial}{\partial u_2}(h_3A_3)-\frac{\partial}{\partial u_3}(h_2A_2)\right]\mathbf{e_1} \\
&+ \frac{1}{h_3h_1}\left[\frac{\partial}{\partial u_3}(h_1A_1)-\frac{\partial}{\partial u_1}(h_3A_3)\right]\mathbf{e_2} \\
&+ \frac{1}{h_1h_2}\left[\frac{\partial}{\partial u_1}(h_2A_2)-\frac{\partial}{\partial u_2}(h_1A_1)\right]\mathbf{e_2}
\end{align*}

By rearranging we can write this as a determinant,

\begin{align*}
  \nabla\times\mathbf{A} = \frac{1}{h_1h_2h_3}
  \begin{vmatrix}
    h_1\mathbf{e_1} & h_2\mathbf{e_2} & h_3\mathbf{e_3} \\[6pt]
    \frac{\partial}{\partial u_1} & \frac{\partial}{\partial u_2} & \frac{\partial}{\partial u_3} \\[6pt]
    h_1A_1 & h_2A_2 & h_3A_3
  \end{vmatrix}
\end{align*}

\section{Divergence in Orthogonal Coordinate Systems: Differential Approach}

In earlier sections we explored how we can apply the divergence and curl in orthogonal coordinate systems by taking integrals over surfaces and lines and then applying Gauss' and Stokes' theorems to find the divergence and curl. In this section we will find the same result by using differentiation only, as well as some vector calculus identities. To see the vector calculus identities, as well as proofs for a few of them, see the separate vector calculus notes.

The trick to finding the divergence and curl in orthogonal coordinate systems is to find combinations of the scale factors $h_i$ and the unit vectors $\mathbf{e_i}$ that have either zero divergence or curl. First we do this by finding the gradient of $u_1$,

\begin{align*}
  \nabla u_1 &= \frac{\mathbf{e_1}}{h_1}\frac{\partial}{\partial u_1}\left(u_1\right) + \frac{\mathbf{e_2}}{h_2}\frac{\partial}{\partial u_2}\left(u_1\right) + \frac{\mathbf{e_3}}{h_3}\frac{\partial}{\partial u_3}\left(u_1\right) \\
             &= \frac{\mathbf{e_1}}{h_1}
\end{align*}

Since $\nabla\times\nabla\Psi = \mathbf{0}$ it follows that,

\begin{align*}
  \nabla\times\left(\frac{\mathbf{e_1}}{h_1}\right) = \mathbf{0}
\end{align*}

We can apply similar reasoning for $u_2$ and $u_3$ to give,

\begin{align*}
  \nabla\times\left(\frac{\mathbf{e_1}}{h_1}\right) = \nabla\times\left(\frac{\mathbf{e_2}}{h_2}\right) = \nabla\times\left(\frac{\mathbf{e_3}}{h_3}\right) = \mathbf{0}
\end{align*}

Using our vector calculus identity $\nabla\cdot(\mathbf{A}\times\mathbf{B}) = \mathbf{B}\cdot(\nabla\times\mathbf{A}) - \mathbf{A}\cdot(\nabla\times\mathbf{B})$ we obtain a combination that leads to zero divergence,

\begin{align*}
  \nabla\cdot\left(\frac{\mathbf{e_1}}{h_2h_3}\right) &= \nabla\cdot\left(\frac{\mathbf{e_2}}{h_2}\times\frac{\mathbf{e_3}}{h_3}\right) \\
                                                      &= \frac{\mathbf{e_3}}{h_3}\cdot\nabla\times\left(\frac{\mathbf{e_2}}{h_2}\right) - \frac{\mathbf{e_2}}{h_2}\cdot\nabla\times\left(\frac{\mathbf{e_3}}{h_3}\right) \\
                                                      &= 0
\end{align*}

Similar reasoning leads us to,

\begin{align*}
  \nabla\cdot\left(\frac{\mathbf{e_1}}{h_2h_3}\right)=\nabla\cdot\left(\frac{\mathbf{e_2}}{h_3h_1}\right)=\nabla\cdot\left(\frac{\mathbf{e_3}}{h_1h_2}\right)=0
\end{align*}

To calculate the divergence then, we take the divergence of a given vector field $\mathbf{A}$ in the orthogonal coordinate system,

\begin{align*}
  \nabla\cdot\mathbf{A}=\nabla\cdot(A_1\mathbf{e_1}+A_2\mathbf{e_2}+A_3\mathbf{e_3})
\end{align*}

We can write the $\mathbf{e_1}$ term as,

\begin{align*}
  \nabla\cdot(A_1\mathbf{e_1})=\nabla(h_2h_3A_1\frac{\mathbf{e_1}}{h_2h_3})
\end{align*}

Now we use identity five, which is that $\nabla\cdot(\Psi\mathbf{F})=\nabla\Psi\cdot\mathbf{F}+\Psi\nabla\cdot\mathbf{F}$. This gives us,

\begin{align*}
  \nabla\cdot\left(h_2h_3A_1\frac{\mathbf{e_1}}{h_2h_3}\right)&=\nabla(h_2h_3A_1)\cdot\frac{\mathbf{e_1}}{h_2h_3}+h_2h_3A_1\nabla\cdot\left(\frac{\mathbf{e_1}}{h_2h_3}\right) \\
                                                              &=\left[\frac{1}{h_1}\frac{\partial}{\partial u_1}(h_2h_3A_1)\mathbf{e_1}\right]\cdot\frac{\mathbf{e_1}}{h_2h_3} + 0 \\
                                                              &=\frac{1}{h_1h_2h_3}\frac{\partial}{\partial u_1}(h_2h_3A_1)
\end{align*}

The last term in the first line can be seen to be zero as we have shown earlier, as $\nabla\cdot\left(\frac{\mathbf{e_1}}{h_2h_3}\right) = 0$. We can get the other two terms of the divergence using cyclic interchange and we see that we get the same result as before,

\begin{align*}
  \nabla\cdot\mathbf{A}=\frac{1}{h_1h_2h_3}\left[\frac{\partial}{\partial u_1}(h_2h_3A_1)+\frac{\partial}{\partial u_2}(h_3h_1A_2)+\frac{\partial}{\partial u_3}(h_1h_2A_3)\right]
\end{align*}

\section{Curl in Orthogonal Coordinate Systems: Differential Approach}

Similarly we can apply the same reasoning to find the curl in an orthogonal coordinate system. First we state the curl of a vector field $\mathbf{A}$ in orthogonal coordinates,

\begin{align*}
  \nabla\times\mathbf{A}=\nabla\times(A_1\mathbf{e_1}+A_2\mathbf{e_2}+A_3\mathbf{e_3})
\end{align*}

We can see that the first term can be written as,

\begin{align*}
  \nabla\times(A_1\mathbf{e_1})=\nabla\times(h_1A_1\frac{\mathbf{e_1}}{h_1})
\end{align*}

Next we use identity six which is that $\nabla\times(\Psi\mathbf{F}) = \nabla\Psi\times\mathbf{F} + \Psi\nabla\times\mathbf{F}$. Using this leads us to,

\begin{align*}
  \nabla\times(A_1\mathbf{e_1}) &= \nabla\times\left((h_1A_1\frac{\mathbf{e_1}}{h_1}\right) = \nabla(h_1A_1)\times\frac{\mathbf{e_1}}{h_1}+h_1A_1\nabla\times\frac{\mathbf{e_1}}{h_1} \\
                                &= \left[\frac{1}{h_2}\frac{\partial}{\partial u_2}(h_1A_1)\mathbf{e_2}+\frac{1}{h_3}\frac{\partial}{\partial u_3}(h_1A_1)\mathbf{e_3}\right]\times\frac{\mathbf{e_1}}{h_1} + \mathbf{0} \\
                                &= \frac{1}{h_3h_1}\frac{\partial}{\partial u_3}(h_1A_1)\mathbf{e_2} - \frac{1}{h_1h_2}\frac{\partial}{\partial u_2}(h_1A_1)\mathbf{e_3} \\
\end{align*}

We see that the last term of the first line goes to zero as we have shown earlier in the last section. We can get the other terms by cyclic interchange,

\begin{align*}
  \nabla\times\mathbf{A}&=\frac{1}{h_2h_3}\left[\frac{\partial}{\partial u_2}(h_3A_3)-\frac{\partial}{\partial u_3}(h_2A_2)\right]\mathbf{e_1} \\
                        &+\frac{1}{h_3h_1}\left[\frac{\partial}{\partial u_3}(h_1A_1)-\frac{\partial}{\partial u_1}(h_3A_3)\right]\mathbf{e_2} \\
                        &+\frac{1}{h_1h_2}\left[\frac{\partial}{\partial u_1}(h_2A_2)-\frac{\partial}{\partial u_2}(h_1A_1)\right]\mathbf{e_3}
\end{align*}

This can also be written in determinant form,

\begin{align*}
  \nabla\times\mathbf{A}=\frac{1}{h_1h_2h_3}
  \begin{vmatrix}
    h1\mathbf{e_1} & h2\mathbf{e_2} & h3\mathbf{e_3} \\
    \frac{\partial}{\partial u_1} & \frac{\partial}{\partial u_2} & \frac{\partial}{\partial u_3} \\
    h_1A_1 & h_2A_2 & h_3A_3
  \end{vmatrix}
\end{align*}

\section{Converting Spherical Polars to Cartesian}

We can write the Cartesian coordinates $(x, y, z)$ in terms of the spherical polar coordinates $(r, \theta, \phi)$,

\begin{align*}
  x &= r\sin\theta\cos\phi \\
  y &= r\sin\theta\sin\phi \\
  z &= r\cos\theta
\end{align*}

From this we can write $(r, \theta, \phi)$ in terms of $(x, y, z)$,

\begin{align*}
  r&=\sqrt{x^2+y^2+z^2} \\
  \theta&=\arctan{\left(\frac{\sqrt{x^2 + y^2}}{z}\right)} \\
  \phi&=\arctan{\left(\frac{y}{x}\right)} \\
\end{align*}

But if we need to take the cosine or sin of either $\theta$ or $\phi$ in terms of $(x, y, z)$ we need to do a difficult calculation (the sine of an arctangent) so it is better if we define the cosine and sine of $\theta$ and $\phi$ without an arctangent,

\begin{alignat*}{2}
  \cos\theta&=\frac{r\cos\theta}{\cos\theta}&&=\frac{z}{r}=\frac{z}{\sqrt{x^2 + y^2 + z^2}}\\
  \sin\theta&=\frac{r\sin\theta}{\sin\theta}&&=\frac{\sqrt{x^2+y^2}}{\sqrt{x^2+y^2+z^2}}\\
  \cos\phi&=\frac{r\sin\theta\cos\phi}{r\sin\theta}&&=\frac{x}{\sqrt{x^2+y^2}}\\
  \sin\phi&=\frac{r\sin\theta\sin\phi}{r\sin\theta}&&=\frac{y}{\sqrt{x^2+y^2}}
\end{alignat*}

\end{document}