\documentclass[12pt]{article} \usepackage{amsmath,amssymb,amsthm}
\usepackage[margin=1in]{geometry} \title{A Naive Bayes Primer / Naive Bayes for
  Text Categorization} \date{November 14, 2016} \author{Nelson Liu\\
  nfliu@uw.edu \and Jonathan Lee\\ jlee27@uw.edu}
\newtheorem{theorem}{Theorem}[section] \newtheorem*{remark}{Remark}

\begin{document}
\maketitle
\section{Motivating Problem}
Consider the following (hypothetical) problem:
\newline

\textit{In our dataset, Hillary Clinton has tweeted 4190 times and Donald Trump
  has tweeted 4246 times. 30\% of Clinton tweets contain the word "climate",
  while only 15\% of Trump tweets contain the word "climate". What is the
  probability that a tweet is written by Trump or Clinton, given that it
  contains the word "climate"?}
\newline

% Let $S$ be the event that a given email is spam, and let $V$ be the event that the email contains the word "viagra". We would like to know $\mathbb{P}(S|V)$, the probability that an email is spam given that it contains "viagra". A straightforward application of Bayes' theorem tells us that

We like to assume that our dataset is fairly accurate sample of the population,
so we'll say that the numbers of tweets we have for each person is an accurate
representation of their tweeting frequency. Let $H$ be the event that a tweet is
written by Hillary Clinton, $T$ be the event that a tweet is written by Donald
Trump. Thus, we can estimate the probability that an arbitrary tweet is written
by Clinton or by Trump from our dataset.

$$\mathbb{P}(H) = \frac{\text{\# Hillary tweets}}{\text{\# total tweets}} =
\frac{4190}{4190+4246} \approx 0.497$$
$$\mathbb{P}(T) = \frac{\text{\# Trump tweets}}{\text{\# total tweets}} =
\frac{4246}{4190+4246} \approx 0.503$$

Let $C$ be the event that the tweet contains the word "climate". We want to
calculate $\mathbb{P}(H|C)$, the probability that a tweet is written by Clinton
given that we know that it contains the word "climate". We can calculate this
probability (called the posterior probability) by applying Bayes' Theorem
(presented in the next section, so don't worry if you don't know what this is).

\begin{equation}
\begin{split}
  \mathbb{P}(H|C) &= \frac{\mathbb{P}(C|H)\mathbb{P}(H)}{\mathbb{P}(C|H)\mathbb{P}(H) + \mathbb{P}(C|T)\mathbb{P}(T)}\\
  &= \frac{0.3 \times 0.497}{0.3 \times 0.497 + 0.15 \times 0.503}\\
  &\approx 0.66
\end{split}
\end{equation}

It's pretty simple to calculate this probability when we're only examining one
word. We would like to extend this sort of thinking to classify entire tweets as
either being written by Hillary Clinton or Donald Trump by examining every word
in the tweet. This is where naive bayes classifiers come in.

\section{Derivation of Bayes' Theorem}
This section seeks to give the reader a theoretical understanding of Bayes'
Theorem. It's difficult to understand how Bayes' theorem works without knowing
what conditional probability is, so we'll begin by giving a recap on that.

\subsection{Conditional Probability: A Recap}
Given two events $A$ and $B$, the probability of $A$ happening given that we
know that $B$ happens (denoted $\mathbb{P}(A|B)$) is defined by:

$$\mathbb{P}(A|B) = \frac{\mathbb{P}(AB)}{\mathbb{P}(B)}$$
$\mathbb{P}(A|B)$ is read ``the probability of $A$ given $B$''. This equation
makes intuitive sense; since you are trying to calculate the probability that
$A$ happens and you know that $B$ has happened, you put the probability that $B$
happens in the denominator (this is your ``sample space'') and the probability
that $A$ and $B$ happen in the numerator. Dividing these thus yields the
probability that $A$ occurs given that $B$ occurs.

\begin{remark}
Notation may shift occasionally; note that $\mathbb{P}(AB) = \mathbb{P}(A,B)$
\end{remark}

\subsection{Derivation}
Before we can derive Bayes' theorem, we need a preliminary result, the law of
total probability. We present it without proof for the sake of brevity, the the
proof can be found online quite easily.
\begin{theorem}[Law of Total Probability]
  Let $A_1, \dots, A_n$ represent events that span the entire sample space (that
  is, the possible outcome events are $A_1, \dots, A_n$). For any event B,
  $$\mathbb{P}(B) = \sum \limits_{i=1}^{k} \mathbb{P}(B|A_i)\mathbb{P}(A_i)$$
\end{theorem}

In our derivation of Bayes' Theorem, let's continue considering the events
$A_1, \dots, A_n$ where $\mathbb{P}(A_i) > 0$ for each $i$. With the formula for
conditional probability, we rearrange the terms a bit and see that:
$$\mathbb{P}(A_i B) = \mathbb{P}(B) \times \mathbb{P}(A_i|B)$$
This signifies that the probability of $A_i$ and $B$ occurring is the
probability of $B$ occurring multiplied by the probability of $A_i$ occurring
given that $B$ occurs.

We can similarly rearrange this the other way around:
$$\mathbb{P}(A_iB) = \mathbb{P}(A_i) \times \mathbb{P}(B|A_i)$$
This signifies that the probability of $A_i$ and $B$ occurring is the
probability of $A_i$ occurring multiplied by the probability of $B$ occurring
given that $A_i$ occurs, which is also a reasonable thing to say.

Because the two equations above equal $\mathbb{P}(A_iB)$, it follows that:
$$\mathbb{P}(B) \times \mathbb{P}(A_i|B) = \mathbb{P}(A_i) \times \mathbb{P}(B|A_i)$$

Now, by dividing $\mathbb{P}(B)$ on both sides, we get the ``simple
formulation'' of Bayes' Theorem:
$$\mathbb{P}(A_i|B) = \frac{\mathbb{P}(A_i) \times \mathbb{P}(B|A_i)}{\mathbb{P}(B)}$$
The simple formulation shows us that the probability of $A$ given $B$ is equal
to the probability of $A$ happening multiplied by the probability of $B$ given
$A$, all divided by the probability of $B$. In reality however, the probability
of $B$ is calculated by applying the aforementioned law of total probability.
Thus, we can apply the law of total probability to the ``simple formulation'' to
yield the formulation of Bayes' Theorem that is commonly found in literature
(simply ``Bayes' Theorem'' from here on out).
\begin{theorem}[Bayes' Theorem]
  Let $A_1, \dots, A_k$ represent the events that span the entire sample space
  such that $\mathbb{P}(A_i) > 0$ for each i. If $\mathbb{P}(B) > 0$ then, for
  each $i = 1, \dots, k$,
  $$\mathbb{P}(A_i|B) = \frac{\mathbb{P}(A_i) \times \mathbb{P}(B|A_i)}{\sum \limits_{i=1}^{k} \mathbb{P}(B|A_i)\mathbb{P}(A_i)}$$
\end{theorem}

\begin{remark}
We call $\mathbb{P}(A_i)$ the \textbf{prior probability of $A$} and
$\mathbb{P}(A_i|B)$ the \textbf{posterior probability of $A$}.
\end{remark}
\section{Theory behind Naive Bayes Classifiers}
Let's represent a tweet as the set of distinct words $(x_1, \dots, x_n)$
contained in the text. We are interested in computing

$$\mathbb{P}(H|x_1, \dots, x_n)$$
where $H$ is the event that the tweet is written by Hillary, and $T$ is the
event that the tweet is written by Trump. Note that the probability of $T$ is
equivalent to $H^c$, or the probability that the tweet is not written by Hillary
(since we're only deciding between two classes, a task called binary
classification). By Bayes' theorem, this is equivalent to
\begin{equation}
\begin{split}
  \mathbb{P}(H|x_1, \dots, x_n) &= \frac{\mathbb{P}(H) \times
                                  \mathbb{P}(x_1, \dots, x_n|H)}{\mathbb{P}(x_1, \dots, x_n)}\\
                                &= \frac{\mathbb{P}(H)\mathbb{P}(x_1, \dots, x_n|H)}{\mathbb{P}(x_1, \dots,
                                  x_n)}\\
                                &= \frac{\mathbb{P}(H)\mathbb{P}(x_1, \dots, x_n|H)}{\mathbb{P}(x_1,
                                  \dots, x_n|H)\mathbb{P}(H)+\mathbb{P}(x_1, \dots, x_n|T)\mathbb{P}(T)}
\end{split}
\end{equation}
Ignoring the denominator for a minute, by definition of conditional probability,
$$\mathbb{P}(H)\mathbb{P}(x_1, \dots, x_n|H) = \mathbb{P}(x_1, \dots, x_n, H)$$

To simplify this equation, we need the chain rule of probability. As a brief
recap, the chain rule is a rearrangement of the generalization of conditional
probability formula to $n$ terms, and not just $2$. We can get the ``product
rule'' by rearranging the formula for conditional probability:
$$\mathbb{P}(A,B) = \mathbb{P}(A|B) \mathbb{P}(B)$$

Extending this for three variables, we can see that:
$$\mathbb{P}(A,B,C) = \mathbb{P}(A|B,C) \mathbb{P}(B,C) = \mathbb{P}(A|B,C)
\mathbb{P}(B|C) \mathbb{P}(C)$$

Applying this to $n$ variables leads to the chain rule.
\begin{theorem}[The Chain Rule of Probability]
$$\mathbb{P}(A_1, A_2, \dots, A_n) = \mathbb{P}(A_1| A_2, \dots,
A_n)\mathbb{P}(A_2| A_3, \dots, A_n) \dots \mathbb{P}(A_{n-1}|A_n) \mathbb{P}(A_n)$$
\end{theorem}

Applying the chain rule to decompose the probability equation we had above,
\begin{equation}
\begin{split}
  \mathbb{P}(x_1, \dots, x_n, H) & = \mathbb{P}(x_1 | x_2, \dots, x_n, H) \mathbb{P}(x_2, \dots, x_n, H) \\
                                 & = \mathbb{P}(x_1 | x_2, \dots, x_n, H) \mathbb{P}(x_2 | x_3, \dots, x_n, H) \mathbb{P}(x_3, \dots, x_n, H) \\
                                 & = \dots \\
                                 & = \mathbb{P}(x_1 | x_2, \dots, x_n, H) \mathbb{P}(x_2 | x_3, \dots, x_n, H) \dots   \mathbb{P}(x_{n-1} | x_n, H) \mathbb{P}(x_n | H) \mathbb{P}(H) \\
\end{split}
\end{equation}
This is a correct formula, but it's not a particularly useful one for us! The
problem is that terms like $\mathbb{P}(x_1 | x_2, \dots, x_n, H)$ have so many
conditions, that it's generally not possible to accurately calculate their
probability. \newline

Think about what that term is really trying to calculate, though: what's the
probability that the word $x_1$ occurs in a Hillary tweet, given that
($x_2, \dots x_n$) also occur? Unless there is another Hillary tweet with all of
those words ($x_2, \dots x_n$) in it that you've already looked at (which is
unlikely unless you have absurd amounts of data), there's no way to know that
probability already. We need to simplify the problem! \newline

Naive Bayes "fixes" this by making the assumption that the words in an tweet
(and more generally, features in data) are \textbf{conditionally independent of
  each other, given that we know whether or not the tweet was written by
  Hillary}. In other words, it assumes that knowing that an tweet has the word
"climate" in it, doesn't tell us anything about whether it has the word "change"
in it, \textbf{if we already factor in whether the tweet was written by Hillary
  or not}. This is why it's called Naive Bayes: it naively assumes independence
where it might not actually (and likely doesn't!) exist. \newline

This assumption, of course, is \textbf{not true} in reality: words often appear
together, and English is not just a random walk through the dictionary. But it's
a useful simplification of the problem that can lead to practical results.
\newline

Let's rewrite those chain rule probabilities using the conditional independence
assumptions.
\begin{equation}
\begin{split}
  \mathbb{P}(x_1, \dots, x_n, H) & = \mathbb{P}(x_1 |  H) \mathbb{P}(x_2 | x_3, \dots, x_n, H) \dots   \mathbb{P}(x_{n-1} | x_n, H) \mathbb{P}(x_n | H) \mathbb{P}(H) \\
                                 & \approx \mathbb{P}(x_1 | H) \mathbb{P}(x_2 | H) \dots   \mathbb{P}(x_{n-1} | H) \mathbb{P}(x_n | H) \mathbb{P}(H) \\
                                 & = \mathbb{P}(H)\prod_{i=1}^n \mathbb{P}(x_i | H)
\end{split}
\end{equation}

By a similar argument,

$$\mathbb{P}(x_1, \dots, x_n, T) \approx \mathbb{P}(T)\prod_{i=1}^n \mathbb{P}(x_i | T)$$
Putting it all together,
$$\mathbb{P}(H|x_1, \dots, x_n) \approx \frac{\mathbb{P}(H)\prod\limits_{i=1}^n \mathbb{P}(x_i | H)}{\mathbb{P}(H)\prod\limits_{i=1}^n \mathbb{P}(x_i | H) + \mathbb{P}(T)\prod\limits_{i=1}^n \mathbb{P}(x_i | T)}$$

Thus, we can calculate the probability of a tweet being authored by Hillary if
we know the underlying probability of tweet being a Hillary-tweet, and the
probabilities of seeing a particular word in either a Hillary or Trump tweet!

\section{How Trump-ish is a word?}
We have a way to compute the probability of an tweet being from Hillary Clinton
or Donald Trump using relatively simple terms, but it's not yet clear how to
compute those conditional probabilities. This classifier works by taking a large
number of tweets that have already been hand-labelled as \textit{Hillary} or
\textit{Trump} and uses that data to compute word-Trump (or word-Hillary)
probabilities, by counting the frequency of each word. \newline

Imagine we're given a labelled training set of $2000$ Hillary tweets and $3000$
Trump tweets (so $\mathbb{P}(H) = 0.4$, and $\mathbb{P}(T) = 0.6$). We'd like to
calculate $\mathbb{P}("woman"|H)$ and $\mathbb{P}("woman"|T)$. The easiest thing
to do would be to count how many Hillary tweets have ``woman'' and divide that by
the total number of Hillary tweets (and do the same thing for Trump). So, if
there were 216 Hillary tweets with ``woman'' and 12 Trump tweets with ``woman'',
we'd say
$$\mathbb{P}("woman"|H) = \frac{216}{2000} \approx 11\%$$
$$\mathbb{P}("woman"|T) = \frac{12}{3000} \approx 0.4\%$$

\section{Important Practical / Engineering Considerations}
While the theory is all dandy and fine, there are some problems that we may run
into when actually implementing the classifier. We detail a few of them below,
and how they are solved.
\subsection{Smoothing}
Our theoretical framework and formulation of the solution has the right idea,
but there's a small problem that we have yet to address: what if there's a word
(say, "Pokemon") that we've only ever seen before in Trump tweets, and not
Hillary tweets? In that case, $\mathbb{P}("Pokemon"|H) = 0$, and the entire
Hillary probability will go to zero, because we're multiplying all of the word
probabilities together and we've never seen "Pokemon" in a Hillary tweet before.
Thus, any tweet that we get that has ``Pokemon'' in it would be classified as a
Trump tweet for sure! We would like to be robust to words we haven't seen
before, or at least words we've only seen in one setting. \newline

The solution is to never let any word probabilities be zero, by smoothing them
upwards. \textbf{Instead of starting each word count at 0, start it at 1}. This
way none of the counts will ever have a numerator of 0. This overestimates the
word probability, so we need to \textbf{add 2 to the denominator}. (We add 2
because we're implicitly keeping track of 2 things: the number of tweets that
contain that word, and the number that don't. The sum of those two things should
be in the denominator, and the 2 accounts for starting both the counters at 1.)
Essentially, you're hallucinating that you've seen the novel word at least once
in your train set. \newline

The smoothed word probabilities for the previous example are now
$$\mathbb{P}("woman"|S) = \frac{217}{2002} \approx 11\%$$
$$\mathbb{P}("woman"|H) = \frac{13}{3002} \approx 0.43\%$$
And our estimate for "Pokemon" in Hillary tweets would now be $\frac{1}{2002}$,
instead of $0$. \newline

This technique is called \textbf{Laplacian smoothing} and was used in the 18th
century to estimate the probability that the sun will rise tomorrow!

\subsection{The Naive Bayes Classification algorithm}
\begin{enumerate}
  \item Iterate over the labelled Hillary tweets, and for each word $w$ seen,
  count how many of the Hillary tweets contain $w$. Compute
  $\mathbb{P}(w | H) = \frac{|\text{\# Hillary tweets containing w}| +
    1}{|\text{\# Hillary Tweets}| + 2}$.
\item Compute $\mathbb{P}(w | T)$ the same way for Trump tweets.
\item Compute $\mathbb{P}(H) = \frac{|\text{\# Hillary tweets}|}{|\text{\#
    Hillary tweets}| + |\text{\# Trump tweets}|}$
\item $\mathbb{P}(T) = \frac{|\text{\# Trump tweets}|}{|\text{\# Hillary
    tweets}| + |\text{\# Trump tweets}|}$
\item Iterate over the unlabelled test tweets:
\begin{enumerate}
  \item Create a set $(x_1, \dots, x_n)$ of the distinct words in the tweet.
  Ignore the words that you haven't seen in the labelled training data.
\item Compute
$$\mathbb{P}(H|x_1, \dots, x_n) \approx \frac{\mathbb{P}(H)\prod\limits_{i=1}^n \mathbb{P}(x_i | H)}{\mathbb{P}(H)\prod\limits_{i=1}^n \mathbb{P}(x_i | H) + \mathbb{P}(T)\prod\limits_{i=1}^n \mathbb{P}(x_i | T)}$$
\item If $\mathbb{P}(H|x_1, \dots, x_n) > 0.5$, output "Hillary", else output
"Trump"
\end{enumerate}
\end{enumerate}
\subsection{Avoiding floating point underflow}
Multiplying a bunch of small probabilities together will probably result in
floating point underflow, where the numbers will become too small to represent
and will go to $0$. Instead of calculating
$$\mathbb{P}(H)\prod\limits_{i=1}^n \mathbb{P}(x_i | H)$$
(which may underflow), consider computing the logarithm of this,
$$\log\Big({\mathbb{P}(H)\prod\limits_{i=1}^n \mathbb{P}(x_i | H)}\Big)$$
which can be written equivalently as
$$\log(\mathbb{P}(H))+\sum\limits_{i=1}^n \log(\mathbb{P}(x_i | H))$$
Which will certainly not underflow. Then, realize that if
$$\log(\mathbb{P}(H))+\sum\limits_{i=1}^n \log(\mathbb{P}(x_i | H)) > \log(\mathbb{P}(T))+\sum\limits_{i=1}^n \log(\mathbb{P}(x_i | T))$$
then, because in general $\log(x) > \log(y)$ means $x > y$,
$$\mathbb{P}(H)\prod\limits_{i=1}^n \mathbb{P}(x_i | H) > \mathbb{P}(T)\prod\limits_{i=1}^n \mathbb{P}(x_i | T)$$
and therefore
$$\mathbb{P}(H|x_1, \dots, x_n) > 0.5$$
and thus the tweet should be classified as ``Hillary'' (or as ``Trump'', if the
sum of the logarithms for ``Trump'' was greater).
\end{document}
%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:


\subsection{Cointegration}

If we have multiple variables, we can explore the order of integration of linear combinations.

If two series have time trends, a linear combination of them could remove this.


\section{Lagrange Multipliers}\label{sec:LagrangeMultipliers}

Many applied max/min problems take the following form:
we want to find an extreme value of a function, like $V=xyz$, subject
to a constraint, like $\ds1=\sqrt{x^2+y^2+z^2}$. Often this can be
done, as we have, by explicitly combining the equations and then
finding critical points. There is another approach that is often
convenient, the method of \dfont{Lagrange multipliers}\index{Lagrange multipliers}.

It is somewhat easier to understand two variable problems, so we begin
with one as an example. Suppose the perimeter of a rectangle is to be
100 units. Find the rectangle with largest area. This is a fairly
straightforward problem from single variable calculus. We write down
the two equations: $A=xy$, $P=100=2x+2y$, solve the second of these
for $y$ (or $x$), substitute into the first, and end up with a
one-variable maximization problem. Let's now think of it differently:
the equation $A=xy$ defines a surface, and the equation $100=2x+2y$
defines a curve (a line, in this case) in the $x$-$y$ plane. If we
graph both of these in the three-dimensional coordinate system, we can
phrase the problem like this: what is the highest point on the surface
above the line? The solution we already understand effectively
produces the equation of the cross-section of the surface above the
line and then treats it as a single variable problem. Instead, imagine
that we draw the level curves (the contour lines) for the surface in
the $x$-$y$ plane, along with the line.

\begin{figure}[H]
%\texonly
\centerline{
\vbox{\beginpicture
\normalgraphs
%\ninepoint
\setcoordinatesystem units <3truecm,3truecm>
\setplotarea x from -1 to 1, y from 0 to 1
\put {\hbox{\epsfxsize6cm\epsfbox{images/lagrange.eps}}} at 0 0
\endpicture}}
%\endtexonly
\caption{Constraint line with contour plot of the surface $xy$.}
\label{fig:lagrange}
\end{figure}

Imagine that the line represents a hiking trail and the contour lines
are, as on a topographic map, the lines of constant altitude. How
could you estimate, based on the graph, the high (or low) points on
the path? As the path crosses contour lines, you know the path must be
increasing or decreasing in elevation. At some point you will see the
path just touch a contour line (tangent to it), and then begin to
cross contours in the opposite order---that point of tangency must be
a maximum or minimum point. If we can identify all such points, we can
then check them to see which gives the maximum and which the minimum
value. As usual, we also need to check boundary points; in this
problem, we know that $x$ and $y$ are positive, so we are interested
in just the portion of the line in the first quadrant, as shown. The
endpoints of the path, the two points on the axes, are not points of
tangency, but they are the two places that the function $xy$ is a
minimum in the first quadrant.

How can we actually make use of this? At the points of tangency that
we seek, the constraint curve (in this case the line) and the level
curve have the same slope---their tangent lines are parallel. This also
means that the constraint curve is perpendicular to the gradient
vector of the function; going a bit further, if we can express the
constraint curve itself as a level curve, then we seek the points at
which the two level curves have parallel gradients.
The curve $100=2x+2y$ can be thought of as a level curve of the
function $2x+2y$; Figure~\ref{fig:lagrange two} shows both sets of
level curves on a single graph. We are interested in those points
where two level curves are tangent---but there are many such points,
in fact an infinite number, as we've only shown a few of the level
curves. All along the line $y=x$ are points at which two level curves
are tangent. While this might seem to be a show-stopper, it is
not.

\begin{figure}[H]
%\texonly
\centerline{
\vbox{\beginpicture
\normalgraphs
%\ninepoint
\setcoordinatesystem units <3truecm,3truecm>
\setplotarea x from -1 to 1, y from 0 to 1
\put {\hbox{\epsfxsize6cm\epsfbox{images/lagrange2.eps}}} at 0 0
\endpicture}}
%\endtexonly
\caption{Contour plots for $2x+2y$ and $xy$.}
\label{fig:lagrange two}
\end{figure}

The gradient of $2x+2y$ is $\langle 2,2\rangle$, and the gradient of
$xy$ is $\langle y,x\rangle$. They are parallel when
$\langle 2,2\rangle=\lambda\langle y,x\rangle$, that is, when
$2=\lambda y$ and $2=\lambda x$. We have two equations in three
unknowns, which typically results in many solutions (as we
expected). A third equation will reduce the number of solutions; the
third equation is the original constraint, $100=2x+2y$. So we have the
following system to solve:
$$2=\lambda y \qquad 2=\lambda x\qquad 100=2x+2y.$$
In the first two equations, $\lambda$ can't be 0, so we may divide by
it to get $x=y=2/\lambda$. Substituting into the third equation we get
\begin{align*}
2{2\over \lambda}+2{2\over \lambda}&=100	\\
{8\over100}&=\lambda	\\
\end{align*}
so $x=y=25$. Note that we are not really interested in the value of
$\lambda$---it is a clever tool, the Lagrange multiplier, introduced
to solve the problem. In many cases, as here, it is easier to find
$\lambda$ than to find everything else without using $\lambda$.

The same method works for functions of three variables, except of
course everything is one dimension higher: the function to be
optimized is a function of three variables and the constraint represents
a surface---for example, the function may represent temperature, and
we may be interested in the maximum temperature on some surface, like
a sphere.
The points we seek are those at which the constraint
surface is tangent to a level surface of the function. Once again, we
consider the constraint surface to be a level surface of some
function, and we look for points at which the two gradients are
parallel, giving us three equations in four unknowns. The constraint
provides a fourth equation.

\begin{example}{Optimization with Constraints}{}
Maximize the function $xyz$ given the constraint $\ds
1=\sqrt{x^2+y^2+z^2}$.
\end{example}
\begin{solution}
%Recall Example~\ref{exa:box diagonal}: the diagonal of a box is 1, we seek to maximize the volume.
The constraint is $\ds
1=\sqrt{x^2+y^2+z^2}$, which is the same as
$1=x^2+y^2+z^2$. The function to maximize is $xyz$. The two gradient
vectors are $\langle 2x,2y,2z\rangle$ and $\langle yz,xz,xy\rangle$,
so the equations to be solved are
\begin{align*}
yz&=2x\lambda	\\
xz&=2y\lambda	\\
xy&=2z\lambda	\\
1&=x^2+y^2+z^2	\\
\end{align*}
If $\lambda=0$ then at least two of $x$, $y$, $z$ must be 0, giving a
volume of 0, which will not be the maximum. If we multiply the first
two equations by $x$ and $y$ respectively, we get
\begin{align*}
xyz&=2x^2\lambda	\\
xyz&=2y^2\lambda	\\
\end{align*}
so $2x^2\lambda=2y^2\lambda$ or $x^2=y^2$; in the same way we can show
$x^2=z^2$. Hence the fourth equation becomes
$1=x^2+x^2+x^2$ or $x=1/\sqrt3$, and so $x=y=z=1/\sqrt3$ gives the
maximum volume. This is of course the same answer we obtained
previously.
\end{solution}

Another possibility is that we have a function of three variables, and
we want to find a maximum or minimum value not on a surface but on a
curve; often the curve is the intersection of two surfaces, so that we
really have two constraint equations, say $g(x,y,z)=c_1$ and
$h(x,y,z)=c_2$. It turns out that at points on the intersection of the
surfaces where $f$ has a maximum or minimum value,
$$\nabla f=\lambda\nabla g+\mu \nabla h.$$
As before, this gives us three equations, one for each component of
the vectors, but now in five unknowns, $x$, $y$, $z$, $\lambda$, and
$\mu$. Since there are two constraint functions, we have a total of
five equations in five unknowns, and so can usually find the solutions
we need.

\begin{example}{Intersection of a Plane with a Cylinder}{}
The plane $x+y-z=1$ intersects the cylinder $x^2+y^2=1$ in an
ellipse. Find the points on the ellipse closest to and farthest from
the origin.
\end{example}
\begin{solution}
We want the extreme values of $f=\sqrt{x^2+y^2+z^2}$ subject to the
constraints  $g=x^2+y^2=1$ and $h=x+y-z=1$. To simplify the algebra,
we may use instead $f=x^2+y^2+z^2$, since this has a maximum or
minimum value at exactly the points at which $\sqrt{x^2+y^2+z^2}$ does.
The gradients are
$$\nabla f =\langle 2x,2y,2z\rangle\qquad
\nabla g = \langle 2x,2y,0\rangle\qquad
\nabla h = \langle 1,1,-1\rangle,$$
so the equations we need to solve are
\begin{align*}
2x&=\lambda 2x+\mu	\\
2y&=\lambda 2y+\mu	\\
2z&=0-\mu	\\
1&=x^2+y^2	\\
1&=x+y-z.	\\
\end{align*}
Subtracting the first two we get
$2y-2x=\lambda(2y-2x)$, so either $\lambda=1$ or $x=y$. If $\lambda=1$
then $\mu=0$, so $z=0$ and the last two equations are
$$1=x^2+y^2\qquad\hbox{and}\qquad 1=x+y.$$
Solving these gives $x=1$, $y=0$, or $x=0$, $y=1$, so the points of
interest are $(1,0,0)$ and $(0,1,0)$, which are both distance 1 from
the origin. If $x=y$, the fourth equation is $2x^2=1$, giving
$x=y=\pm1/\sqrt2$, and from the fifth equation we get
$z=-1\pm\sqrt2$. The distance from the origin to
$(1/\sqrt2,1/\sqrt2,-1+\sqrt2)$ is $\sqrt{4-2\sqrt2}\approx 1.08$ and
the distance from the origin to
$(-1/\sqrt2,-1/\sqrt2,-1-\sqrt2)$ is $\sqrt{4+2\sqrt2}\approx 2.6$.
Thus, the points $(1,0,0)$ and $(0,1,0)$ are closest to the origin and
$(-1/\sqrt2,-1/\sqrt2,-1-\sqrt2)$ is farthest from the origin.
%The Java
%\expandafter\url\expandafter{\liveurl lagrange_two_constraints.html}%
%applet \endurl shows the cylinder, the plane, the four points of
%interest, and the origin.
\end{solution}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Opensolutionfile{solutions}[ex]
\section*{Exercises for \ref{sec:LagrangeMultipliers}}

\begin{enumialphparenastyle}

\begin{ex}
A six-sided rectangular box is to hold $1/2$ cubic meter;
what shape should the box be to minimize surface area?
\begin{sol}
a cube, $\root 3 \of {1/2}\times\root 3 \of {1/2}\times\root 3 \of {1/2}$
\end{sol}
\end{ex}

\begin{ex}
The post office will accept packages whose combined length
and girth are at most 130 inches (girth is the maximum distance around
the package perpendicular to the length). What is the largest volume
that can be sent in a rectangular box?
\begin{sol}
$65/3\cdot 65/3\cdot 130/3=2\cdot 65^3/27$
\end{sol}
\end{ex}

\begin{ex}
The bottom of a rectangular box costs twice as much per unit
area as the sides and top. Find the shape for a given volume that will
minimize cost.
\begin{sol}
It has a square base, and is one and one half times as tall as wide.
If the volume is $V$ the dimensions are $\root 3 \of {2V/3}\times
\root 3 \of {2V/3}\times \root 3\of {9V/4}$.
\end{sol}
\end{ex}

\begin{ex}
Using Lagrange multipliers, find the shortest
distance from the point $(x_0,y_0,z_0)$ to the plane $ax+by+cz=d$.
\begin{sol}
$|ax_0+by_0+cz_0-d|/\sqrt{a^2+b^2+c^2}$
\end{sol}
\end{ex}

\begin{ex}
Find all points on the surface $xy-z^2+1=0$ that are closest
to the origin.
\begin{sol}
$(0,0,1)$, $(0,0,-1)$
\end{sol}
\end{ex}

\begin{ex}
The material for the bottom of an aquarium costs half as
much as the high strength glass for the four sides. Find the shape of
the cheapest aquarium that holds a given volume $V$.
\begin{sol}
$\root 3\of{4V}\times\root 3\of{4V}\times\root 3\of{V/16}$
\end{sol}
\end{ex}

\begin{ex}
The plane $x-y+z=2$ intersects the cylinder $x^2+y^2=4$ in an
ellipse. Find the points on the ellipse closest to and farthest from
the origin.
\begin{sol}
Farthest: $(-\sqrt2,\sqrt2,2+2\sqrt2)$; closest:
$(2,0,0)$, $(0,-2,0)$
\end{sol}
\end{ex}

\begin{ex}
Find three positive numbers whose sum is 48 and whose
product is as large as possible.
\begin{sol}
$x=y=z=16$
\end{sol}
\end{ex}

\begin{ex}
Find all points on the plane $x+y+z = 5$ in the first octant at
which $\ds f(x,y,z) = xy^2z^2$ has a maximum value.
\begin{sol}
$(1,2,2)$
\end{sol}
\end{ex}

\begin{ex}
Find the points on the surface $x^2 -yz = 5$ that are closest to the
origin.
\begin{sol}
$\ds (\sqrt{5},0,0)$, $\ds (-\sqrt{5},0,0)$
\end{sol}
\end{ex}

\begin{ex}
A manufacturer makes two models of an item, standard and deluxe.  It
costs \$40 to manufacture the standard model and \$60 for the deluxe.  A
market research firm estimates that if the standard model is priced at $x$
dollars and the deluxe at $y$ dollars, then the manufacturer will sell
$500(y-x)$ of the standard items and $45,000+500(x-2y)$ of the deluxe each
year.  How should the items be priced to maximize profit?
\begin{sol}
standard \$65, deluxe \$75
\end{sol}
\end{ex}

\begin{ex}
A length of sheet metal is to be made into a
water trough by bending up two sides as shown in
Figure~\ref{fig:trough two}.  Find $x$ and $\phi$ so that the
trapezoid--shaped cross section has maximum area, when
the width of the metal sheet is 27 inches (that is, $2x+y=27$).
\begin{sol}
$x=9$, $\phi=\pi/3$
\end{sol}

\begin{figure}[H]
%\texonly
\centerline{
\vbox{\beginpicture
\normalgraphs
%\ninepoint
\setcoordinatesystem units <1truecm,1truecm>
\setplotarea x from 0 to 5, y from -0.5 to 1
\plot 0 1 1 0 4 0 5 1 /
\put {$x$} [bl] <2pt,2pt> at 0.5 0.5
\put {$x$} [br] <-2pt,2pt> at 4.5 0.5
\put {$y$} [t] <0pt,-3pt> at 2.5 0
\put {$\phi$} at 4.7 0.35
\put {$\phi$} at 0.3 0.35
\circulararc 45 degrees from 4.5 0 center at 4 0
\circulararc -45 degrees from 0.5 0 center at 1 0
\setdashes
\putrule from 4 0 to 5.3 0
\putrule from -0.3 0 to 1 0
\endpicture}}
%\endtexonly
\caption{Cross-section of a trough.}
\label{fig:trough two}
\end{figure}
\end{ex}

\begin{ex}
Find the maximum and minimum values of $f(x,y,z)=6x+3y+2z$ subject
to the constraint $\ds g(x,y,z) = 4x^2+2y^2 + z^2 - 70 = 0$.
\begin{sol}
$35$, $-35$
\end{sol}
\end{ex}

\begin{ex}
Find the maximum and minimum values of $f(x,y)=e^{xy}$ subject
to the constraint $g(x,y) = x^3+y^3 - 16 = 0$.
\begin{sol}
maximum $e^4$, no minimum
\end{sol}
\end{ex}

\begin{ex}
Find the maximum and minimum values of $\ds f(x,y) = xy +
\sqrt{9-x^2-y^2}$ when $\ds x^2+y^2 \leq 9$.
\begin{sol}
$5$, $-9/2$
\end{sol}
\end{ex}

\begin{ex}
Find three real numbers whose sum is 9 and the sum of whose squares
is a small as possible.
\begin{sol}
$3$, $3$, $3$
\end{sol}
\end{ex}

\begin{ex}
Find the dimensions of the closed rectangular box with maximum volume
that can be inscribed in the unit sphere.
\begin{sol}
a cube of side length $\ds 2/\sqrt{3}$
\end{sol}
\end{ex}

\end{enumialphparenastyle}


\subsection{Categories}\label{subsec:categories}

\begin{definition}\label{def:category}\mcite[def. 1.1.1]{Leinster2016Basic}
  A \term{category} is a \hyperref[def:quiver]{quiver} \( \cat{C} \) equipped with a \hyperref[def:partial_function]{partial operation} \( \bincirc \) on the arrows of \( \cat{C} \) and another operation \( \id \) that selects a distinguished arrow for each vertex.

  In tradition regarding \hyperref[def:concrete_category]{forgetful functors}, we denote the underlying quiver of \( \cat{C} \) by \( U(\cat{C}) \).

  \begin{thmenum}[series=def:category]
    \thmitem{def:category/objects} We call the vertices of the quiver \term{objects} and denote the set of all objects by \( \obj(\cat{C}) \). We will often write \( A \in \cat{C} \) as a shorthand for \( A \in \obj(\cat{C}) \).

    \thmitem{def:category/morphisms} We call the arrows of the quiver \term{morphisms} or sometimes \term{maps}. If \( f \) is a morphism, we call its head its \term{domain} \( \dom(f) \) and its tail its \term{codomain} \( \co\dom(f) \). We denote a morphism from \( A \) to \( B \) by \( f: A \to B \) or \( A \reloset f \to B \).

    We call the set \( \cat{C}(A, B) \) of all morphisms from \( A \) to \( B \) a \term{morphism set} or \term{\( \hom \)-set}. We use the shorthand \( \cat{C}(A) \) for \( \cat{C}(A, A) \). Another established notation is \( \op{hom}(A, B) \) instead of \( \cat{C}(A, B) \).

    Both of these notations highlight that \( \cat{C}(A, B) \), when parameterized by \( A \) and \( B \), is a \hyperref[def:functor]{functor}, as discussed in \fullref{def:hom_functor}.

    \thmitem{def:category/composition} We require the \term{composition} \( \bincirc \) of the arrows \( f \) and \( g \) to be defined only if \( \co\dom(f) = \dom(g) \). In this case, we require \( g \bincirc f \) to be a morphism from \( \dom(f) \) to \( \co\dom(g) \).

    Note how the order of \( f \) and \( g \) may seem confusing: we write the composition of \( f: A \to B \) and \( g: B \to C \) as \( g \bincirc f: A \to C \). This is set up so that it matches \hyperref[def:multi_valued_function/composition]{function composition}. The order may seem different compared to multiplication in \hyperref[def:group]{groups}, for example, however \fullref{def:monoid_delooping} shows that this is actually a generalization of multiplication.

    This order of composition is used in \cite[7]{MacLane1994}, \cite[def. 1.1.1]{Leinster2016Basic} and \cite[def. I.3.1]{Aluffi2009}.

    \thmitem{def:category/identity} We denote the \term{identity morphism} of an object \( A \) by \( \id_A \).
  \end{thmenum}

  The definition of a category additionally requires the following conditions to hold:
  \begin{thmenum}[resume=def:category]
    \thmitem[def:category/C1]{C1} For any morphism \( f: A \to B \), the identities \( \id_A \) and \( \id_B \) must satisfy
    \begin{equation}\label{eq:def:category/C1}\tag{\logic{C1}}
      f \bincirc \id_A = \id_B \bincirc f = f.
    \end{equation}

    \thmitem[def:category/C2]{C2} Composition must be associative. That is, for each triple of morphism \( f: A \to B \), \( g: B \to C \) and \( h: C \to D \), the following must hold:
    \begin{equation}\label{eq:def:category/C2}\tag{\logic{C2}}
      (h \bincirc g) \bincirc f = h \bincirc (g \bincirc f).
    \end{equation}
  \end{thmenum}
\end{definition}

\begin{example}\label{ex:def:category}
  Examples of categories include:

  \begin{itemize}
    \item The category \( \cat{Set} \) of \hyperref[def:large_and_small_sets]{small} \hyperref[def:set]{sets} and \hyperref[def:function]{functions} defined in \fullref{def:category_of_small_sets}.

    \item The category \( \cat{Cat} \) of small categories defined in \fullref{def:category_of_small_categories}.

    \item All the \hyperref[def:category_of_small_first_order_models]{categories of small first-order models} listed in \fullref{ex:def:category_of_small_first_order_models}

    \item The category \( \cat{Top} \) of small \hyperref[def:topological_space]{topological spaces} and \hyperref[def:global_continuity]{continuous functions} defined in \fullref{def:category_of_small_topological_spaces}.

    \item For every topological space, the fundamental groupoid defined in \fullref{def:fundamental_groupoid}.

    \item The category \( \cat{Quiv} \) of small \hyperref[def:quiver]{quivers} defined in \fullref{def:category_of_small_quivers}.

    \item For every quiver, the free category defined in \fullref{def:quiver_free_category}.

    \item For every \hyperref[def:preordered_set]{preordered set}, the induced category defined in \fullref{thm:order_category_isomorphism}.
  \end{itemize}
\end{example}

\begin{definition}\label{def:category_size}
  As can be seen from \fullref{ex:def:category}, some of the categories we are working with, like \( \cat{Set} \), contain as objects all \hyperref[def:large_and_small_sets]{small sets}. As mentioned in \fullref{def:large_and_small_sets}, the concept of a small set is defined relative to the smallest Grothendieck universe that suits our needs.

  \Fullref{thm:russels_paradox} demonstrates that the set of all sets easily leads to a paradox, which is the reason we restrict our attention only to sets within some Grothendieck universe. This universe is implicit by default, however we will occasionally need to make it explicit.

  We will say that the category \( \cat{C} \) is \term{locally \( \mscrU \)-small} if the morphism set \( \cat{C}(A, B) \) is \( \mscrU \)-small for every pair of objects \( A \) and \( B \). If, in addition, the set \( \obj(\cat{C}) \) of objects is also \( \mscrU \)-small, we will say that the category \( \cat{C} \) is \term{\( \mscrU \)-small}. If a category is not \( \mscrU \)-small, we say that it is \term{\( \mscrU \)-large}.

  In particular, \term{finite} and \term{locally finite} categories are ones who are \( V_\omega \)-small and \( V_\omega \)-locally small for the universe of hereditary finite sets \hyperref[def:universe_of_hereditary_finite_sets]{\( V_\omega \)}. This notion of local finiteness is unrelated to local finiteness of graphs defined in \fullref{def:hypergraph/degree}.

  Universes are crucial to be able to do a lot of categorical constructions within set theory, most importantly \( \mscrU \)-large \hyperref[def:functor_category]{functor categories} but also \hyperref[def:product_category]{product categories} and, as discussed in \fullref{rem:functor_size}, even the \hyperref[def:functor]{functors} themselves.

  Note that, even if a category is \( \mscrU \)-small, the category itself as the tuple \( (Q, \bincirc, \id) \) from \fullref{def:category} may not be a \( \mscrU \)-small set.

  Also note that, in a locally small category, it is possible for the set of all morphisms to be \( \mscrU \)-large. This is impossible for small categories due to \ref{def:grothendieck_universe/union}.

  We sometimes skip the prefix \enquote{\( \mscrU \)-} if it is unimportant, and simply speak of \enquote{large categories} or \enquote{locally small categories}.
\end{definition}

\begin{definition}\label{def:category_of_small_sets}
  Suppose that we are given a \hyperref[def:grothendieck_universe]{Grothendieck universe} \( \mscrU \), which is safe to assume to be the smallest suitable one as explained in \fullref{def:large_and_small_sets}.

  We denote the \hyperref[def:category]{category} of \( \mscrU \)-small \hyperref[def:set]{sets} by \( \ucat{Set} \) or, if the universe is clear from the context, simply by \( \cat{Set} \). See \fullref{def:category_size} for a further discussion of universes and categories.

  \begin{itemize}
    \item The \hyperref[def:category/objects]{set of objects} \( \obj(\cat{Set}) \) is the set of all \( \mscrU \)-small sets, i.e. all members of \( \mscrU \).

    \item The \hyperref[def:category/morphisms]{set of morphisms} \( \cat{Set}(A, B) \) from \( A \) to \( B \) is the set \hyperref[def:function/set_of_functions]{\( \fun(A, B) \)} of all total single-valued functions from \( A \) to \( B \).

    \item The \hyperref[def:category/composition]{composition of morphisms} is the usual \hyperref[def:multi_valued_function/composition]{function composition}.

    \item The \hyperref[def:category/identity]{identity morphism} on the set \( A \) is the \hyperref[def:multi_valued_function/identity]{identity function}
    \begin{equation*}
      \begin{aligned}
        &\id_A: A \to A \\
        &\id_A(x) \coloneqq A.
      \end{aligned}
    \end{equation*}
  \end{itemize}
\end{definition}
\begin{defproof}
  To see that \( \ucat{Set} \) is indeed a category, we verify the conditions \ref{def:category/C1} and \ref{def:category/C2}.

  \SubProofOf{def:category/C1} For every two sets \( A, B \in \mscrU \) and every function \( f: A \to B \), for all \( x \in A \) we have
  \begin{equation*}
    [\id_B \bincirc f](x)
    =
    \id_B(f(x))
    =
    f(x)
    =
    f(\id_A(x))
    =
    [f \bincirc \id_A](x).
  \end{equation*}

  Therefore, \( \id_A \) and \( \id_B \) satisfy \eqref{eq:def:category/C1}.

  \SubProofOf{def:category/C2} Associativity of function composition is proved in \fullref{thm:def:multivalued_function/associative}.
\end{defproof}

\begin{proposition}\label{thm:category_of_small_sets_properites}
  We collect here important properties of the category \hyperref[def:category_of_small_sets]{\( \ucat{Set} \)} of \( \mscrU \)-small sets. Most of them require forward references.

  \begin{thmenum}
    \thmitem{thm:category_of_small_sets_properites/large} It is a \( \mscrU \)-large category in the sense of \fullref{def:category_size} because \( \mscrU \) itself is the set of objects and, defined as a \hyperref[def:quiver]{quiver} with additional operations, the category is a \( \mscrU \)-large set in the sense of \fullref{def:large_and_small_sets}.

    \thmitem{thm:category_of_small_sets_properites/locally_small} It is a \hyperref[def:category_size]{\( \mscrU \)-locally small category} because \( \mscrU \) is a model of \hyperref[def:zfc]{\( \logic{ZFC} \)} and \fullref{thm:zfc_existence_theorems/set_of_functions} holds.

    \thmitem{thm:category_of_small_sets_properites/morphism_invertibility} All \hyperref[def:morphism_invertibility/right_cancellative]{epimorphisms} and \hyperref[def:multi_valued_function/empty]{nonempty} \hyperref[def:morphism_invertibility/left_cancellative]{monomorphisms} \hyperref[def:morphism_invertibility/left_invertible]{split} and are precisely the \hyperref[def:function_invertibility/surjective]{surjective} and nonempty \hyperref[def:function_invertibility/injective]{injective functions}, respectively.

    This is stated in \fullref{thm:function_invertibility_categorical}. See also \fullref{thm:epimorphisms_split_in_set}.

    \thmitem{thm:category_of_small_sets_properites/universal_objects} The empty set \( \varnothing \) is an \hyperref[def:universal_objects/initial]{initial object} and the singleton set \( \set{ A } \) is a \hyperref[def:universal_objects/terminal]{terminal object} for every \( A \in \ucat{Set} \). No \hyperref[def:universal_objects/zero]{zero objects} exist in \( \ucat{Set} \) by \fullref{thm:def:universal_objects/no_zero}.

    This is discussed in \fullref{ex:def:universal_objects}.

    \thmitem{thm:category_of_small_sets_properites/discrete_category} The \hyperref[def:discrete_category]{discrete category} functor \( D: \ucat{Set} \to \ucat{Cat} \) is left adjoint to the forgetful functor \( U: \ucat{Cat} \to \ucat{Set} \)

    This is discussed in \fullref{ex:def:category_adjunction/set_cat}.

    \thmitem{thm:category_of_small_sets_properites/limits} The \hyperref[def:discrete_category_limits]{products} and \hyperref[def:discrete_category_limits]{coproducts} are the \hyperref[def:cartesian_product/product]{Cartesian products} and the \hyperref[def:disjoint_union]{disjoint unions}, respectively.

    This is stated in \fullref{thm:discrete_category_limits_in_set}.
  \end{thmenum}
\end{proposition}

\begin{definition}\label{def:opposite_category}\mcite[def. 1.1.9]{Leinster2016Basic}
  The \term{opposite} category of \( \cat{C} \) is obtained by \enquote{reversing} all arrows. This reversing is merely a relabeling of the domain and codomain --- the underlying morphisms are the same. This concept is quite powerful because it allows performing constructions and proofs by duality --- see \fullref{thm:categorical_principle_of_duality}.

  Formally, the category \( \cat{C}^{\opcat} \) is defined as follows:
  \begin{itemize}
    \item The \hyperref[def:category/objects]{set of objects} \( \obj(\cat{C}^{\opcat}) \) is the set of objects \( \obj(\cat{C}) \) of \( \cat{C}^{\opcat} \).

    \item The \hyperref[def:category/morphisms]{set of morphisms} \( \cat{C}(A, B) \) is the set \( \cat{C}(B, A) \). Thus, any morphism \( f^{\opcat}: A \to B \) in the opposite category \( \cat{C}^{\opcat} \) is a morphism \( f: B \to A \) in \( \cat{C}^{\opcat} \).

    The superscript here is used solely to distinguish between \( f \) being regarded as a morphism of \( \cat{C} \) and of \( \cat{C}^{\opcat} \) --- the morphisms in \( \cat{C} \) are exactly those of \( \cat{C}^{\opcat} \), simply relabeled.

    \item The \hyperref[def:category/composition]{composition of the morphisms}
    \begin{align*}
      f^{\opcat} &\in \cat{C}^{\opcat}(A, B) = \cat{C}(B, A) \\
      g^{\opcat} &\in \cat{C}^{\opcat}(B, C) = \cat{C}(C, B)
    \end{align*}
    is the morphism
    \begin{equation*}
      \underbrace{g^{\opcat} \bincirc f^{\opcat}}_{\cat{C}^{\opcat}(A, C)} \coloneqq \underbrace{f \bincirc g}_{\cat{C}(C, A)}.
    \end{equation*}

    \item The \hyperref[def:category/identity]{identity morphism} on the object \( A \in \cat{C} \) is again \( \id_A \).
  \end{itemize}
\end{definition}

\begin{remark}\label{rem:double_opposite_category}
  The double-opposite of a category or morphism is obviously the original. This is made precise with the oppositization functor defined in \fullref{def:opposite_functor}.
\end{remark}

\begin{example}\label{ex:def:opposite_category}
  A morphism \( f^{\opcat}: A \to B \) in the category \( \cat{Set}^{\opcat} \) is a function from the set \( B \) to the set \( A \). We cannot apply \( f \) to a point in \( B \) unless \( B \subseteq A \). Thus, we cannot regard, in general, the morphism \( f^{\opcat} \) as a function, although only the \hyperref[def:multi_valued_function]{signature} of \( f \) is different from that of \( f^{\opcat} \) --- their \hyperref[def:multi_valued_function/graph]{graphs} are the same.
\end{example}

\begin{proposition}\label{thm:categorical_principle_of_duality}
  We can extend the principle of duality for preordered sets discussed in \fullref{def:preordered_set/duality} to categories. Since we have defined categories in \hyperref[def:axiom_of_universes]{\logic{ZFC+U}} rather than as a first-order theory, we will state this principle informally:
  \begin{displayquote}
    If a statement holds for every category, its dual statement obtained by \enquote{reversing} all morphisms as in \fullref{def:opposite_category}, also holds for every category.
  \end{displayquote}

  See \fullref{thm:def:morphism_invertibility/split_epimorphism} for how this principle can be utilized easily.

  We list here results that heavily utilize this principle. Note that it is now always obvious what exactly needs to reversed in order for this principle to hold. For example, as discussed in \fullref{def:opposite_functor}, for opposite functors we have
  \begin{equation*}
    [F \bincirc G]^{\opcat} = F^{\opcat} \bincirc G^{\opcat},
  \end{equation*}
  which is somewhat unexpected.

  \begin{thmenum}
    \thmitem{thm:categorical_principle_of_duality/morphism_invertibility} \Fullref{thm:morphism_invertibility_duality}: A morphism \( f: A \to B \) in \( \cat{C} \) is a \hyperref[def:morphism_invertibility/left_invertible]{(split)} \hyperref[def:morphism_invertibility/left_cancellative]{monomorphism} if and only if \( f^{\opcat}: B \to A \) in the opposite category \( \cat{C}^{\opcat} \) is a \hyperref[def:morphism_invertibility/right_invertible]{(split)} \hyperref[def:morphism_invertibility/right_cancellative]{epimorphism}.

    In particular, \( f \) is an \hyperref[def:morphism_invertibility/isomorphism]{isomorphism} in \( \cat{C} \) if and only if \( f^{\opcat} \) is an isomorphism in \( \cat{C}^{\opcat} \).

    \thmitem{thm:categorical_principle_of_duality/universal_objects} \Fullref{thm:universal_object_duality}: An object is \hyperref[def:universal_objects/initial]{initial} if and only if it is a \hyperref[def:universal_objects/terminal]{terminal object} the \hyperref[def:opposite_category]{opposite category}.

    \thmitem{thm:categorical_principle_of_duality/functor_categories} \Fullref{thm:opposite_of_functor_category}: For the \hyperref[def:opposite_category]{opposite} of the \hyperref[def:functor_category]{functor category} \( [\cat{C}, \cat{D}] \) we have
    \begin{equation*}
      [\cat{C}, \cat{D}]^{\opcat} = [\cat{C}^{\opcat}, \cat{D}^{\opcat}].
    \end{equation*}

    \thmitem{thm:categorical_principle_of_duality/equivalences} \Fullref{thm:opposite_of_category_equivalence}: The \hyperref[def:opposite_category]{duals} of \hyperref[def:category_equivalence]{equivalent categories} are equivalent.

    \thmitem{thm:categorical_principle_of_duality/adjunctions} \Fullref{thm:category_adjunction_duality}: The functor \( F \) is \hyperref[def:category_adjunction]{left adjoint} to \( G \) if and only if the \hyperref[def:opposite_functor]{dual functor} \( F^{\opcat} \) is right adjoint to \( G^{\opcat} \).

    \thmitem{thm:categorical_principle_of_duality/limits} \Fullref{thm:categorical_limit_duality}: For every \hyperref[def:category_of_cones/cone]{cone} \( (A, \alpha) \) of the \hyperref[def:categorical_diagram]{diagram} \( D \) in \( \cat{C} \), \( (A, \alpha^{\opcat}) \) is a \hyperref[def:category_of_cones/cone]{cocone} of \( D^{\opcat} \) in \( \cat{C}^{\opcat} \).

    Even more, for every \hyperref[def:category_of_cones/limit]{limit} \( (L, \pi) \) of \( D \) in \( \cat{C} \), \( (L, \pi^{\opcat}) \) is a \hyperref[def:category_of_cones/colimit]{colimit} of \( D^{\opcat} \) in \( \cat{C}^{\opcat} \).
  \end{thmenum}
\end{proposition}

\begin{definition}\label{def:morphism_invertibility}
  In connection with \fullref{def:function_invertibility} and \fullref{def:first_order_homomorphism_invertibility}, we introduce the following terminology:
  \begin{thmenum}
    \thmitem{def:morphism_invertibility/left_cancellative} The morphism \( g: B \to C \) is \term{left-cancellative} if, for any pair of morphisms \( f_1, f_2: A \to B \), the equality \( g \bincirc f_1 = g \bincirc f_2 \) implies \( f_1 = f_2 \).

    Left-cancellative morphisms are also called \term{monic morphisms} or \term{monomorphisms}.

    \thmitem{def:morphism_invertibility/left_invertible} The morphism \( f: A \to B \) is \term{left-invertible} if there exists a morphism \( g: B \to A \) such that \( g \bincirc f = \id_A \). We call \( g \) a \term{left inverse} of \( f \).

    Using forward references to \fullref{def:categorical_diagram}, we can restate this condition by saying that the following diagram commutes:
    \begin{equation}\label{eq:def:morphism_invertibility/left_invertible}
      \begin{aligned}
        \includegraphics[page=1]{output/def__morphism_invertibility.pdf}
      \end{aligned}
    \end{equation}

    Left-invertible morphisms are sometimes called \term{split monomorphisms} because they \enquote{split} the identity \( \id_A \) into a composition of \( f \) and \( g \).

    \thmitem{def:morphism_invertibility/right_cancellative} \hyperref[thm:categorical_principle_of_duality]{Dually}, the morphism \( f: A \to B \) is \term{right-cancellative} if, for any pair of morphisms \( g_1, g_2: B \to C \), the equality \( g_1 \bincirc f = g_2 \bincirc f \) implies \( g_1 = g_2 \).

    Right-cancellative morphisms are also called \term{epic morphisms} or \term{epimorphisms}.

    \thmitem{def:morphism_invertibility/right_invertible} The morphism \( g: B \to A \) is \term{right-invertible} if there exists a morphism \( f: A \to B \) such that \( f \bincirc g = \id_B \). We call \( g \) a \term{right inverse} of \( f \).

    Using forward references to \fullref{def:categorical_diagram}, we can restate this condition by saying that the following diagram commutes:
    \begin{equation}\label{eq:def:morphism_invertibility/right_invertible}
      \begin{aligned}
        \includegraphics[page=2]{output/def__morphism_invertibility.pdf}
      \end{aligned}
    \end{equation}

    Right-invertible morphisms are sometimes called \term{split epimorphisms} because they \enquote{split} the identity \( \id_B \) into a composition of \( g \) and \( f \).

    \thmitem{def:morphism_invertibility/isomorphism} The morphism \( f: A \to B \) is \term{fully invertible} it is both left-invertible and right-invertible. By \fullref{thm:def:morphism_invertibility/left_and_right}, in this case, there exists a unique morphism \( f^{-1}: B \to A \) that is a \term{two-sided inverse}, i.e. it is both a left inverse and a right inverse.

    A fully invertible morphism is usually called an \term{isomorphism}. If there exists an isomorphism between \( A \) and \( B \), we say that they are \term{isomorphic} and write \( A \cong B \).

    \thmitem{def:morphism_invertibility/endomorphism} A morphism \( f: A \to A \) from an object to itself is called an \term{endomorphism}.

    \thmitem{def:morphism_invertibility/automorphism} A morphism that is both an endomorphism and an isomorphism is called an \term{automorphism}.
  \end{thmenum}
\end{definition}

\begin{example}\label{ex:def:morphism_invertibility}
  \Fullref{thm:function_invertibility_categorical} characterizes the cancellative and invertible morphisms defined in \fullref{def:morphism_invertibility} for \hyperref[def:category_of_small_sets]{\( \cat{Set} \)} in terms of \hyperref[def:function_invertibility/injective]{injectivity} and \hyperref[def:function_invertibility/injective]{surjectivity}.

  A very simple example of a monomorphism which does not split is the empty function with nonempty domain. These are discussed in \fullref{thm:function_invertibility_categorical/empty}.

  \Fullref{thm:surjective_functions_are_right_invertible} is important enough to have a categorical interpretation via \fullref{thm:epimorphisms_split_in_set}, where its relation to the \hyperref[def:zfc/choice]{axiom of choice} is also discussed.
\end{example}

\begin{proposition}\label{thm:morphism_invertibility_duality}
  A morphism \( f: A \to B \) in \( \cat{C} \) is a \hyperref[def:morphism_invertibility/left_invertible]{(split)} \hyperref[def:morphism_invertibility/left_cancellative]{monomorphism} if and only if \( f^{\opcat}: B \to A \) in the opposite category \( \cat{C}^{\opcat} \) is a \hyperref[def:morphism_invertibility/right_invertible]{(split)} \hyperref[def:morphism_invertibility/right_cancellative]{epimorphism}.

  In particular, \( f \) is an \hyperref[def:morphism_invertibility/isomorphism]{isomorphism} in \( \cat{C} \) if and only if \( f^{\opcat} \) is an isomorphism in \( \cat{C}^{\opcat} \).

  This is part of the duality principles listed in \fullref{thm:categorical_principle_of_duality}.
\end{proposition}
\begin{proof}
  Trivial.
\end{proof}

\begin{proposition}\label{thm:def:morphism_invertibility}
  Morphisms have the following basic properties regarding their \hyperref[def:morphism_invertibility]{invertibility} (compare to \fullref{thm:function_composition_invertibility}):

  \begin{thmenum}
    \thmitem{thm:def:morphism_invertibility/split_monomorphism} Any \hyperref[def:morphism_invertibility/left_invertible]{left-invertible morphism} is \hyperref[def:morphism_invertibility/left_cancellative]{left-cancellative}.

    In more categorical terms, every split monomorphism is a monomorphism.

    \thmitem{thm:def:morphism_invertibility/split_epimorphism} Any \hyperref[def:morphism_invertibility/right_invertible]{right-invertible morphism} is \hyperref[def:morphism_invertibility/right_cancellative]{right-cancellative}.

    In more categorical terms, every split epimorphism is an epimorphism.

    \thmitem{thm:def:morphism_invertibility/at_most_one_inverse}\mcite[exer. 1.1.13]{Leinster2016Basic} Any morphism has at most one two-sided inverse.

    \thmitem{thm:def:morphism_invertibility/left_and_right} If a morphism is both left-invertible and right-invertible, the two inverses are equal, and the morphism is fully invertible.

    \thmitem{thm:def:morphism_invertibility/inverse_interchanges} The morphism \( f: A \to B \) is a right inverse of \( g: B \to A \) if and only if \( g \) is a left inverse of \( f \).

    \thmitem{thm:def:morphism_invertibility/monomorphism_and_split_epimorphism} If a morphism left-cancellative and right-invertible, it is an isomorphism.

    \thmitem{thm:def:morphism_invertibility/split_monomorphism_and_epimorphism} If a morphism left-invertible and right-cancellative, it is an isomorphism.

    \thmitem{thm:def:morphism_invertibility/cancellative_composition} The composition of two monomorphisms (resp. epimorphisms) is again a monomorphism (resp. epimorphism).

    \thmitem{thm:def:morphism_invertibility/invertible_composition} The composition of two split monomorphisms (resp. epimorphisms) is again a split monomorphism (resp. epimorphism).
  \end{thmenum}
\end{proposition}
\begin{proof}
  \SubProofOf{thm:def:morphism_invertibility/split_monomorphism} Suppose that \( g: B \to C \) is left-invertible with inverse \( h: C \to B \). Suppose that \( f_1, f_2: A \to B \) are morphisms such that
  \begin{equation*}
    g \bincirc f_1 = g \bincirc f_2.
  \end{equation*}

  Then
  \begin{equation*}
    f_1
    \reloset {\eqref{eq:def:category/C1}} =
    \id_B \bincirc f_1
    =
    (h \bincirc g) \bincirc f_1
    \reloset {\eqref{eq:def:category/C2}} =
    h \bincirc (g \bincirc f_1)
    =
    h \bincirc (g \bincirc f_2)
    =
    \cdots
    =
    f_2.
  \end{equation*}

  \SubProofOf{thm:def:morphism_invertibility/split_epimorphism} This is an exemplar proof using duality. By \fullref{thm:morphism_invertibility_duality}, every split epimorphism \( f: A \to B \) in \( \cat{C} \) is a split monomorphism in \( \cat{C}^{\opcat} \). By \fullref{thm:def:morphism_invertibility/split_monomorphism}, \( f^{\opcat} \) is a monomorphism. Then gain by \fullref{thm:morphism_invertibility_duality}, \( f \) is an epimorphism.

  \SubProofOf{thm:def:morphism_invertibility/at_most_one_inverse} If \( f: A \to B \) has no inverse, it vacuously has at most one inverse.

  Now assume that \( f: A \to B \) has two inverses \( g_1: B \to A \) and \( g_2: B \to A \):
  \begin{align*}
    g_1 \bincirc f = \id_A &&& f \bincirc g_1 = \id_B, \\
    g_2 \bincirc f = \id_A &&& f \bincirc g_2 = \id_B.
  \end{align*}

  Then
  \begin{equation*}
    g_1
    \reloset {\eqref{eq:def:category/C1}} =
    g_1 \bincirc \id_B
    =
    g_1 \bincirc (f \bincirc g_2)
    \reloset {\eqref{eq:def:category/C2}} =
    (g_1 \bincirc f) \bincirc g_2
    =
    \id_A \bincirc g_2
    \reloset {\eqref{eq:def:category/C1}} =
    g_2.
  \end{equation*}

  \SubProofOf{thm:def:morphism_invertibility/left_and_right} Suppose that \( f: A \to B \) has a left-inverse \( l: B \to A \) and a right-inverse \( r: B \to A \). Then

  \SubProofOf{thm:def:morphism_invertibility/inverse_interchanges} Trivial.

  \SubProofOf{thm:def:morphism_invertibility/monomorphism_and_split_epimorphism} Let \( g: B \to A \) be left-cancellative and right-invertible. Let \( f: A \to B \) be a right inverse of \( g \). Then
  \begin{equation*}
    f
    =
    \reloset {\eqref{eq:def:category/C1}} =
    f \bincirc \id_A
    =
    f \bincirc (g \bincirc f)
    =
    (f \bincirc g) \bincirc f.
  \end{equation*}

  Because \( g \) is a left inverse of \( f \), from \fullref{thm:def:morphism_invertibility/split_monomorphism} it follows that \( f \) is left-cancellative. Since we have
  \begin{equation*}
    \id_B \bincirc f
    =
    (f \bincirc g) \bincirc f,
  \end{equation*}
  it follows that \( f \bincirc g = \id_B \).

  Therefore, \( f \) is a left inverse of \( g \) and hence an isomorphism.

  \SubProofOf{thm:def:morphism_invertibility/split_monomorphism_and_epimorphism} The proof is analogous to \fullref{thm:def:morphism_invertibility/monomorphism_and_split_epimorphism}.

  \SubProofOf{thm:def:morphism_invertibility/cancellative_composition} Let \( g: B \to C \) and \( h: C \to D \) be monomorphisms (left-cancellative).

  Let \( f_1, f_2: A \to B \) be two arbitrary morphisms with codomain \( B \). Suppose that
  \begin{equation*}
    (h \bincirc g) \bincirc f_1 = (h \bincirc g) \bincirc f_2.
  \end{equation*}

  Then, by \ref{def:category/C2},
  \begin{equation*}
    h \bincirc (g \bincirc f_1) = h \bincirc (g \bincirc f_2).
  \end{equation*}

  Since \( h \) is left-cancellative, it follows that
  \begin{equation*}
    g \bincirc f_1 = g \bincirc f_2.
  \end{equation*}

  Since \( g \) is also left-cancellative, \( f_1 = f_2 \).

  Therefore, \( h \bincirc g \) is a monomorphism.

  The proof for composition of epimorphisms is identical.

  \SubProofOf{thm:def:morphism_invertibility/invertible_composition} Let \( f: A \to B \) and \( g: B \to C \) be split monomorphisms (left-invertible).

  Then there exist left inverses \( l_f: B \to A \) and \( l_g: C \to B \) of \( f \) and \( g \), respectively. We have
  \begin{equation*}
    (l_f \bincirc l_g) \bincirc (g \bincirc f)
    \reloset {\eqref{eq:def:category/C2}} =
    l_f \bincirc (l_g \bincirc g) \bincirc f
    =
    l_f \bincirc \id_B \bincirc f
    \reloset {\eqref{eq:def:category/C1}} =
    l_f \bincirc f
    =
    \id_A.
  \end{equation*}

  Therefore, \( g \bincirc f \) is also left-invertible.

  The proof for composition of split epimorphisms is identical.
\end{proof}

\begin{theorem}[Epimorphisms split in Set]\label{thm:epimorphisms_split_in_set}
  Every \hyperref[def:morphism_invertibility/right_cancellative]{epimorphism} in \hyperref[def:category_of_small_sets]{\( \cat{Set} \)} splits. That is, all epimorphisms in \( \cat{Set} \) are \hyperref[def:morphism_invertibility/right_invertible]{split epimorphisms}.

  Assuming the existence of the \hyperref[def:grothendieck_universe]{Grothendieck universe} containing \( \cat{Set} \), in \hyperref[def:zfc]{\logic{ZF}} this theorem is equivalent to the \hyperref[def:zfc/choice]{axiom of choice} --- see \fullref{thm:axiom_of_choice_equivalences/epimorphisms}.

  Since not every epimorphism splits in a general category, this theorem is sometimes considered to be a categorical statement of the axiom of choice, which holds in some categories but not in others.
\end{theorem}
\begin{proof}
  By \fullref{thm:function_invertibility_categorical/right_cancellative}, a function is an epimorphism if and only if it is surjective. Thus, the theorem is equivalent to \fullref{thm:surjective_functions_are_right_invertible}.
\end{proof}

\begin{definition}\label{def:universal_objects}\mcite[def. 2.1.7]{Leinster2016Basic}
  Fix a category \( \cat{C} \).

  \begin{thmenum}
    \thmitem{def:universal_objects/initial} We call the object \( I \in \cat{C} \) an \term{initial object} if for any other object \( A \in \cat{C} \) there exists a unique morphism \( f: I \to A \),

    \thmitem{def:universal_objects/terminal} \hyperref[thm:categorical_principle_of_duality]{Dually}, we call the object \( T \in \cat{C} \) a \term{terminal object} or \term{final object} if for any other object \( A \in \cat{C} \) there exists a unique morphism \( f: A \to T \).

    The initial and terminal objects are collectively called \term{universal objects}.

    \thmitem{def:universal_objects/zero}\mcite{nLab:pointed_category} If \( Z \) is both an initial and a terminal object, we say that \( Z \) is a \term{zero object}. A category with a zero object is called a \term{pointed category}.
  \end{thmenum}
\end{definition}

\begin{example}\label{ex:def:universal_objects}
  \begin{thmenum}
    \thmitem{ex:def:universal_objects/set} In the category \hyperref[def:category_of_small_sets]{\( \cat{Set} \)} of small sets, for any set \( A \) there is a unique \hyperref[def:multi_valued_function/empty]{empty function} from \( \varnothing \) to \( A \). Therefore, \( \varnothing \) is an \hyperref[def:universal_objects/initial]{initial object} in \( \cat{Set} \).

    For any set \( A \), there is a unique function that contracts any set \( B \) to \( \set{ A } \). Therefore, every singleton set is a \hyperref[def:universal_objects/terminal]{final object} in \( \cat{Set} \).

    We often denote the initial and terminal objects in \( \cat{Set} \) by \( 0 \) and \( 1 \) respectively, which corresponds to their definition as \hyperref[def:ordinal]{ordinals}.

    By \fullref{thm:def:universal_objects/no_zero}, \( \cat{Set} \) has no zero object.

    \thmitem{ex:def:universal_objects/grp} In the category \hyperref[def:group/category]{\( \cat{Grp} \)} of small groups, the \hyperref[def:group/trivial]{trivial group} is a \hyperref[def:universal_objects/zero]{zero object}. This holds more generally for pointed sets rather than groups.

    Indeed, it can be embedded into any other group and any group can be contracted into the corresponding trivial group. Furthermore, all trivial groups are isomorphic.
  \end{thmenum}
\end{example}

\begin{proposition}\label{thm:universal_object_duality}
  An object is \hyperref[def:universal_objects/initial]{initial} if and only if it is a \hyperref[def:universal_objects/terminal]{terminal object} the opposite category.

  This is part of the duality principles listed in \fullref{thm:categorical_principle_of_duality}.
\end{proposition}
\begin{proof}
  Trivial.
\end{proof}

\begin{proposition}\label{thm:def:universal_objects}
  \hfill
  \begin{thmenum}
    \thmitem{thm:def:universal_objects/initial} An \hyperref[def:universal_objects/initial]{initial object} is unique up to an isomorphism.
    \thmitem{thm:def:universal_objects/terminal} \hyperref[thm:categorical_principle_of_duality]{Dually}, a \hyperref[def:universal_objects/initial]{terminal object} is also unique up to an isomorphism.
    \thmitem{thm:def:universal_objects/zero} If a category has an initial and a terminal object and if they are isomorphic, then both are zero objects.

    In particular, a zero object is unique up to an isomorphism.

    \thmitem{thm:def:universal_objects/no_zero} If an initial and a terminal object exists and are not isomorphic, then there exist no zero objects.
  \end{thmenum}
\end{proposition}
\begin{proof}
  \SubProofOf{thm:def:universal_objects/initial} Suppose that \( A \) and \( B \) are both initial objects in \( \cat{C} \). Then there exist morphisms \( f: A \to B \) and \( g: B \to A \). Their composition \( g \bincirc f \) is an \hyperref[def:morphism_invertibility/endomorphism]{endomorphism} on \( A \).

  But there exists a unique \hyperref[def:morphism_invertibility/endomorphism]{endomorphism} on \( A \), which must be the identity \( \id_A \). Thus, \( g \bincirc f = \id_A \) and \( g \) is a left inverse of \( f \).

  We can analogously show that \( g \) is a right inverse of \( f \). Therefore, \( f \) is fully invertible, and \( A \) and \( B \) are isomorphic.

  \SubProofOf{thm:def:universal_objects/terminal} If \( T' \) and \( T^\dprime \) are terminal objects in \( \cat{C} \), by \fullref{thm:universal_object_duality}, they are initial objects in \( \cat{C}^{\opcat} \). By \fullref{thm:def:universal_objects/initial}, they are isomorphic in \( \cat{C}^{\opcat} \) and by \fullref{thm:morphism_invertibility_duality}, they are isomorphic in \( \cat{C} \).

  \SubProofOf{thm:def:universal_objects/zero} Suppose that \( A \) is an initial object and that \( B \) is a final object in \( \cat{C} \). Let \( f: A \to B \) be an isomorphism between them.

  Let \( C \in \cat{C} \) be any other object and let \( g: C \to B \) be the unique morphism to \( B \). Then \( f^{-1} \bincirc g: C \to A \) is a morphism from \( C \) to \( A \). The inverse \( f^{-1}: B \to A \) is unique by \fullref{thm:def:morphism_invertibility/at_most_one_inverse}, therefore its composition with \( g: C \to B \) is also unique. Hence, any object has a unique morphism to \( A \). This makes \( A \) a terminal object and thus a zero object.

  We can analogously show that \( B \) is a zero object.

  \SubProofOf{thm:def:universal_objects/no_zero} By \fullref{thm:def:universal_objects/zero}, all zero objects are isomorphic. By \fullref{thm:def:universal_objects/initial}, all initial objects are isomorphic and analogously for terminal objects. Hence, if a zero object exists, all initial objects are isomorphic to all terminal objects.

  If some initial object is not isomorphic to some terminal object, then by contraposition it follows that no zero object exists.
\end{proof}


\section{Orbits}
The study of orbits it is motivated by the motion of heavenly bodies under the infludence of the gravitational force due to e.g. a star.
Of course, we want to study a conservative field $-\nabla V$ where $V$ is a potential of a central force (so it only depends on $r=|\underline{r}|$), that is
$$m\underline{\ddot{r}}=-\nabla V(r)$$
We shall study the case when the central body is much more massive than the orbiting body, so that the central body can be regarded as fixed.\\
Recall that $\underline{L}=m\underline{r}\times\underline{\dot{r}}$, and in the case for a central force $\underline{\dot{L}}=0$, so $\underline{L}$ is constant.
Also we always have $\underline{L}\cdot\underline{r}=0$, so we can regard the motion as if it is in a plane.
\subsection{Polar Coordinates in a Plane}
In an orbit problem, we of course want to use polar coordinate to simplify calculation.
Since we can regard the problem as two-dimensional, we can use the plane polar coordinates $x=r\cos\theta,y=r\sin\theta$, so we define the unit vectors
$$\underline{e_r}=(\cos\theta,\sin\theta)^\top,\underline{e_\theta}=(-\sin\theta,\cos\theta)^\top$$
So we can use $\underline{e_r},\underline{e_\theta}$ as two basis vectors, but note that they are dependent of the position.
Note that $\underline{e_r}$ is always in the direction of the position and $\underline{e_\theta}$ the direction of rotation.
Also note that $\mathrm d\underline{e_r}/\mathrm d\theta=\underline{e_\theta},\mathrm d\underline{e_\theta}/\mathrm d\theta=-\underline{e_r}$, so
$$\frac{\mathrm d\underline{e_r}}{\mathrm dt}=\underline{e_\theta}\dot{\theta},\frac{\mathrm d\underline{e_\theta}}{\mathrm dt}=-\underline{e_r}\dot{\theta}$$
Now we turn to consider the implications for the velocity of the particle given some acceleration.
Write $\underline{r}=r\underline{e_r}$.
Consider this as a function of time, then $\underline{v}=\underline{\dot{r}}=\dot{r}\underline{e_r}+r\underline{e_\theta}\dot{\theta}$.
$\dot{r}$ is the radial component of the velocity while $\dot{\theta}$ is the angular component of it.
So $\dot{\theta}$ has dimension $T^{-1}$.\\
As for accelerations, we have
$$\underline{\ddot{r}}=\underline{\dot{v}}=\ddot{r}\underline{e_r}+\dot{r}\underline{e_\theta}\dot{\theta}+(\dot{r}\dot{\theta}+r\ddot{\theta})\underline{e_\theta}-r\dot{\theta}\underline{e_r}\dot{\theta}=(\ddot{r}-r\dot{\theta}^2)\underline{e_r}+(2\dot{r}\dot{\theta}+r\ddot{\theta})\underline{e_\theta}$$
\begin{example}
    Consider the circular motion with a constant angular velocity, then $r=a,\dot{\theta}=\omega$, so $\dot{r}=\ddot{\theta}=0$, hence
    $$\underline{\ddot{r}}=(\ddot{r}-r\dot{\theta}^2)\underline{e_r}+(2\dot{r}\dot{\theta}+r\ddot{\theta})\underline{e_\theta}=-a\omega^2\underline{e_r}=-\omega^2\underline{r}$$
    which is the familiar centripetal acceleration.
    Newton's Second Law requires a force to be applied to cause this acceleration, which is called the centripetal force, which is in the direction of $-\underline{e_r}$.
    For the special case that it is actually a mass on a string, then when the string broke, the mass will move in a straight line that is tangential to the point where the string broke.
\end{example}
\subsection{Motion in a Constant Force Field}
We know
$$m\underline{\ddot{r}}=\underline{F}=-\nabla V(r)=-\frac{\mathrm dV}{\mathrm dr}\underline{e_r}$$
for a force field that is symmetric wrt the origin.
Note that
$$-\frac{\mathrm dV}{\mathrm dr}\underline{e_r}=\underline{F}=m(\ddot{r}-r\dot{\theta}^2)\underline{e_r}+m(2\dot{r}\dot{\theta}+r\ddot{\theta})\underline{e_\theta}$$
By looking at the $\underline{e_\theta}$ component, we have $2\dot{r}\dot{\theta}+r\ddot{\theta}=0$, so
$$\frac{1}{r}\frac{\mathrm d}{\mathrm dt}(mr^2\dot{\theta})=0$$
Hence the quantity $mr^2\dot{\theta}$ is constant, but $\underline{L}=\underline{r}\times (m\underline{\dot{r}})=mr^2\dot{\theta}\underline{e_z}$.
Where $\underline{e_z}$ is the normal to the plane of motion.
Hence the angular momentum is constant in magnitude.
We write $h=|\underline{L}|/m=r^2\dot{\theta}$.\\
Going to the radial part $\mathrm dV/\mathrm dr=-m(\ddot{r}-h^2/r^3)$, rearranging gives
$$m\ddot{r}=-\frac{\mathrm dV}{\mathrm dr}+\frac{mh^2}{r^3}=-\frac{\mathrm dV_{\rm eff}}{\mathrm dr},V_{\rm eff}=V+\frac{mh^2}{2r^2}$$
So the motion of the particle is as if we are considering one dimensional motion under the influence of a modified potential $V_{\rm eff}$.\\
The energy of the particle is then
$$E=T+V=\frac{1}{2}m|\underline{\dot{r}}|^2+V(r)=\frac{1}{2}m\dot{r}^2+V_{\rm eff}(r)$$
\begin{example}
    For gravity, we have
    $$V(r)=-\frac{GMm}{r},V_{\rm eff}(r)=-\frac{GMm}{r}+\frac{mh^2}{2r^2}$$
    So $V_{\rm eff}$ is minimum at $r=h^2/GM$ and minimum energy is
    $$E_{\rm min}=-m(GM)^2/(2h^2)$$
    At the minimum (which is a stable equilibrium), both $r,\dot\theta$ are constants.\\
    At any $E_{\rm min}<E<0$, the particle oscillates.
    Let $r_0$ be the point where $V_{\rm eff}=0$, then $r_0<r_{\rm min}\le r\le r_{\rm max}$.
    So it gives a bounded non-circular orbit with $\dot\theta$ varying when $r$ varies.
    $r_{\rm min}$ is called the periapsis and $r_{\rm max}$ is called apoapsis.\\
    For $E>0$, the particle can move from a long distance and escape, which is an unbounded orbit.
\end{example}
\subsection{Stability of Circular Orbits}
Consider the potential $V(r)$, we want to investigate whether a circular orbit exists and whether it is stable.\\
Assume that the angular momentum $h$ is given and is nonzero.
For circular orbit $r(t)=r_\star$ is a constant, then
$$\ddot{r}=0\implies V_{\rm eff}^\prime(r_\star)=0$$
which is the condition for circular orbit.
Note that if $V_{\rm eff}^{\prime\prime}(r_\star)>0$, then it is a minimum, so $r_\star$ is a stable fixed point.
So if we express it as $V(r)$, we get
$$0=V_{\rm eff}^\prime(r_\star)=V^\prime(r_\star)-\frac{mh^2}{r_\star^3}=0\implies V^\prime(r_\star)=\frac{mh^2}{r_\star^3}$$
And it is stable if
$$0<V_{\rm eff}^\prime(r_\star)=V^{\prime\prime}(r_\star)+\frac{3mh^2}{r_\star^4}=V^{\prime\prime}(r_\star)+\frac{3V^\prime(r_\star)}{r_\star}$$
So in terms of $F(r)$, we have
$$F^{\prime}(r_\star)+\frac{3F(r_\star)}{r_\star}<0$$
\begin{example}
    If we take $V(r)=-km/rp$ for $k,p>0$ for a circular orbit with radius $r_\star$, we can solve the above equation to get $r_\star=(pk/h^2)^{1/(p-2)}$.
    So unless $p=2$, there exists a circular orbit.\\
    As for stability, we have
    $$V^{\prime\prime}(r_\star)+\frac{3V^\prime(r_\star)}{r_\star}=\frac{p(2-p)k}{r_\star^{p+2}}>0$$
    which is positive iff $p<2$.
\end{example}
\subsection{The Orbit Equation}
The shape of the orbit is obviously governed by the joint variation of $r$ and $\theta$ (both as functions of $t$).
In principle, the energy equation can be helpful to determine $r(t)$, i.e.
$$E=\frac{1}{2}m\dot{r}^2+V_{\rm eff}(r)\implies t=\pm\sqrt{\frac{m}{2}}\int\frac{\mathrm dr}{\sqrt{E-V_{\rm eff}(r)}}$$
Given $r(t)$, since we already know the conservation of angular momentum $r^2\dot\theta=h$, we can then deduce $\theta(t)$.
But this might not always yield an analytic solution.\\
An interesting approach if one is only interested in the trajectory is to use $\theta$ as the dependent variable.
We can write
$$\frac{\mathrm d}{\mathrm dt}=\dot\theta\frac{\mathrm d}{\mathrm d\theta}=\frac{h}{r^2}\frac{\mathrm d}{\mathrm d\theta}$$
So plugging in Newton's Second Law,
$$m\frac{h}{r^2}\frac{\mathrm d}{\mathrm d\theta}\left( \frac{h}{r^2}\frac{\mathrm dr}{\mathrm d\theta} \right)-\frac{mh^2}{r^3}=F(r)$$
which then becomes, by substituting $u=1/r$,
$$\frac{\mathrm d^2u}{\mathrm d\theta^2}+u=-\frac{1}{mh^2u^2}F\left(\frac{1}{u}\right)$$
This is called the orbit equation.
We can then solve for $u$ as a function of $\theta$, and then $\dot\theta=hu^2$ can help us to deduce the time evolution.
\subsection{The Kepler Problem}
We want to solve the case for gravitational central force given by
$$F(r)=-\frac{mk}{r^2}$$
So the orbit equation becomes
$$\frac{\mathrm d^2u}{\mathrm d\theta^2}+u=\frac{k}{h^2}$$
Which is linear in $u$.
We know how to solve this.
Indeed, the general solution is given by
$$u=\frac{k}{h^2}+A\cos(\theta-\theta_0)$$
WLOG we assume $A\ge 0$.\\
If $A=0$, then $u$ is constant hence we obtain a circular orbit.
If $A>0$, $u$ obtains its maximum (hence $r$ obtains its minimum) at $\theta=\theta_0$.
We may choose $\theta_0=0$, then
$$r=\frac{1}{u}=\frac{\ell}{1+e\cos\theta},\ell=\frac{h^2}{k},e=\frac{Ah^2}{k}$$
Which is the polar coordinate form of a conic section with focus at the origin.
$e$ is called the eccentricities, which determines the shape of the trajectory.
By rearranging we obtain (since $r=\ell-ex$ and $r\cos\theta=y$)
$$(1-e^2)x^2+2elx+y^2=\ell^2$$
Therefore if $e\in[0,1)$, it is an ellipse that is bounded by
$$\frac{\ell}{1+e}\le r\le\frac{\ell}{1-e}$$
Or analytically we can rewrite the equation as
$$\frac{(x+ea)^2}{a^2}+\frac{y^2}{b^2}=1,a=\frac{\ell}{1-e^2},b=\frac{\ell}{\sqrt{1-e^2}}\le a$$
$a,b$ represents the semimajor and semiminor axes respectively.
In particular, for $e=0$, the path is a circle with center being the central mass.\\
For $e>1$, the equation gives a hyperbola, so $r\to\infty$ when $\theta\to\pm\alpha$ where $\alpha=\cos^{-1}(-1/e)\in (\pi/2,\pi)$.
We can also transform the equation ot the standard equation for hyperbola
$$\frac{(x-ea)^2}{a^2}-\frac{y^2}{b^2}=1$$
with $a=\ell/(e^2-1),b=\ell/\sqrt{e^2-1}$.
This case represents incoming body with large velocity which is deflected by gravitational force.
By simple calculations the asymptotes are $y=\mp b(x-ea)/a$, so $bx\pm ay=eba$.
And the normal vectors are $\underline{n}=(b,\pm a)/\sqrt{a^2+b^2}$.\\
Now consider the perpendicular distance between incoming mass and orgin, we have
$$\underline{r}\cdot\underline{n}=(x,y)\cdot\left( \frac{b}{\sqrt{a^2+b^2}},\pm\frac{a}{\sqrt{a^2+b^2}} \right)=\frac{eba}{\sqrt{a^2+b^2}}=b$$
This is sometimes called the impact parameter.\\
The marginal case that $e=1$ yields a parabola with equation
$$r=\frac{\ell}{1+\cos\theta}$$
where $r\to \infty$ as $\theta\to\pm\pi$.
In Cartesians this reduces to $y=2\ell(\ell-x)$.\\
On the other hand, we might want to analyze the linkge between the energy and the eccentricity of the trajectory.
Recall that
\begin{align*}
    E&=\frac{1}{2}m(\dot{r}^2+r^2\dot{\theta}^2)-\frac{mk}{r}\\
    &=\frac{1}{2}mh^2\left( \left( \frac{\mathrm du}{\mathrm d\theta} \right)^2+u^2 \right)-mku\\
    &=\frac{mk}{2\ell}(e^2-1)
\end{align*}
Hence bounded orbits have $e<1,E<0$ and unbounded ones have $e>1,E>0$.
The marginal case is then $e=E=0$.
\begin{law}[Kepler's Laws of Planetary Motion]
    1. Orbit of planet is ellipe with the Sun at focus.\\
    2. Line between the planet and the sun sweeps cut equal area in equal time.\\
    3. Square of period $P$ is proportional to cube of semimajor axis.
\end{law}
1 is consistent with the solutrion to the orbit equation that we have obtained earlier, and $2$ follows from the conservation of angular momentum (since the rate of change of area is approximately $r^2\dot\theta/2=h/2$).
Hence the area of the ellipse is $A=hP/2$ where $P$ is the period.
Therefore $\pi ab=hP/2$, rearranging gives the third statement.
\subsection{Rutherford Scattering}
Consider the motion in a repulsive force under inverse square law:
$$V(r)=\frac{mk}{r},F(r)=\frac{mk}{r^2}$$
Then the orbit equation solves to give
$$\frac{1}{r}=u=-\frac{k}{h^2}+A\cos(\theta-\theta_0)$$
WLOG $\theta_0=0,A\ge 0$.
So
$$r=\frac{\ell}{e\cos\theta-1},\ell=\frac{h^2}{k},e=\frac{Ah^2}{k}$$
If there is sometime where $r>0$, then we necessarily have $e>1$, therefore the trajectory is a hyperbola.
As previously known, $r\to\infty$ as $\theta\to\pm\alpha$ where $\alpha=\cos^{-1}(1/e)\in(0,\pi/2)$ and in Cartesian,
$$\frac{(x-ea)^2}{a^2}-\frac{y^2}{b^2}=1,a=\frac{\ell}{e^2-1},b=\frac{\ell}{\sqrt{e^2-1}}$$
Suppose the speed of the particle from far away be $v$, that is
With $x$-axis parallel to the incoming asymptote, as $t\to-\infty$
$$\underline{r}(t)\to (x(t),b,0),\underline{\dot{r}}(t)\to(-v,0,0)$$
Then $\underline{r}\times\underline{\dot{r}}\to(0,0,bv)$
Therefore the angular momentum per unit mass is $bv$, so
$$b=\frac{h^2}{k}\frac{k}{bv^2}=\frac{h^2}{k}\tan\frac{\beta}{2}=\frac{b^2v^2}{k}\tan\frac{\beta}{2},\beta=2\tan^{-1}\left( \frac{k}{bv^2} \right)$$
Rutherfold (1911) fired $\alpha$ particles at gold leaf to obtain experimental results of the scattering.
But Scattering angles greater than $\pi/2$ is observed in the experiment, from which he concluded that the positive charge must be highly concentrated.


\documentclass{article}
\usepackage{amsmath,amssymb,amsthm,fullpage,enumerate,hyperref,graphicx}
\usepackage[color]{changebar}
\cbcolor{blue}
\theoremstyle{definition}
\newtheorem{defi}{Definition}
\newtheorem{theo}{Theorem}
\newtheorem{prop}{Proposition}
\newtheorem{claim}{Claim}
\begin{document}
\section{Starting point}
We take a lattice
\[L=\mathbb{Z}\omega_1+\mathbb{Z}\omega_2,\]
for some $\omega_1,\omega_2\in\mathbb{C}$ not on the same line. Then $\mathbb{C}/L$ is topologically a torus.

Since we're going to do differential geometry, we want to find an explicit map. We recall that the origin-centered torus is given parametrically as
\begin{align*}
  x(\theta,\phi)&=(R+r\cos(\theta))\cos(\phi)\\
  y(\theta,\phi)&=(R+r\cos(\theta))\sin(\phi)\\
  z(\theta,\phi)&=r\sin(\theta),
\end{align*}
where $0\leq\theta,\phi\leq 2\pi$ and $R$ is the distance from the center of the tube to the center of the torus, and $r$ is the radius of the tube. We denote this torus by $\mathbb{T}^2(R,r)$. If you plot $\mathbb{T}^2(R,r)$ you'll notice that $\theta$ specifies angle in relation to the center of the tube, and $\phi$ specifies angle in relation to the center of the torus. As in the below picture:

\begin{figure}[h]
  \centering
  \includegraphics[width=0.6\textwidth]{theta_phi.png}
\end{figure}

To figure out the map from $\mathbb{C}/L$ to $\mathbb{T}^2(R,r)$ we essentially just have to figure out how to map a point on the ``parallellogram with sides identified'' to a point on $\mathbb{T}^2(R,r)$ for some appropriate $R$ and $r$.

There are several ways of doing this, and here's but one. Let $F:\mathbb{C}\to\mathbb{T}^2(1,1)$ be given by
\[F(\alpha\omega_1+\beta\omega_2)=(x,y,z)(\alpha\cdot2\pi,\beta\cdot2\pi).\]
Since $\omega_1$ and $\omega_2$ are linearly independent, this is well-defined, and since $\cos$ and $\sin$ are $2\pi$-periodic, $F$ descends to a function $\tilde{F}:\mathbb{C}/L\to\mathbb{T}^2(R,r)$. It is clear that viewed as function from $\mathbb{R}^2$ to $\mathbb{R}^3$, $F$ is a diffeomorphism.

By the way, I'm making the gentle assumption that $\omega_1$ and $\omega_2$ are both in the first quadrant.

Excellent, so let's now bring in some definitions from Spivak so that we can make sense of $\mathrm{d}z$ -- the translation invariant one-form.
\section{Definitions from Spivak}
We begin with a $k$-dimensional manifold.
\begin{defi}
  A subset $M\subseteq\mathbb{R}^n$ is called a $k$-dimensional manifold if for
  every $x\in M$ it holds that
  \begin{quotation}
    there exists an open set $U\ni x$ and an open set $V\subseteq\mathbb{R}^n$ and
    a diffeomorphism $h:U\to V$ such that
    \[h(U\cap M)=V\cap(\mathbb{R}^k\times\{\mathbf{0}\}).\]
  \end{quotation}
\end{defi}
To see that $\mathbb{T}^2(R,r)$ is a $2$-manifold, we have to think a little bit. Take $v\in M$ arbitrary, say with $v=(x(\theta',\phi'),y(\theta',\phi),z(\theta',\phi'))$ with $0\leq\theta',\phi<2\pi$. Let $0\leq\epsilon<\pi/4$ be arbitrary and put
\[P=\{(x(\theta'+2\pi\delta_{\theta',0}+s,\phi'+t),y(\theta'+2\pi\delta_{\theta',0}+s,\phi'+t),z(\theta'+2\pi\delta_{\theta',0}+s,\phi'+t)):-\epsilon<s,t<\epsilon\}.\]
Here $P$ stands for ``patch''. Let
\[w=(x(\pi/2,\phi'),y(\pi/2,\phi'),z(\pi/2,\phi')),\]
and let
\[U=\{p+rw:p\in P,-1/2<r<1/2\}.\]
Hence $U\cap M=P$. I take as geometrically clear that $U$ is indeed an open set. Let now
\[V=\{(s,t,u):-\epsilon<s,t<\epsilon,-1/2<u<1/2\},\]
and define $h:U\to V$ by
\[h(x(\theta'+2\pi\delta_{\theta',0}+s,\phi'+t)+rw_1,y(\theta'+2\pi\delta_{\theta',0}+s,\phi'+t)+rw_2,z(\theta'+2\pi\delta_{\theta',0}+s,\phi'+t)+rw_3)=(s,t,r).\]
Since any point in in $U$ is determined uniquely by the parameters $s$, $t$, and $r$, we have that $h$ is well-defined, and some first year calculus shows that it's smooth.

We see that
\[h(U\cap M)=h(P)=\{(s,t,0):-\epsilon<s,t<\epsilon\}=V\cap\mathbb{R}^2\times\{0\},\]
and so we conclude that $\mathbb{T}^2(R,r)$ indeed is a $2$-manifold.

I will take the view that $\mathbb{C}/L$ is simply identified with $\mathbb{T}^2(1,1)$ through $\tilde{F}$. This is because Spivak doesn't include an ambient space. {\tt Frankly, when giving the talk I will probably just use Lee's ``Smooth Manifolds'' instead or better yet Wells' ``Differential Analysis on Complex Manifolds'' because it's a bit difficult to translate notions all the time. On the other hand, Wells' and Lee kind of go off the deep end into some algebraic geometry. So there's that.}

Before continuing to {\it forms}, let's talk about coordinate systems. Spivak gives the following theorem.
\begin{theo}
  A subset $M$ of $\mathbb{R}^n$ is a $k$-dimensional manifold iff for every $x\in M$ the following condition is satisfied:
  \begin{quotation}
    There is an open set $U\ni x$ and an open set $W\subseteq\mathbb{R}^k$ and an injective smooth function $f:W\to\mathbb{R}^n$ such that
    \begin{enumerate}
      \item $f(W)=M\cap U$,
      \item $f'(y)$ has rank $k$ for every $y\in W$,
      \item $f^{-1}:f(W)\to W$ is continuous.
    \end{enumerate}
  \end{quotation}
  Such a function $f$ is called a coordinate system around $x$.
\end{theo}
I'm pretty sure that in modern terminology -- this corresponds to having an atlas.
\section{Forms}
Before we treat forms on manifolds, we need to recall what forms are in Euclidean space. For this, we need the concept of tensors, alternating tensors, and the wedge product. From Spivak's point of view, this isn't too bad. {\tt Although he seems to consciously skip the concept of vector and tangent bundles, which makes it a bit harder to define things.}

Let's start with tensors.
\begin{defi}[Tensor]
  Let $V$ be an $\mathbb{R}$-vector space, and let $k$ be a positive integer. Then a $k$-tensor on $V$ is a multilinear function $T:V^k\to\mathbb{R}$. The set of all $k$-tensors on $V$ is denoted by $\mathcal{T}^k(V)$. It forms a vector space given
  \[(S+T)(v)=S(v)+T(v),\]
  and
  \[(rS)(v)=rS(v),\]
  for $S,T\in\mathcal{T}^k(V)$ and $r\in\mathbb{R}$ and $v\in V$.
\end{defi}
In other words, $\mathcal{T}^k(V)=(V^k)^\vee$. If $T\in\mathcal{T}^k(V)$ it's called {\it alternating} if
\[T(v_1,\dots,v_i,\dots,v_j,\dots,v_k)=-T(v_1,\dots,v_j,\dots,v_i,\dots,v_k),\]
for any $i<j$. The space of all alternating $k$-tensors on $V$ is denoted by $\Lambda^k(V)$.

In this setting, the tensor product $\otimes$ is defined as follows.
\begin{defi}
  Let $V$ be an $\mathbb{R}$-vector space. Let $S\in\mathcal{T}^k(V)$ and $T\in\mathcal{T}^l(V)$. Then we define
  \[(S\otimes T)(v_1,\dots,v_k,v_{k+1},\dots,v_{k+l})=S(v_1,\dots,v_k)T(v_{k+1},\dots,v_{k+l}),\]
  and notice $S\otimes T\in\mathcal{T}^{k+l}(V)$.
\end{defi}
We can construct alternating tensors from tensors using the $\mathrm{Alt}$-map, defined as follows. Let $T\in\mathcal{T}^k(V)$, then put
\[\mathrm{Alt}(T)(v)=\frac{1}{k!}\sum_{\sigma\in S_k}\mathrm{sgn}(\sigma)T(\sigma.v),\]
where $\sigma$ acts by permutating the canonical basis. Clearly $\mathrm{Alt}$ is a generalization of the determinant.

It's a fact that if $T\in\mathcal{T}^k(V)$ then $\mathrm{Alt}(T)\in\Lambda^k(V)$ and $\mathrm{Alt}(\mathrm{Alt}(T))=T$, and if $\omega\in\Lambda^k(V)$, then $\mathrm{Alt}(\omega)=\omega$.

We can form products also on the space of alternating tensors.
\begin{defi}[Wedge product]
  Let $\omega\in\Lambda^k(V)$ and $\eta\in\Lambda^l(V)$ for some $\mathbb{R}$-vector space. Then we define
  \[\omega\land\eta=\frac{(k+l)!}{k!l!}\mathrm{Alt}(\omega\otimes\eta).\]
  It holds that $\omega\land\eta\in\Lambda^{k+l}(V)$.
\end{defi}
We can use the wedge product to get a basis for $\Lambda^k(V)$.
\begin{prop}
  Let $\{e_i\}_{i=1}^n$ be a basis for $V$. Then
  \[e_{i_1}^\vee\land\dots\land e_{i_k}^\vee,\]
  where $1\leq i_1<i_2<\dots<i_k\leq n$, forms a basis for $\Lambda^k(V)$.
\end{prop}

Now we're really close to defining forms. First we need the concept of a tangent space.
\begin{defi}[Tangent space]
  Let $n$ be a positive integer and let $p\in\mathbb{R}^n$. Then we define
  \[\mathbb{R}^n_p=\{(p,v):v\in\mathbb{R}^n\},\]
  and $(p,v)+(p,w)=(p,v+w)$ and $a(p,v)=(p,av)$. This turns $\mathbb{R}^n_p$ into a vector space, which we call the tangent space at $p$.

  We usually write $v_p$ for $(p,v)$.
\end{defi}
We will probably need the notion of vector fields, so I'll define that too.
\begin{defi}[Vector field]
  A vector field is a function
  \[F:\mathbb{R}^n\to\bigsqcup_{p\in\mathbb{R}^n}\mathbb{R}^n_p,\]
  such that $F(p)\in\mathbb{R}^n_p$ for every $p\in\mathbb{R}^n$.
\end{defi}
Notice that $(e_i)_p$ is a basis for $\mathbb{R}^n_p$, and thus we have
\[F(p)=\sum_{i=1}^nF^i(p)(e_i)_p,\]
for some component functions $F^i$. We call $F$ continuous or smooth if the $F^i$ are continuous or smooth; respectively.

And here we go, now we can define a differential form.
\begin{defi}[Differential form]
  A differential $k$-form on $\mathbb{R}^n$ is a function $\omega:\mathbb{R}^n\to\bigsqcup_{p\in\mathbb{R}^n}\Lambda^k(\mathbb{R}^n_p)$ with $\omega(p)\in\Lambda^k(\mathbb{R}^n_p)$.
\end{defi}
Since we know a basis for $\Lambda^k(V)$, we see that any $k$-form $\omega$ can be written as
\[\omega(p)=\sum_{1\leq i_1<\dots<i_k\leq n}\omega_{i_1,\dots,i_k}(p)\phi_{i_1}(p)\land\dots\land\phi_{i_k}(p),\]
for some coefficients $\omega_{i_1,\dots,i_k}(p)\in\mathbb{R}$, and $\phi_i=(e_i)_p^\vee$.

If the coefficient functions $p\mapsto\omega_{i_1,\dots,i_k}(p)$ are continuous or smooth, then we call the $k$-form $\omega$ continuous, or smooth; respectively.

Suppose now that $f:\mathbb{R}^n\to\mathbb{R}$ is smooth.
\begin{claim}
  Then $Df(p)\in\Lambda^1(\mathbb{R}^n)$.
\end{claim}
\begin{proof}
  By definition $Df(p)=\lambda:\mathbb{R}^n\to\mathbb{R}$ is the unique linear transformation satisfying
  \[\lim_{h\to 0}|f(p+h)-f(h)-\lambda(h)|/|h|=0.\]
  So $\lambda$, because it's linear, is a $1$-tensor. But then it's alternating (vacuously).
\end{proof}
Using this, we can define $df$, for $f:\mathbb{R}^n\to\mathbb{R}$ smooth.
\begin{defi}
  Let $f:\mathbb{R}^n\to\mathbb{R}$ be smooth. Then we define $df:\mathbb{R}^n\to\bigsqcup_{p\in\mathbb{R}^n}\Lambda^1(\mathbb{R}^n_p)$ by
  \[df(p)(v_p)=Df(p)(v).\]
\end{defi}
Let in particular $\pi^i:\mathbb{R}^n\to\mathbb{R}$ be defined by $\pi^i(x)=x^i$. Notice now that
\[d\pi^i(p)(v_p)=D\pi^i(p)(v)=e_i^Tv=v^i,\]
So $d\pi^i(p)((e_j)_p)=[i=j]$, so $d\pi^i(p)=(e_i)_p^\vee$. Usually we write $x^i$ for $\pi^i$, and thus any differential $k$-form can be written as
\[\omega(p)=\sum_{1\leq i_1<\dots<i_k\leq n}\omega_{i_1,\dots,i_k}(p)dx^{i_1}\land\dots\land dx^{i_k}.\]
When defining differential forms on manifolds, we want to induce smooth functions $\mathbb{R}^n\to\mathbb{R}^m$ to linear transformations $\mathbb{R}^n_p\to\mathbb{R}^n_{f(p)}$. This is how we do it.
\begin{defi}
  Let $f:\mathbb{R}^n\to\mathbb{R}^m$ be a smooth. Then we define $f_\ast:\mathbb{R}^n_p\to\mathbb{R}^m_{f(p)}$ by
  \[f_\ast(v_p)=(Df(p)(v))_{f(p)}.\]
\end{defi}
\begin{claim}
  It makes sense.
\end{claim}
\begin{proof}
  We have that $Df(p)$ is a linear transformation $\mathbb{R}^n\to\mathbb{R}^m$, so $Df(p)(v)\in\mathbb{R}^m$ and thus $(Df(p)(v))_{f(p)}$. Linearity should be fine.
\end{proof}
From $f_\ast$ we can induce further, to get a transformation
\[f^\ast:\Lambda^k(\mathbb{R}^m_{f(p)})\to\Lambda^k(\mathbb{R}^n_p),\]
in the usual way -- that is:
\[f^\ast(\omega)=\omega\circ f_\ast.\]
\begin{claim}
  This also makes sense.
\end{claim}
\begin{proof}
  Take $v_p\in\mathbb{R}^n_p$, then $f_\ast(v_p)\in\mathbb{R}^m_{f(p)}$, so is in the domain of $\omega$. Cool, we're fine.
\end{proof}
That's all we need for forms. So let's go to {\Huge\tt manifold town!}
\section{Forms on manifolds}
Let $M$ be a $k$-dimensional in $\mathbb{R}^n$ and let $f:W\to\mathbb{R}^n$ be a coordinate system around $x=f(a)$.
\begin{claim}
  It holds that $f_\ast:\mathbb{R}^k_a\to\mathbb{R}^n_x$ is injective, and $f_\ast(\mathbb{R}^k_a)$ is $k$-dimensional subspace of $\mathbb{R}^n_x$.
\end{claim}
\begin{proof}
  We have that $f_\ast(v_a)=(Df(a)(v))_x$. Since $f'(a)$ has rank $k$, the linear transformation $Df(a):W\to\mathbb{R}^k$ is an isomorphism. Therefore, $(Df(a)(v))_x=0$ implies $Df(a)(v)=0$ implies $v=0$ implies $v_a=0$. So also $f_\ast$ is injective.

  It's not surjective, but at least $f_\ast(\mathbb{R}^k_a)$ is an isomorphic copy of $\mathbb{R}^k_a$, so it is of dimension $k$.
\end{proof}
Let $g:V\to\mathbb{R}^n$ be any other coordinate system around $x=g(b)$. {\tt We assume, I think WLOG, that the $U\ni x$ is the same for both.}
\begin{claim}
  Its range is the same.
\end{claim}
\begin{proof}
  \cbstart
  Let's first prove that $g_\ast=f_\ast\circ(f^{-1}\circ g)_\ast$. This follows basically by the chain rule. Note that
  \[f^{-1}(g(b))=f^{-1}(x)=a,\]
  and therefore
  \[Df(a)D(f^{-1}\circ g)(b)=Df(f^{-1}(g(b)))D(f^{-1}\circ g)(b)=D(f\circ(f^{-1}\circ g))(b)=Dg(b).\]
  This means that
  \[f_\ast(f^{-1}\circ g)_\ast(v_b)=f_\ast((D(f^{-1}\circ g)(b)v)_a)=(Df(a)(D(f^{-1}\circ g)(b)v))_x=(Dg(b)v)_x=g_\ast(v_b).\]
  Let's now take a peek at
  \[(f^{-1}\circ g)_\ast(\mathbb{R}^k_b).\]
  We have that
  \[(f^{-1}\circ g)_\ast(\mathbb{R}^k_b)=\{(Df^{-1}g(b)\circ Dg(b)v)_a:v\in\mathbb{R}^k\}.\]
  Since $Dg(b)$ has rank $k$, we have that $Dg(b)v$ spans a $k$-dimensional subspace of $\mathbb{R}^n$. Since $Df(a)$ has rank $k$, it should be the case that $Df^{-1}(g(b))$ (which exists because reasons) also has rank $k$, and thus $Df^{-1}(g(b))Dg(b)v$ should be $k$-dimensional, and thus $\mathbb{R}^k$ itself.
  \cbend

  {\tt BTW:} The correct way is to say that $(g^{-1}\circ f)_\ast$ is an inverse to $(f^{-1}\circ g)_\ast$. No smoothness needed!
\end{proof}
{\tt Fishy argument. I need to fix this.}

The best way to fix this is to just use Lee. But, I'll let it slide for now. Let's just assume that it works. {\tt Damn it!}

{\tt Found a fix in ``Comprehensive Introduction'' volume 1. Yes!}

If $f:W\to\mathbb{R}^n$ is a coordinate system. We write
\[M_x=f_\ast(\mathbb{R}^k_a),\]
and call this the tangent space of $M$ at $x$. We can now define a differential $p$-form.
\begin{defi}[Form on a manifold]
  Let $\omega:M\to\bigsqcup_{x\in M}\Lambda^p(M_x)$ satisfy $\omega(x)\in\Lambda^p(M_x)$ for every $x\in M$. Then we call $\omega$ a $p$-form on $M$.

  If $f:W\to\mathbb{R}^n$ is a coordinate system, then $f^\ast\omega$ is a $p$-form on $W$, and we say that $\omega$ is smooth if $f^\ast\omega$ is.
\end{defi}
Next Spivak claims we can write $\omega$ as
\[\omega=\sum_{1\leq i_1<\dots<i_p\leq n}\omega_{i_1,\dots,i_p}dx^{i_1}\land\dots\land dx^{i_p},\]
and this means we need to make sense of
\[dx^{i_j}(p),\]
for $p\in M$. Can we? Previously we defined it as
\[d\pi^i(v)(w\in M_x)=D\pi^i(v)(w),\]
but we probably need to use coordinate systems now.

And yes we do, but recall that $M_x=f_\ast(\mathbb{R}^k_a)$ actually sits inside $\mathbb{R}^n_x$, so we in fact use the same $d\pi^i$ as before.

But, when we want to carry over $dz$ to $\mathbb{T}^2$ we get a problem. It's the case that
\[f_\ast(\mathbb{R}^2_a)\subset\mathbb{R}^3_x,\]
so we can impossible make sense of $(dx,dy)$, because $dx$ and $dy$ only make sense in $\mathbb{R}^2_x$. Here's where our grand plan stops making sense.
\end{document}