%begin-include

\section{ZFC Set theory}

\begin{para}[ZFC set theory]
The standard axiomatisation of set theory is known as ZFC. The Z and the F stand, respectively, for the mathematicians Ernst Zermelo and Abraham Fraenkel. The C stands for the controversial axiom of choice, so, yes, there is an axiomatisation of set theory known as ZF that does not include this axiom.

The formal system $\mathsf{ZFC}$ in which we are going to formalise set theory uses first-order logic with equality and is built on the first-order language with set of symbols $S = \{=,\in,\emptyset\}$ and signature $\sigma(=) = -2$,  $\sigma(\in) = -2$ and $\sigma(\emptyset) = 0$. Both $=$ and $\in$ are used with infix notation.
If two elements $x$ and $y$ satisfy $x = y$ or $x\in y$, we will say, respectively that ``$x$ is equal to $y$'' and that ``$x$ is an \emph{element} of $y$''.
We can also say that ``$x$ is \emph{contained} in $y$'' to mean that $x\in y$.
The constant $\emptyset$ will be referred to as the \emph{empty set}.
The elements of the domain of any model of $\mathsf{ZFC}$ are called \emph{sets}.

For any two variables (sets) $x$ and $y$, we introduce the notation $x\subseteq y$ as an abbreviation of the formula $(\forall a)\qsep a\in x \limplies a \in y$ for any variable $a$ not occurring anywhere in the formula where $x\subseteq y$ is being used.
If two sets $x$ and $y$ satisfy $x\subseteq y$, we say ``$x$ is \emph{included} in $y$''.

We will now present all the non-logical axioms of $\mathsf{ZFC}$ and analyse them from a semantic perspective.
\begin{axioms}[ZF]
\item \label{extensionality} Axiom of extensionality: $(\forall x)(\forall y)\qsep ((\forall a)\qsep (a \in x \liff a\in y)) \limplies x = y$.
\end{axioms}
In layman's terms, the axiom of extensionality means that any two sets with the same elements are equal.
In addition, using this axiom, \ref{axe4}, \ref{foiff} and the generalisation rule, one can easily deduce that
\[(\forall x)(\forall y)\qsep x = y \liff ((\forall a)\qsep a\in x \liff a \in y).\]

\begin{axioms}
\item \label{emptyset} Axiom of the empty set: $(\forall x)\qsep \lnot(x\in\emptyset)$.
\end{axioms}
The axiom of the empty set, as anyone should expect, simply states that $\emptyset$ has no elements. Furthermore, it follows from \ref{extensionality} that any element verifying this property is equal to $\emptyset$.

\begin{axioms}
\item Axiom of union: $(\forall X)(\exists U)(\forall x)(\forall a)\qsep (x\in X \land a \in x) \limplies a\in U$.
\label{union}
\end{axioms}
The axiom of union states that, given any set $X$, there exists a set $U$ containing all the elements of the sets in $X$.
Informally, this means that, given any collection of sets $X$, there exists a set that contains the union of all the sets in $X$.

\begin{axioms}
\item \label{infinity} Axiom of infinity: $(\exists X)\qsep \emptyset \in X \land (\forall x)\qsep$\\
$x\in X \limplies (\exists y)(y \in X \land (\forall a)\qsep a\in y \liff (a = x \lor (\forall b)(b\in a \liff b = x)))$.
\end{axioms}
I know what you are thinking and you are right; the axiom of infinity is a mess.
There are simpler ways to present this axiom, but this is by far the most formal of all, and, you know, if we are going to do a formal treatment of set theory, let us do it properly! You and I are warriors, not soldiers.

If you take your time to analyse it, you will see that it postulates the existence of a set $X$ having $\emptyset \in X$ and verifying, for every $x\in X$, $x\cup \{x\} \in X$.
Notice that we have not yet defined what $x\cup \{x\}$ means in $\mathsf{ZFC}$, but we have already done an informal treatment of set theory and you should know that, with that, I (informally) mean ``the set having as elements $x$ and a set that only has $x$ as an element''.

You may wonder what the point of this axiom is. Turns out this will enable us to define the natural numbers and, from there on, the rationals, the reals\ldots you name it!

\begin{axioms}
\item Power set axiom: $(\forall x)(\exists P)(\forall a)\qsep a\subseteq x \limplies a\in P$.
\label{powerset}
\end{axioms}

The power set axiom --- which, after going through the axiom of infinity, looks ridiculously simple --- simply establishes the existence, for any set $x$, of a set containing each subset of $x$, i.e., its power set.

\begin{axioms}
\item \label{regularity} Axiom of regularity: $(\forall x)\qsep \lnot(x=\emptyset) \limplies (\exists y)(y \in x \land \lnot (\exists z) (z\in y \land z \in x))$.
\end{axioms}
This axioms guarantees the existence in any non-empty set $x$ of an element $y$ containing no elements of $x$. To put it in perhaps clearer terms, it says that no set $x$ can only consist of sets having elements of $x$.

The best way in which you can see why the axiom of regularity need be true is by trying to construct a set contradicting it.

\begin{axioms}
\item \label{replacement} Axiom schema of replacement:
for any formula $\psi$ in which there are only free occurrences of $x$ and $y$ and in which there are no quantifications over $X$ and $Y$,
$(\forall x)(\exists! y) \psi \limplies (\forall X)(\exists Y)(\forall y)\qsep y\in Y \liff (\exists x)(x\in X \land \psi)$.
\end{axioms}
What the axiom of replacement tells us is that, if we are given a formula\footnote{We are using the informal notation $\psi(x,y)$ to represent an arbitrary formula $\psi$ having two free variables $x$ and $y$.} $\psi(x,y)$ that, for each value of $x$, is true for one unique value of $y$ --- in other words, if $\psi(x,y)$ behaves like a function, --- then, given any set $X$, there exists a set $Y$ containing, exclusively, all the elements $y$ for which there exists an $x\in X$ verifying $\psi(x,y)$.

Putting it simpler terms, this means that if a ``function'' $f$ taking any set $x$ to a set $y$ can be expressed as a formula $A(x,y)$ that is true if and only if $f(x) = y$, then, for any set $X$, there exists a set containing the image of $X$ under $f$.
Of course, these formulas are not functions within set theory: a function within $\mathsf{ZFC}$ need be an object of $\mathsf{ZFC}$ and, therefore, a set, not a formula!
We will later on define the concept of a function in $\mathsf{ZFC}$ in a precise and formal manner.

\begin{axioms}
\item \label{choice} Axiom of choice: $(\forall X) \qsep (\forall x)(x\in X \limplies (\lnot(x = \emptyset) \land$\\
\hspace*{1em}$(\forall y)\qsep y \in X \land\lnot(x = y) \limplies \lnot(\exists a)\qsep a \in x \land a \in y))\limplies$\\
$((\exists S)(\forall s)\qsep s\in S \limplies ((\exists x)\qsep x\in X \land s \in x \land(\forall z)((\lnot (z=a)\land z\in S )\limplies \lnot (z \in x))))$.
\end{axioms}
What this famous and controversial axiom of choice is telling us is something that is, well, obvious.
It is simply stating that, given any collection $X$ of non-empty sets that have no elements in common, there exists a set $S$ containing one and only one element from every set in $X$.
This assumption is extremely natural and so are the consequences that are derived from it --- at least from my personal perspective --- but the poor axiom of choice is rejected by some people (see \cite{Brown} for reference).
Please, give him some love and say with me ``I choose choice!''.

By the way, if you were, by any chance, hoping that there could be a way to prove that either the axiom of choice or its negations would make $\mathsf{ZF}$ inconsistent and thus settle this debate of choice versus no choice once and for all\ldots I have bad news. The axiom of choice is independent from $\mathsf{ZF}$ and, if $\mathsf{ZF}$ is consistent, so is $\mathsf{ZF}$ with the axiom of choice and so is $\mathsf{ZF}$ with the negation of the axiom of choice. So, at the end of the day, accepting or negating the axiom of choice is not a mathematical issue, but a philosophical one. This is one of those things that makes mathematics look more like a religion than like a science.

We shall now introduce a pretty solid collection of definitions and results. All those definitions and results that follow are done within $\mathsf{ZFC}$ unless otherwise stated.
\end{para}

\begin{proposition}
\label{setelementary}
The following elementary properties of sets are true:
\begin{statements}
\item Let $X$ be a set. Both $X$ and $\emptyset$ are subsets of $X$.
\item \label{doubleinclussion} Two sets $X$ and $Y$ satisfy $X = Y$ if and only if $X\subseteq Y$ and $Y\subseteq X$.
\end{statements}
\label{<+label+>}
\end{proposition}

\begin{proof}
\begin{parlist}
\item It is obvious that $X\subseteq X$. In regard to $\emptyset$, for it to be a subset of $X$, the formula $(\forall x)(x\in \emptyset \limplies x \in X)$ needs to be satisfied.
Since, according to \ref{emptyset}, no set $x$ can verify $x\in \emptyset$, the formula $x\in \emptyset \limplies x \in X$ is satisfied for any $x$ and $X$. Thus, the empty set is a subset of any set.

\item Immediate from \ref{extensionality} and from the definition of $\subseteq$.
\end{parlist}
\end{proof}

\begin{theorem}[Schema of specification]
Let $\phi$ be a formula in the language of set theory with a single free variable. Given any set $X$, there exist the subset $Y\subseteq X$ of elements $x\in X$ verifying $\phi(x)$.
Such a set is described using \emph{set-builder notation}: $Y = \{x\in X \mid \phi(x)\}$.
\label{specification}
\end{theorem}

\begin{proof}
If there are no elements $x\in X$ satisfying $\phi(x)$, then $Y = \emptyset$, which certainly exists.

Let us then assume the existence of an element $y_0 \in Y$. We can consider the formula $\psi(x,y)$ given by
\[ (\phi(x) \land y = x) \lor (\lnot \phi(x) \land y = y_0).\]
It is clear that, for any set $x$, there exists a unique $y$ satisfying $\psi(x,y)$: if $\phi(x)$ holds, that $y$ is $x$ itself, whereas, if it does not, that $y$ is $y_0$.
Thus, we can apply \ref{replacement} to conclude that the collection $Y'$ of all the elements $y$ for which there exists an $x\in X$ verifying $\psi(x,y)$ is a set. Furthermore, this set is $Y$. Let us prove it by double inclusion, i.e., using \ref{setelementary}\ref{doubleinclussion}.

Given any $y\in Y$, the formula $\psi(y,y)$ holds and --- since $Y\subseteq X$ and, therefore, $y\in X$ --- we have $y \in Y'$, which shows that $Y\subseteq Y'$.
Conversely, given any $y'\in Y'$, we there know to exist an $x\in X$ such that $\psi(x,y')$ holds.
If $\phi(x)$ is satisfied, then $y'=x$, so $y'\in Y$.
If $\phi(x)$ is not satisfied, then $y' = y_0$, which, by hypothesis, belongs to $Y$.
In either case, $y'\in Y$, which shows that $Y'\subseteq Y$.
\end{proof}

\begin{para}
Do you remember when back in \ref{logifail} we said that, in modern day set theory, we could identify predicates with sets --- just as people wanted to do in the early days --- provided we did it with care?
The scheme of specification has just made that notion precise: given any set $X$ we can identify each unary predicate $\phi$ with the set $\{x \in X \mid \phi(x)\}$.
What makes this approach different from the one that led to Russel's paradox is our requiring the ``domain'' over which we define $\phi$ to be a set complying with the axioms of $\mathsf{ZFC}$, and that enables us to escape from any paradox.

We will shortly analyse some details regarding the schema of specification  and how it avoids the paradoxes of primitive set theory.
\end{para}

\begin{para}[Definition-Proposition]
\label{setops}
Let $X$ and $Y$ be any two arbitrary sets.
\begin{statements}
\item \label{setunion} There exists a set $\cup X$ containing, exclusively, all the elements of the sets contained in $X$. This set is known as the \emph{union} of the elements of $X$.
\item There exists a set $\cap X$ containing, exclusively, the elements that belong to all the sets contained in $X$. This set is called the \emph{intersection} of the elements of $X$.
\item There exists a set $X\setminus Y$ containing, exclusively, the elements of $X$ that do not belong to $Y$. The set $X\setminus Y$ is known as the \emph{difference} of $X$ and $Y$. In particular, if $Y\subseteq X$, the set $X\setminus Y$ is said to be the \emph{complement} of $Y$ in $X$.
\item There exists a set $\mathcal{P}(X)$ containing, exclusively, all the subsets of $X$. This set is known as the \emph{power set} of $X$.
\end{statements}
\label{<+label+>}
\end{para}

\begin{proof}
According to \ref{union}, there exists a set $U$ that contains all the elements of the sets contained in $X$. Applying \ref{specification}, it follows that
\begin{gather*}
\cup X = \{u \in U \mid (\exists x \in X) u \in x\},\\
\cap X = \{u \in U \mid (\forall x \in X) u \in X\},
\end{gather*}

In regard to $X\setminus Y$, it suffices to use \ref{specification} again and consider $X\setminus Y = \{x\in X \mid x\not\in Y\}$.

Lastly, regarding the power set, \ref{powerset} guarantees the existence of a set $P$ containing every subset of $X$, so we just need to apply \ref{specification} once more and take
\[\mathcal{P}(X) = \{p \in P \mid p \subseteq X\}.\]
\end{proof}

\begin{proposition}
Any element $a$ of a set $X$ is a set.
\label{elisset}
\end{proposition}

\begin{proof}
It suffices to apply \ref{specification} and \ref{setops}\ref{setunion} to conclude that $a = \cup \{x \in X \mid x = a\}$ is a set.
\end{proof}

\begin{proposition}[Pairing]
Let $a$ and $b$ be sets. The set $\{a,b\}$ containing, exclusively, the elements $a$ and $b$ exists. In particular, if $a=b$, the \emph{singleton} $\{a\}$ exists.
\label{pairing}
\end{proposition}

\begin{proof}
According to \ref{infinity}, we there know to exist a set that contains, in particular, the element $X = \{\emptyset, \{\emptyset\}\}$. Thus, using \ref{elisset}, we know $X$ to be a set.
We can then consider the formula $\psi(x,y)$ defined by
\[ (x = \emptyset \land y = a) \lor (x = \{\emptyset\} \land y = b) \lor (\lnot (x \in X) \limplies x = y).\]
A direct application of the axiom schema of replacement \ref{replacement} shows that the set $Y$ of elements $y$ for which there exists an $x\in X$ verifying $\psi(x,y)$ is a set. Since $Y = \{a,b\}$, we have shown the set $\{a,b\}$ to exist.
\end{proof}

\begin{para}[Notation]
Let $X$ and $Y$ be sets. By \ref{pairing}, $\{X,Y\}$ is a set. We define $X\cup Y = \cup\{X,Y\}$ and $X\cap Y = \cap \{X\cap Y\}$.
If $X\cap Y = \emptyset$, we say that $X$ and $Y$ are \emph{disjoint}. Furthermore, given any collection $C$ of sets, we say that the sets in $C$ are \emph{pairwise disjoint} if, for any $X,Y\in C$ with $X\neq Y$, the sets $X$ and $Y$ are disjoint.

Let $n$ be a natural number and let $X_1,\ldots,X_n$ be sets. Through a recursive application of \ref{pairing} and \ref{setops}\ref{setunion}, we know the set $X$ with elements $X_1,\ldots, X_n$ to exist. This set is represented as $\{X_1,\ldots,X_n\}$.
\end{para}

\begin{para}
Now that we know singletons to exist, we can add a final touch on our discussion on the scheme of specification and Russel's paradox.

We can illustrate the robustness of our system with a simple example.
If we consider \emph{any} set $X$ and try to recreate Russel's paradox by defining the set $R = \{x \in X \mid x\not\in x\}$, we would have
\[ R \in R \iff (R \in X \land R \not \in R).\]
Does it look suspicious? It is actually harmless.
Let $A$ be any set. Applying the axiom of regularity \ref{regularity} to $\{A\}$, we know that $A \not\in A$. Therefore, it is clear that $R = X$ and, therefore, that $R\not\in X$.
It then follows that both sides of the equivalence are false: everything fits nicely and we are paradox-free.

You may then wonder: and what if I take $X$ to be a ``universe'' set containing all the sets in $\mathsf{ZFC}$? Well, it is immediate from the axiom of regularity \ref{regularity} that such a set cannot exist in $\mathsf{ZFC}$, so there is nothing to worry about.
\end{para}

\begin{definition}
Let $x$ and $y$ be sets. The \emph{ordered pair} with \emph{first coordinate} $x$ and \emph{second coordinate} $y$ is the set
\[ (x,y) = \{ \{x\}, \{x,y\} \}.\]
Obviously, $(x,y) \neq (y,x)$ unless $y = x$, hence the name ordered pair.
Moreover, if $x_1,x_2,y_1,y_2$ are some arbitrary sets, $(x_1,y_1) = (x_2,y_2)$ if and only if $x_1 = x_2$ and $y_1 = y_2$.

Let us assume that, for some sets $X$ and $Y$, $x\in X$ and $y\in Y$. The ordered pair $(x,y)$ belongs to the set $\mathcal{P}(X\cup Y)$, so we can define the \emph{binary cartesian product} of $X$ and $Y$ as the set
\[ X\times Y = \{p \in \mathcal{P}(X\cup Y) \mid (\exists x)(\exists y)\qsep x\in X \land y\in Y \land p = (x,y)\}.\]
To put it simply, $X\times Y$ is the set of all ordered pairs $(x,y)$ with $x\in X$ and $y\in Y$.
\label{<+label+>}
\end{definition}

\begin{para}
Just as every unary predicate (unary relation) on a set $X$ was characterised by a subset of $X$, every binary relation on two sets $X$ and $Y$ will be characterised by a subset of $X\times Y$.
The ability to work with relations in $\mathsf{ZFC}$ will enable us to introduce functions as mere set-theoretical objects, and, in return, functions will enable us --- among many other things --- to define $n$-ary cartesian products and, therefore, to introduce $n$-ary relations in set theory.
\end{para}

\begin{definition}
\label{binrel}
Let $X$ and $Y$ be sets. A \emph{binary relation} over the sets $X$ and $Y$ is a subset $R$ of the cartesian product $X\times Y$.
If a subset $R\subseteq X\times Y$ is being regarded as a binary relation and not as a mere subset, it is customary to write $xRy$ in lieu of $(x,y) \in R$.

Any formula $\phi$ with a free variable can induce a binary relation according to \ref{specification}, for it suffices to consider $R = \{(x,y)\in X\times Y \mid \phi ( ( x,y) )\}$.
This kind of definition is often done in an implicit manner as ``let $R$ be the relation on $X$ and $Y$ that is satisfied if and only if $\phi( (x,y))$''.

Very often, relations will be subsets of $X\times X$. In this case, we say that $R$ is a binary relation over $X$: it would be redundant to say that it is a binary relation over $X$ and $X$.

Writing $xRy$ might look a little bit confusing, that is why binary relations are often represented by fancy symbols such as $\sim$ or $\equiv$.
In addition, some relations may use a more complex notational artefact than just the structure ``element symbol element''.
As always, if there is something inexact and non-universal about mathematics, that is its notation!

\label{<+label+>}
\end{definition}

\begin{example}
\begin{parlist}
\item Let $X$ be any set.
The empty relation $R = \emptyset$ is a binary relation over $X$.
It is, obviously, not satisfied by any $(a,b)\in X\times X$.
\item Let $X$ be any set.
The subset $R = \{(a,b)\in X\times X \mid a = b\}$ is a binary relation over $X$.
The relation $R$ can equivalently be defined as the binary relation on $X$ such that $aRb$ if and only if $a= b$.
\item Let $X$ and $Y$ be sets.
The set $R = X\times Y$ is a binary relation over $X$ and $Y$.
Of course, any $x\in X$ and $y\in Y$ verify $xRy$.
\end{parlist}
\end{example}

\begin{definition}
A \emph{function} $f$ from a set $X$ to a set $Y$ is a binary relation over $X$ and $Y$ such that, for every $x\in X$, there exists a unique $y\in Y$ verifying $xfy$. Instead of writing $xfy$, we will use $f(x) = y$.
The fact that $f$ is a function from $X$ to $Y$ is denoted by $f:X\longrightarrow Y$. In addition, $f(x) = y$ can also be written as $f:x\longmapsto y$.

Given $f:X\longrightarrow Y$, the set $X$ is said to be the \emph{domain} of $f$ ($X = \op{dom} f$) whereas $Y$ is known as the \emph{codomain} of the function. The \emph{image} $\op{im} f$ of $f$ is the subset of $Y$ containing the elements $y\in Y$ for which there exists an $x\in X$ such that $f(x) = y$, i.e.,
\[\op{im} f = \{y\in Y \mid (\exists x \in X)(y = f(x)\},\]
where, as in \ref[prel]{pseudoquan}, $(\exists x \in X)\theta(x)$ means $(\exists x)(x\in X \limplies \theta(x))$.

We say that $f$ is injective if, for every $y\in \op{im}f$, there exists a unique $x$ such that $f(x) = y$, or if, equivalently, for every $a,b\in X$, $f(a) = f(b)$ implies $a = b$.
In addition, $f$ is said to be \emph{surjective} if $\op{im} f = Y$.
If a function is both injective and surjective, it is said to be \emph{bijective}.

Given any $A\subseteq X$, the \emph{image} under $f$ of $A$ is the set
\[ f[A] = \{y \in Y \mid (\exists a \in A)\qsep f(a) = y\}.\]
Notice how $\op{im} f = f[X]$.
Similarly, given any $B\subseteq Y$, the \emph{inverse image} of $B$ under $f$ is the set
\[ f^{-1}[B]  = \{x\in X \mid f(x) \in B\}.\]

If $f$ is injective, we can define a function $f^{-1}: \op{im} f\longrightarrow X$ mapping every $y\in Y$ to $\cup f^{-1}[\{y\}] \in X$, i.e., to the only element $x\in X$ satisfying $f(x) = y$.

Given any set $X$, the \emph{identity} function on $X$ is the function $\op{id}_X :X\longrightarrow X$ taking any $x\in X$ to itself: $f(x) = x$.

Given any two functions $f:X\longrightarrow Y$ and $g:Y\longrightarrow Z$, the \emph{composition} of $f$ with $g$ is the function
\begin{align*}
g\circ f: X &\longrightarrow Z \\
x &\longmapsto g\left( f(x) \right).
\end{align*}

Lastly, given a subset $A\subseteq X$ and a function $f:X\longrightarrow Y$, we define the \emph{restriction} of $f$ to $A$ as the function
\begin{align*}
f\vert_A : A &\longrightarrow Y\\
a &\longmapsto f(a).
\end{align*}
\label{<+label+>}
\end{definition}

\begin{proposition}
Let $f:X\longrightarrow Y$ and $g:Y\longrightarrow Z$ be functions.
\begin{statements}
\item The function $g\circ f$ is injective only if $f$ is injective.
\item The function $g\circ f$ is surjective only if $g$ is surjective.
\end{statements}
\label{compinjsur}
\end{proposition}

\begin{proof}
We will prove the contrapositive of each of the statements.

\begin{parlist}
\item If $f$ is not injective, there exist $a,b\in X$ such that $a\neq b$ and $f(a) = f(b)$; consequently, $g\circ f(a) = g \circ f(b)$. Thus, $g\circ f$ is not injective.
\item Analogously, if $g$ is not surjective, no $y\in Y$ satisfies $f(y) = z_0$ for some $z_0\in Z$. Thus, given any $x\in X$, since $f(x) \in Y$, we have $g\circ f(x) \neq z_0$.
\end{parlist}
\end{proof}

\begin{proposition}
Let $f: X\longrightarrow Y$. The function $f$ is bijective if and only if there exists a function $g:Y\longrightarrow X$ such that $f\circ g = \op{id}_X$ and $g\circ f =\op{id}_Y$.
\label{<+label+>}
\end{proposition}

\begin{proof}
If $f$ is bijective, the function $g$ we are looking for is $f^{-1} : \op{Im} f = Y \longrightarrow X$. It is immediate that $f \circ f^{-1} = \op{id}_X$ and that $f^{-1}\circ f = \op{id}_Y$.

Conversely, since both $\op{id}_X = f\circ g$ and $\op{id}_Y = g\circ f$ are bijective, the result follows from \ref{compinjsur}.
\end{proof}

\begin{para}[Definition-proposition]
We say that a set $X$ is \emph{inductive} if it verifies the formula $\Omega(X)$ given by
\[ \emptyset \in X \land (\forall x)(x\in X \limplies x \cup \{x\} \in X),\]
where, for the sake of clarity, we have used $x\cup\{x\}\in X$ in lieu of
\[(\exists y)\qsep y \in X \land (\forall t)\qsep t\in y\liff(t = x \lor (\forall s)(s\in t \liff s = x )).\]
There exists a \emph{minimal} inductive set $\omega$, i.e., an inductive set that is included in every inductive set.
\label{<+label+>}
\end{para}

\begin{proof}
By \ref{infinity}, we know an inductive set $V$ to exist. The set $\omega$ can be constructed using \ref{specification} as
\[ \omega = \{v\in V\mid (\forall X)(\Omega(X)\limplies v\in X)\}.\]
In plain English, $\omega$ is the set containing, exclusively, the elements that are common to all inductive sets.
Informally, we could have defined it as the ``intersection of all the inductive sets''; nonetheless, since we have not proven the existence of such thing as a set of all inductive sets, we have had to create this custom definition.

The set $\omega$ is clearly inductive. On the one hand, any inductive set $X$ verifies $\emptyset \in X$, so $\emptyset\in \omega$.
On the other hand, if $v\in\omega$, then $v$ belongs to any inductive set $X$, and, by definition, so does $v\cup\{v\}$. Thus, $v\cup\{v\}\in \omega$.

Lastly, it is trivial that $\omega$ is included in any inductive set for the elements in $\omega$ belong, by definition, to any inductive set.
\end{proof}

\begin{para}
If set theory has any intention of becoming a foundational system for mathematics, it better let us work with natural numbers!
Now, how could we possibly implement natural numbers in $\mathsf{ZFC}$? How could we construct a set of natural numbers?
The answer is, surprise surprise, we have already done it! The set $\omega$ is the set we have been looking for! Informally, if we let $0 = \emptyset\in \omega$, we can define
\[ 1 = \{0\}\in \omega,\qquad 2 = \{0,1\} \in \omega, \qquad 3 = \{0,1,2\}\in \omega,\]
and so on. In general, given any natural number $n$ representing a set $n\in \omega$, we define its successor as $s(n) = n \cup \{n\} = \{1,\ldots,n\}\in \omega$.

In this construction, the set of natural numbers would be $\mathbb{N} = \omega \setminus \{\emptyset\}$.

I know. This looks weird. Having $4\subseteq 5$ seems like an odd property; nonetheless, as you will later see, these oddities will not have any visible effect in our daily-life arithmetic.
In fact, these very oddities will prove themselves very useful in enabling us to properly incorporate number systems into our beautiful foundational theory.

From now on, unless otherwise stated, we will use Arabic numerals in order to refer to their corresponding elements in $\omega$. This is just notation.

Shortly, we will define the basic operations and relations in $\omega$, but we need to go through a very significant result before we can get there.
\end{para}

\begin{theorem}[Principle of recursive definition]
Let $X$ be a set with $a \in X$. Let $u:X\longrightarrow X$ be a function. There exists a unique function $f:\omega \longrightarrow X$ verifying $f(0) = a$ and $f(s(n)) = u(f(n))$.
\label{<+label+>}
\end{theorem}

\begin{proof}
We will prove this theorem by explicitly constructing $f$ as a set of ordered pairs in $\omega\times X$.

For that purpose, let us consider the set $F$ of all subsets $S\subseteq \omega \times X$ such that $(0,a)\in S$ and such that, whenever $(n,x)\in S$, then $(s(n),u(x)) \in S$.
The set $F$ is clearly non-empty for $\omega \times X\in F$.

We now take $f = \cap F$.
It should be clear that $f\in F$ and that $f$ is included in every element of $f$.
Thus, if we showed $f$ to be a function, we would have proved the result.
The reasoning behind this is simple.
If there existed another function $g$ verifying the conditions of the theorem, $g\in F$ and, therefore, $f\subseteq g$; nonetheless, assuming that $f$ is a function from $\omega$, $g$ can only be a function from $\omega$ if it is equal to $f$, because if it contained an additional ordered pair, we would have, for a certain $n\in \omega$ and $x,y\in X$ with $x\neq y$, both $(n,x)\in g$ and $(n,y)\in g$, which would mean that $g$ would not be a function.

Let us then show $f$ to be function! We can consider the set $N\subseteq \omega$ of elements $n\in \omega$ for which there exists a unique ordered pair in $f$ with first coordinate $n$.

If we had $0\not\in N$, then there would exist an element of the form $(0,b)\in f$ with $b\neq a$. Nonetheless, then $f' = f\setminus \{(0,b)\}\in F$ with $f'\subset f$, which would contradict our hypothesis that $f = \cap F$.
Analogously, If we had, for some $n\in \omega$, $n\in N$ but $s(n)\not\in N$, then there would exist a unique $x\in X$ such that $(n,x)\in f$ and there would exist an $y \in X$ distinct from $u(x)$ such that $(s(n), y)\in f$.
Were this the case, then it is trivial that $f\setminus \{(s(n),y)\}\in F$ and we would once again reach a contradiction.

Since $\omega$ is the smallest inductive set and we have shown that, for a set $N\subseteq \omega$, we have $0\in N$ and, whenever $n\in N$, $s(n) \in N$, we can conclude \emph{by induction} that $N = \omega$. This completes the proof.
\end{proof}

\begin{definition}
Let $m\in \omega$. Applying the principle of recursive definition, we can define two functions $s_m$ and $p_m$ from $\omega$ to $\omega$ verifying, on the one hand, $s_m(0) = m$ and $s_m(s(n)) = s(s_m(n))$, and, on the other hand, $p_m(0) = 0$ and $p_m(s(n)) = s_{p_m(n)}(n)$.

This enables us to define \emph{addition} $+$ and \emph{multiplication} $\cdot$ in $\omega$ as
\begin{align*}
+:\omega \times \omega & \longrightarrow \omega & \cdot : \omega \times \omega & \longrightarrow \omega \\
(x,y) &\longmapsto x+y = s_x(y), & (x,y) & \longmapsto x\cdot y = p_x(y),
\end{align*}
where, of course, we have used infix notation.

Just to complete our definitions in $\omega$, let us the relation $\leq$ in $\omega$ as $x\leq y$ if and only if $x\in y$ or $x = y$. Analogously, we define a relation $<$ such that $x < y$ if and only if $x \in y$. As was to be expected, if $x\leq y$ we say that $x$ is \emph{smaller than or equal to} $y$ and if $x < y$, it is said that $x$ is \emph{smaller than} $y$.
\label{<+label+>}
\end{definition}

\begin{definition}
Let $X$ be a set. An \emph{internal $n$-ary law of composition} on $X$ is a function $*:\times_{i=1}^n X \longrightarrow X$. The adjective ``internal'' is often dropped. Furthermore, for binary laws of composition, the adjective ``binary'' is often omitted too.
Laws of compositions are commonly referred to as operations.

Infix notation is often used with laws of composition: thus, $*(x,y)$ is written as $x * y$. We say that a law of composition $*$ is
\begin{itemize}
\item \emph{associative} if, for any $x,y,z\in X$, we have $(x*y)*z = x * (y * z)$,
\item \emph{commutative} if, for any $x,y \in X$, we have $x * y = y * x$,
\item \emph{left distributive} with respect to another law of composition $\square$ if, for any $x,y,a\in X$, $a * (x \square y) = (a * x) \square (a * y)$,
\item \emph{right distributive} with respect to another law of composition $\square$ if, for any $x,y,a\in X$, $ (x \square y) * a = (x * a) \square (y * a)$,
\item and \emph{distributive} with respect to another law of composition if it is both left distributive and right distributive.
\end{itemize}
Laws of composition often use \emph{additive notation} (i.e., use the symbol $+$ and infix notation) or \emph{multiplicative notation} (i.e., use the symbol $\cdot$ and infix notation). In multiplicative notation, the use of $\cdot$ may be replaced by juxtaposition: thus, $xy$ would be read as $x\cdot y$.

Clearly, the operations $+$ and $\cdot$ that we have defined on $\omega$ are laws of composition using, respectively, additive and multiplicative notation.
\label{<+label+>}
\end{definition}

\begin{lemma}
Let $n\in \omega$. The following seemingly irrelevant statements are true:
\begin{statements}
\item \label{wsubse} If $m\in n$, then $n\not\subseteq m$. In other words, no element of $\omega$ is a subset of any of its elements.
\item \label{wtrans} The set $\omega$ is transitive: if $m\in n$, then $m\subseteq n$.
\end{statements}
\label{lemmapeano}
\end{lemma}

\begin{proof}
\begin{parlist}
\item Let $N\subseteq \omega$ be the set of elements $n\in\omega$ such that, for any $m\in n$, we have $n\not\subseteq m$. It is clear that $0 \in N$ for $0 = \emptyset$. In addition, if $n \in N$, we can easily show that $s(n) = n \cup \{n\}\in N$.

The elements of $n \cup \{n\}$ are those of $n$ and $n$ itself. Can $n\cup\{n\}$ be a subset of $n$? For that to be the case, we would need to have $\{n\}\subseteq n$ and, therefore, $n\in n$, but, as $n\in N$ and $n\subseteq n$, it is clear that $n\not\in n$. Can $n\cup \{n\}$ be a subset of $x\in n$? Since $n\subseteq n \cup\{n\}$, if this were the casee, $n$ would also be a subset of $x$ thus contradicting the hypothesis that $n\in N$. We can conclude that $s(n) \in N$ and, by induction, that $N = \omega$.

\item We proceed, again, inductively. Let $N\subseteq\omega$ be the set of elements $n$ for which the result holds.
It is out of question that $0\in N$.
In addition, if we assume $n\in N$, we can easily prove that $s(n)\in N$.
The elements of $s(n)$ are $n$ and all the elements of $n$.
Clearly, since all the elements of $n$ belong to $s(n)$, we have $n\subseteq s(n)$.
Moreover, according to the inductive hypothesis, all the elements $x\in n$ are subsets of $n$, and, since $n \subseteq s(n)$, they must also be subsets of $s(n)$.
\end{parlist}
\end{proof}

\begin{theorem}
Let us consider the interpretation of the formal system $\mathsf{PA}$ of Peano Arithmetic using $\mathbb{N}$ as domain and having $\{\emptyset\} = 1$ as $\und{1}$; the set-theoretic relation $=$ as $\und{=}$; the set-theoretic function $s$ as $\und{s}$, and the set-theoretic functions $+\vert_{\mathbb{N}\times\mathbb{N}}$ and $\cdot\vert_{\mathbb{N}\times\mathbb{N}}$ as $\und{+}$ and $\und{\cdot}$ respectively.
This interpretation is a model.

Consequently, if $\mathsf{ZFC}$ is consistent, so is $\mathsf{PA}$.
\label{<+label+>}
\end{theorem}

\begin{proof}
We need to show that, in the interpretation of $\mathsf{PA}$ that we have constructed, all the axioms are true.
The axioms of equality are clearly verified, so we can get to work with the remaining non-logical axioms.

It is trivial that \ref{axpa1}, \ref{axpa2}, \ref{axpa3}, \ref{axpa4} and \ref{axpa5} are true by how we have defined the elements of the interpretation. Furthermore, the principle of mathematical induction captured in \ref{axpa6} trivially holds by considering, for any property $\phi(x)$, the set $\{x\in \omega\mid \phi(x)\}$ and noting that $\omega$ is inductive.

The only axiom that remains to be shown to be true is \ref{axpa1bis}.
Let us assume that two elements $m,n\in \omega$ verify $s(m) = s(n)$.
Under these conditions,
\[ m\in s(m) = s(n) = n \cup \{n\} \quad \tx{and} \quad n\in s(n) = s(m) = m \cup \{m\},\]
so either $m = n$ or both $m\in n$ and $n\in m$. In the latter case, we have $m\in n \in m$, but, applying \ref{lemmapeano}\ref{wtrans}, this implies that $n\subseteq m$ and, therefore, yields a contradiction with \ref{lemmapeano}\ref{wsubse}.
\end{proof}

\begin{para}
The following statements about $\omega$ can be shown to be true in any model of $\mathsf{ZFC}$ in an identical way to their analogues in $\mathbb{N}$:
\begin{axioms}[$\omega$]
\item \label{axw1} $(\forall x\in \omega)\qsep\lnot(s(x) = 0)$.
\item \label{axw2} $(\forall x,y\in \omega)\qsep s(x) = s(y) \limplies x = y$.
\item \label{axw3} $(\forall x\in \omega)\qsep x + 0 = x$.
\item \label{axw4} $(\forall x,y\in \omega)\qsep x + s(y) = s(x+y)$.
\item \label{axw5} $(\forall x\in \omega)\qsep x \cdot 0 = 0$.
\item \label{axw6} $(\forall x,y\in \omega)\qsep x \cdot s(y) = x\cdot y + x$.
\item \label{axw7} $(\forall A \subseteq \omega)\qsep (0 \in \omega \land  (\forall n\in \omega) (s(n) \in \omega) \limplies A = \omega)$.
\end{axioms}
It goes without saying that \ref{axw7} implies that any formula $P(x)$
is true for any $x\in \omega$ if proved to hold for $x = 0$ and, given an arbitrary $n\in \omega$, for $x = s(n)$ assuming $P(n)$.
\end{para}

\begin{lemma}
Assuming addition in $\omega$ to be associative, the following statements about $\omega$ are true:
\begin{statements}
\item \label{xp0i0} $(\forall x \in \omega)\qsep 0 + x = 0$.
\item \label{switchs} $(\forall x,y\in \omega)\qsep s(x) + y = s(x+y)$.
\end{statements}
\label{lemmaomplus}
\end{lemma}

\begin{proof}
\begin{parlist}
\item Let us show that $0 + n = 0$ for every $n\in \omega$ by induction on $n$.
For $n = 0$, the result is immediate from \ref{axw3}.
If we now assume this property to hold for an arbitrary $n\in \omega$, it follows from \ref{axw3} and \ref{axw4} that it is verified by $s(n)$ as
\[ 0 + s(n) = 0 + s(n+0) = 0 + n + s(0) = n + s(0) = s(n).\]
Thus, the result is true by the principle of mathematical induction, i.e., by \ref{axw7}.
Notice how we have also made an implicit use of the associativity hypothesis.

\item We shall prove that $s(x) + n = s(x+n)$ for any $x,n\in \omega$ using induction on $n$.
The result $n = 0$ is a direct consequence of \ref{axw3}. Assuming it to hold for an arbitrary $n\in \omega$, we can deduce from \ref{axw3} and \ref{axw4} that
\begin{align*}
s(x) + s(n) &= s(x) + s(n+0) = s(x) + n + s(0) = s(x+n) + s(0) \\
&= (x+s(n)) + s(0) = s( (x+s(n)) + 0) = s(x+s(n)),
\end{align*}
and, therefore, the result is true by induction.
\end{parlist}
\end{proof}

\begin{proposition}
Addition $+$ on $\omega$ is associative and commutative.
\end{proposition}

\begin{proof}
We shall first prove associativity: we will prove, by induction on $n\in \omega$, that $(x+y)+n = x + (y + n)$. For $n=0$, the result is obvious since, according to \ref{axw3},
\[ (x+y) + 0 = x+ y = x + (y + 0).\]
Let us then assume associativity to hold for an arbitrary $n\in \omega$. Regarding $s(n)$, the inductive hypothesis together with a repeated application of \ref{axw4} yields
\begin{align*}
(x+y) + s(n) &= s( (x+y) + n ) = s(x + (y + n)) = x + s(y+n) \\
&= x + (y+ s(n)).
\end{align*}

Now that we have associativity assured, let us deal with commutativity. We will show that any $x,n\in \omega$ verify $x+n = n+x$ using, once again, induction on $n$. The property is obvious for $n=0$ as $x + 0 = x$ and $0 + x = x$. These equalities are direct applications of \ref{axw3} and \ref{lemmaomplus}\ref{xp0i0}.
Assuming as inductive hypothesis that $x+n = n+x$, we have
\[ x+ s(n) = s(x+n) = s(n+x) = s(n) + x.\]
The fact that $s(n+x) = s(n) + x$ was shown in \ref{lemmaomplus}\ref{switchs}.
\end{proof}

\begin{lemma}
The following statements about $\omega$ are true:
\begin{statements}
\item \label{lemmaw0x} $(\forall x \in \omega)\qsep 0 \cdot x = 0$.
\item \label{lemmawps} $(\forall x,y\in \omega)\qsep s(x) \cdot y = x \cdot y + y$.
\end{statements}
\label{lemmawprod}
\end{lemma}

\begin{proof}
\begin{parlist}
\item As you should have expected, we proceed by induction on $x$. It is clear that $0\cdot 0 = 0$ from \ref{axw5}. If we now assume the result to hold for an arbitrary $x$, it follows from \ref{axw6} and \ref{axw3} that
\[ 0 \cdot s(x) = 0\cdot x + 0 = 0 + 0 = 0.\]

\item Once again, we proceed by induction on $y$. The result is immediate for $y = 0$ and, assuming it to hold for an arbitrary $y\in \omega$, we have
\[ s(x) \cdot s(y) = s(x) \cdot y + s(x) = x\cdot y + y + s(x) = x \cdot s(y) + s(y),\]
which shows that the result holds for $s(y)$.
\end{parlist}
\end{proof}

\begin{proposition}
Multiplication $\cdot$ on $\omega$ is distributive over addition, associative and commutative.
\label{<+label+>}
\end{proposition}

\begin{proof}
We will first prove left distributivity and use it in the proof of associativity and commutativity.
Then, right distributivity will follow from commutativity.

We need to show that, for every $n,x,y\in \omega$, we have $n\cdot (x+y) = n\cdot x + n\cdot y$.
As our notation suggests, we will proceed by induction on $n$. Using \ref{lemmawprod}\ref{lemmaw0x}, the base case $n=0$ for distributivity is trivial as $0\cdot (x+y) = 0 = 0\cdot x + 0\cdot y$.
Assuming left distributivity to hold for an arbitrary $n\in \omega$, we have
\begin{align*}
s(n) \cdot (x + y) &= n\cdot(x+y) + (x+y) = n\cdot x + n\cdot y + x + y \\
&= s(n) \cdot x + s(n) \cdot (y),
\end{align*}
where we have made implicit use of properties such as the associativity and commutativity of addition, \ref{lemmawprod}\ref{lemmawps} and \ref{axw6}.

In regard to associativity, let us show that $x\cdot(y \cdot n) = (x\cdot y) \cdot n$ by induction on $n$. For $n=0$, it follows from \ref{axw5} that
\[ x \cdot (y \cdot 0) = x\cdot 0 = 0 = (x\cdot y)\cdot 0.\]
Assuming associativity to hold for an arbitrary value of $n\in \omega$, we have, using left distributivity
\begin{align*}
x\cdot (y\cdot s(n) ) &= x\cdot (y \cdot n + y) = x \cdot (y \cdot n) + x \cdot y = (x\cdot y)\cdot n + x\cdot y \\
&= (x\cdot y) \cdot s(n),
\end{align*}
where we have also used \ref{axw6}.

Lastly, regarding commutativity, let us prove that $x \cdot n = n \cdot x$ for any by induction on $n$. The result is trivial for $n = 0$ and, assuming it to hold for an arbitrary $n$,
\[ x \cdot s(n) = x \cdot n + x = n \cdot x + x = s(n) \cdot x,\]
where we have made use of \ref{axw6} and \ref{lemmawprod}\ref{lemmawps}.
\end{proof}

\begin{para}
What we have just done is quite significant. We have proven some facts about $+$ and $\cdot$ without making any direct reference to the way in which they are defined: we have only used a collection of assumptions.

What we have seen in these proofs is an example of how the axiomatic method is used ``in practice'' within $\mathsf{ZFC}$.
\end{para}

\begin{definition}
Let $I$ and $X$ be sets. A function $x:I\longrightarrow X$ may be regarded as a \emph{family} of elements of $X$ \emph{indexed} by the set $I$.
If that is the case, given $i\in I$, we write $x_i$ instead of $x(i)$ and we denote the function $x$ by $\{x_i\}_{i\in I}$ or by $\{x_i\}_i$ for short.
On some occasions, the context might make it acceptable to also use the notation $\{x_i\}$.

Given a family $\{X_i\}_{i\in I}$ of sets, its union is defined as $\cup_{i\in I} X_i = \cup \op{im} X$.
If $I\neq \emptyset$, we define its intersection as $\cap_{i\in I} X_i = \cap \op{Im} X$.
Lastly, the cartesian product of the family is the set $\times_{i\in I} X_i$ of families $\{x_i\}_{i\in I}$ such that, for every $i\in I$, $x_i \in X_i$; or, in more symbolic terms,
\[ \bigtimes_{i\in I} X_i= \left\{x : I \longrightarrow \bigcup_{i\in i} X_i \mmid (\forall i\in\{1,\ldots,n)(x_i\in X_i)\right\}.\]
If the context allows for it, we may write $\cup_i$, $\cap_i$ and $\times_i$ instead of $\cup_{i\in I}$, $\cap_{i\in I}$ and $\times_{i\in I}$.
Naturally, a family $\{X_i\}_{i\in I}$ of sets is said to be pairwise disjoint if, for any $i,j\in I$ with $i\neq j$, $X_i \cap X_j = \emptyset$. Notice how we are only requiring that $i\neq j$, not that $X_i\neq X_j$.

If the index set of a family $\{x_i\}$ is $\mathbb{N}$, the family is said to be a \emph{sequence} of elements in $X$.

Of particular interest is the case where the index set is $I = \{1,\ldots,n\}\subseteq \mathbb{N}$.
In this situation, a family $\{x_i\}_{i\in I}$ is said to be an $n$-\emph{tuple} and it is represented writing down its terms explicitly as $(x_1,\ldots,x_n)$. Notice that tuples generalise ordered pairs.
If $(X_1,\ldots,X_n)$ is a tuple of sets, we can write
\begin{gather*}
\bigcup_{i\in I} X_i = \bigcup_{i=1}^n X_i = X_1\cup \cdots \cup X_n,\qquad \bigcap_{i\in I} X_i = \bigcap_{i=1}^n X_i = X_1 \cap \cdots \cap X_n,\\
\bigtimes_{i\in I} X_i = \bigtimes_{i=1}^n X_i = X_1\times\cdots\times X_n.
\end{gather*}

Given any tuple of sets $(X_1,\ldots,X_n)$, its cartesian product $X_1\times \cdots\times X_n$ is, according to the definition, the set of all tuples $(x_1,\ldots,x_n)$ having, for every $i\in\{1,\ldots,n\}$, $x_i\in X_i$.
The subsets of these cartesian products enable us to generalise the binary relations introduced in \ref{binrel} to $n$-ary relations.

While it might seem awkward to have two distinct objects representing the same concepts (2-tuples vs ordered pairs and 2-ary relations vs binary relations), this redundancy has no noticeable consequences in practice.
The different implementations of these concepts behave identically, so we might as well ignore their very nature.
In addition, we will use their different denominations interchangeably.
\label{<+label+>}
\end{definition}

\begin{definition}
Let $X$ be a set. A binary relation $\sim$ defined over $X$ is said to be an \emph{equivalence relation} if it satisfies the following properties:
\begin{itemize}
\item Reflexivity: for every $a\in X$, we have $a\sim a$.
\item Symmetry: for every $a,b \in X$, if $a\sim b$ then $b\sim a$.
\item Transitivity: for every $a,b,c\in X$, if $a\sim b$ and $b\sim c$, then $a\sim c$.
\end{itemize}
Given any $a\in X$, we define its \emph{equivalence class} as the set $[a] = \{x\in X \mid a\sim x\}$.
The \emph{quotient set} $X/{\sim}$ of $X$ by $\sim$ is the set of all the equivalence classes in $X$.
\label{<+label+>}
\end{definition}

\begin{definition}
Given a set $X$, a \emph{partition} $P$ of $X$ is a collection of pairwise disjoint subsets of $X$ such that $\cup P = X$.
\end{definition}

\begin{proposition}
Let $X$ be a set.
\begin{statements}
\item If $\sim$ is an equivalence relation over $X$, the quotient set $X/\sim$ is a partition of $X$.
\item If $P$ is a partition of $X$, then the equivalence relation $\sim$ defined in such a way that $x\sim y$ if and only if $x$ and $y$ belong to the same set in $P$ is an equivalence relation.
\end{statements}
\end{proposition}

\begin{proof}
\begin{parlist}
\item It is clear from reflexivity that, given any $a\in X$, we have $a\in [a]$.
Thus, it is obvious that $\cup (X/{\sim}) = X$, so we just need to show that any two distinct equivalence classes are disjoint.

Let $a,b\in X$ and let us assume the existence of an element $x\in [a]\cap [b]$.
Under these assumptions, we know that $x\sim a$ and $x\sim b$ and, therefore, by transitivity, $a\sim b$. Given any $x\in [a]$, we will have $x\sim a$, so, by transitivity, $x\sim b$ and, therefore, $x\in [b]$.
This proves that $[a]\subseteq [b]$ and it can be shown analogously that $[b] \subseteq [a]$.
Thus, two sets in the quotient set can only have a non-empty intersection if they are the same set, i.e., the elements of $X/{\sim}$ are pairwise disjoint.

Notice, by the way, how we have been constantly using the symmetry of equivalence relations throughout the proof.

\item Symmetry and reflexivity are obvious. Regarding transitivity, let $a,b, c\in X$ be such that $a\sim b$ and $a\sim c$. If we let $A\in P$ be the only subset of the partition to which $a$ belongs, as $a\sim b$, we know that $b\in A$. In addition, since $b\sim c$, that must mean that $c\in A$ and, therefore, that $a\sim c$. This proves that $\sim$ is an equivalence relation.
\end{parlist}
\end{proof}

\begin{definition}
Let $X$ be a set. A binary relation $\preceq$ defined on $X$ is said to be a \emph{partial order} if it verifies the following properties:
\begin{itemize}
\item Reflexivity: for any $a\in X$, we have $a\preceq a$.
\item Antisymmetry: for any $a,b\in X$, if $a\preceq b$ and $a\neq b$, then $b\not\preceq a$.
\item Transitivity: for any $a,b,c\in X$, if $a\preceq b$ and $b\preceq c$, then $a\preceq c$.
\end{itemize}
A partial order is said to be a \emph{total order} if, for any $a,b\in X$, we have $a\preceq b$ or $b\preceq a$.

A \emph{strict partial order} $\prec$ is a binary relation that verifies antisymmetry, transitivity and \emph{irreflexivity}, i.e., that for any $a\in A$, $a\not\prec a$.
Any partial order $\preceq$ can be used to define a strict partial order $\prec$ as $a \prec b$ if and only if $a\preceq b$ and $a\neq b$.
Conversely, any strict partial order defines a partial order in the obvious way.

Given a set $X$ endowed with a partial order $\preceq$, an element $a\in A$ is said to be a \emph{minimal} element if there exists no $x\in X$ such that $x\preceq a$.
Analogously, $a$ is said to be \emph{maximal} if no $x\in X$ exists such that $a\preceq x$.
The element $a$ is said to be a \emph{minimum} in $X$ if, for any $x\in X$, $a\preceq x$; moreover, it is said to be a \emph{maximum} if any $x\in X$ satisfies $x\preceq a$.
Notice how all these definitions need to be understood with respect to a particular partial order $\preceq$: an element may be maximal, minimal, a maximum or a minimum with respect to one partial order but not to another.

A total order $\preceq$ on a set $X$ is said to be a \emph{well-order} if any $A\subseteq X$ has a minimum. If a set $X$ has a well-order $\preceq$, it is said that $X$ is \emph{well-ordered} (by $\preceq$).
\label{<+label+>}
\end{definition}


%
% Chapter 2.7
%

\section*{2.7 Rates of Change Problems}

The average rate of change of \(y\) with respect to \(x\) over the interval \([x_1, x_2]\) is
$$ \frac{\Delta y}{\Delta x} = \frac{f(x_2)-f(x_1)}{x_2-x_1} $$
Its limit as \(\Delta x \to 0\) is the derivative \(f'(x)\), which is the instantaneous rate of change of \(y\) with respect to \(x\) written as
$$ \frac{dy}{dx} = \lim_{\Delta x \to 0}\frac{\Delta y}{\Delta x} $$

\subsection*{Physics}

If \(s = f(t)\) is the position function of a particle that is moving in a straight line, then \(\frac{\Delta s}{\Delta t}\) represents the \textbf{average velocity over a time period} \(\Delta t\).\\
\(v=\frac{ds}{dt}\) represents the instantaneous velocity (the rate of change of displacement with respect to time).\\
\(a=\frac{dv}{dt}\) represents the instantaneous acceleration (the rate of change of velocity with respect to time).
$$ a(t)=v'(t)=s''(t) $$

\subsection*{Chemistry}

Consider the reaction where \(A + B \to C\). The \textbf{instantaneous rate of reaction} is obtained by taking the limit of the average rate of reaction as the time interval \(\Delta t\) approaches 0.
$$ \text{rate of reaction} = \lim_{\Delta t \to 0}\frac{\Delta [C]}{\Delta t} = \frac{d[C]}{dt} $$

\subsection*{Biology}

Let \(n=f(t)\) be the number of individuals in an animal or plant population at time \(t\).
$$ \text{average rate of growth} = \frac{\Delta n}{\Delta t} = \frac{f(t_2)-f(t_1)}{t_2-t_1} $$
The \textbf{instantaneous rate of growth} is obtained from the average rate of growth by letting the time period \(\Delta t\) approach 0:
$$ \text{growth rate} = \lim_{\Delta t \to 0}\frac{\Delta n}{\Delta t} = \frac{dn}{dt} $$


\section{Encoding polytope placement into SMT}

Let's now focus on the task of placing polytopes into a parallelepiped.
Informally, the placement is carried out by means of a vector
for each polytope, which translates the polytope with respect to the
origin. The polytope packing problem is then the problem of finding
such placement vectors, which may or may not exist, depending on the
particular instance to solve.

Formally, given a set of polytopes $P_1, \ldots, P_n$, we need to find
placement vectors $\point{v_1}, \ldots, \point{v_n}$ such that for $0\leq i<j \leq n$
$$(\point{v_i}+P_i) \cap (\point{v_j}+P_j) \not= \emptyset$$
which means that for $0 \leq i<j \leq n$
$$(\point{v_j} - \point{v_i}) \not\in (P_i \ominus P_j).$$

The latter condition can be translated into a constraint satisfaction
problem. In particular $P_i \ominus P_j$ is a polytope that can be
defined by the intersection of a set of linear inequalities
\begin{eqnarray}
\nonumber
&        & c_{1_x} x + c_{1_y} y + c_{1_z} z \leq c_1 \\
\nonumber
& \wedge & c_{2_x} x + c_{2_y} y + c_{2_z} z \leq c_2 \\
\nonumber
& \wedge & \ldots \\
\nonumber
& \wedge & c_{n_x} x + c_{n_y} y + c_{n_z} z \leq c_n
\end{eqnarray}
each representing a faced of the polytope. The
area ``outside'' $P_i \ominus P_j$ is therefore
\begin{eqnarray}
\nonumber
&      & c_{1_x} x + c_{1_y} y + c_{1_z} z \geq c_1 \\
\nonumber
& \vee & c_{2_x} x + c_{2_y} y + c_{2_z} z \geq c_2 \\
\nonumber
& \vee & \ldots \\
\nonumber
& \vee & c_{n_x} x + c_{n_y} y + c_{n_z} z \geq c_n
\end{eqnarray}
Technically we should use strict inequalities $>$, however
we may allow polytopes to ``touch'' on a common point. At last
we can specify that $\point{v_j} - \point{v_i}$ is in the
area outside $P_i \ominus P_j$ with the following substitution
\begin{eqnarray}
\nonumber
&      & c_{1_x} (v_{j_x} - v_{i_x}) + c_{1_y} (v_{j_y} - v_{i_y}) + c_{1_z} (v_{j_z}-v_{i_z}) \geq c_1 \\
\nonumber
& \vee & c_{2_x} (v_{j_x} - v_{i_x}) + c_{2_y} (v_{j_y} - v_{i_y}) + c_{2_z} (v_{j_z}-v_{i_z}) \geq c_2 \\
\label{eq:smt1}
& \vee &\ldots \\
\nonumber
& \vee & c_{n_x} (v_{j_x} - v_{i_x}) + c_{n_y} (v_{j_y} - v_{i_y}) + c_{n_z} (v_{j_z}-v_{i_z}) \geq c_n
\end{eqnarray}
In addition to the constraints above, we need to specify the ``borders'' of the parallelepiped. Suppose
that the parallelepiped measures $l$,$w$,$h$ of length, witdth, and height respectively, and let
$x_\downarrow(P_i), x_\uparrow(P_i)$, the lowest and highest $x$ coordinate of $P_i$,
$y_\downarrow(P_i), y_\uparrow(P_i)$, the lowest and highest $y$ coordinate of $P_i$,
$z_\downarrow(P_i), z_\uparrow(P_i)$, the lowest and highest $z$ coordinate of $P_i$. Then we
need to encode for all $i$
\begin{eqnarray}
\nonumber
&& 0 \leq v_{i_x} + x_\downarrow(P_i) \wedge v_{i_x} + x_\uparrow(P_i) \leq l \\
\label{eq:smt2}
&& 0 \leq v_{i_y} + y_\downarrow(P_i) \wedge v_{i_y} + y_\uparrow(P_i) \leq w \\
\nonumber
&& 0 \leq v_{i_z} + z_\downarrow(P_i) \wedge v_{i_z} + z_\uparrow(P_i) \leq h
\end{eqnarray}
By encoding (\ref{eq:smt1}) and (\ref{eq:smt2}) into the SMT2~\cite{SMTLIB} language, we can find
values of $\point{v_i}$ for each $P_i$ that represent a polytope placement such
that the polytopes (may touch but) do not intersect and such that
is contained in the given parallelepiped.


\documentclass[12pt]{article}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage[lastexercise]{exercise}

\renewcommand{\QuestionNB}{(\alph{Question})\ }

\begin{document}

\section{Induction Proofs}
\begin{ExerciseList}
	\Exercise Show that for any \(n\in\mathbb N_1\)

	\[
		\frac{1}{1 \cdot 3} + \frac{1}{3 \cdot 5} + \cdots + \frac{1}{(2n-1)(2n+1)} = \frac{n}{2n+1}
	\]

	\Exercise
	Consider the sequence \((a_n)\) defined by
	\[
		\begin{cases}
			a_1=6, \\
			a_{n+1}  = \frac{a_n}{3}(1-e^{-a_n^2}), & \mbox{if \(n\ge 1\)} \\
		\end{cases}
	\]
	\Question{Show that \(a_n\in (0,6]\) for any \(n\in\mathbb N_1\)}
	\Question{Show that the sequence is decreasing}
	\Question{Prove that the sequence is convergent and determine it's limit}

	\Exercise
	Consider a sequence \((b_n)\) defined by
	\[
		\begin{cases}
			b_1=\frac{1}{2},\\
			b_{n+1}  = b_n(b_n - 1) + 1, & \mbox{if $n\ge 1$} \\
		\end{cases}
	\]
	\Question{Show that \(b_n\in (0,1)\) for any \(n\ge 1\)}
	\Question{Show that the sequence is increasing}
	\Question{Prove that the sequence is convergent and determine it's limit}

	\Exercise
	Consider the sequence \((u_n)\) defined by
	\[
		\begin{cases}
			u_1=1,\\
			u_{n+1}  = 1 + \frac{u_n}{2}, & \mbox{if $n\ge 1$} \\
		\end{cases}
	\]
	\Question{Show that $u_n \le 2$ for any $n\in\mathbb N_1$}
	\Question{Show that the sequence is increasing}
	\Question{Prove that the sequence is convergent and determine it's limit}

	\Exercise
	Let $\alpha\in [0,1]$, consider the sequence $(b_n)$ given by
	\[
		\begin{cases}
			b_1=\alpha,\\
			b_{n+1}  = b_n - b_n^4, & \mbox{if $n\ge 1$} \\
		\end{cases}
	\]
	\Question{Show that $b_n$ is monotonically decreasing}
	\Question{Show that $b_n\in [0,1]\forall n\in\mathbb N_1$}
	\Question{Prove that $b_n$ is convergent and calculate $\lim b_n$}

	\Exercise
	Consider the sequence $(a_n)$ defined by
	\[
		\begin{cases}
			a_1=1,\\
			a_{n}  = \frac{3a_{n-1}}{n}, & \mbox{if $n\ge 2$} \\
		\end{cases}
	\]
	Show that
	$$a_n=\frac{3^{n-1}}{n!}\quad\quad\forall n\ge 1$$
\end{ExerciseList}

\section{Limits I}
\begin{ExerciseList}
	\Exercise Calculate or show that it does not exist in $\overline{\mathbb R}$
	\Question $\lim \frac{(n+1)!-n!}{n!(n+2)}$
	\Question $\lim (-1)^n \frac{10^n}{n!}$
	\Question $\lim \frac{5n!+5n}{n^n + 2}$
	\Question $\lim \sqrt{\frac{e^n+2}{n!}}$
	\Question $\lim \frac{(-1)^n n}{n!+4}$
	\Question $\lim \frac{\sqrt[3]{n+4}}{\sqrt[3]{n}+4}$
	\Question $\lim \sqrt[n]{\frac{n+2^n}{2+5^n}}$
	\Question $\lim \frac{3n^4-2n}{(n^2 + 3)(1+5n^2)}$
	\Question $\lim (\frac{2}{3}+\cos(3n\pi))^n$
	\Question $\lim \frac{3+\cos(e^{-n})}{n+\sqrt{n!}}$
	\Question $\lim \frac{(-1)^n n }{n! + 5}$
	\Question $\lim \frac{\cos{(n\pi)}}{\sin{(1/n\pi)}+1}$
	\Question $\lim \frac{e^{3n}+1}{2^n+n^2}$
	\Question $\lim \frac{\sqrt{n} + n^3}{(n+\sqrt{n})(n^2 + n^{3/2})}$
	\Question $\lim (\cos{(\frac{\pi}{4}+n\pi)}+1)^n$
	\Question $\lim \frac{(3n)!}{n!(2n)!}$
	\Question $\lim \frac{\arccos(1/n)}{\cos(\pi/n)}$
	\Question $\lim \frac{2n! + 3^n}{n^{50} + n!}$
	\Question $\lim \sqrt[n]{\frac{\arctan n}{1+e^n}}$
\end{ExerciseList}

\section{Limits II}
\begin{ExerciseList}
	\Exercise Calculate or show that it does not exist in $\overline{\mathbb R}$
	\Question $\lim_{x\to 0}\frac{e^x - 1}{x-e^{3x} + 1}$
	\Question $\lim_{x\to +\infty}(3x^2+1)^{1/x}$
	\Question $\lim_{x\to 1}\frac{x-1}{\tan(x-1)}$
	\Question $\lim_{x\to 2}(x-1)^{\frac{1}{2-x}}$
	\Question $\lim_{x\to -\infty}xe^{x^2}$
	\Question $\lim_{x\to +\infty}(\log(2x))^{1/x}$
	\Question $\lim_{x\to 1}\frac{1}{x-1}\int_0^{\log x}{e^{\sin t}}dt$
	\Question $\lim_{x\to 0}(x^2 + 1)^{\frac{1}{x^2}}$
	\Question $\lim_{x\to1+}\frac{\arctan(x-1)}{x^3-3x+2}$
	\Question $\lim_{x\to 0}\frac{\arctan{2x}}{\tan x}$
	\Question $\lim_{x\to +\infty}(2e^x + 1)^{\frac{1}{x}}$
	\Question $\lim_{x\to 0}\frac{1-\cos{x}}{\arcsin{x}}$
	\Question $\lim_{x\to 0} x(\log x)^2$
	\Question $\lim_{x\to 0}\frac{2\int_0^x(1-e^{t^2})dt}{x^2}$
\end{ExerciseList}

\section{Functions}
\begin{ExerciseList}
	\Exercise Consider the function $f\colon \mathbb R \to \mathbb R$
	$$f(x) = \begin{cases} -x e^x, & \mbox{if $x<0$} \\ \alpha\arctan(x^2-2x), &\mbox{if $x\ge 0$} \\ \end{cases}$$
	Where $\alpha$ is a real constant.
	\Question Calculate $\lim_{x\to -\infty}f(x)$ and $\lim_{x\to +\infty}f(x)$
	\Question Justify whether $f$ is continuous
	\Question Let $f'_+(0)=-2\alpha$, determine $\alpha$ such that $f$ is differentiable at $x=0$. Justify that $f$ is differentiable in $\mathbb R$ and calculate it's derivative
	\Question Determine the local extrema and the monotonous intervals of $f$ (with $\alpha = 1/2$)
	\Question Indicate the co-domain of $f$ (with $\alpha=1/2$)

	\Exercise The function $h\colon \mathbb R \to \mathbb R$
	$$h(x)=\begin{cases}x^2, & \mbox{if $x\in\mathbb Q$} \\ 0, & \mbox{if $x\in\mathbb R\setminus\mathbb Q$}\end{cases}$$
	is continuous at only one point. Is it differentiable at that point?

	\Exercise Consider the function $f\colon \mathbb R\setminus\{0\}\to\mathbb R$
	$$f(x) = \begin{cases} \frac{e^{-x}}{x+1}, & \mbox{if $x>0$} \\ 2+\log(1-x), & \mbox{if $x<0$} \\ \end{cases}$$
	\Question Show that $f$ is continuous. Will $f$ be extensible by continuity on the point $x=0$?
	\Question Calculate in $\overline{\mathbb R}$
	$$\lim_{x\to -\infty}f(x), \qquad \lim_{x\to -\infty}f(x)$$
	\Question Show that $f$ is differentiable, and calculate $f'(x)\quad\forall x\in\mathbb R\setminus \{ 0\}$.
	Use the result to determine the monotonous intervals of $f$
	\Question Determine the co-domain of $f$

	\Exercise Consider the function $f\colon \mathbb R\to \mathbb R$ such that
	$$f(x)=\begin{cases}-x^2e^x, & \mbox{if $x\le 0$} \\ (1-\cos x)e^{-x} & \mbox{if $x>0$} \\ \end{cases}$$
	\Question Calculate, if it exists in $\overline{\mathbb R}$, $\lim_{x\to +\infty}f(x)$
	\Question Show that $f$ is continuous at the point $x=0$ and calculate,if it exists, $f'(0)$
	\Question Determine the differentiability domain of $f$, and calculate it's derivative.
	\Question Show that $f$ has one and only one local extremum on the interval $(-\infty, 0)$.
	Prove that it's an absolute minimum of $f$ in $\mathbb R$.
	\Question Show that the co-domain of $f$ is a closed and limited interval.

	\Exercise Given a function $f\colon \mathbb R\setminus\{ 0 \} \to \mathbb R$ such that
	$$f(x)=\begin{cases}\alpha + \log{\frac{1}{1+x^2}} & \mbox{if $x<0$} \\ \arctan{\frac{1}{x}} & \mbox{if $x>0$} \\ \end{cases}$$
	\Question Calculate, if it exists, $\lim_{x\to -\infty}$ and $\lim_{x\to +\infty}$
	\Question Determine $\alpha\in\mathbb R$ such that $f$ is extensible by continuity at $x=0$
	\Question Determine the domain of differentiability of $f$, and calculate it's derivative.
	\Question Determine the monotony intervals of $f$, as well as it's extrema, if they exist.
	\Question Assuming $\alpha=0$, what will be the co-domain of $f$?

	\Exercise Consider the function $f\colon\mathbb R\setminus\{ -\pi/2,\pi/2 \}\to\mathbb R$ given by
	$$f(x)=\begin{cases}\frac{\pi}{2(\sin{x}+1)}, & \mbox{if $|x| < \pi/2$} \\ \arctan(e^{x-\pi/2}), & \mbox{if $|x|>\pi/2$} \\ \end{cases}$$
	\Question Study $f$ with regards to continuity.
	\Question Calculate $\lim_{x\to -\infty}f(x)$ and $\lim_{x\to +\infty}f(x)$
	\Question Decide whether $f$ extendable by continuity at $-\pi/2$ and $\pi/2$
	\Question Calculate the derivative $f'$, and determine it's monotonous intervals.

	\Exercise Consider the function $f\colon (-1,+\infty)\to\mathbb R$:
	$$f(x)=\begin{cases}\log{\sqrt{1-x^2}}, & \mbox{if $-1<x\le 0$} \\ x^2e^{1-x^2}, & \mbox{if $x>0$} \\ \end{cases}$$
	\Question Study $f$ with regards to continuity
	\Question Calculate $\lim_{x\to -1^+}f(x)$ and $\lim_{x\to +\infty}f(x)$
	\Question Determine the derivative $f'$
	\Question Determine the monotony intervals of $f$, as well as the local maxima and minima.

\end{ExerciseList}

\section{Primitives}
\begin{ExerciseList}
	\Exercise Calculate the primitive of the function $$\frac{2x+3}{x^2+2}$$ that vanishes at $x=0$

	\Exercise Determine a primitive for the following functions:
	\Question $\frac{1}{\sqrt{e^x - 1}}$
	\Question $\frac{3x}{1+x^4}$
	\Question $\frac{3x}{\sqrt{4+x^2}}$
	\Question $\frac{1}{x^2+4}$
	\Question $\frac{\sin{\sqrt[3]{x}}}{\sqrt[3]{x}}$
	\Question $\frac{\cos(\log x)}{x}$
	\Question $\log(1+x)^2$

	\Exercise Determine the function $g\colon \mathbb R \to \mathbb R$ that satisfies
	$$\begin{cases}g'(x)=\frac{3x^2 + 1}{x^2 + 2} & \mbox{$\forall x\in \mathbb R$} \\ g(0)=2\end{cases}$$

	\Exercise Calculate the function $f\colon \mathbb R\to\mathbb R$ that satisfies
	$$\forall x\in\mathbb R\quad f'(x)=\frac{1+x}{9+x^2}\quad\mbox{and}\quad f(0)=\log(3)$$

	\Exercise Write the general expression for the primitives in $(0,+\infty)$ of
	$$\frac{1}{x\sqrt{x+1}}$$
\end{ExerciseList}

\section{Integration}
\begin{ExerciseList}
	\Exercise Calculate the area of region delimited by:
	\Question $\{(x,y)\in\mathbb R^2 \colon x\le y\le -x^2+2\}$
	\Question $\{ (x,y)\in\mathbb R^2 \colon x^2-\pi x\le y \le -\sin{x} \}$
	\Question $\{ (x,y)\in\mathbb R^2 \colon 0\le x\le 1,\frac{\pi}{4}x\le y\le\arctan x \}$
	\Question The triangle described by the lines $y=x,\enspace y=2x,\enspace y=3x-2$

	\Exercise Compute the following integrals
	\Question $\int_0^{\frac{\pi}{2}}\frac{\sin x \cos x}{2+\sin^2x}dx$
	\Question $\int_0^1{\frac{4x}{4+x^2}}dx$
	\Question $\int_{\log 2}^{\log 3}{\frac{e^x}{(e^x-1)^2}}dx$
	\Question $\int_1^2{\frac{1}{x(4+log^2(x))}}dx$
	\Question $\int_{-1}^0\frac{\log(x+2)}{(x+2)^2}dx$
	\Question $\int_0^1{(\frac{x}{2}+x^3)\arctan{x}}\: dx$
	\Question $\int_1^{\log{2}}{\frac{e^x}{\sqrt{e^x - 1}}}dx$
	\Question $\int_2^7{\frac{1}{(x+1)\sqrt{x+2}}}dx$

	\Exercise Determine the value of the constant $c\in\mathbb R$ such that $f'(0) = 0$, with
	$$f(x)=\int_x^{cx+2}{e^{-t^2}}dt$$

	\Exercise Let $g\in C(\mathbb R)$ be an odd function, and $f$ the function given by
	$$f(x)=\int_1^{x^2-1}{xg(t)}dt$$
	Calculate $f'(x)$ for any $x\in\mathbb R$, and show that $f'(0)=0$

	\Exercise Determine a function $f\colon\mathbb R \to \mathbb R$, differentiable and non-null, such that
	$$f^3(x)=\int_{\pi/2}^{x}{\frac{\cos{t}}{2-\sin{t}}}dt$$

\end{ExerciseList}

\section{Taylor Polynomial}
\begin{ExerciseList}
	\Exercise Let $f\in C^2(\mathbb R)$ and $g(x)=f(e^x)\forall x\in\mathbb R$.
	Let $3-x+2(x-1)^2$ be the second order Taylor polynomial of $f$ relative
	to point $1$, determine the second order MacLaurin polynomial of $g$.

	\Exercise Let $f\in C^4(\mathbb R)$ be such that its third degree Taylor
	polynomial at point $2$ is constant. Given that $f^{(4)}(2)=1$, justify that
	$f(2)$ is an extremum of $f$ and classify it.

	\Exercise Let $h\colon \mathbb R \to \mathbb R$ be a differentiable function
	, and consider $f\colon \mathbb R\to\mathbb R$ given by
	$$f(x)=\int_x^{x^2}{h(t)}dt$$
	\Question Show that $f''(1)-2f'(1)=3h'(1)$
	\Question If $p_1(x)=x-1$ is the first order Taylor polynomial of $h$ at point $1$,
	show that $f$ has a local minimum at $x=1$
\end{ExerciseList}

\section{Series}
\begin{ExerciseList}
	\Exercise Analyse the following series and determine whether they are absolutely convergent, conditionally convergent, or divergent
	\Question $\sum_{n=1}^{+\infty} \frac{1}{3+2n}$
	\Question $\sum_{n=1}^{+\infty} \frac{\cos(2n)}{n^3}$
	\Question $\sum_{n=1}^{+\infty} \frac{n^3}{e^{2n}}$
	\Question $\sum_{n=1}^{\infty}  \frac{2+n\sqrt{n}}{1+3n^4}$
	\Question $\sum_{n=2}^{\infty}  \frac{3^n(1+(-1)^n)}{(2\pi)^{n+1}}$
	\Question $\sum_{n=2}^{+\infty} \log(\arctan(n+1))-\log(\arctan n)$
	\Question $\sum_{n=2}^{+\infty} \frac{(-1)^n}{\sqrt[4]{n^3 + 1}}$
	\Question $\sum_{n=1}^{\infty}  (\frac{n}{n+1} - \frac{n+1}{n+2})$
	\Question $\sum_{n=0}^{\infty}  2^{-3n}$
	\Question $\sum_{n=1}^{+\infty}\frac{\sqrt[3]{n^3+2}}{n^2\sqrt{n+9}}$
	\Question $\sum_{n=1}^{+\infty}\frac{\arctan{n}}{e^n}$
	\Question $\sum_{n=1}^{+\infty}(-1)^{n+1}\frac{\sin(2n)}{n^3}$

	\Exercise Show that if the series $\sum_{n=1}^{+\infty}a_n$ converges, then
	$\sum_{n=1}^{+\infty}\frac{2n-1}{2n}$ also converges.

	\Exercise Determine the values of $x\in\mathbb R$ for which the following series
	is absolutely convergent, conditionally convergent, or divergent
	\Question $\sum_{n=1}^{+\infty}\frac{n^n(x-1)^n}{(n+3)^n}$
	\Question $\sum_{n=0}^{\infty}\frac{(x-2)^n}{\sqrt{n}+1}$
	\Question $\sum_{n=1}^{\infty}\frac{\log n}{2^n}(x-2)^n$
\end{ExerciseList}

\section{Proofs}
\begin{ExerciseList}
	\Exercise Show that for any $x>0$
	$$\frac{x}{1+x} < \log(1+x) < x$$
	Hint: Mean Value Theorem

	\Exercise Let $h\in C(\mathbb R)$ such that $h(x) = h(x+2)$ for all $x\in\mathbb R$, and
	$$\phi(x) = \int_0^x{h(t)dt}-\int_0^{x+2}{h(t)dt}$$
	Prove that $\phi$ is identically $0$ if and only if $\int_0^2 h(t)dt = 0$

	\Exercise Let $(a_n)$ be a limited sequence with terms in $(1,+\infty)$, and
	$(b_n)$ another sequence such that $$b_n = \frac{na_n}{n+a_n}\qquad n\in\mathbb N_1$$
	Show that $(b_n)$ has convergent subsequences.

	\Exercise Let $f$ and $g$ be real functions, defined and continuous on the interval $[a,b]$, such that
	$$\int_a^b{f(t)}dt = 2\int_a^b{g(t)}dt$$
	Show that there is $c\in[a,b]$ such that $f(c)=2g(c)$

	\Exercise Let $h\colon [0,+\infty)\to\mathbb R$ be a continuous function
	for which $\lim_{x\to +\infty}h(x)=c\in\mathbb R$.
	\Question Show that there is at least one solution for $h(x)=\frac{x^2-1}{x}$
	in the interval $(0,+\infty)$
	\Question Suppose $h$ is differentiable in $(0,+\infty)$ and
	$$\forall x\in(0,+\infty)\quad h'(x<1)$$
	Show that $h(x)=\frac{x^2-1}{x}$ has one and only one solution

	\Exercise Let $f\colon [0,1]\to\mathbb R$ be a continuous function. Show that
	for all $x\in[0,1]$
	$$\int_0^x{\int_0^u{f(t)}dt}du=\int_0^x{(x-t)f(t)}dt$$

	\Exercise Let $f$ be a function integrable in $[0,1]$. Show that
	$$\lim_{n\to\infty}\int_0^1{x^{n+1}f(x)}dx = 0$$

	\Exercise Let $G\colon \mathbb R\to \mathbb R$ be a continuous function.
	Suppose there is $m\in\mathbb R$ such that $\{ x\in\mathbb R\colon G(x)\le m \}$
	is limited and not empty.
	Show that $G$ has an absolute minimum.

	\Exercise Let $g\colon \mathbb R \to \mathbb R$ be a continuous function such that
	$$\lim_{x\to -\infty}g(x)=\alpha>0$$
	Show that
	$$\lim_{x\to -\infty}\int_x^0 e^{x-t}g(t)dt=\alpha$$
	Hint: Show that $\lim_{x\to -\infty}\int_x^0 e^{-t}g(t)dt=+\infty$

	\Exercise Let $\psi\colon\mathbb R \to \mathbb R$ be a continuous function
	such that $\lim_{x\to +\infty}\psi(x)=+\infty$ and $\lim_{x\to -\infty}\psi(x)=-\infty$.
	Decide if $G\colon\mathbb R \to \mathbb R$ defined as
	$$G(x)=\frac{\psi(x)}{1+\psi^2(x)}$$
	Has a maximum and a minimum.

	\Exercise Show that if $\sum_{n=1}^{+\infty}a_n$ and $\sum_{n=1}^{+\infty}b_n$
	are convergent series with positive terms, then $\sum_{n=1}^{+\infty}a_n b_n$
	is a convergent series. Will this hold true if $\sum_{n=1}^{+\infty}a_n$ and $\sum_{n=1}^{+\infty}b_n$
	are series with an oscillating sign?

	\Exercise Let $I$ be an open interval, $a\in I$ and $\rho\colon I\to\mathbb R$
	a function 2-times differentiable, such that $\rho''(x)>0$ for any $x\in I$. Also let
	$$g(x)=\rho'(a)(x-a)+\rho(a)$$
	Show that $\rho(x)>g(x)$ for all $x\in I\setminus\{ a \}$
\end{ExerciseList}
\section{Notation}
\begin{enumerate}
	\item  $[a,b]$ for a closed interval, $(a,b)$ for an open one.
	\item $\overline{\mathbb R} = \mathbb R \cup \{-\infty, +\infty\}$
\end{enumerate}
\end{document}


\subsection{Clarke generalized gradients}\label{subsec:clarke_gradients}

Let \( X \) be a Banach space and \( f: X \to \BbbR \) be locally Lipschitz.

\begin{definition}\label{def:clarke_gradient}\mcite[def. 10.3]{Clarke2013}
  Let \( x \in X \) and \( U \subseteq X \) be a neighborhood of x where \( f \) is \( L \)-Lipschitz, i.e.

  \begin{equation*}
    \forall y, z \in U, \abs{f(y) - f(z)} \leq L \norm{y - z}.
  \end{equation*}

  We use the Clarke generalized \hyperref[def:nonsmooth_derivatives/clarke]{derivative},
  \begin{equation*}
    f^\circ(x)(h) \coloneqq \limsup_{\substack{y \to x \\ t \downarrow 0}} \frac {f(y + th) - f(y)} t
  \end{equation*}

  We define the \term{generalized gradient of \( f \) at \( x \)} to be the set
  \begin{equation*}
    \partial_C f(x) \coloneqq \{ x^* \in X^* \colon \forall h \in X, f^\circ(x)(h) \geq \inprod {x^*} h \}.
  \end{equation*}

  We say that the vector \( h \) is a \term{descent direction of \( f \) at \( x \)} if
  \begin{equation*}
    \limsup_{t \downarrow 0} \frac {f(x + th) - f(x)} t < 0.
  \end{equation*}
\end{definition}

\begin{proposition}\label{thm:clarke_derivative_exists}
  The generalized derivative of a locally Lipschitz function \( f: X \to \BbbR \) exists for every \( x \in X \).
\end{proposition}
\begin{proof}
  Let \( x, h \in X \) and let \( U \) be a neighborhood of \( x \) where the Lipschitz condition holds with the constant \( L_U \). Then there exists \( \delta_0 > 0 \) such that \( B(x, \delta_0) \subseteq U \).

  Define \( \delta_1 \coloneqq \frac 1 2 \min \left\{\delta_0, \frac {\delta_0} {\norm h} \right\} < \delta_0 \), so that for \( y \in B(x, \delta_1) \) and \( t \in (0, \delta_1) \) we have
  \begin{balign*}
    \norm{(y + th) - x}
    \leq
    \norm{y - x} + t \norm h
    \leq
    \delta_1 + \delta_1 \norm h
    \leq
    \begin{cases}
      \frac {\delta_0} 2 (1 + \norm h),           & \norm h \leq 1 \\
      \frac {\delta_0} {2 \norm h} (1 + \norm h), & \norm h > 1.
    \end{cases}
  \end{balign*}

  In both cases we get that \( y + th \in B(x, \delta_0) \).

  The generalized derivative in \( x \) in the direction \( h \in X \) is then norm-bounded by
  \begin{balign*}
    \abs{f^\circ(x)(h)}
    =
    \abs{\limsup_{\substack{y \to x                             \\ t \downarrow 0}} \frac {f(y + th) - f(y)} t}
    =
    \abs{\lim_{\delta \to 0} \sup_{\substack{y \in B(x, \delta) \\ t \in (0, \delta)}} \frac {f(y + th) - f(y)} t}
    \leq                                                        \\ \leq
    \abs{\sup_{\substack{y \in B(x, \delta_1)                   \\ t \in (0, \delta_1)}} \frac {f(y + th) - f(y)} t}
    \leq
    \sup_{\substack{y \in B(x, \delta_1)                        \\ t \in (0, \delta_1)}} \frac {\abs{f(y + th) - f(y)}} t
    \leq                                                        \\ \leq
    \sup_{\substack{y \in B(x, \delta_1)                        \\ t \in (0, \delta_1)}} \frac {\norm{(y + th) - (y)}} t
    =
    \norm h.
  \end{balign*}

  The fact that \( f \) is locally Lipschitz gave us that the supremum is taken over a bounded set and thus the generalized derivative exists.
\end{proof}


\subsection{$F_1$ Score}
\label{chp:fundamentals:sec:metrics:subsec:f1_score}

We previously introduced the two metrics \textit{precision} and \textit{recall}.
From their definition in \cref{eq:precision} and \cref{eq:recall} one concludes immediately that both are only optimal if an algorithm manages to select \textit{exclusively} \acp{TP} which would lead to $prec = rec = 1$.
However, in most scenarios an algorithm does not select \acp{TP} exclusively.
If that is the case, it is proven that a trade-off exists among whether one wants to identify all available relevant items (recall) or all of the selected items should be relevant (precision) \parencite{Gordon:1989}.
Here we do not focus on the formal derivation of this well known trade-off, instead excellent derivations can be found in \textcites{Gordon:1989}{Zhu:2004}.
To measure and express this trade-off one can build the \textit{harmonic mean} of precision and recall which is called \textit{$F_1$ score} \parencite{Powers:2011}.
This metric is defined by the following equation:

\begin{equation}\label{eq:f1_score}
    F_1 = 2 \frac{prec \cdot rec}{prec+rec}
\end{equation}

For the example data of \cref{fig:metrics:tp_vis} the algorithm achieves an F$_1$ score of $F_1 = 2 \frac{\frac{3}{7} \cdot \frac{1}{4}}{\frac{3}{7}+\frac{1}{4}} \approx 0.32$.
The $F_1$ score's generalization for a multi-class scenario is called \textit{macro $F_1$ score} \parencite{Opitz:2019}.


 In this model, we suppose that the data was generated using linear recursive formula
 \begin{equation}
  \label{eq:varx_stationary}
	x_t = \mu + \sum\limits_{q=1}^{\mathrm{xmem}} A_q x_{t-q} + \sum\limits_{p=0}^{\mathrm{umem}} B_p u_{t-p} + \varepsilon_t, \forall t = \mathrm{xmem}, \mathrm{xmem}+1, \dots, T-1,
 \end{equation}
 where given data $x_t \in \mathbb{R}^{\mathrm{xdim}}, t = 0,\dots,T-1$ are stored in column vectors, $A_q \in \mathbb{R}^{\mathrm{xdim},\mathrm{xdim}}$ are unknown coefficients (matrices) corresponding to previous $\mathrm{xmem}$ time-steps and \todo{write here something funny about variables in the model}.

 Let us denote the number of equations in \eqref{eq:varx_stationary} by $m = T-\mathrm{xmem}$.
 Moreover, we define
 \begin{displaymath}
  \begin{array}{rcl}
   X & = & [x_{\mathrm{xmem}}, x_{\mathrm{xmem}+1}, \dots, x_{T-1}] \in \mathbb{R}^{\mathrm{xdim},m} \\[5mm]
   M & = & [\mu, A_1, A_2, \dots, A_{\mathrm{xmem}}, B_0, B_1, \dots, B_{\mathrm{umem}} ] \in \mathbb{R}^{\mathrm{xdim},1+\mathrm{xmem}\cdot\mathrm{xdim}+(\mathrm{umem}+1)\cdot\mathrm{udim}}\\[5mm]
   Z & = & \left[
	\begin{array}{ccccc}
	 1 & 1 & 1 & & 1 \\ \hdashline[2pt/2pt]
	 x_{\mathrm{xmem}-1} & x_{\mathrm{xmem}} & x_{\mathrm{xmem}+1} & & x_{T-1} \\
	 \vdots & \vdots & \vdots & \dots & \vdots \\
	 x_0 & x_1 & x_2 & & x_{T-\mathrm{xmem}} \\ \hdashline[2pt/2pt]
	 u_{\mathrm{xmem}} & u_{\mathrm{xmem}+1} & u_{\mathrm{xmem}+2} & & u_{T-1} \\
	 u_{\mathrm{xmem}-1} & u_{\mathrm{xmem}} & u_{\mathrm{xmem}+1} & & u_{T-2} \\
	 \vdots & \vdots & \vdots & & \vdots
    \end{array}
   \right] \in \mathbb{R}^{1+\mathrm{xmem}\cdot\mathrm{xdim}+(\mathrm{umem}+1)\cdot\mathrm{udim},m} \\[5mm]
   \varepsilon & = & [\varepsilon_{\mathrm{xmem}}, \varepsilon_{\mathrm{xmem}+1}, \dots, \varepsilon_{T-1}] \in \mathbb{R}^{\mathrm{xdim},m}
  \end{array}
 \end{displaymath}

 Then \eqref{eq:varx_stationary} is equivalent to\footnote{please, notice that both of left side and right side are matrices}
 \begin{equation}
  \label{eq:varx_stationary_matrix}
  X = MZ + \varepsilon,
 \end{equation}
 where $M$ is matrix of unknown parameters of the model \eqref{eq:varx_stationary}.
 Now we will find $M$ as \emph{the best} solution, i.e. we minimize the size of error $\varepsilon$ in \eqref{eq:varx_stationary_matrix}\footnote{please, notice that we are talking about matrix norms}
 \begin{displaymath}
  \label{eq:eq:varx_stationary_matrix_eps}
  \Vert \varepsilon \Vert = \Vert X - MZ \Vert ~~ \rightarrow ~~ \min\limits_{M}.
 \end{displaymath}
 or equivalently\footnote{\todo{the trace and matrix norms should be discussed}}
 \begin{displaymath}
   \bar{M} =  \arg \min\limits_{M} \Vert X - MZ \Vert =  \arg \Vert X - MZ \Vert^2 =  \arg \min\limits_{M} \underbrace{\tr\Vert X - MZ \Vert^2}_{= L(M)}.
 \end{displaymath}
 The optimization problem with object function $L(M): \mathbb{R}^{\mathrm{xdim},1+\mathrm{xmem}\cdot\mathrm{xdim}+(\mathrm{umem}+1)\cdot\mathrm{udim}} \rightarrow \mathbb{R}^{+}_0$ could be simplified
 \begin{displaymath}
  \begin{array}{rcl}
   \min L(M) & = & \min \tr \Vert X - MZ \Vert^2 = \min \tr (X-MZ)^T(X-MZ) \\
        & = & \min \tr \left( X^TX - X^TMZ - (MZ)^TX + (MZ)^T MZ \right) \\
        & = & \min \tr \left( X^TX - X^TMZ - Z^TM^TX + Z^T M^T MZ \right) \\
        & = & \min \tr (X^TX) - \tr (X^TMZ) - \tr (Z^TM^TX) + \tr (Z^T M^T MZ)
  \end{array}
 \end{displaymath}
 We consider the neccessary optimality condition $\frac{\partial L(M)}{\partial M} = 0$, therefore we have to compute the derivatives of addends in the previous formula.
 These derivatives follow (using \cite{matrix_cookbook}\todo{add cookbook reference}).
 \begin{displaymath}
  \begin{array}{rcl}
   \frac{\partial \tr (X^TX)}{\partial M} & = & 0 \\
   \frac{\partial \tr (X^TMZ)}{\partial M} & = & XZ^T \\
   \frac{\partial \tr (Z^TM^TX)}{\partial M} & = & XZ^T \\
   \frac{\partial \tr (Z^TM^T M Z)}{\partial M} & = & M(ZZ^T) + M (ZZ^T) = 2MZZ^T
  \end{array}
 \end{displaymath}
 Therefore the neccessary optimality condition of the problem \eqref{eq:eq:varx_stationary_matrix_eps} is given by
 \begin{displaymath}
  \frac{\partial L(M)}{\partial M} = 0 ~~~ \Leftrightarrow ~~~ -2XZ^T + 2M(ZZ^T) = 0,
 \end{displaymath}
 which could be written in the form of the system of linear equations with multiple right-hand side vectors as
 \begin{equation}
  \label{eq:varx_stationary_system}
  (ZZ^T) M^T = ZX^T,
 \end{equation}
 where $M^T$ is the matrix of unknown parameters of the original model \eqref{eq:varx_stationary}.

 \subsection{Checking equations by example}

 Let us consider a problem with $\mathrm{xdim} = 2, \mathrm{udim} = 1, \mathrm{xmem} = 2, \mathrm{umem} = 0, T = 5$. Then $m = 3$ and
 \begin{displaymath}
  X \in \mathbb{R}^{2,3}, ~ M \in \mathbb{R}^{2,6}, ~ Z \in \mathbb{R}^{6,3}, ~\varepsilon \in \mathbb{R}^{2,3}.
 \end{displaymath}
 Please, see Fig. \ref{fig:varx1}, where we present given data (time-series and external forces) and Fig. \ref{fig:varx2} to visualize objects in the problem.

 \begin{figure}[h!]
  \centering
    \includegraphics[scale=0.2]{figure/varx1.pdf}
  \caption{Given data in VarX problem; time-series values $x_0,x_1,x_2,x_3,x_4$ and external forces $u_0,u_1,u_2,u_3,u_4$.}
  \label{fig:varx1}
 \end{figure}

 \begin{figure}[h!]
  \centering
    \includegraphics[scale=0.2]{figure/varx2.pdf}
  \caption{Objects in the VarX problem.}
  \label{fig:varx2}
 \end{figure}

 The most complicated operation in equation \eqref{eq:varx_stationary_matrix} is matrix multiplication $MZ$. The graphical analysis of this operation could be found in Fig. \ref{fig:varx3}.
 Here, we used general property
 \begin{displaymath}
  \forall A \in \mathbb{R}^{m,n} \forall v_1,v_2 \in \mathbb{R}^n: A\left[ v_1, v_2 \right ] = \left[ A v_1, Av_2 \right] ,
 \end{displaymath}
 i.e. multiplication by matrix could be applied into columns.

 \begin{figure}[h!]
  \centering
    \includegraphics[scale=0.2]{figure/varx3.pdf}
  \caption{Multiplication $MZ$; the dashed line represents the separation between columns.}
  \label{fig:varx3}
 \end{figure}

 Now we are ready to assemble $X = MZ + \varepsilon$ (which we actually will not demonstrate, because the operation addition on the right side is an operation between columns of matrices, it is trivial, and it will be clear from following).
 Afterwards, we can compare columns on the left and right side of equation $X = MZ + \varepsilon$, see Fig. \ref{fig:varx4} and we obtain the original equations in VarX model, see equations \eqref{eq:varx_stationary}.
 Therefore, in this case, equations \eqref{eq:varx_stationary} and \eqref{eq:varx_stationary_matrix} are equivalent.

 \begin{figure}[h!]
  \centering
    \includegraphics[scale=0.2]{figure/varx4.pdf}
  \caption{The definition of original VarX problem.}
  \label{fig:varx4}
 \end{figure}


\section*{Chapter 6: Abstraction in Mathematics}

\paragraph{Exercise 6.3}
Prove that any group has at least one element.

\begin{proof}
Any group, by definition, has an identity element $e$.
\end{proof}

\paragraph{Exercise 6.4}
What is the order of $e$? Prove that $e$ is the only element of such order.

\begin{proof}[Answer]
Let $G$ be a group with identity element $e$. The order of $e$ is 1 given that
$e^1 = e$. Suppose that $x \in G$ has also order 1. Then, $x^1 = x = e$.
\end{proof}

\paragraph{Exercise 6.5}
Prove that if $a$ is an element of order $n$, then $a^{-1} = a^{n-1}$.

\begin{proof}
We know that $a^n = a \, a^{n-1} = e$. Since inverses are unique, then
it must be $a^{-1} = a^{n-1}$.
\end{proof}

\paragraph{Exercise 6.7}
Prove that any subgroup of a cyclic group is cyclic.

\begin{proof}
Let $G$ be a cyclic group and $S$ a subgroup of $G$. Let $x$
be a generator of $G$, and let
$i_0 = \min{\{ i \leq |G| \, / \, x^i \in S \ \}}$.
Consider the element $y = x^{i_0} \in S$, and suppose that its order
$m$ is such that $m < |S|$. Then, let $z \in S$ be an element such that
$z \neq y^j$. Being $x$ a generator of $G$, we have that $z = x^{j_0}$
for some $j_0$. By the division algorithm, we can write $j_0$ as
$j_0 = i_0 q + r$, for some $0 \leq r < i_0$. Then,
\begin{eqnarray*}
z &=& x^{j_0} \\
  &=& x^{i_0 q + r} \\
  &=& x^{i_0 q} \, x^r \\
  &=& (x^{i_0})^q \, x^r \\
  &=& y^q \, x^r
\end{eqnarray*}
This implies that $x^r = (y^q)^{-1} \, z \in S$, but this contradicts
the minimality of $i_0$. Hence, such $z$ cannot exist, which proves that
$y$ is a generator of $S$ and, consequently, that $S$ is cyclic.
\end{proof}

\paragraph{Exercise 6.8}
Prove that any cyclic group is abelian.

\begin{proof}
Let $G$ be a cyclic group, and let $x$ be a generator of $G$. Given
$a, b \in G$, we know that there exist $i, j \in \Nat$ such that
$a = x^i$ and $b = x^j$. Then, using the fact that the group
operation is associative and that integer addition commutes,
$$ab = x^i \, x^j = x^{i+j} = x^{j+i} = x^j \, x^i = ba$$
\end{proof}

\paragraph{Exercise 6.10}
Prove that every group of prime order is cyclic.

\begin{proof}
Let $G$ be a group such that its order $p$ is prime. Since $p > 1$,
there must be at least one element in $G$ whose order is greater than 1.
Let $x$ be one such element, and let $n$ be its order. The set
$S = \{x^i \, / \, 1 \leq i \leq n \}$ (equipped with $G$'s operation)
is a subgroup of $G$, and so, by Lagrange's theorem,
$\Divides{n = |S|}{|G| = p}$, which implies that $n = p$. Thus, $x$
generates $G$.
\end{proof}


\chapter{Vector}

\section{Vector space as Cartesian power of ring}

A ring \(R\) give rises to an
\index{vector space}%
\emph{\(n\)-dimensional vector space} \(R^n\) and its vector operations.
Each element of \(R^n\) is a
\index{vector}%
\emph{vector} \((x_1,\ldots,x_n)\)
where each \(x_k \in R\).
We say that \(R^n\) is a vector space \emph{over} ring \(R\).

\paragraph{Operations}
Let \(a \in R\) and \(x,y\in R^n\).
The
\index{scalar multiplication}%
\index{multiplication!scalar and vector}%
\index{vector!scalar multiplication}%
\emph{scalar multiplication} of \(a\) and \(x\) is \(ax \in R^n\) where \((ax)_k = a x_k\).
The
\index{vector!addition}%
\index{addition!vector}%
\emph{vector addition} of \(x\) and \(y\) is \((x+y) \in R^n\) where \((x+y)_k = x_k+y_k\).
The
\index{inner product!vector}%
\index{vector!inner product}%
\emph{inner product} of \(x\) and \(y\) is \(\langle x,y \rangle = x \cdot y = \sum_{k=1}^n x_k y_k\).

The \emph{\(i\)th standard basis} is \(e_i\) where \((e_i)_k = \delta_{ik}\)
where \(\delta_{ik} = 1_R\) iff \(i=k\) and \(0_R\) otherwise.
\(0_R\) is the additive identity of \(R\)
and \(1_R\) multiplicative identity of \(R\).

% FIXME use standard terms
Such vector space is also an inner product space.
Such vector space is the \emph{natural \(n\)-vectorization of \(R\)}.
The \emph{component ring} of that vector space is \(R\).

\section{Dual vector space}

Let \(V\) be a vector space over ring \(R\).

The
\index{dual!vector space}%
\index{vector space!dual}%
\emph{dual} of \(V\) is \(V^* = V \to R\).
A
\index{covector}%
\emph{covector} is an element of \(V^*\).
Another names for covector are
\emph{linear form}, \emph{linear functional}, and \emph{one-form}.

\(V^*\) is also a vector space
where scalar multiplication is
\((cf)(x) = c \cdot f(x)\)
and vector addition is
\((f+g)(x) = f(x)+g(x)\)
where \(c \in R\) and \(f,g \in V^*\).
\(f(x) = \langle a,x\rangle\) defines a bijection involving \(f:V^*\) and \(a:V\).

A \emph{linear map} is a function \(f\) such that \(f(x+y) = f(x) + f(y)\)
and \(f(c \cdot x) = c \cdot f(x)\) where \(c\) is a scalar.
A linear map \(T : R^m \to R^n\) can be thought
as a vector of covectors \(t : (R^m \to R)^n\) where
\(
(T(x))_i = t_i(x) = \langle a_i, x \rangle
\).
There is a bijection between matrices and linear maps:
\(Mx = T(x)\) where
\[
M = \begin{bmatrix}\tp(a_1)\\\vdots\\\tp(a_m)\end{bmatrix}
= \begin{bmatrix}a_{11} & \ldots & a_{1m} \\ \vdots & \ddots & \vdots \\ a_{m1} & \ldots & a_{mm}\end{bmatrix}
\]
where \(\tp(x)\) is the transpose of \(x\).
Iff $m = n$, then \(T\) is an \emph{operator}.

The linear combination of \(n\) vectors in the same \(n\)-dimensional vector space:
\(x\) is a linear combination of \(e_1,\ldots,e_n\)
iff there exists \(a = (a_1,\ldots,a_n)\)
such that \(x = a_1 \cdot e_1 + \ldots + a_n \cdot e_n\)
where each \(a_k\) is a scalar and each \(e_k\) is a vector.
We can also write the latter equation as
\[
x = E~a = a_1 \cdot e_1 + \ldots + a_n \cdot e_n,
\]
which we can also spell out into $n$ equations, each like this:
\[
x_i = E_i~a = a_1 \cdot (e_1)_i + \ldots + a_n \cdot (e_n)_i,
\]
which suggests that each $E_i$ is a covector
(you should be able to show that $E_i$ is linear).
We can also write the equation using matrices:
\[
x =
\begin{bmatrix}
e_1 & \ldots & e_n
\end{bmatrix}
\begin{bmatrix}
a_1 \\ \vdots \\ a_n
\end{bmatrix}
\]
which we can write out in full as
\[
x =
\begin{bmatrix}
e_{11} & \ldots & e_{n1}
\\ \vdots & \ddots & \vdots
\\ e_{1n} & \ldots & e_{nn}
\end{bmatrix}
\begin{bmatrix}
a_1 \\ \vdots \\ a_n
\end{bmatrix}
\]
and thus $E~a = ea$,
where $e_k$ becomes the $k$th column of $e$.
We say that $e$ is the matrix corresponding to $E$.
This is an example of using matrix multiplication to specify
a linear combination of a set of vectors;
this matrix equation summarizes $n$ equations into one equation.
Thus there is an isomorphism between a linear endofunction in a vector space
and multiplication of a matrix and a column vector;
every such endofunction has a corresponding matrix.

\section{Vector space over a field}

\index{vector space}%
\index{space!vector}%
A \emph{vector space} over a field \(F\) is a set \(V\) and the
\index{vector space axioms}%
\emph{vector space axioms} \cite{wpvectorspace}\cite{roman2005advanced}:
For all \(a, b \in F\) and \(x, y \in V\):
\begin{enumerate*}[label={(\arabic*)}]
    \item \(V\) forms an additive group,
    \item \(1 x = x\) where \(1\) is the multiplicative identity of \(F\),
    \item \((ab)x = a(bx)\),
    \item \(a(x+y) = ax+ay\),
    \item \((a+b)x = ax+by\).
\end{enumerate*}
Therefore, a vector space \(V\) over a field \(F\) is
an additive group \(V\) and a \emph{scalar multiplication} \(F \to V \to V\).

\index{vector}%
A \emph{vector} is an element of a vector space.

\index{vector!concatenation}%
\index{concatenation!vector}%
\index{vector concatenation}%
The \emph{vector concatenation} of \(a : \Real^m\) and \(b : \Real^n\)
is \(a|b : \Real^p\)
where \(p = m + n\),
\(a|b = (a_1 , \ldots , a_m , b_1 , \ldots , b_n)\),
and a scalar is treated as a vector of length 1.

\index{vector!column}%
\index{column vector}%
A \emph{column vector} of length \(n\) is a \(n \times 1\) matrix.

\index{vectors!orthogonal}%
\index{orthogonal vectors}%
Two vectors \(x\) and \(y\) are \emph{orthogonal} iff \(x \cdot y = 0\).
\index{vectors!parallel}%
\index{parallel vectors}%
Two vectors \(x\) and \(y\) are \emph{parallel} iff \(x \cdot y = \norm{x} \cdot \norm{y}\).

\index{dot product}%
\index{vectors!dot product}%
Relationship between length and dot product: \(\norm{x}^2 = x \cdot x\).
Dot product distributes addition: \(x \cdot (y+z) = x \cdot y + x \cdot z\).
Geometric interpretation of dot product: \(a \cdot b = \norm{a} \cdot \norm{b} \cdot \cos \theta\).

\index{unit vector}%
\index{vectors!unit}%
A \emph{unit vector} is a vector whose length is 1.

\index{vector projection}%
\index{vectors!projection}%
\index{projection!of a vector to another vector}%
The \emph{projection} of \(a\) to \(b\) is \((a \cdot b) \cdot b / |b|^2\).

\section{Matrix}

\index{scalar-matrix multiplication}%
\emph{Scalar-matrix multiplication} is \((ka)_{ij} = k \cdot a_{ij}\).
\index{matrix!addition}%
\index{addition!matrix}%
\emph{Matrix addition} is \((a + b)_{ij} = a_{ij} + b_{ij}\).
\index{matrix!multiplication}%
\index{multiplication!matrix}%
\emph{Matrix multiplication} is \((ab)_{ij} = \sum_{k=1}^n a_{ik} b_{kj}\) where
\(a : R^{m \times n}, b : R^{n \times p}, c : R^{m \times p}\).

\index{coefficient matrix}%
\index{matrix!coefficient}%
\index{system of linear equations}%
\index{unknown}%
A \emph{system of linear equations} is a matrix equation \(A x = b\)
where \(A\) is the \emph{coefficient matrix} and \(x\) is the \emph{unknown}.
\index{overdetermined system of linear equations}%
\index{system of linear equations!overdetermined}%
That system is \emph{overdetermined} iff \(A\) has more rows than columns.

\section{Basis}

A
\index{basis!of vector space}%
\index{vector space!basis}%
\emph{basis} of an \(n\)-dimensional vector space \(V\) is a set of vectors \(\{e_1,\ldots,e_n\}\) where each \(e_k \in V\)
such that for every \(v \in V\) there is \(a \in V\) such that \(v = \sum_{k=1}^n a_k e_k\).

The
\index{standard basis}%
\index{natural basis}%
\emph{standard basis} for \(R^n\) is \(\{ e_1, \ldots, e_n\}\) where
\( (e_k)_i = [ k = i ] \).
It is also called \emph{natural basis}.

\section{Matrix as linear operator}

The
\index{span (of a matrix)}%
\emph{span} of \(A \in \Real^{n\times n}\) is $\fspan(A) = \{ Ax ~|~ x \in \Real^n \}$.
If we treat \(A\) as a linear functional,
then \(\fspan(A)\) is the range of \(A\).

\section{Least-squares solution}

\index{least-squares!solution of an overdetermined system of linear equations}%
\index{system of linear equations!overdetermined!least-squares solution}%
\index{system of linear equations!least-squares solution}%
If \(A : R^{m \times n}\) and \(b : R^m\),
then the \emph{least-squares solution} of \(A x = b\)
is the \(y\) that minimizes \(\norm{A y - b}^2\).
\index{normal equation}%
That \(y\) is also the solution of the corresponding \emph{normal equation}
\((A^T A) y = A^T b\).

\section{Hyperplane}

\index{hyperplane}%
\index{hyperplane!below}%
\index{hyperplane!below-or-on}%
\index{hyperplane!on}%
\index{hyperplane!above}%
\index{hyperplane!above-or-on}%
A \emph{hyperplane}
\(h : \Real^\infty \to \Real\)
is \(h~x = n \cdot (x - p)\)
where \(n\) is the \emph{normal} of \(h\)
and \(p\) is a point on \(h\).
The point \(x\)
is \emph{below} \(h\) iff \( h~x < 0 \),
is \emph{below-or-on} \(h\) iff \( h~x \le 0 \),
is \emph{on} \(h\) iff \( h~x = 0 \),
is \emph{above} \(h\) iff \( h~x > 0 \),
and
is \emph{above-or-on} \(h\) iff \( h~x \ge 0 \).

\index{hyperplane equation!matrix form}%
\index{matrix form of hyperplane equation}%
The \emph{matrix form} of the hyperplane equation \(f~x = a \cdot x + b\)
is \(f~x = (a|b)^T (x|1)\).

The \emph{distance} of a point \(x\) to hyperplane \(h = n \cdot (x - p)\)
is the length of the projection of \(x-p\) to \(n\).

\section{Matrix unary operations}

The
\index{transpose}%
\index{transpose!of matrix}%
\index{matrix!transpose}%
\emph{transpose} of \(M\) is \((M^T)_{ij} = M_{ji}\).

The
\index{conjugate transpose}%
\index{conjugate transpose!of matrix}%
\index{transpose!conjugate}%
\index{matrix!conjugate transpose}%
\emph{conjugate transpose} of \(M\) is \((M^*)_{ij} = (M_{ji})^*\).

\section{Special matrices}

\index{identity matrix}%
\index{matrix!identity}%
A matrix \(I : \Real^{n \times n}\) is \emph{identity} iff
\(\forall (A : \Real^{n \times n}) (IA = AI = A)\).
The \emph{\(n\times n\) identity matrix} is
\((I_n)_{ij} = \delta_{ij}\) where \(\delta\) is the
\index{Kronecker delta}%
Kronecker delta \(\delta_{ij} = [i=j]\).

A matrix \(M\) is
\index{unitary matrix}%
\index{matrix!unitary}%
\emph{unitary} iff \(M^*M = MM^* = I\)
where \(I\) is an identity matrix.

\section{Singular value decomposition}

The
\index{singular value decomposition}%
\index{matrix decomposition!singular value}%
\emph{singular value decomposition} of \(M\) is \(U S V^* = M\) where \(V^*\) is the conjugate transpose of \(V\).

\section{QR decomposition}

\(M\) is an
\index{triangular matrix}%
\index{matrix!triangular}%
\emph{upper triangular matrix} iff ...

\(M\) is an
\index{orthogonal matrix}%
\index{matrix!orthogonal}%
\emph{orthogonal matrix} iff ...

The
\index{QR decomposition}%
\index{matrix decomposition!QR}%
\emph{QR decomposition} of \(M\) is \(M = QR\) where
\(Q\) is an orthogonal matrix and
\(R\) is an upper triangular matrix.