\subsection{Partially ordered sets}\label{subsec:partially_ordered_sets}

\hyperref[def:preordered_set]{Preordered sets} are simple to define and arise naturally (e.g. \fullref{def:lindenbaum_tarski_algebra}), but they have uniqueness considerations (see \fullref{ex:preorder_nonuniqueness}) that we usually want to factor out. \Fullref{thm:preorder_to_partial_order} shows that the result obtained from this factorization is a partially ordered set, which this section is dedicated to.

\begin{definition}\label{def:partially_ordered_set}
  A \term{partial order} on a set \( \mscrP \) can be defined in the following equivalent ways:
  \begin{thmenum}[series=def:partially_ordered_set]
    \thmitem{def:partially_ordered_set/nonstrict} A \hyperref[def:preordered_set]{preorder} \( \leq \) on \( \mscrP \) such that \( \leq \) is \hyperref[def:binary_relation/antisymmetric]{antisymmetric} in addition to being \hyperref[def:binary_relation/reflexive]{reflexive} and \hyperref[def:binary_relation/transitive]{transitive}. This definition is the more common one. If we wish to distinguish it from the other definition, we call such a relation a \term{nonstrict partial order}.

    \thmitem{def:partially_ordered_set/strict} An \hyperref[def:binary_relation/irreflexive]{irreflexive} and \hyperref[def:binary_relation/transitive]{transitive} binary relation \( < \) on \( \mscrP \). This relation is called a \term{strict partial order}.
  \end{thmenum}

  If both relations are present, in order for them to be equivalent, \( \leq \) must be the union of \( < \) and the \hyperref[def:binary_relation/diagonal]{diagonal} \( \Delta \). This condition corresponds to the following axiom:
  \begin{equation}\label{def:partially_ordered_set/compatibility_nonstrict}
    (x \leq y) \leftrightarrow \parens[\Big]{(x < y) \vee (x = y)}.
  \end{equation}

  By adding \( \anon \wedge \neg (x = y) \) to both sides of \eqref{def:partially_ordered_set/compatibility_nonstrict}, using \fullref{thm:de_morgans_laws} and taking irreflexivity of \( < \) into account, we obtain
  \begin{equation}\label{def:partially_ordered_set/compatibility_strict}
    (x < y) \leftrightarrow \parens[\Big]{(x \leq y) \wedge \neg (x = y)}.
  \end{equation}

  A set \( \mscrP \) with any of the two types of partial orders is called a \term{partially ordered set} or simply a \term{partially ordered set}.

  The elements \( x, y \in \mscrP \) are called \term{comparable} if either \( x \leq y \) or \( y \leq x \). That is, they are comparable if they are related by \( \leq \).

  Partially ordered sets have the following metamathematical properties:
  \begin{thmenum}[resume=def:partially_ordered_set]
    \thmitem{def:partially_ordered_set/theory} Since we can interdefine nonstrict and strict orders, it makes little sense to study different theories for the two.

    In order to define the theory of partially ordered sets, we extend the language of the \hyperref[def:preordered_set/theory]{theory of preordered sets} with two binary infix predicate symbol --- \( < \) and \( > \). We then add to the axioms of the theory \eqref{def:binary_relation/antisymmetric} for \( \leq \) and either of the compatibility conditions \eqref{def:partially_ordered_set/compatibility_nonstrict} or \eqref{def:partially_ordered_set/compatibility_strict} (it is unnecessary to add both).

    We can also add \eqref{def:binary_relation/irreflexive} and \eqref{def:binary_relation/transitive} for \( < \), but that would be redundant.

    \thmitem{def:partially_ordered_set/homomorphism} We have two types of \hyperref[def:first_order_homomorphism]{first-order homomorphisms} between partially ordered sets. The homomorphisms are often called \term{order homomorphisms} or \term{monotone maps}.

    A \term{nonstrict monotone map} from \( (\mscrP, \leq_\mscrP) \) to \( (\mscrQ, \leq_\mscrQ) \) is a homomorphism for the theory containing only \( \leq \) and \( \geq \):
    \begin{equation}\label{eq:def:partially_ordered_set/homomorphism/nonstrict}
      x <_\mscrP y \T{implies} f(x) \leq_\mscrQ f(y).
    \end{equation}

    Nonstrict homomorphisms are used extensively in the theory of \hyperref[subsec:partially_ordered_sets]{partially ordered sets}, in particular in \hyperref[subsec:lattices]{lattice theory}.

    A \term{strict monotone map} is instead a homomorphism for the full theory:
    \begin{equation}\label{eq:def:partially_ordered_set/homomorphism/strict}
      x <_\mscrP y \T{implies} f(x) <_\mscrQ f(y).
    \end{equation}

    Strict homomorphisms are used in the theory of \hyperref[subsec:partially_ordered_sets]{totally ordered sets}, in particular for \hyperref[subsec:well_ordered_sets]{well-ordered sets} and \hyperref[subsec:ordinals]{ordinals}.

    See \fullref{thm:monotone_map_converse} for the converse implications to those in \eqref{eq:def:partially_ordered_set/homomorphism/nonstrict} and \eqref{eq:def:partially_ordered_set/homomorphism/strict}.

    In particular, if \( \mscrP \) is the preordered set of \hyperref[rem:peano_arithmetic_zero/nonnegative]{positive integers}, then we speak of \term{monotone sequences}
    \begin{equation}\label{eq:def:partially_ordered_set/homomorphism/sequence}
      \seq{ x_k }_{k=1}^\infty,
    \end{equation}
    where \( x_{k-1} \leq_Q x_k \) for all \( k \in 1, 2, 3, \ldots \).

    To elaborate, order homomorphisms can either be strict or nonstrict depending on whether we include \( < \) and \( > \) as predicate symbols in the language of the theory. We will use the convention that even when the language does include \( < \) and \( > \), by \enquote{order homomorphism} we will understand nonstrict monotone.

    Furthermore, \fullref{thm:totally_ordered_strong_homomorphism} shows that for totally ordered sets, strict order homomorphisms are precisely the strong order homomorphisms (in the sense of \fullref{rem:first_order_strong_homomorphism}).

    \thmitem{def:partially_ordered_set/submodel} As for preordered sets, any subset of a partially ordered set is itself a partially ordered set.

    \thmitem{def:partially_ordered_set/trivial} The \hyperref[thm:substructures_form_complete_lattice/bottom]{trivial partially ordered set} is the empty set (see \fullref{rem:empty_models} regarding allowing empty sets as first-order structures)..

    \thmitem{def:partially_ordered_set/category} We denote the \hyperref[def:category_of_small_first_order_models]{category of \( \mscrU \)-small models} by \( \ucat{Pos} \). It is a full subcategory of the category \hyperref[def:preordered_set/category]{\( \ucat{PreOrd} \)} of \( \mscrU \)-small preordered sets.

    The category \( \ucat{Pos} \) is equivalent to that of \( \mscrU \)-small thin skeletal categories --- see \fullref{thm:order_category_isomorphism/partially_ordered}.

    \thmitem{def:partially_ordered_set/duality} The \hyperref[def:preordered_set/duality]{principle of duality for preordered sets} holds for partially ordered sets if we also swap \( < \) and \( > \).
  \end{thmenum}
\end{definition}
\begin{proof}
  \ImplicationSubProof{def:partially_ordered_set/nonstrict}{def:partially_ordered_set/strict} Let \( \leq \) be a nonstrict partial order. We will show that \( < \) is a strict partial order.

  \begin{itemize}
    \item The relation \( < \) is \hyperref[def:binary_relation/transitive]{transitive}. To see this, let \( x < y \) and \( y < z \). In particular, \( x \leq y \) and \( y \leq z \). From transitivity, we have \( x \leq z \).

    Additionally, \( x \neq y \) and \( y \neq z \). Assume that \( x = z \). From reflexivity of \( \leq \) we have \( z \leq x \) and, since \( y \leq z \), from transitivity we obtain \( y \leq x \). But since \( x \leq y \), from the antisymmetry of \( \leq \), we have \( x = y \), which contradicts the assumption that \( x < y \).

    Therefore, \( x < z \).

    \item \hyperref[def:binary_relation/irreflexive]{Irreflexivity} of \( < \) follows directly from reflexivity of \( \leq \) and the compatibility condition.
  \end{itemize}

  Since the right side is false, the left side \( x < x \) is also false.

  \ImplicationSubProof{def:partially_ordered_set/strict}{def:partially_ordered_set/nonstrict} Let \( < \) be a strict partial order. We will show that \( \leq \) is a nonstrict partial order.

  \begin{itemize}
    \item To see \hyperref[def:binary_relation/reflexive]{reflexivity}, fix \( x \in \mscrP \) and assume that \( x \not\leq x \). Then \( x \neq x \) which contradicts the reflexivity of equality. Hence, \( x \leq x \).

    \item To see \hyperref[def:binary_relation/antisymmetric]{antisymmetry}, let \( x \leq y \) and \( y \leq x \), that is, either \( x = y \) or both \( x < y \) and \( y < x \) hold. Assume the latter. By the transitivity of \( \leq \), we have \( x < x \), which contradicts the irreflexivity of \( < \). Hence, \( x = y \).

    \item To see \hyperref[def:binary_relation/transitive]{transitivity}, let \( x \leq y \) and \( y \leq z \). Then we have four cases depending on which of \( x \), \( y \) and \( z \) are equal. Since both relations \( < \) and \( = \) are transitive, it follows that in all four cases \( x \leq z \).
  \end{itemize}
\end{proof}

\begin{definition}\label{def:hasse_diagram}
  It is usually easier to define small finite partially ordered sets by drawing graphs than by enumerating all relation pairs. Let \( (\mscrP, \leq) \) be a finite partially ordered set. The relation \( \leq \) may also be regarded as the set of edges of a \hyperref[def:quiver/simple]{directed graph}. The graph \( (\mscrP, \red^T(\leq)) \), whose edges are the \hyperref[def:relation_closures/transitive]{transitive reduction} of \( \leq \), is called the \term{Hasse graph} or \term{Hasse diagram} of \( \mscrP \).

  The term \enquote{Hasse diagram} is usually associated with drawings. By convention, no arrowheads for denoting directions are drawn on the Hasse graph despite the graph being directed; instead, edges always point upwards. See \fullref{ex:def:hasse_diagram}.
\end{definition}

\begin{example}\label{ex:def:hasse_diagram}
  Consider the partial order over \( \set{ a, b, c, d, e } \) defined via
  \begin{equation}\label{eq:ex:def:hasse_diagram/partially ordered set}
    \underline{a \leq c},\quad \underline{a \leq d},\quad a \leq e,\quad \underline{b \leq d},\quad b \leq e,\quad \underline{d \leq e}.
  \end{equation}

  The corresponding Hasse graph includes only the underlined edges. The rest of the edges can be restored from transitivity. In this case, the Hasse graph has edges
  \begin{equation}\label{eq:ex:def:hasse_diagram/hasse_graph}
    \set{ a \to c, a \to d, b \to d, d \to e }
  \end{equation}

  \begin{figure}
    \centering
    \includegraphics[page=1]{output/ex__def__hasse_diagram.pdf}
    \caption{A drawing of the Hasse diagram \eqref{eq:ex:def:hasse_diagram/hasse_graph}}
    \label{fig:ex:def:hasse_diagram}
  \end{figure}
\end{example}

\begin{example}\label{ex:preorder_nonuniqueness}
  Consider the preordered set \( \mscrP \) in \cref{fig:ex:preorder_nonuniqueness} in which \( b \leq c \) and \( c \leq b \), but \( b \neq c \). We cannot properly draw a \hyperref[def:hasse_diagram]{Hasse diagram} because we have the restriction that \( c \) is drawn (strictly) higher than \( b \) if \( c > b \) and that \( c \) is drawn lower than \( b \) if \( c < b \). We face a similar problem formally, for example in the definition of a \hyperref[def:lindenbaum_tarski_algebra]{Lindenbaum-Tarski algebra} of a \hyperref[def:first_order_theory]{logical theory}, where the preorder \( \vdash \) allows \( \varphi \vdash \psi \) and \( \psi \vdash \varphi \), but still \( \varphi \neq \psi \). Thus, we have nonuniqueness --- every tautology is a largest element with respect to \( \vdash \), while we want to have a single largest element for the sake of building a tidier theory.

  If we are only interested in members of \( \mscrP \) up to the equivalence \eqref{eq:thm:preorder_to_partial_order/equivalence}, it is easy to factor \( \mscrP \) by the equivalence relation \eqref{eq:thm:preorder_to_partial_order/equivalence} and obtain a partially ordered set. In the language of graph theory, if we have \hyperref[def:quiver_path/cycle]{directed cycles} that we may wish to avoid, we can contract each directed cycle into a single vertex, at which point the graph becomes acyclic. This corresponds to \hyperref[def:quiver_condensation]{graph condensation}.

  The formulation and proof of correctness of this process can be found in \fullref{thm:preorder_to_partial_order} and an example can be found in \cref{fig:ex:preorder_nonuniqueness}.

  \begin{figure}
    \hfill
    \includegraphics[page=1]{output/ex__preorder_nonuniqueness.pdf}
    \hfill
    \includegraphics[page=2]{output/ex__preorder_nonuniqueness.pdf}
    \hfill\hfill
    \caption{A preordered set and its induced partially ordered set.}
    \label{fig:ex:preorder_nonuniqueness}
  \end{figure}
\end{example}

\begin{proposition}\label{thm:preorder_to_partial_order}
  Let \( (\mscrP, \leq) \) be a preordered set. Define the relation \( \cong \) by
  \begin{equation}\label{eq:thm:preorder_to_partial_order/equivalence}
    x \cong y \T{if and only if} x \leq y \T{and} y \leq x.
  \end{equation}

  That is, \( \cong \) is the intersection of the relation \( \leq \) with its \hyperref[def:binary_relation/converse]{inverse}.

  Since \( \cong \) is an \hyperref[def:equivalence_relation]{equivalence relation} we can for the the quotient set \( \mscrP / \cong \). Define the relation \( \preceq \) on this quotient set by
  \begin{equation*}
    [x] \preceq [y] \T{if and only if} x \leq y.
  \end{equation*}

  The pair \( (\mscrP / \cong, \preceq) \) is then a \hyperref[def:partially_ordered_set]{partially ordered set}.
\end{proposition}
\begin{proof}
  The relation \( \preceq \) is well-defined. Indeed, let \( x \cong x' \) and \( y \cong y' \), that is, both \( x \leq x' \) and \( x' \leq x \) and similarly for \( y \). If \( x \leq y \), from transitivity \( x \leq y \leq y' \). But \( x' \leq x \), hence \( x' \leq y' \).

  It is then clear that \( \preceq \) is a partial order because it inherits reflexivity and transitivity from \( \leq \) and antisymmetry is imposed by taking quotient sets --- equality in \( \mscrP / \cong \) holds precisely when \( \cong \) holds in \( \mscrP \).

  Thus, \( (\mscrP / \cong, \preceq) \) is indeed a partially ordered set.
\end{proof}

\begin{proposition}\label{thm:order_embedding_is_strict}
  For any two \hyperref[def:preordered_set]{preordered sets} \( (\mscrP, \leq_\mscrP) \) and \( (\mscrQ, \leq_\mscrQ) \), every \hyperref[def:partially_ordered_set/homomorphism]{order embedding} of \( \mscrP \) into \( \mscrQ \) is a \hyperref[def:partially_ordered_set/homomorphism]{strict order embedding}.

  Compare this to \fullref{thm:total_order_embedding_iff_strict}
\end{proposition}
\begin{proof}
  Let \( f: \mscrP \to \mscrQ \) be an order embedding.

  Let \( x <_\mscrP y \) for some members \( x \) and \( y \) of \( \mscrP \). Since \( f \) is an order homomorphism, we have \( f(x) \leq_\mscrQ f(y) \). Since it is also injective, \( f(x) = f(y) \) implies \( x = y \), which contradicts our previous assumption.

  Therefore, \( f(x) <_\mscrQ f(y) \) and \( f \) is a strict order homomorphism.
\end{proof}

\begin{proposition}\label{thm:monotone_map_converse}
  For any (even nonstrict) monotone map \( f: \mscrP \to \mscrQ \) between partially ordered sets, if \( x \) and \( y \) are comparable elements of \( \mscrP \) we have
  \begin{equation}\label{eq:thm:monotone_map_converse}
    x <_\mscrP y \T{if and only if} f(x) <_\mscrQ f(y).
  \end{equation}
\end{proposition}
\begin{proof}
  Let \( f(x) <_\mscrQ f(y) \) and suppose that \( x \geq y \). Since \( f \) is a monotone map, we have \( f(x) \geq_\mscrQ f(y) \), which is a contradiction.
\end{proof}

\begin{definition}\label{def:partially_ordered_set_extremal_points}
  We introduce the following terminology for extremal elements of a partially ordered set \( \mscrP \). Analogous definition can be given for preordered sets, but the nonuniqueness problems outlined in \fullref{ex:preorder_nonuniqueness} highlight that there are sometimes difficulties in doing, so.

  The notions on the left and on the right are \hyperref[def:partially_ordered_set/duality]{dual}, but we discuss both nonetheless.

  \begin{thmenum}
    \thmitem{def:partially_ordered_set_extremal_points/upper_and_lower_bounds}\mcite[2]{Gratzer1978}
    \begin{minipage}[t]{0.45\textwidth}
      An \term{upper bound} for the set \( A \subseteq \mscrP \) is an element \( x_0 \in \mscrP \) such that \( x \leq x_0 \) for every \( x \in A \). Note that \( x_0 \) does not in general belong to \( A \). An upper bound is called \term{strict} if it does not belong to \( A \).

      If \( A \) has at least one upper bound, it is called \term{bounded from above}.

      Every element is vacuously an upper bound of \( A = \varnothing \).

      In \cref{fig:ex:def:hasse_diagram}, the set \( A = \set{ a, b } \) is bounded from above by both \( d \) and \( e \), but the entire partially ordered set has no upper bound.
    \end{minipage}
    \hspace{0.02\textwidth}
    \begin{minipage}[t]{0.45\textwidth}
      Dually, \( x_0 \in \mscrP \) is a \term{lower bound} of \( A \) if \( x_0 \leq x \) for every \( x \in A \). A lower bound is called \term{strict} if it does not belong to \( A \). If \( A \) has a lower bound, it is called \term{bounded from below}.

      If \( A \) is bounded both from below and from above, we say that \( A \) is \term{bounded}.

      Every element is vacuously a lower bound of \( A = \varnothing \). Hence, the empty set is bounded.

      In \cref{fig:ex:def:hasse_diagram}, the entire partially ordered set has no lower bound. The set \( A = \set{ c, d } \) is bounded from below by \( a \), but not from above, hence \( A \) is not bounded.
    \end{minipage}

    \thmitem{def:partially_ordered_set_extremal_points/maximal_and_minimal_element}
    \begin{minipage}[t]{0.45\textwidth}
      A \term{maximal element} for the set \( A \subseteq \mscrP \) is a member \( x_0 \) of \( A \) such that there is no greater element in \( A \) than \( x_0 \). More precisely, \( x_0 \) is a maximal element of \( A \) if for every element \( x \in A \) such that \( x \leq x_0 \) we have \( x = x_0 \).

      In \cref{fig:ex:def:hasse_diagram}, the entire partially ordered set has two incomparable maximal elements --- \( c \) and \( e \).
    \end{minipage}
    \hspace{0.02\textwidth}
    \begin{minipage}[t]{0.45\textwidth}
      The member \( x_0 \in A \) is a \term{minimal element} of \( A \) if for every \( x \in A \) such that \( x \leq x_0 \) we have \( x = x_0 \).

      The empty set cannot have maximal or minimal elements because it has no members.

      In \cref{fig:ex:def:hasse_diagram}, the entire partially ordered set has two incomparable minimal elements --- \( a \) and \( b \).
    \end{minipage}

    \thmitem{def:partially_ordered_set_extremal_points/maximum_and_minimum}
    \begin{minipage}[t]{0.45\textwidth}
      The \term{maximum} or \term{greatest element} of \( A \subseteq \mscrP \), if it exists, is an upper bound of \( A \) that belongs to \( A \). A maximum is necessarily a maximal element because \( x_0 \leq x \) only holds for \( x = x_0 \), which also demonstrates uniqueness of \( x_0 \). See \fullref{ex:unique_maximal_element_that_is_not_maximum} for a unique maximal element that is not a maximum.
    \end{minipage}
    \hspace{0.02\textwidth}
    \begin{minipage}[t]{0.45\textwidth}
      The \term{minimum} of \( A \), also called \term{smallest element} or \term{least element}, is a lower bound that belongs to \( A \).

      The empty set cannot have a maximum or minimum because it has no members.

      In \cref{fig:ex:def:hasse_diagram}, the entire partially ordered set has no maximum, but the set \( A = \mscrP \setminus \set{ b } = \set{ a, b, d, e } \) has \( a \) as its minimum.
    \end{minipage}

    \thmitem{def:partially_ordered_set_extremal_points/supremum_and_infimum}\mcite[2]{Gratzer1978}
    \begin{minipage}[t]{0.45\textwidth}
      The \term{supremum} \( \sup A \) of \( A \subseteq \mscrP \), if it exists, is its least upper bound of \( A \), i.e. the \hyperref[def:partially_ordered_set_extremal_points/maximum_and_minimum]{minimum} of the set of its \hyperref[def:partially_ordered_set_extremal_points/upper_and_lower_bounds]{upper bounds}.

      In \cref{fig:ex:def:hasse_diagram}, the entire partially ordered set has no upper bound, so it cannot possibly have a supremum. Obviously every supremum is a maximum, but the converse is not true. We already noted that both \( d \) and \( e \) are upper bounds of the set \( \set{ a, b } \) and since \( d \leq e \), we conclude that \( d \) is the supremum of \( \set{ a, b } \), yet the set \( \set{ a, b } \) has no maximum.
    \end{minipage}
    \hspace{0.02\textwidth}
    \begin{minipage}[t]{0.45\textwidth}
      The \term{infimum} \( \inf A \) of \( A \subseteq \mscrP \) is its greatest lower bound.

      An infimum may fail to exist because the set of lower bounds is nonempty, but has no maximum. This can happen if, for example, we had \( b \leq c \) in \cref{fig:ex:def:hasse_diagram}, in which case both \( a \) and \( b \) would be maximal lower bounds of \( \set{ c, d } \), but none of them would be a greatest lower bound because they are incomparable.
    \end{minipage}

    \thmitem{def:partially_ordered_set_extremal_points/top_and_bottom}\mcite[2]{Gratzer1978}
    \begin{minipage}[t]{0.45\textwidth}
      If it exists, the maximum of the entire partially ordered set \( \mscrP \) is usually denoted by \( \top \) and called the \term{global maximum} or \term{top} element of \( \mscrP \). Since \( \top \) is the maximum of \( \mscrP \), it is also the supremum of \( \mscrP \).

      Since every member of \( \mscrP \) is a lower bound of \( \varnothing \), the greatest lower bound is the maximum of \( \mscrP \).

      In conclusion,
      \begin{equation*}
        \top = \max \mscrP = \sup \mscrP = \inf \varnothing.
      \end{equation*}

      If \( \top \) exists, we say that the partially ordered set \( \mscrP \) itself is bounded from above.
    \end{minipage}
    \hspace{0.02\textwidth}
    \begin{minipage}[t]{0.45\textwidth}
      Dually, the minimum of \( \mscrP \) is usually denoted by \( \bot \) and called the \term{global minimum} or \term{bottom} element of \( \mscrP \).

      The supremum of the empty set is the least of the upper bounds of the empty set, i.e. the minimum of \( \mscrP \), which is \( \bot \).

      In conclusion,
      \begin{equation*}
        \bot = \min \mscrP = \inf \mscrP = \sup \varnothing.
      \end{equation*}

      If \( \bot \) exists, we say that the partially ordered set \( \mscrP \) itself is bounded from below.
    \end{minipage}
  \end{thmenum}
\end{definition}

\begin{example}\label{ex:unique_maximal_element_that_is_not_maximum}\mcite{MathSE:unique_maximal_element_that_is_not_maximum}
  For a more extreme example of the interplay between maximal elements and maxima, adjoin \( \BbbZ \) under the usual order with a new sentinel element \( \star \). Define \( \star \) to satisfy reflexivity, but not be in relation with any integer. Then \( \star \) is a unique maximal element of \( \BbbZ \cup \set{ \star } \), yet the latter set has no largest element.

  This phenomenon is impossible in finite partially ordered sets where a unique maximal element is always a maximum.
\end{example}

\begin{definition}\label{def:partially_ordered_set_chain_and_antichain}\mcite[2]{Gratzer1978}
  A \term{chain} in a partially ordered set is a subset in which every two elements are comparable. The \term{length} of a chain \( A \) is the \hyperref[def:cardinal]{cardinal number} \( \card(A) - 1 \). The \term{length} of a partially ordered set, if it exists, is the maximum among the lengths of all its chains. The length of a partially ordered set is also called its \term{height} because of how Hasse diagrams are drawn..

  An \term{antichain} is a subset in which no two elements are comparable. The \term{width} of a partially ordered set, if it exists, is the maximum among the cardinalities of its antichains.

  The height of the partially ordered set in \cref{fig:ex:def:hasse_diagram} is \( 2 \) and it is reached by the chains \( \set{ a, d, e } \) and \( \set{ b, d, e } \). The width is \( 2 \) are is reached by \( \set{ a, b } \), \( \set{ b, e } \), \( \set{ c, d } \) and \( \set{ c, e } \).
\end{definition}

\begin{definition}\label{def:partially_ordered_set_interval}
  Fix a \hyperref[def:partially_ordered_set]{partially ordered set set} \( (\mscrP, \leq) \). For any \( a, b \in P \) with \( a \leq b \), we define the following related partially ordered sets:

  \begin{thmenum}
    \thmitem{def:partially_ordered_set_interval/ray} The \term{open rays}, also called the \term{open initial segment} and \term{open final segment}, are defined as
    \begin{equation*}
      \begin{aligned}
        (a, \infty) \coloneqq \set{ x \in \mscrP \given b > a } \eqqcolon \mscrP_{>a},
        \\
        (-\infty, b) \coloneqq \set{ x \in \mscrP \given x < b } \eqqcolon \mscrP_{<b}.
      \end{aligned}
    \end{equation*}

    The notation on the left assumes that the sentinel symbols \( \infty \) and \( -\infty \) are adjoined to \( \mscrP \) as in the case of the \hyperref[def:extended_real_numbers]{extended real numbers}. This convention is widespread for unbounded ordered rings representing numbers --- for example \hyperref[def:set_of_integers]{\( \BbbZ \)}, \hyperref[def:set_of_rational_numbers]{\( \BbbQ \)} and \hyperref[def:set_of_real_numbers]{\( \BbbR \)}. The term \enquote{ray} is used in this context due to the connection with \hyperref[def:geometric_ray]{geometric rays}.

    The notation on the right is more general and is widespread for abstract partial orders, most notably \hyperref[def:well_ordered_set]{well-ordered sets}. In the latter context, they are usually referred to as \enquote{initial/final segments}.

    The \term{closed rays} and \term{closed initial/final segments} are defined analogously as
    \begin{equation*}
      \begin{aligned}
        [a, \infty) \coloneqq \set{ x \in \mscrP \given x \geq a } \eqqcolon \mscrP_{\geq a},
        \\
        (-\infty, b] \coloneqq \set{ x \in \mscrP \given x \leq b } \eqqcolon \mscrP_{\leq b}.
      \end{aligned}
    \end{equation*}

    \thmitem{def:partially_ordered_set_interval/closed} The \term{closed interval} with endpoints \( a \) and \( b \) is
    \begin{equation*}
      [a, b] \coloneqq \set{ x \in \mscrP \given a \leq x \leq b } = \mscrP_{\geq a} \cap \mscrP_{\leq b}.
    \end{equation*}

    We implicitly assume that \( a \leq b \), but this is not strictly necessary --- \( [a, b] \) is an empty set otherwise.

    \thmitem{def:partially_ordered_set_interval/open} The \term{open interval} with endpoints \( a \) and \( b \) is
    \begin{equation*}
      (a, b) \coloneqq \set{ x \in \mscrP \given a < x < b } = \mscrP_{> a} \cap \mscrP_{< b}.
    \end{equation*}

    We implicitly assume that \( a < b \), but this is also not strictly necessary.

    \thmitem{def:partially_ordered_set_interval/half_open} The \term{half-open intervals} are
    \begin{equation*}
      \begin{aligned}
        (a, b] \coloneqq \set{ x \in \mscrP \given a < x \leq b } = \mscrP_{> a} \cap \mscrP_{\leq b},
        \\
        [a, b) \coloneqq \set{ x \in \mscrP \given a \leq x < b } = \mscrP_{\geq a} \cap \mscrP_{< b}.
      \end{aligned}
    \end{equation*}
  \end{thmenum}
\end{definition}

\begin{proposition}\label{thm:partially_ordered_cofinal_equivalences}
  Let \( (\mscrP, \leq) \) be a \hyperref[def:partially_ordered_set_extremal_points/upper_and_lower_bounds]{bounded from above} totally ordered set and let \( A \subseteq \mscrP \). Then \( A \) is \hyperref[def:cofinal_set]{cofinal} if and only if it contains the \hyperref[def:partially_ordered_set_extremal_points/top_and_bottom]{top element} \( \top \).

  Compare this result with \Fullref{thm:totally_ordered_cofinal_equivalences}.
\end{proposition}
\begin{proof}
  \SufficiencySubProof Let \( A \) be a cofinal set. Then \( A \) must contain an element \( x \) such that \( \top \leq x \). But \( \top \) is a maximum and hence \( x = \top \) and thus \( \top \in A \).

  \NecessitySubProof Let \( A \) be a set containing \( \top \). Then for any \( x \in \mscrP \) we have \( x \leq \top \) and hence \( A \) is cofinal.
\end{proof}

\begin{definition}\label{def:lexicographic_order}
  Let \( (\mscrP, \leq_\mscrP) \) and \( (\mscrQ, \leq_\mscrQ) \) be partially ordered sets.

  The \term{lexicographic order} on \( \mscrP \times \mscrQ \), also know as the \term{dictionary order}, is defined as
  \begin{equation}\label{eq:def:lexicographic_order}
    (a, b) \prec (c, d) \thickspace \T{if and only if} \thickspace \parens[\Big]{ a <_\mscrP c \T{or} \parens[\Big]{ a = c \T{and} b <_\mscrQ d } }.
  \end{equation}

  The \term{reverse lexicographic order} is
  \begin{equation}\label{eq:def:lexicographic_order/reverse}
    (a, b) \prec (c, d) \thickspace \T{if and only if} \thickspace \parens[\Big]{ b <_\mscrQ d \T{or} \parens[\Big]{ b = d \T{and} a <_\mscrP c } }.
  \end{equation}

  The lexicographic order on \( \mscrP \times \mscrQ \) inherits some important properties from \( (\mscrP, \mscrP) \) and \( (\mscrQ, \leq_\mscrQ) \) as can be seen in \fullref{thm:lexicographic_order_is_partial_order,thm:total_lexicographic_order_is_total_order,thm:well_ordered_lexicographic_order_is_well_ordered}.

  We can use natural number recursion to extend this to arbitrary \( n \)-tuples --- see \fullref{ex:def:lexicographic_order} for an example.
\end{definition}

\begin{example}\label{ex:def:lexicographic_order}
  A key example for \hyperref[def:lexicographic_order]{lexicographic ordering} is real-world dictionary like a thesaurus. It is obvious that \enquote{homomorphism} should come after \enquote{axiom} and the lexicographic order on a Cartesian power of Latin alphabets suggests that \enquote{homeomorphism} should also come after \enquote{homeomorphic}.

  A slightly more relevant example for mathematics is the lexicographic ordering
  \begin{equation*}
    AB < AD < BC < CD
  \end{equation*}
  of the names of the edges of a rectangle. The reverse lexicographic ordering is
  \begin{equation*}
    AB < BC < AD < CD.
  \end{equation*}

  For the sides of a \hyperref[def:triangle]{triangle}, we have \( AB < AC < BC \) for both orderings.

  The edges of the graph in \fullref{eq:fig:def:quiver} are numbered in lexicographic order, which also happens to be the reverse lexicographic order.

  \Fullref{thm:ordinal_addition_disjoin_union} and \fullref{thm:ordinal_multiplication_cartesian_product} contain more interesting applications of lexicographic orders.
\end{example}

\begin{proposition}\label{thm:lexicographic_order_is_partial_order}
  The \hyperref[eq:def:lexicographic_order]{lexicographic} and \hyperref[eq:def:lexicographic_order/reverse]{reverse lexicographic} orders are \hyperref[def:partially_ordered_set/strict]{strict partial order} relations.

  Compare this result to \fullref{thm:total_lexicographic_order_is_total_order} and \fullref{thm:well_ordered_lexicographic_order_is_well_ordered}.
\end{proposition}
\begin{proof}
  \SubProofOf[def:binary_relation/irreflexive]{irreflexivity} Trivial.

  \SubProofOf[def:binary_relation/transitive]{transitivity} Let \( \prec \) be a lexicographic order on \( \mscrP \times \mscrQ \).

  If \( (a, b) \prec (c, d) \) and \( (c, d) \prec (e, f) \), then:
  \begin{itemize}
    \item If \( a < c \), then \( a < c \leq e \) and thus \( (a, b) \prec (e, f) \).

    \item If \( a = c \) and \( b < d \), then \( a \leq e \) and \( b < d \leq f \) and thus \( (a, b) \prec (e, f) \).
  \end{itemize}

   The proof for the reverse lexicographic order is analogous.
\end{proof}

\begin{definition}\label{def:inflationary_function}\mcite{MathOF:closure_operator_on_poset}
  An \hyperref[def:multi_valued_function/endofunction]{endofunction} on a \hyperref[def:partially_ordered_set]{partially ordered set} \( (\mscrP, \leq) \) is said to be \term{inflationary} if \( x \leq \cl(x) \) for every \( x \in \mscrP \).
\end{definition}

\begin{definition}\label{def:closure_operator}\mcite{MathOF:closure_operator_on_poset}
  Let \( (\mscrP, \leq) \) be a \hyperref[def:partially_ordered_set]{partially ordered set}. We say that the function \( \cl: \mscrP \to \mscrP \) is a \term{closure operator} if it is \hyperref[def:inflationary_function]{inflationary}, \hyperref[def:magma/idempotent]{idempotent} and \hyperref[eq:def:partially_ordered_set/homomorphism/nonstrict]{nonstrictly order-preserving}.

  We say that \( x \) is \term{closed} with respect to \( \cl \) if \( x = \cl(x) \).
\end{definition}

\begin{proposition}\label{thm:closure_operator_minimality}
  For any \hyperref[def:closure_operator]{closure operator} \( \cl: \mscrP \to \mscrP \) on any \hyperref[def:partially_ordered_set]{partially ordered set} \( (\mscrP, \leq) \) and any \( x \in \mscrP \) it holds that \( \cl(x) \) is the smallest closed element of \( \mscrP \) containing \( x \).
\end{proposition}
\begin{proof}
  Since \( f \) is \hyperref[def:inflationary_function]{inflationary}, it is clear that \( x \leq \cl(x) \).

  Let \( y \in \mscrP \) be a closed element such that \( x \leq y \leq \cl(x) \). From the monotonicity of \( f \) we have that
  \begin{equation*}
    \cl(x) \leq \underbrace{\cl(y)}_{y} \leq \underbrace{\cl(\cl(x))}_{\cl(x)}.
  \end{equation*}

  Therefore, \( y = \cl(x) \).
\end{proof}

\begin{theorem}[Zorn's lemma]\label{thm:zorns_lemma}\mcite[63]{Gratzer1978}
  If every \hyperref[def:partially_ordered_set_chain_and_antichain]{chain} in a \hyperref[def:partially_ordered_set]{partially ordered set} has an \hyperref[def:partially_ordered_set_extremal_points/upper_and_lower_bounds]{upper bound}, then the entire set has a \hyperref[def:partially_ordered_set_extremal_points/maximal_and_minimal_element]{maximal element}.

  Zorn's lemma is usually stated and used only in a \hyperref[thm:boolean_algebra_of_subsets]{lattice of sets}, however it is a more general statement in order theory.

  In \hyperref[def:zfc]{\logic{ZF}} this theorem is equivalent to the \hyperref[def:zfc/choice]{axiom of choice} --- see \fullref{thm:axiom_of_choice_equivalences/zorns_lemma}.
\end{theorem}
\begin{proof}
  \ImplicationSubProof[def:zfc/choice]{the axiom of choice}[thm:zorns_lemma]{Zorn's lemma} Let \( (\mscrP, \leq) \) be a partially ordered set in which every chain has an upper bound. Aiming at a contradiction, suppose that \( \mscrP \) has no maximal element.

  Denote by \( \mscrC \) the set of all chains of \( \mscrP \) and define the multi-valued map \( F: \mscrC \to \mscrP \) that assigns to each chain the set of all strict upper bounds.

  Since \( \mscrP \) has no maximal element, every chain has a strict upper bound. That is, the function \( F \) is a total multi-valued map. By \fullref{thm:existence_of_multi_valued_function_selection}, there exists a single-valued selection \( f: \mscrC \to \mscrP \) of \( F \). We have indirectly used the axiom of choice via \fullref{thm:existence_of_multi_valued_function_selection}.

  By \fullref{thm:hartogs_lemma}, there exists a smallest \hyperref[def:ordinal]{ordinal} \( \gamma \) such that no function from \( \gamma \) to \( \mscrP \) is injective. Using \fullref{thm:bounded_transfinite_recursion}, we can define
  \begin{equation*}
    \begin{aligned}
      &g: \gamma \to A \\
      &g(\delta) \coloneqq f(\set{ g(\varepsilon) \given \varepsilon < \delta }).
    \end{aligned}
  \end{equation*}

  By construction, for every \( \delta < \gamma \) the value \( g(\delta) \) is a strict upper bound of the set \( \set{ g(\varepsilon) \given \varepsilon < \delta } \). Hence, the function \( f \) is injective, which directly contradicts our choice of \( \gamma \).

  The obtained contradiction shows that \( \mscrP \) has a maximal element.

  \ImplicationSubProof[thm:zorns_lemma]{Zorn's lemma}[def:zfc/choice]{axiom of choice} Let \( \mscrA \) be a family of nonempty sets. Let \( \mscrF \) be the set of all \hyperref[def:partial_function]{partial single-valued functions} from \( \mscrA \) to \( \bigcup \mscrA \) with the subset ordering. That is, \( f \leq g \) if \( \dom(f) \subseteq \dom(g) \) for \( f, g \in \mscrF \).

  Clearly every chain has a maximum - a total single-valued function. Then \( \mscrF \) itself has a maximal element by Zorn's lemma. This maximal element is necessarily a total function because otherwise it would not be maximal.

  Then this is the desired choice function for the family \( \mscrA \).
\end{proof}


\chapter{Neural Networks}

\section*{Exercise 5.1 $\star \star$}
Consider a two-layer network function of the form $\eqref{eq:5.7}$
in which the hidden-unit nonlinear activation functions $h(\cdot)$
are given by logistic sigmoid functions of the form
\begin{equation}\label{eq:5.191}\tag{5.191}
    \sigma(a) = \frac{1}{1 + \exp(-a)}.
\end{equation}
Show that there exists an equivalent network, which computes
exactly the same function, but with hidden activation functions
given by $\tanh(a)$ where the $\tanh$ function is defined by
(5.59). Hint: first find the relation between $\sigma(a)$
and $\tanh(a)$, and then show that the parameters of the two
networks differ by linear transformations.

\vspace{1em}

\begin{proof}
    The considered two-layer network has the form
    \begin{equation}\label{eq:5.7}\tag{5.7}
        y_k(\mathbf{x}, \mathbf{w})
        = \sigma\bigg(\sum_{j=1}^{M} w_{kj}^{(2)}
        h\bigg(\sum_{i=1}^{D} w_{ji}^{(1)}x_i + w_{j0}^{(1)}\bigg) + w_{k0}^{(2)}\bigg)
    \end{equation}
    Now, we've proved in Exercise 3.1 that
    \[
        \sigma(x) = \frac{1}{2} \tanh \frac{x}{2} + \frac{1}{2}
    \]
    Therefore, we can rewrite $y_k$ as
    \begin{align*}
        y_k(\mathbf{x}, \mathbf{w})
        &= \sigma\bigg(\frac{1}{2}\sum_{j=1}^{M} w_{kj}^{(2)}
            \tanh\bigg(\frac{1}{2}\sum_{i=1}^{D} w_{ji}^{(1)}x_i + \frac{1}{2} w_{j0}^{(1)}\bigg)
            + \frac{1}{2} \sum_{j=1}^{M} w_{kj}^{(2)} + w_{k0}^{(2)}\bigg) \\
        &= \sigma\bigg(\sum_{j=1}^{M} \omega_{kj}^{(2)}
        h\bigg(\sum_{i=1}^{D} \omega_{ji}^{(1)}x_i + \omega_{j0}^{(1)}\bigg) + \omega_{k0}^{(2)}\bigg)
    \end{align*}
    where
    \newline
    \begin{minipage}[b]{0.25\textwidth}
    \[
       \omega_{ji}^{(1)} = \frac{1}{2} w_{ji}^{(1)}
    \]
    \end{minipage}
    \begin{minipage}[b]{0.25\textwidth}
    \[
        \omega_{j0}^{(1)} = \frac{1}{2} w_{j0}^{(1)}
    \]
    \end{minipage}
    \begin{minipage}[b]{0.25\textwidth}
    \[
        \omega_{kj}^{(2)} = \frac{1}{2} w_{kj}^{(2)}
    \]
    \end{minipage}
    \begin{minipage}[b]{0.25\textwidth}
        \[
            \omega_{k0}^{(2)} = \frac{1}{2}\sum_{j = 1}^M w_{kj}^{(2)} + w_{k0}^{(2)}
        \]
    \end{minipage}
    Both new parameter sets can be obtained as linear transformations of the old ones,
    so there exists an equivalent two-layer network using $\tanh$ hidden activation functions,
    but different parameters.
\end{proof}

\section*{Exercise 5.2 $\star$}
Show that maximizing the likelihood function under the conditional
distribution \eqref{eq:5.16} for a multioutput network is equivalent to minimizing
the sum-of-squares error function (5.11).

\vspace{1em}

\begin{proof}
    The likelihood function is given by
    \[
        p(\mathbf{T} | \mathbf{X}, \mathbf{w}, \beta)
        = \prod_{n = 1}^N p(\mathbf{t}_n | \mathbf{x}_n, \mathbf{w}, \beta)
    \]
    The target variables are assumed to be distributed normally
    \begin{equation}\label{eq:5.16}\tag{5.16}
        p(\mathbf{t}_n | \mathbf{x}_n, \mathbf{w}, \beta)
        = \mathcal{N}(\mathbf{t}_n | \mathbf{y}(\mathbf{x}_n, \mathbf{w}), \beta^{-1}\mathbf{I})
    \end{equation}
    and since
    \[
        \ln \mathcal{N}(\mathbf{t}_n | y(\mathbf{x}_n, \mathbf{w}), \beta^{-1}\mathbf{I})
        = -\frac{N}{2} \ln \beta - \frac{NK}{2} \ln(2\pi) - \frac{\beta}{2}
        ||\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n||^2
    \]
    the negative log-likelihood is given by
    \[
        -\ln p(\mathbf{t} | \mathbf{X}, \mathbf{w}, \beta)
        = \frac{\beta}{2}\sum_{n=1}^{N}
        ||\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n||^2 + \text{const}
    \]
    where we grouped the terms that don't depend on $\mathbf{w}$ under the constant term.
    Maximization of the likelihood function is equivalent to minimizing the negative
    log-likelihood.
    Therefore, one can easily find that this is equivalent to minimizing the error function
    \begin{equation}\label{eq:5.11}\tag{5.11}
        E(\mathbf{w}) = \frac{1}{2} \sum_{n=1}^{N}
        ||\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n||^2
    \end{equation}
\end{proof}

\section*{Exercise 5.3 $\star \star$}
Consider a regression problem involving multiple target variables in which
it is assumed that the distribution of the targets, conditioned on the input
vector $\mathbf{x}$, is a Gaussian of the form
\begin{equation}\label{eq:5.192}\tag{5.192}
    p(\mathbf{t} | \mathbf{x}, \mathbf{w})
    = \mathcal{N}(\mathbf{t} | \mathbf{y}(\mathbf{x}, \mathbf{w}), \mathbf{\Sigma})
\end{equation}
where $\mathbf{y(x, w)}$ is the output of a neural network with input vector
$\mathbf{x}$ and weight vector $\mathbf{w}$, and $ \mathbf{\Sigma}$ is the
covariance of the assumed Gaussian noise on the targets. Given a set
of independent observations of $\mathbf{x}$ and $\mathbf{t}$, write down
the error function that must be minimized in order to find the maximum
likelihood solution for $\mathbf{w}$, if we assume that $ \mathbf{\Sigma}$
is fixed and known. Now assume that $ \mathbf{\Sigma}$ is also to be determined
from the data, and write down an expression for the maximum likelihood
solution for $ \mathbf{\Sigma}$. Note that the optimizations of
$\mathbf{w}$ and $\mathbf{\Sigma}$ are now coupled, in contrast to the
case of independent target variables discussed in Section 5.2.

\vspace{1em}

\begin{proof}
    The negative log-likelihood is given by
    \begin{align*}
    -\ln p(\mathbf{T} | \mathbf{X}, \mathbf{w})
    &= -\sum_{i=1}^{N} \ln \mathcal{N}(\mathbf{t}_n | \mathbf{y}(\mathbf{x}_n, \mathbf{w}), \mathbf{\Sigma}) \\
    &= \frac{NK}{2} \ln(2\pi) + \frac{N}{2} \ln|\mathbf{\Sigma}|
    + \frac{1}{2}  \sum_{n=1}^{N} \big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big)^T
    \mathbf{\Sigma}^{-1}\big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big)
    \end{align*}
    Maximizing the likelihood is equivalent to minimizing the negative
    log-likelihood. Therefore, the error function that must be minimized
    to obtain maximum likelihood is given by
    \[
        E(\mathbf{w}, \mathbf{\Sigma})
        = \frac{N}{2} \ln|\mathbf{\Sigma}|
        + \frac{1}{2}  \sum_{n=1}^{N} \big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big)^T
        \mathbf{\Sigma}^{-1}\big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big)
    \]
    In the case when $\mathbf{\Sigma}$ is known, we can simply treat
    the determinant term as a constant, so minimizing
    the error function
    \[
        E(\mathbf{w}) =
        \frac{1}{2}  \sum_{n=1}^{N} \big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big)^T
        \mathbf{\Sigma}^{-1}\big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big)
    \]
    would yield the maximum likelihood solution $\mathbf{w}_\text{ML}$.
    If $ \mathbf{\Sigma}$ is unknown, we can't do that and the
    determination of $\mathbf{w}_{\text{ML}}$ would use $\mathbf{\Sigma}$,
    so that's why this time the optimizations of $\mathbf{w}$
    and $\mathbf{\Sigma}$ are coupled. The MLE for
    the covariance matrix is obtained by taking the
    derivative of the negative log-likelihood wrt. $\mathbf{\Sigma}^{-1}$, equalizing
    it to 0 and then solving for $\mathbf{\Sigma}$. Taking the
    derivative of the negative log-likelihood yields
    \begin{align*}
        \pdv{\mathbf{\Sigma}^{-1}} \ln p(\mathbf{T} | \mathbf{X}, \mathbf{w})
        &= \frac{N}{2} \pdv{\mathbf{\Sigma}^{-1}} \ln |\mathbf{\Sigma}| +
        \frac{1}{2}  \sum_{n=1}^{N} \pdv{\mathbf{\Sigma}^{-1}}
        \big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big)^T
        \mathbf{\Sigma}^{-1}\big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big) \\
        &= -\frac{N}{2} \pdv{\mathbf{\Sigma}^{-1}} \ln |\mathbf{\Sigma}^{-1}| +
        \frac{1}{2}  \sum_{n=1}^{N} \pdv{\mathbf{\Sigma}^{-1}}
        \Trace\big\{\big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big)^T
        \mathbf{\Sigma}^{-1}\big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big)\big\} \\
        &= -\frac{N}{2} \mathbf{\Sigma}
        + \frac{1}{2}  \sum_{n=1}^{N} \pdv{\mathbf{\Sigma}^{-1}}
        \Trace\big\{\big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big)^T
        \big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big) \mathbf{\Sigma}^{-1}\big\}\\
        &= -\frac{N}{2} \mathbf{\Sigma} + \frac{1}{2} \sum_{n=1}^{N}
        \big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big)^T
        \big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big)
    \end{align*}
    where we've used the cyclic property of the trace operator and the fact that
    \[
        \pdv{\mathbf{A}} \ln |A| = A^{-T}
    \]
    Now, equalizing
    the derivative with 0 and solving for $\mathbf{\Sigma}$ gives
    the MLE for the covariance matrix:
    \[
        \mathbf{\Sigma}_\text{ML}
        = \frac{1}{N} \sum_{n=1}^{N}
        \big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big)^T
        \big(\mathbf{y}(\mathbf{x}_n, \mathbf{w}) - \mathbf{t}_n\big)
    \]
\end{proof}

\section*{Exercise 5.4 $\star \star$}
Consider a binary classification problem where the target values
are $t \in \{0, 1\}$, with a network output $y(\mathbf{x}, \mathbf{w})$
that represents $p(t = 1 | \mathbf{x})$, and suppose that
there is a probability $\epsiilon$ that the class label
on a trainining data point has been incorrectly set.
Assuming independent and identically distributed data,
write down the error function corresponding to the negative
log likelihood. Verify that the error function $\eqref{eq:5.21}$
is obtained when $\epsilon = 0$. Note that this error function
makes the model robust to incorrectly labelled data, in contrast
to the usual error function.

\vspace{1em}

\begin{proof}
    We're going to model the problem similarly with what we've done in
    Section 4.2.2, but this time taking into account the mislabelled training data
    probability. As a result, let $r \in \{0, 1\}$ the real target values, considering
    mislabelling. Therefore, we can find the label probabilities by weighting in the error
    chance:
    \begin{align*}
        p(r = 1 | \mathbf{x}, \mathbf{w})
        &= (1 - \epsilon)p(t = 1 | \mathbf{x}, \mathbf{w})
        + \epsilon p(t = 0 | \mathbf{x}, \mathbf{w})
        = (1 - \epsilon)y(\mathbf{x}_n, \mathbf{w})
        + \epsilon \big(1 - y(\mathbf{x}_n, \mathbf{w})\big)
    \end{align*}
    \vspace{-2em}
    \begin{align*}
        p(r = 0 | \mathbf{x}, \mathbf{w})
        &= (1 - \epsilon)p(t = 0 | \mathbf{x}, \mathbf{w})
        + \epsilon p(t = 1 | \mathbf{x}, \mathbf{w})
        = (1 - \epsilon)\big(1 - y(\mathbf{x}_n, \mathbf{w})\big)
        + \epsilon y(\mathbf{x}_n, \mathbf{w})
    \end{align*}
    We can combine both of these into
    \begin{align*}
        p(r | \mathbf{x}, \mathbf{w})
        &= p(r = 1 | \mathbf{x}, \mathbf{w})^r p(r = 0 | \mathbf{x}, \mathbf{w})^{1 - r} \\
        &= \big[(1 - \epsilon)y(\mathbf{x}_n, \mathbf{w}) + \epsilon \big(1 - y(\mathbf{x}_n, \mathbf{w})\big)\big]^r
        \big[(1 - \epsilon)\big(1 - y(\mathbf{x}_n, \mathbf{w})\big)
        + \epsilon y(\mathbf{x}_n, \mathbf{w})\big]^{1 - r}
    \end{align*}
    Therefore, the negative log-likelihood is given by
    \begin{align*}
        - \ln p(\mathbf{r} | \mathbf{\mathbf{X}}, \mathbf{w})
        = - \ln \prod_{i=1}^N p(r_n | \mathbf{x}_n, \mathbf{w})
        = - \sum_{i=1}^{N} \{r_n \ln p(r_n = 1 | \mathbf{x}_n, \mathbf{w})
            + (1 - r_n) \ln p(r_n = 0 | \mathbf{x}, \mathbf{w})\}
    \end{align*}
    As a result, this is equivalent to minimizing the error function
    \[
        E(\mathbf{w}) = -\sum_{i=1}^{N}
        \big[r_n \ln\big\{(1 - \epsilon)y(\mathbf{x}_n, \mathbf{w})
            + \epsilon\big(1 - y(\mathbf{x}_n, \mathbf{w})\big)\big\}
            + (1 - r_n) \ln\big\{(1 - \epsilon)\big(1 - y(\mathbf{x}_n, \mathbf{w})\big)
            + \epsilon y(\mathbf{x}_n, \mathbf{w})\big]
    \]
    which for $\epsilon = 0$ is equivalent to $\eqref{eq:5.21}$.
\end{proof}

\section*{Exercise 5.5 $\star$}
Show that maximizing likelihood for a multiclass neural network
model in which the network outputs have the interpretation
$y_k(\mathbf{x}, \mathbf{w}) = p(t_k = 1 | \mathbf{x})$ is equivalent to minimization
of the cross-entropy function \eqref{eq:5.24}.

\vspace{1em}

\begin{proof}
    Let's consider the binary target variables $t_k \in \{0, 1\}$ have a 1-of-$K$
    coding scheme indicating the class. If we assume the class labels are independent,
    given the input vector, the conditional distribution of the targets is
    \[
        p(t_k | \mathbf{x}) = \prod_{k=1}^K p(t_k = 1 | \mathbf{x})^{t_k}
    \]
    As a result, the corresponding negative log likelihood is given by
    \[
        -\ln p(\mathbf{T} | \mathbf{X}, \mathbf{w})
        = -\ln \prod_{n=1}^N \prod_{k=1}^K p(t_{nk} = 1 | \mathbf{x}_n)^{t_{nk}}
        = -\ln \sum_{n=1}^{N} \sum_{k=1}^{K} t_{nk} \ln p(t_{nk} = 1 | \mathbf{x}_n)
    \]
    Therefore, maximizing the likelihood of the model is equivalent to minimization
    of the cross entropy function
    \begin{equation}\label{eq:5.24}\tag{5.24}
        E(\mathbf{w})
        = -\sum_{n=1}^{N} \sum_{k=1}^{K} t_{nk} \ln p(t_{nk} = 1 | \mathbf{x}_n)
    \end{equation}
\end{proof}

\section*{Exercise 5.6 $\star$}
Show the derivative of the error function (5.21) with respect
to the activation $a_k$ for output units having a softmax activation
function satisfies \eqref{eq:5.18}.

\vspace{1em}

\begin{proof}
    The general result for the derivative of the softmax function with respect to the
    activation $a_k$ was proved in Exercise 4.17 and is given by $\eqref{eq:4.106}$.
    Therefore, we have that
    \[
        \pdv{y_k}{a_k} = y_k(1 - y_k)
    \]
    Taking the derivative of
    \begin{equation}\label{eq:5.21}\tag{5.21}
        E(\mathbf{w}) = -\sum_{n=1}^{N} \{t_n \ln y_n + (1 - t_n) \ln(1 - y_n)\}
    \end{equation}
    with respect to $a_k$ yields
    \begin{align*}
        \pdv{a_k} E(\mathbf{w}) &= -t_k \pdv{a_k} \ln y_k - (1 - t_k) \pdv{a_k} \ln(1 - y_k)
        = -t_k(1 - y_k) + y_k(1 - t_k)y_k
        = y_k - t_k
    \end{align*}
    As a result,
    \begin{equation}\label{eq:5.18}\tag{5.18}
        \pdv{E}{a_k} = y_k - t_k
    \end{equation}
\end{proof}

\section*{Exercise 5.7 $\star$}
Show the derivative of the error function $\eqref{eq:5.21}$ with respect
to the activation $a_k$ for an output unit having a logistic sigmoid activation
function satisfies $\eqref{eq:5.18}$.

\vspace{1em}

\begin{proof}
    We've seen in Exercise 4.12 that
    \begin{equation}\tag{4.88}
        \pdv{a} \sigma(a) = \sigma(a)(1 - \sigma(a)
    \end{equation}
    Since the output unit has a logistic sigmoid activation function, then
    \[
        y_k = \sigma(a_k)
    \]
    Therefore, using $\eqref{eq:4.88}$ gives
     \[
         \pdv{y_k}{a_k} = \sigma(a_k)\big(1 - \sigma(a_k)\big) = y_k(1 - y_k)
    \]
    Analogously to Exercise 5.6, one can quickly reach that $\eqref{eq:5.18}$ holds.
\end{proof}

\section*{Exercise 5.8 $\star$}
We saw in $\eqref{eq:4.88}$ that the derivative of the logistic sigmoid
activation function can be expressed in terms of the function value itself.
Derive the corresponding result for the '$\tanh$' activation function defined
by (5.59).

\vspace{1em}

\begin{proof}
    Taking the derivative of the '$\tanh$' function is straightforward:
    \[
        \pdv{a} \tanh(a)
        = \pdv{a} \bigg(\frac{e^a - e^{-a}}{e^a + e^{-a}}\bigg)
        = \frac{\big(e^a + e^{-a}\big)^2 - \big(e^a - e^{-a}\big)^2}{\big(e^a + e^{-a}\big)^2}
        = 1 - \bigg(\frac{e^a - e^{-a}}{e^a + e^{-a}}\bigg)^2
        = 1 - \tanh(a)^2
    \]
    Notice that the derivative of the `$\tanh$` function can also be expressed
    as a function of itself.
\end{proof}

\section*{Exercise 5.9 $\star$}
The error function $\eqref{eq:5.21}$ for binary classification problems
was derived for a network having a logistic-sigmoid output activation
function, so that $0 \leq y(\mathbf{x}, \mathbf{w}) \leq 1$, and data having
target values $t \in \{0, 1\}$. Derive the corresponding error function
if we consider a network having an output $-1 \leq y(\mathbf{x}, \mathbf{w}) \leq 1$
and target values $t = 1$ for class $\mathcal{C}_1$ and $t = -1$ for
class  $\mathcal{C}_2$. What would be the appropiate choice of output
unit activation function?

\vspace{1em}

\begin{proof}
    The hyperbolic tangent is the appropiate choice for the ouput unit
    activation function, because `$\tanh$` is a sigmoid function and its
    values range between $-1$ and $1$. Let's consider the case of binary
    classification in which we interpret the network output $y(\mathbf{x}, \mathbf{w})$
    as the conditional probability $p(\mathcal{C}_1 | \mathbf{x})$, with
    $p(\mathcal{C}_2 | \mathbf{x})$ given by $1 - y(\mathbf{x}, \mathbf{w})$. The
    conditional distribution of targets given inputs is then of the form
    \[
        p(t | \mathbf{x}, \mathbf{w})
        = y(\mathbf{x}, \mathbf{w})^{\frac{1+t}{2}}
        \big\{1 - y(\mathbf{x}, \mathbf{w})\big\}^{\frac{1-t}{2}}
    \]
    Taking the negative log-likelihood then yields
    \[
        -\ln p(\mathbf{t} | \mathbf{X}, \mathbf{w})
        = -\ln \prod_{n = 1}^N p(t_n | \mathbf{x}_n, \mathbf{w})
        = -\sum_{n=1}^{N} \bigg\{\frac{1+t}{2} \ln y(\mathbf{x}, \mathbf{w})
        + \frac{1-t}{2} \ln\big(1 - y(\mathbf{x}, \mathbf{w})\big)\bigg\}
    \]
    As a result, maximizing the likelihood is equivalent to minimizing
    the error function
    \[
        E(\mathbf{w}) = -\sum_{n=1}^{N} \bigg\{\frac{1+t}{2} \ln y_n + \frac{1-t}{2} \ln(1 -  y_n)\bigg\}
    \]
    where $y_n$ denotes $y(\mathbf{x}_n, \mathbf{w})$.
\end{proof}

\section*{Exercise 5.10 $\star$}
Consider a Hessian matrix $\mathbf{H}$ with eigenvector equation $\eqref{eq:5.33}$. By
setting the vector $\mathbf{v}$ in (5.39) equal to each of the eigenvectors
$\mathbf{u}_i$ in turn, show that $\mathbf{H}$ is positive definite if,
and only if, all of its eigenvalues are positive.

\vspace{1em}

\begin{proof}
    Consider the eigenvector equation
    \begin{equation}\label{eq:5.33}\tag{5.33}
        \mathbf{H}\mathbf{u}_i = \lambda_i \mathbf{u}_i
    \end{equation}
    \begin{enumerate}
        \item [\to] Assume that $\mathbf{H}$ is positive definite. Then,
    \[
        \mathbf{u}_i^T \mathbf{H} \mathbf{u}_i = \lambda_i ||\mathbf{u}_i||^2 > 0
    \]
    which happens only if the eigenvalues $\lambda_i$ are positive.
    \vspace{1em}
    \item [\leftarrow] Suppose that the eigenvalues $\lambda_i$ are positive.
        Since the eigenvectors form an orthonormal basis, an arbitrary
        vector $\mathbf{v}$ can be written in the form
        \begin{equation}\label{eq:5.38}\tag{5.38}
            \mathbf{v} = \sum_{i} c_i \mathbf{u}_i
        \end{equation}
        Therefore,
        \[
            \mathbf{v}^T\mathbf{H}\mathbf{v}
            = \bigg(\sum_{i} c_i \mathbf{u}_i\bigg)^T \mathbf{H}
            \bigg(\sum_{i} c_i \mathbf{u}_i\bigg)
            = \bigg(\sum_{i} c_i \mathbf{u}_i\bigg)^T
            \bigg(\sum_{i} c_i \lambda_i \mathbf{u}_i\bigg)
            = \sum_{i} \sum_{j} \lambda_j c_i c_j \mathbf{u}_i^T \mathbf{u}_j
            = \sum_{i} \lambda_i c_i^2
        \]
        Since the eigenvalues $\lambda_i$ are positive,
        \[
            \mathbf{v}^T\mathbf{H}\mathbf{v} = \sum_{i} \lambda_i c_i^2 > 0
        \]
        for all $\mathbf{v}$, which proves that $\mathbf{H}$ is positive definite.
    \end{enumerate}
\end{proof}

\section*{Exercise 5.11 $\star \star ++++++++++++$}
Consider a quadratic error function defined by $\eqref{eq:5.32}$, in which the
Hessian matrix $\mathbf{H}$ has an eigenvalue equation given by $\eqref{eq:5.33}$.
Show that the contours of constant error are ellipses whose axes are aligned
with eigenvectors $\mathbf{u}_i$ with lengths that are inversly proportional
to the square root of the corresponding eigenvalues $\lambda_i$.

\vspace{1em}

\begin{proof}
    Analogously to what we've seen in Section 5.3.2, we're going
    to rewrite
    \begin{equation}\label{eq:5.32}\tag{5.32}
        E(\mathbf{w}) \approxeq E(\mathbf{w}^\star) +
        \frac{1}{2}(\mathbf{w} - \mathbf{w}^\star)^T\mathbf{H}(\mathbf{w} - \mathbf{w}^\star)
    \end{equation}
    as
    \begin{equation}\label{eq:5.36}\tag{5.36}
        E(\mathbf{w}) \approxeq E(\mathbf{w}^\star) + \frac{1}{2} \sum_{i} \lambda_i \alpha_i^2
    \end{equation}
    where we've expanded $(\mathbf{w} - \mathbf{w}^\star)$ as a linear combination
    of $\mathbf{H}$'s eigenvectors:
    \begin{equation}\label{eq:5.35}\tag{5.35}
       \mathbf{w} - \mathbf{w}^\star = \sum_{i} \alpha_i \mathbf{u}_i
    \end{equation}
    Now, since $\mathbf{w}$ and $\mathbf{w}^\star$ are fixed, let
    $\xi = 2E(\mathbf{w}) - 2E(\mathbf{w}^\star)$.
    Therefore, one can rewrite $\eqref{eq:5.36}$ as
     \[
         \xi \approxeq \sum_{i} \lambda_i\alpha_i^2
        \sum_{i} \frac{\alpha_i^2}{\lambda_i^{-1/2}}
    \]
    This equation describes an ellipsoid with axes aligned to
    the eigenvectors $\mathbf{u}_i$ (since we use the coordinate system
    defined in $\eqref{eq:5.35}$) with axes lengths that are inversly
    proportional to the square root of the corresponding eigenvalues
    $\lambda_i$.
\end{proof}

\section*{Exercise 5.12 $\star \star +++++++++++++$}
By considering the local Taylor expansion ($\ref{eq:5.32}$) of
an error function about a stationary point $\mathbf{w}^\star$, show
that the necessary and sufficient condition for the stationary point to be
a local minimum of the error function is that the Hessian matrix
$\mathbf{H}$, defined by $\eqref{eq:5.30}$ with $\widehat{\mathbf{w}} = \mathbf{w}^\star$,
be positive definite.

\vspace{1em}

\begin{proof}
    $ $
    \begin{enumerate}
        \item [\to] Suppose that $\mathbf{H}$ is positive definite. From
            $\eqref{eq:5.32}$ one could then find that
             \[
                 E(\mathbf{w}) - E(\mathbf{w}^\star) > 0
            \]
            for $\mathbf{w} \neq  \mathbf{w}^\star$. Therefore, $E(\mathbf{w}^\star)$
            would be the minimum value of E.
        \vspace{1em}
        \item [\leftarrow] Assume that $\mathbf{w}^\star$
            is a local minimum of E. Then,
            \[
                E(\mathbf{w}) -E(\mathbf{w}^\star) > 0
            \]
            which would mean that
            \[
                \frac{1}{2} (\mathbf{w} - \mathbf{w}^\star)^T \mathbf{H}
                (\mathbf{w} - \mathbf{w}^\star) > 0
            \]
            for $\mathbf{w} \neq \mathbf{w}^\star$, i.e.
            $\mathbf{H}$ is positive definite, since
            $\mathbf{w}$ respectively $\mathbf{w} - \mathbf{w}^\star$
            can be chosen arbitrarily.
    \end{enumerate}
\end{proof}


\subsection{Algebraic dual spaces}\label{subsec:algebraic_dual_spaces}

\begin{definition}\label{def:dual_vector_space}\mcite[50]{Knapp2016BasicAlgebra}
  Let \( V \) be a \hyperref[def:vector_space]{vector space} over the \hyperref[def:field]{field} \( \BbbK \). By \fullref{thm:functions_over_algebra}, the set \( \hom(V, \BbbK) \) of all \hyperref[def:semimodule/homomorphism]{linear maps} from \( V \) to the underlying field \( \BbbK \) also form a vector space over \( \BbbK \).

  We call this space the \term{algebraic dual space} of \( V \) and denote it by \( V^* \). We call the functions in \( V^* \) \term{linear functionals}. The prefix \enquote{algebraic} is important when confusion is possible with \hyperref[def:continuous_dual_space]{continuous dual spaces}.
\end{definition}

\begin{remark}\label{rem:functional}
  The term \enquote{functional} as a noun has no definite meaning.

  \begin{itemize}
    \item In the context of linear algebra, and in particular \fullref{def:dual_vector_space}, the term \enquote{functional} refers to \enquote{linear functional}, i.e. a \hyperref[def:semimodule/homomorphism]{linear map} from a \hyperref[def:vector_space]{vector space} to its base field.

    This terminology can be found, for example, in \cite[50]{Knapp2016BasicAlgebra} and \cite[sec. 26.1]{Тыртышников2004Лекции}.

    \item In the context of functional analysis, \enquote{linear functional} may refer to either \hyperref[def:continuous_dual_space]{continuous linear functionals} from some \hyperref[def:topological_vector_space]{topological vector space} to its base field, or to arbitrary linear functionals.

    This terminology can be found, for example, in \cite[def. 3.1]{Rudin1991Functional} and \cite[sec. 1.3]{Clarke2013}.

    An arbitrary map from a topological vector space to its field may also be called a functional --- for example, \cite[102]{KufnerFucik1980} and \cite[223]{Deimling1985} refer to \enquote{nonlinear functionals}. \hyperref[def:minkowski_functional]{Minkowski functionals} are notoriously nonlinear.

    \item In the context of recursive functions, for example in \cite{StanfordPlato:recursive_functions}, functionals are defined as \enquote{operations which map one or more functions of type \( \BbbN^k \to \BbbN \) (possibly of different arities) to other functions}.
  \end{itemize}

  The commonality between linear algebra and functional analysis is that \enquote{functional} refers to a map from a vector space to its base field. The commonality between functional analysis and logic is that \enquote{functional} refers to a map acting on a set of functions.
\end{remark}

\begin{definition}\label{def:duality_pairing}\mimprovised
  A \term{duality pairing} \( \inprod \anon \anon: U \times V \to \BbbK \) is a \hyperref[def:multilinear_function]{bilinear function}.

   such that, if \( \inprod u v = 0 \) for all \( u \), then \( v = 0 \).
\end{definition}

\begin{definition}\label{thm:natural_duality_pairing}
  Given a vector space \( V \), the following function is \hyperref[def:multilinear_function]{bilinear}:
  \begin{equation*}
    \begin{aligned}
      &\inprod \cdot \cdot: V^* \times X \to \BbbK \\
      &\inprod {x^*} x \mapsto x^*(x).
    \end{aligned}
  \end{equation*}
\end{definition}

\begin{proposition}\label{thm:algebraic_dual_basis}
  Fix a \hyperref[def:vector_space]{vector space} \( V \) over \( \BbbK \) and a \hyperref[def:hamel_basis]{basis} \( B \) of \( V \). For every basis vector \( e \), the \term{projection functional} \( \pi_e: V \to \BbbK \), defined in \fullref{def:basis_decomposition}, maps an arbitrary vector \( v \) to its \( e \)-th coordinate.

  \begin{thmenum}
    \thmitem{thm:algebraic_dual_basis/independent} Given basis vectors \( x \) and \( y \) from \( B \), \( \pi_x \) and \( \pi_y \) are \hyperref[def:linear_dependence]{linearly independent} in \( V^* \).

    \thmitem{thm:algebraic_dual_basis/finite} Furthermore, if \( e_1, \ldots, e_n \) is a basis of \( V \), then \( \pi_{e_1}, \ldots, \pi_{e_n} \) is a basis for the \hyperref[def:dual_vector_space]{dual space} \( V^* \).

    \thmitem{thm:algebraic_dual_basis/infinite} The set \( \set{ \pi_e \given e \in B } \) spans \( V^* \) if and only if \( V \) is finite dimensional.
  \end{thmenum}
\end{proposition}
\begin{proof}
  \SubProofOf{thm:algebraic_dual_basis/independent} Let \( t_x \) and \( t_y \) be scalars such that
  \begin{equation*}
    t_x \pi_x + t_y \pi_y = 0_V.
  \end{equation*}

  Then
  \begin{equation*}
    0 = t_x \pi_x(x) + t_y \pi_y(x) = t_x \cdot 1 + t_y \cdot 0.
  \end{equation*}

  Analogously, \( t_y = 0 \). Therefore, the functionals \( \pi_x \) and \( \pi_y \) are linearly independent.

  \SubProofOf{thm:algebraic_dual_basis/finite} Let \( l \) be an arbitrary linear functional. Then
  \begin{equation*}
    l(y)
  \end{equation*}

  \SubProofOf{thm:algebraic_dual_basis/infinite}
\end{proof}

\begin{proposition}\label{def:double_dual_canonical_embedding}
  Fix a vector space \( V \). We define the canonical embedding into the double dual \( V^{**} \) of \( V \) by
  \begin{balign*}
     & \Phi: V \to V^{**}                              \\
     & \Phi(x) \coloneqq (\varphi \mapsto \varphi(x)),
  \end{balign*}
  where \( \varphi \in V^* \).
\end{proposition}

\begin{proposition}\label{thm:finite_dimensional_dual_space_is_isomorphic}
  The dual vector space of a finite-dimensional vector space has the same dimension.
\end{proposition}
\begin{proof}
  Let \( V \) be an \( n \)-dimensional vector space over \( F \) and let \( B \) be a basis of \( V \). For each \( b \in B \), define its dual vector on \( V^* \) as the linear \hyperref[thm:quotient_module_universal_property]{extension} of the functions
  \begin{balign*}
     & \varphi: B \to F                               \\
     & \varphi(x) \coloneqq \begin{cases}
      1, & x = b    \\
      0, & x \neq b
    \end{cases}
  \end{balign*}
  from the basis to the whole space. Denote the dual basis vector of \( b \) by \( b^* \).

  We will now show that the set \( B^* \coloneqq \{ b^* \colon b \in B \} \) forms a basis of \( V^* \).

  Fix \( x^* \in V^* \). Define
  \begin{equation*}
    y^* \coloneqq \sum_{b \in B} x^*(b) b^*.
  \end{equation*}

  The linear functions \( x^* \) and \( y^* \) evidently agree on the basis \( B \). Hence, they agree on the whole space.

  Hence, \( B^* \) is a basis of \( V^* \). Note that it has the same cardinality as the basis of \( B \).
\end{proof}

\begin{remark}\label{rem:finite_dimensional_dual_space_isomorphism}
  By \fullref{thm:finite_dimensional_spaces_are_isomorphic}, the vector space \( F^n \) is isomorphic to its dual \( {F^n}^* \).

  In practice, it is sometimes useful to distinguish between vectors and functionals. This is why we regard functionals as either
  \begin{itemize}
    \item functions
    \item column vectors
    \item row vectors
  \end{itemize}
  depending on what interpretation suits us best.

  This is consistent with \fullref{thm:matrix_and_linear_function_algebras}, where we regard linear operators as matrices that act on vectors by multiplication.

  For example, if we have the \hyperref[def:differentiability]{differentiable} function \( f(x, y) = xy \), we can regard its gradient at the point \( (\overline x, \overline y) \) as the row vector
  \begin{balign*}
    f'(\overline x, \overline y) =
    \begin{pmatrix}
      \overline y & \overline x
    \end{pmatrix}.
  \end{balign*}

  This is a linear functional that can acts on regular (column) vector by multiplying them from the left.
\end{remark}

\begin{definition}\label{def:dual_linear_operator}
  We define the \term{dual linear operator} of \( L: U \to V \) as
  \begin{balign*}
     & L^*: V^* \to U^*                \\
     & L^*(v^*) \coloneqq v^* \circ L.
  \end{balign*}
\end{definition}

\begin{definition}\label{def:vector_space_annihilator}\mcite[52]{Knapp2016BasicAlgebra}
  Fix a subset \( S \subseteq V \) of a vector space \( V \) over \( F \). We define the \term{annihilator} of \( S \) as the vector space of functionals
  \begin{equation*}
    \op{ann}(S) \coloneqq \{ x^* \in V^* \colon x^*(x) = 0_F \quad\forall x \in S \}.
  \end{equation*}
\end{definition}


\documentclass[a4paper,10pt]{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath,amssymb,amstext,amsthm}
\usepackage{xcolor}

\DeclareMathOperator*{\E}{\mathbb{E}_{\pi}}   % rbp
\newcommand{\Ps}{ P_{ss'}^a}   % rbp
\newcommand{\R}{ R_{ss'}^a}   % rbp

\begin{document}

\section*{V and Q recursions}

\begin{align}
 V^{\pi}(s)   &= \E \left\{ \sum\limits_{k=0}^{\infty} \gamma^k r_{t+k+1} | s_t = s \right\} \label{vr-value} \\
 Q^{\pi}(s,a) &= \E \left\{ \sum\limits_{k=0}^{\infty} \gamma^k r_{t+k+1} | s_t = s, a_t = a \right\} \label{qr-value}
\end{align}

From Sutton's book page 70, Equation (3.10), recursive formulation of the value function update is:

\begin{align}
 V^{\pi}(s) &= \sum\limits_a \pi(s,a) \underbrace{\sum\limits_{s'} \Ps \left[ \R + \gamma V^{\pi}(s')\right]}_{Q^{\pi}(s,a)} \\
	    &= \sum\limits_a \pi(s,a) Q^{\pi}(s,a) \label{v-rec}
\end{align}

As an exercise (Exercise 3.8, page 72 in Suttons book), the same recursion has to be derived, but this time for
the Q-Value function, Equation \ref{qr-value}.

\begin{align*}
 Q^{\pi}(s,a) &= \E \left\{ \sum\limits_{k=0}^{\infty} \gamma^k  r_{t+k+1}  | s_t = s, a_t = a \right\}  \\
	      &= \E \left\{ r_{t+1} + \gamma \sum\limits_{k=0}^{\infty} \gamma^k r_{t+k+2} | s_t = s, a_t = a \right\} \\
	      &= \E \left\{ r_{t+1} |  s_t = s, a_t = a \right\} + \gamma\, \E \left\{ \sum\limits_{k=0}^{\infty} \gamma^k r_{t+k+1} | \textcolor{red}{s_{t}} = s, a_{t} = a  \right\} \\
	      &= \sum\limits_{s'} \Ps \R  		 + \gamma \textcolor{red}{\sum\limits_{s'} \Ps} \E \left\{ \sum\limits_{k=0}^{\infty} \gamma^k r_{t+k+2} |  \textcolor{red}{s_{t+1}} = s', \textcolor{blue}{a_{t}} = a  \right\} \\
	      &= \sum\limits_{s'} \Ps \R 		 + \gamma \sum\limits_{s'} \Ps  \textcolor{blue}{ \sum\limits_{a'} \pi(s',a')} \E \left\{ \sum\limits_{k=0}^{\infty} \gamma^k r_{t+k+2}  |  s_{t+1} = s', \textcolor{blue}{a_{t+1}} = a'  \right\} \\
      	      &= \sum\limits_{s'} \Ps \R 		 + \gamma \sum\limits_{s'} \Ps  \sum\limits_{a'} \pi(s',a') Q(s',a')  \\
      	      &= \sum\limits_{s'} \Ps \Big[ \R	  	 + \gamma \underbrace{ \sum\limits_{a'} \pi(s',a') Q(s',a')}_{V^{\pi}(s')} \Big]
\end{align*}

We not that the last term in the expansion of Q, contains Equation \ref{v-rec}. So we can substitute it back in and get:

\begin{equation}
  Q^{\pi}(s,a) = \sum\limits_{s'} \Ps \Big[ \R	 + \gamma V^{\pi}(s')  \Big]
\end{equation}

\end{document}


\subsection{Cauchy sequences}

\subsubsection{Cauchy sequence}

A cauchy sequence is a sequence such that for an any arbitrarily small number \(\epsilon\), there is a point in the sequence where all possible pairs after this are even closer together.

\((\forall \epsilon >0)(\exists N\in \mathbb{N}: \forall m,n \in \mathbb{N} >N)( |a_m - a_n|<\epsilon)\)

This last term gives a distance between two entries. In addition to the number line, this could be used on vectors, where distances are defined.

As a example, \(\dfrac{1}{n}\) is a cauchy sequence, \(\sum_i \dfrac{1}{n}\) is not.

\subsubsection{Completeness}

Cauchy sequences can be defined on some given set. For example given all the numbers between \(0\) and \(1\) there are any number of different cauchy sequences converging at some point.

If it is possible to define a cauchy sequence on a set where the limit is not in the set, then the set is incomplete.

For example, the numbers between \(0\) and \(1\) but not including \(0\) and \(1\) are not complete. It is possible to define sequences which converge to these missing points.

More abstractly, you could have all vectors where \(x^2+y^2<1\). This is incomplete (or open) as sequences on these vectors can converge to limits not in the set.

Cauchy sequences are important when considering real numbers. We could define a sequence converging on \(\sqrt 2\), but as this number is not in the set, it is incomplete.


\chapter{Unit 5}
\section{Area Between Two Curves}
Let $f(x)$ and $g(x)$ both be curves defined $\forall{x}\in[a,b]$. Let $A$ be
the area between these two curves.

We must always assume that $f(x)$ is the larger curve between the two. Should
this change, you must split the integral.

\begin{align}
  A &= \int_a^b [f(x)-g(x)] dx \\
    &= \int_c^d [f(y)-g(y)] dy
\end{align}

\section{Volume: The Disk Method}
\subsection{Disks}
\begin{align}
  V &= \int_a^b A(x) dx \\
    &= \pi\int_a^b [r(x)]^2 dx
\end{align}

\subsection{Washers}
\begin{align}
  V &= \int_a^b A(x) dx \\
    &= \int_a^b \pi(R^2-r^2) dx \\
    &= \pi\int_a^b (R^2-r^2) dx
\end{align}

\section{Volume: The Shell Method}
Let $r(x)$ be the distance of the slice to the axis of revolution, and $h(x)$ be
the width or height of that slice.

\begin{equation}
  V = 2\pi\int_a^b r(x)h(x) dx
\end{equation}

\section{Arc Length}
Let $L$ be the arc-length of a curve defined over an interval $[a,b]$.

\begin{equation}
  L_{\text{arc}} = \int_a^b \sqrt{1+[f'(x)]^2} dx
\end{equation}

\subsection{Surfaces}
Let $r(x)$ be the distance from the rotating arc to the axis of rotation.

\begin{equation}
  S_{\text{rev}} = 2\pi\int_a^b r(x)\sqrt{1+[f'(x)]^2} dx
\end{equation}

\section{Work}
\begin{align}
  W &= \vec{F}\cdot{\vec{D}}=FD\cos\theta \\
    &= \int_a^b F(x) dx
\end{align}

\subsection{Springs}
Some work has a special definition. For example, springs follow Hooke's Law:
\begin{equation}
  F=kd
\end{equation}

\subsection{Slabs of Liquid}
\begin{align}
  \Delta F &= \text{Volume} \cdot \text{Density} \\
           &= A\Delta{y}\rho \\
  \Delta W &= \Delta{F}\cdot x
\end{align}

\subsection{Chains and Lifting}
\begin{align}
  W &= W_c + W_{nc} \\
    &= (\text{weight}\cdot\text{distance}) + (\lambda{d}\Delta{d}) + \int_a^b
  \lambda(h-y) dy
\end{align}

\section{Moments, Centers of Mass}
\subsection{Discrete, Linear Systems}
\begin{equation}
  M_O = \frac{\sum_{i=1}^{n} m_ix_i}{\sum_{i=1}^n m_i}
\end{equation}

\subsection{Two-Dimensional Systems}
\begin{equation}
  M_y = \sum_{i=1}^{n} m_ix_i
\end{equation}

\begin{equation}
  M_x = \sum_{i=1}^{n} m_iy_i
\end{equation}

\begin{equation}
  (\bar{x}, \bar{y}) = \left( \frac{M_y}{m}, \frac{M_x}{m} \right)
\end{equation}

\subsection{Planar Lamina}
\begin{align}
  m &= Da \\
    &= \int_a^b \rho(x)[f(x)-g(x)] dx
\end{align}

\begin{equation}
  M_x = \int_a^b \rho(x)\left[ \frac{f(x)+g(x)}{2} \right]\left[ f(x)-g(x) \right] dx
\end{equation}

\begin{equation}
  M_y = \int_a^b \rho(x)\left[ f(x)-g(x) \right] dx
\end{equation}

\section{Fluid Pressure and Force}
\begin{description}
  \item[Pressure] $P=wh$
  \item[Force] $F=PA=whA$
  \item[Pascal's Principle] pressure exerted by a fluid at a depth h is
    transmitted equally in all directions.
\end{description}


\chapter{Bonus: Fourier analysis}
\label{ch:fourier}
Now that we've worked hard to define abstract inner product spaces,
I want to give an (optional) application:
how to set up Fourier analysis correctly, using this language.

For fun, I also prove a form of Arrow's Impossibility Theorem
using binary Fourier analysis.

In what follows, we let $\TT = \RR/\ZZ$ denote the ``circle group'',
thought of as the additive group of ``real numbers modulo $1$''.
There is a canonical map $e \colon \TT \to \CC$ sending $\TT$ to the
complex unit circle, given by
\[ e(\theta) = \exp(2\pi i \theta). \]

\section{Synopsis}
Suppose we have a domain $Z$ and are interested in functions $f \colon Z \to \CC$.
Naturally, the set of such functions form a complex vector space.
We like to equip the set of such functions
with an positive definite \emph{inner product}.
% usually something like $\left< f,g \right> = \EE_{x \in Z} f(x) \ol{g(x)}$.
%which makes the set of such functions into a normed vector space.
%(Here $\EE$ is an ``average'', which is a finite sum if $|Z| < \infty$
%but otherwise is usually an integral.)
%In particular, $\left< f,f\right>$ is the average of $|f(x)|^2$;
%and thus gives a positive definite inner form.

The idea of Fourier analysis is to then select an \emph{orthonormal basis}
for this set of functions, say $(e_\xi)_{\xi}$,
which we call the \vocab{characters};
the indexing $\xi$ are called \vocab{frequencies}.
In that case, since we have a basis, every function $f : Z \to \CC$
becomes a sum
\[ f(x) = \sum_{\xi} \wh f(\xi) e_\xi \]
where $\wh f(\xi)$ are complex coefficients of the basis;
appropriately we call $\wh f$ the \vocab{Fourier coefficients}.
The variable $x \in Z$ is referred to as the \vocab{physical} variable.
This is generally good because the characters are deliberately chosen
to be nice ``symmetric'' functions,
like sine or cosine waves or other periodic functions.
Thus we decompose an arbitrarily complicated function into a sum of nice ones.

\section{A reminder on Hilbert spaces}
For convenience, we record a few facts about orthonormal bases.
\begin{proposition}
	[Facts about orthonormal bases]
	\label{prop:orthonormal}
	Let $V$ be a complex Hilbert space
	with inner form $\left< -,-\right>$
	and suppose $x = \sum_\xi a_\xi e_\xi$ and $y = \sum_\xi b_\xi e_\xi$
	where $e_\xi$ are an orthonormal basis.
	Then
	\begin{align*}
		\left< x,x \right> &= \sum_\xi |a_\xi|^2 \\
		a_\xi &= \left< x, e_\xi \right> \\
		\left< x,y \right> &= \sum_\xi a_\xi \ol{b_\xi}.
	\end{align*}
\end{proposition}
\begin{exercise}
	Prove all of these.
	(You don't need any of the preceding section,
	it's only there to motivate the notation with lots of scary $\xi$'s.)
\end{exercise}

In what follows,
most of the examples will be of finite-dimensional inner product spaces
(which are thus Hilbert spaces),
but the example of ``square-integrable functions''
will actually be an infinite dimensional example.
Fortunately, as I alluded to earlier,
this is no cause for alarm
and you can mostly close your eyes and not worry about infinity.

\section{Common examples}
\subsection{Binary Fourier analysis on $\{\pm1\}^n$}
Let $Z = \{\pm 1\}^n$ for some positive integer $n$,
so we are considering functions $f(x_1, \dots, x_n)$ accepting binary values.
Then the functions $Z \to \CC$ form a $2^n$-dimensional vector space $\CC^Z$,
and we endow it with the inner form
\[ \left< f,g \right> = \frac{1}{2^n} \sum_{x \in Z} f(x) \ol{g(x)}. \]
In particular,
\[ \left< f,f \right>
	= \frac{1}{2^n} \sum_{x \in Z} \left\lvert f(x) \right\rvert^2 \]
is the average of the squares;
this establishes also that $\left< -,-\right>$ is positive definite.

In that case, the \vocab{multilinear polynomials} form a basis of $\CC^Z$,
that is the polynomials
\[ \chi_S(x_1, \dots, x_n) = \prod_{s \in S} x_s. \]
\begin{exercise}
	Show that they're actually orthonormal under $\left< -,-\right>$.
	This proves they form a basis, since there are $2^n$ of them.
\end{exercise}
Thus our frequency set is actually the subsets $S \subseteq \{1, \dots, n\}$.
Thus, we have a decomposition
\[ f = \sum_{S \subseteq \{1, \dots, n\}} \wh f(S) \chi_S. \]
\begin{example}
	[An example of binary Fourier analysis]
	Let $n = 2$.
	Then binary functions $\{ \pm 1\}^2 \to \CC$ have a basis
	given by the four polynomials
	\[ 1, \quad x_1, \quad x_2, \quad x_1x_2. \]
	For example, consider the function $f$
	which is $1$ at $(1,1)$ and $0$ elsewhere.
	Then we can put
	\[ f(x_1, x_2) = \frac{x_1+1}{2} \cdot \frac{x_2+1}{2}
		= \frac14 \left( 1 + x_1 + x_2 + x_1x_2 \right). \]
	So the Fourier coefficients are $\wh f(S) = \frac 14$
	for each of the four $S$'s.
\end{example}
This notion is useful in particular for
binary functions $f : \{\pm1\}^n \to \{\pm1\}$;
for these functions (and products thereof),
we always have $\left< f,f \right> = 1$.

It is worth noting that the frequency $\varnothing$ plays a special role:
\begin{exercise}
	Show that
	\[ \wh f(\varnothing) = \frac{1}{|Z|} \sum_{x \in Z} f(x). \]
\end{exercise}

\subsection{Fourier analysis on finite groups $Z$}
This time, suppose we have a finite abelian group $Z$,
and consider functions $Z \to \CC$;
this is a $|Z|$-dimensional vector space.
The inner product is the same as before:
\[ \left< f,g \right> = \frac{1}{|Z|} \sum_{x \in Z} f(x) \ol{g(x)}. \]

To proceed, we'll need to be able to multiply two elements of $Z$.
This is a bit of a nuisance since it actually won't really
matter what map I pick, so I'll move briskly;
feel free to skip most or all of the remaining paragraph.
\begin{definition}
We select a \emph{symmetric non-degenerate bilinear form}
\[ \cdot \colon Z \times Z \to \TT \]
satisfying the following properties:
\begin{itemize}
	\ii $\xi \cdot (x_1 + x_2) = \xi \cdot x_1 + \xi \cdot x_2$
	and $(\xi_1 + \xi_2) \cdot x = \xi_1 \cdot x + \xi_2 \cdot x$
	(this is the word ``bilinear'')
	\ii $\cdot$ is symmetric,
	\ii For any $\xi \neq 0$, there is an $x$ with $\xi \cdot x \neq 0$
	(this is the word ``nondegenerate'').
\end{itemize}
\end{definition}
\begin{example}
	[The form on $\Zc n$]
	If $Z = \Zc n$ then $\xi \cdot x = (\xi x)/n$ satisfies the above.
\end{example}
In general, it turns out finite abelian groups
decompose as the sum of cyclic groups (see \Cref{sec:FTFGAG}),
which makes it relatively easy to find such a $\cdot$;
but as I said the choice won't matter, so let's move on.

Now for the fun part: defining the characters.
\begin{proposition}
	[$e_\xi$ are orthonormal]
	For each $\xi \in Z$ we define the character
	\[ e_\xi(x) = e(\xi \cdot x). \]
	The $|Z|$ characters form an orthonormal basis of the
	space of functions $Z \to \CC$.
\end{proposition}
\begin{proof}
	I recommend skipping this one, but it is:
	\begin{align*}
		\left< e_{\xi}, e_{\xi'} \right>
		&= \frac{1}{|Z|} \sum_{x \in Z} e(\xi \cdot x) \ol{e(\xi' \cdot x)} \\
		&= \frac{1}{|Z|} \sum_{x \in Z} e(\xi \cdot x) e(-\xi' \cdot x) \\
		&= \frac{1}{|Z|} \sum_{x \in Z} e\left( (\xi-\xi') \cdot x \right).
	\end{align*}
\end{proof}

In this way, the set of frequencies is also $Z$,
but the $\xi \in Z$ play very different roles from the ``physical'' $x \in Z$.
Here is an example which might be enlightening.
\begin{example}
	[Cube roots of unity filter]
	Suppose $Z = \Zc3$, with the inner form given by $\xi \cdot x = (\xi x)/3$.
	Let $\omega = \exp(\frac 23 \pi i)$ be a primitive cube root of unity.
	Note that
	\[ e_\xi(x) = \begin{cases}
			1 & \xi = 0 \\
			\omega^x & \xi = 1 \\
			\omega^{2x} & \xi = 2.
		\end{cases} \]
	Then given $f \colon Z \to \CC$ with $f(0) = a$, $f(1) = b$, $f(2) = c$,
	we obtain
	\[ f(x) = \frac{a+b+c}{3} \cdot 1
		+ \frac{a + \omega^2 b + \omega c}{3} \cdot \omega^x
		+ \frac{a + \omega b + \omega^2 c}{3} \cdot \omega^{2x}.  \]
	In this way we derive that the transforms are
	\begin{align*}
		\wh f(0) &= \frac{a+b+c}{3} \\
		\wh f(1) &= \frac{a+\omega^2 b+ \omega c}{3} \\
		\wh f(2) &= \frac{a+\omega b+\omega^2c}{3}.
	\end{align*}
\end{example}
\begin{exercise}
	Show that in analogy to $\wh f(\varnothing)$
	for binary Fourier analysis, we now have
	\[ \wh f(0) = \frac{1}{|Z|} \sum_{x \in Z} f(x). \]
\end{exercise}
Olympiad contestants may recognize the previous example
as a ``roots of unity filter'', which is exactly the point.
For concreteness, suppose one wants to compute
\[ \binom{1000}{0} + \binom{1000}{3} + \dots + \binom{1000}{999}. \]
In that case, we can consider the function
\[ w : \ZZ/3 \to \CC. \]
such that $w(0) = 1$ but $w(1) = w(2) = 0$.
By abuse of notation we will also think of $w$
as a function $w : \ZZ \surjto \ZZ/3 \to \CC$.
Then the sum in question is
\begin{align*}
	\sum_n \binom{1000}{n} w(n)
	&= \sum_n \binom{1000}{n} \sum_{k=0,1,2} \wh w(k) \omega^{kn} \\
	&= \sum_{k=0,1,2} \wh w(k) \sum_n \binom{1000}{n} \omega^{kn} \\
	&= \sum_{k=0,1,2} \wh w(k) (1+\omega^k)^n.
\end{align*}
In our situation, we have $\wh w(0) = \wh w(1) = \wh w(2) = \frac13$,
and we have evaluated the desired sum.
More generally, we can take any periodic weight $w$
and use Fourier analysis in order to interchange the order of summation.

\begin{example}
	[Binary Fourier analysis]
	Suppose $Z = \{\pm 1\}^n$, viewed as an abelian group
	under pointwise multiplication
	hence isomorphic to $(\ZZ/2\ZZ)^{\oplus n}$.
	Assume we pick the dot product defined by
	\[ \left< \xi, x \right>
		= \half \sum_i \frac{\xi_i-1}{2} \cdot \frac{x_i-1}{2} \]
	where $\xi = (\xi_1, \dots, \xi_n)$ and $x = (x_1, \dots, x_n)$.

	We claim this coincides with the first example we gave.
	Indeed, let $S \subseteq \{1, \dots, n\}$
	and let $\xi \in \{\pm1\}^n$ which is $-1$ at positions in $S$,
	and $+1$ at positions not in $S$.
	Then the character $\chi_S$ from the previous example
	coincides with the character $e_\xi$ in the new notation.
	In particular, $\wh f(S) = \wh f(\xi)$.

	Thus Fourier analysis on a finite group $Z$ subsumes
	binary Fourier analysis.
\end{example}

\subsection{Fourier series for functions $L^2([-\pi, \pi])$}
This is the most famous one, and hence the one you've heard of.
\begin{definition}
	The space $L^2([-\pi, \pi])$ consists of all functions
	$f \colon [-\pi, \pi] \to \CC$ such that
	the integral
	$\int_{[-\pi, \pi]} \left\lvert f(x) \right\rvert^2 \; dx$
	exists and is finite,
	modulo the relation that a function which is zero ``almost everywhere''
	is considered to equal zero.\footnote{We won't define this, yet,
		as it won't matter to us for now.
		But we will elaborate more on this in the parts on measure theory.

		There is one point at which this is relevant.
		Often we require that the function $f$ satisfies $f(-\pi) = f(\pi)$,
		so that $f$ becomes a periodic function,
		and we can think of it as $f \colon \TT \to \CC$.
		This makes no essential difference
		since we merely change the value at one point.}

	It is made into an inner product space according to
	\[ \left< f,g \right>
		= \frac{1}{2\pi} \int_{[-\pi, \pi]} f(x) \ol{g(x)} \; dx. \]
\end{definition}
It turns out (we won't prove) that this is an
(infinite-dimensional) Hilbert space!

Now, the beauty of Fourier analysis is that
\textbf{this space has a great basis}:
\begin{theorem}
	[The classical Fourier basis]
	For each integer $n$, define
	\[ e_n(x) = \exp(inx). \]
	Then $e_n$ form an orthonormal basis
	of the Hilbert space $L^2([-\pi, \pi])$.
\end{theorem}
Thus this time the frequency set $\ZZ$ is infinite, and we have
\[ f(x) = \sum_n \wh f(n) \exp(inx)
	\quad\text{almost everywhere} \]
for coefficients $\wh f(n)$
with $\sum_n \left\lvert \wh f(n) \right\rvert^2 < \infty$.
Since the frequency set is indexed by $\ZZ$,
we call this a \vocab{Fourier series}
to reflect the fact that the index is $n \in \ZZ$.
\begin{exercise}
	Show once again
	\[ \wh f(0) = \frac{1}{2\pi} \int_{[-\pi, \pi]} f(x) \; dx. \]
\end{exercise}

\section{Summary, and another teaser}
We summarize our various flavors of Fourier analysis in the following table.
\[
	\begin{array}{llll}
		\hline
		\text{Type} & \text{Physical var} & \text{Frequency var}
			& \text{Basis functions} \\ \hline
		\text{Binary} & \{\pm1\}^n
			& \text{Subsets } S \subseteq \left\{ 1, \dots, n \right\}
			& \prod_{s \in S} x_s \\
		\text{Finite group} & Z & \xi \in Z, \text{ choice of } \cdot
			& e(\xi \cdot x) \\
		\text{Fourier series} & \TT \text{ or } [-\pi, \pi] & n \in \ZZ
			& \exp(inx) \\
		\text{Discrete} & \Zc n
			& \xi \in \Zc n
			& e(\xi x / n) \\
	\end{array}
\]
I snuck in a fourth row with $Z  = \Zc n$,
but it's a special case of the second row, so no cause for alarm.

Alluding to the future, I want to hint at how \Cref{ch:pontryagin} starts.
Each one of these is really a statement
about how functions from $G \to \CC$
can be expressed in terms of functions $\wh G \to \CC$,
for some ``dual'' $\wh G$.
In that sense, we could rewrite the above table as:
\[
	\begin{array}{llll}
		\hline
		\text{Name} & \text{Domain }G & \text{Dual }\wh G
			& \text{Characters} \\ \hline
		\text{Binary} & \{\pm1\}^n
			& S \subseteq \left\{ 1, \dots, n \right\}
			& \prod_{s \in S} x_s \\
		\text{Finite group} & Z
			& \xi \in \wh Z \cong Z & e( i \xi \cdot x) \\
		\text{Fourier series} & \TT \cong [-\pi, \pi]  & n \in \ZZ
			& \exp(inx) \\
		\text{Discrete} & \ZZ/n\ZZ & \xi \in \ZZ/n\ZZ
			& e(\xi x / n) \\
	\end{array}
\]
It will turn out that in general
we can say something about many different domains $G$,
once we know what it means to integrate a measure.
This is the so-called \emph{Pontryagin duality};
and it is discussed as a follow-up bonus in \Cref{ch:pontryagin}.

\section{Parseval and friends}
Here is a fun section in which you get to learn a lot of big names quickly.
Basically, we can take each of the three results
from Proposition~\ref{prop:orthonormal},
translate it into the context of our Fourier analysis
(for which we have an orthonormal basis of the Hilbert space),
and get a big-name result.

\begin{corollary}
	[Parseval theorem]
	Let $f \colon Z \to \CC$, where $Z$ is a finite abelian group.
	Then \[ \sum_\xi |\wh f(\xi)|^2 = \frac{1}{|Z|} \sum_{x \in Z} |f(x)|^2. \]
	Similarly, if $f \colon [-\pi, \pi] \to \CC$ is square-integrable then
	its Fourier series satisfies
	\[ \sum_n |\wh f(n)|^2 = \frac{1}{2\pi} \int_{[-\pi, \pi]} |f(x)|^2 \; dx. \]
\end{corollary}
\begin{proof}
Recall that $\left< f,f\right>$ is equal to the
square sum of the coefficients.
\end{proof}

\begin{corollary}
	[Fourier inversion formula]
	Let $f : Z \to \CC$, where $Z$ is a finite abelian group.
	Then \[ \wh f(\xi) = \frac{1}{|Z|} \sum_{x \in Z} f(x) \ol{e_\xi(x)}. \]
	Similarly, if $f : [-\pi, \pi] \to \CC$ is square-integrable then
	its Fourier series is given by
	\[ \wh f(n) = \frac{1}{2\pi} \int_{[-\pi, \pi]} f(x) \exp(-inx) \; dx. \]
\end{corollary}
\begin{proof}
Recall that in an orthonormal basis $(e_\xi)_\xi$,
the coefficient of $e_\xi$ in $f$ is $\left< f, e_\xi\right>$.
\end{proof}
\begin{ques}
	What happens when $\xi = 0$ above?
\end{ques}

\begin{corollary}
	[Plancherel theorem]
	Let $f : Z \to \CC$, where $Z$ is a finite abelian group.
	Then \[ \left< f,g \right> = \sum_{\xi \in Z} \wh f(\xi) \ol{\wh g(\xi)}. \]
	Similarly, if $f : [-\pi, \pi] \to \CC$ is square-integrable then
	\[ \left< f,g \right> = \sum_n \wh f(n) \ol{\wh g(n)}. \]
\end{corollary}
\begin{ques}
	Prove this one in one line (like before).
\end{ques}

\section{Application: Basel problem}
One cute application about Fourier analysis on $L^2([-\pi, \pi])$
is that you can get some otherwise hard-to-compute sums,
as long as you are willing to use a little calculus.

Here is the classical one:
\begin{theorem}
	[Basel problem]
	We have
	\[ \sum_{n \ge 1} \frac{1}{n^2} = \frac{\pi^2}{6}. \]
\end{theorem}
The proof is to consider the identity function $f(x) = x$,
which is certainly square-integrable.
Then by Parseval, we have
\[
	\sum_{n \in \ZZ} \left\lvert \wh f(n) \right\rvert^2
	= \left< f,f\right>
	= \frac{1}{2\pi} \int_{[-\pi, \pi]} \left\lvert f(x) \right\rvert^2 \; dx.
\]
A calculus computation gives
\[  \frac{1}{2\pi} \int_{[-\pi, \pi]} x^2 \; dx = \frac{\pi^2}{3}. \]
On the other hand, we will now compute all Fourier coefficients.
We have already that
\[ \wh f(0) = \frac{1}{2\pi} \int_{[-\pi, \pi]} f(x) \; dx
	= \frac{1}{2\pi} \int_{[-\pi, \pi]} x \; dx = 0. \]
For $n \neq 0$, we have by definition
(or ``Fourier inversion formula'', if you want to use big words)
the formula
\begin{align*}
	\wh f(n) &= \left< f, \exp(inx) \right> \\
	&= \frac{1}{2\pi} \int_{[-\pi, \pi]} x \cdot \ol{\exp(inx)} \; dx \\
	&= \frac{1}{2\pi} \int_{[-\pi, \pi]} x \exp(-inx) \; dx.
\end{align*}
The anti-derivative is equal to
$\frac{1}{n^2} \exp(-inx) (1+inx)$,
which thus with some more calculation gives that
\[ \wh f(n) = \frac{(-1)^n}{n} i. \]
So
\[ \sum_n \left\lvert \wh f(n) \right\rvert^2
	= 2 \sum_{n \ge 1} \frac{1}{n^2} \]
implying the result.

\section{Application: Arrow's Impossibility Theorem}
As an application of binary Fourier analysis,
we now prove a form of
\href{https://en.wikipedia.org/wiki/Arrow's_impossibility_theorem}{Arrow's theorem}.

Consider $n$ voters voting among $3$ candidates $A$, $B$, $C$.
Each voter specifies a tuple $v_i = (x_i, y_i, z_i) \in \{\pm1\}^3$ as follows:
\begin{itemize}
	\ii $x_i = 1$ if person $i$ ranks $A$ ahead of $B$, and $x_i = -1$ otherwise.
	\ii $y_i = 1$ if person $i$ ranks $B$ ahead of $C$, and $y_i = -1$ otherwise.
	\ii $z_i = 1$ if person $i$ ranks $C$ ahead of $A$, and $z_i = -1$ otherwise.
\end{itemize}
Tacitly, we only consider $3! = 6$ possibilities for $v_i$:
we forbid ``paradoxical'' votes of the form $x_i = y_i = z_i$
by assuming that people's votes are consistent
(meaning the preferences are transitive).

For brevity, let $x_\bullet = (x_1, \dots, x_n)$
and define $y_\bullet$ and $z_\bullet$ similarly.
Then, we can consider a voting mechanism
\begin{align*}
	f \colon \{\pm1\}^n &\to \{\pm1\} \\
	g \colon \{\pm1\}^n &\to \{\pm1\} \\
	h \colon \{\pm1\}^n &\to \{\pm1\}
\end{align*}
such that
\begin{itemize}
	\ii $f(x_\bullet)$ is the global preference of $A$ vs.\ $B$,
	\ii $g(y_\bullet)$ is the global preference of $B$ vs.\ $C$,
	\ii and $h(z_\bullet)$ is the global preference of $C$ vs.\ $A$.
\end{itemize}
We'd like to avoid situations where the global preference
$(f(x_\bullet), g(y_\bullet), h(z_\bullet))$ is itself paradoxical.

Let $\EE f$ denote the average value of $f$ across all $2^n$ inputs.
Define $\EE g$ and $\EE h$ similarly.
We'll add an assumption that $\EE f = \EE g = \EE h = 0$,
which provides symmetry
(and e.g.\ excludes the possibility that $f$, $g$, $h$
are constant functions which ignore voter input).
With that we will prove the following result:
\begin{theorem}
	[Arrow Impossibility Theorem]
	Assume that $(f,g,h)$ always avoids paradoxical outcomes,
	and assume $\EE f = \EE g = \EE h = 0$.
	Then $(f,g,h)$ is either a dictatorship or anti-dictatorship:
	there exists a ``dictator'' $k$ such that
	\[ f(x_\bullet) = \pm x_k, \qquad g(y_\bullet) = \pm y_k,
		\qquad h(z_\bullet) = \pm z_k \]
	where all three signs coincide.
\end{theorem}
Unlike the usual Arrow theorem, we do \emph{not} assume
that $f(+1, \dots, +1) = +1$ (hence possibility of anti-dictatorship).

\begin{proof}
	Suppose the voters each randomly select one of the $3!=6$
	possible consistent votes.
	In \Cref{prob:arrow_lemma} it is shown
	that the exact probability of a paradoxical outcome
	for any functions $f$, $g$, $h$ is given exactly by
	\[ \frac14 + \frac14 \sum_{S \subseteq \{1, \dots, n\}}
		\left( -\frac13 \right)^{\left\lvert S \right\rvert}
		\left( \wh f(S) \wh g(S) + \wh g(S) \wh h(S) + \wh h(S) \wh f(S) \right).
		\]
	Assume that this probability (of a paradoxical outcome) equals $0$.
	Then, we derive
	\[ 1 = \sum_{S \subseteq \{1, \dots, n\}}
		-\left( -\frac13 \right)^{\left\lvert S \right\rvert}
		\left( \wh f(S) \wh g(S) + \wh g(S) \wh h(S) + \wh h(S) \wh f(S) \right). \]
	But now we can just use weak inequalities.
	We have $\wh f(\varnothing) = \EE f = 0$ and similarly for $\wh g$ and $\wh h$,
	so we restrict attention to $|S| \ge 1$.
	We then combine the famous inequality $|ab+bc+ca| \le a^2+b^2+c^2$
	(which is true across all real numbers) to deduce that
	\begin{align*}
		1 &= \sum_{S \subseteq \{1, \dots, n\}}
		-\left( -\frac13 \right)^{\left\lvert S \right\rvert}
		\left( \wh f(S) \wh g(S) + \wh g(S) \wh h(S) + \wh h(S) \wh f(S) \right) \\
		&\le \sum_{S \subseteq \{1, \dots, n\}}
		\left( \frac13 \right)^{\left\lvert S \right\rvert}
		\left( \wh f(S)^2 + \wh g(S)^2 + \wh h(S)^2 \right) \\
		&\le \sum_{S \subseteq \{1, \dots, n\}} \left( \frac13 \right)^1
		\left( \wh f(S)^2 + \wh g(S)^2 + \wh h(S)^2 \right) \\
		&= \frac13 (1+1+1) = 1.
	\end{align*}
	with the last step by Parseval.
	So all inequalities must be sharp, and in particular $\wh f$, $\wh g$, $\wh h$
	are supported on one-element sets, i.e.\ they are linear in inputs.
	As $f$, $g$, $h$ are $\pm 1$ valued, each $f$, $g$, $h$ is itself
	either a dictator or anti-dictator function.
	Since $(f,g,h)$ is always consistent, this implies the final result.
\end{proof}

\section{\problemhead}

\begin{problem}
	[For calculus fans]
	Prove that
	\[ \sum_{n \ge 1} \frac{1}{n^4} = \frac{\pi^4}{90}. \]
	\begin{hint}
		Use Parseval again, but this time on $f(x) = x^2$.
	\end{hint}
\end{problem}

\begin{problem}
	\gim
	\label{prob:arrow_lemma}
	Let $f,g,h \colon \{\pm1\}^n \to \{\pm1\}$
	be any three functions.
	For each $i$, we randomly select $(x_i, y_i, z_i) \in \{\pm1\}^3$
	subject to the constraint that not all are equal
	(hence, choosing among $2^3-2=6$ possibilities).
	Prove that the probability that
	\[ f(x_1, \dots, x_n) = g(y_1, \dots, y_n) = h(z_1, \dots, z_n) \]
	is given by the formula
	\[ \frac14 + \frac14 \sum_{S \subseteq \{1, \dots, n\}}
		\left( -\frac13 \right)^{\left\lvert S \right\rvert}
		\left( \wh f(S) \wh g(S) + \wh g(S) \wh h(S) + \wh h(S) \wh f(S) \right)
		\]
	\begin{hint}
		Define the Boolean function $D : \{\pm 1\}^3 \to \RR$ by
		$D(a,b,c) = ab+bc+ca$.
		Write out the value of $D(a,b,c)$ for each $(a,b,c)$.
		Then, evaluate its expected value.
	\end{hint}
	\begin{sol}
	Define the Boolean function $D : \{\pm 1\}^3 \to \RR$ by
	\[ D(a,b,c) = ab + bc + ca
		= \begin{cases}
			3 & a,b,c \text{ all equal} \\
			-1 & a,b,c \text{ not all equal}.
		\end{cases}.
	\]
	Thus paradoxical outcomes arise when
	$D(f(x_\bullet), g(y_\bullet), h(z_\bullet)) = 3$.
	Now, we compute that for randomly selected
	$x_\bullet$, $y_\bullet$, $z_\bullet$ that
	\begin{align*}
		\EE D(f(x_\bullet), g(y_\bullet), h(z_\bullet))
		&= \EE \sum_S \sum_T
			\left( \wh f(S) \wh g(T) + \wh g(S) \wh h(T) + \wh h(S) \wh f(T) \right)
			\left( \chi_S(x_\bullet)\chi_T(y_\bullet) \right) \\
		&= \sum_S \sum_T
			\left( \wh f(S) \wh g(T) + \wh g(S) \wh h(T) + \wh h(S) \wh f(T) \right)
			\EE\left( \chi_S(x_\bullet)\chi_T(y_\bullet) \right).
	\end{align*}
	Now we observe that:
	\begin{itemize}
		\ii If $S \neq T$, then $\EE \chi_S(x_\bullet) \chi_T(y_\bullet) = 0$,
		since if say $s \in S$, $s \notin T$ then $x_s$ affects
		the parity of the product with 50\% either way,
		and is independent of any other variables in the product.
		\ii On the other hand, suppose $S = T$.
		Then
		\[ \chi_S(x_\bullet) \chi_T(y_\bullet)
			= \prod_{s \in S} x_sy_s. \]
		Note that $x_sy_s$ is equal to $1$ with probability $\frac13$
		and $-1$ with probability $\frac23$
		(since $(x_s, y_s, z_s)$ is uniform from $3!=6$ choices,
		which we can enumerate).
		From this an inductive calculation on $|S|$ gives that
		\[
			\prod_{s \in S} x_sy_s
			=
			\begin{cases}
				+1 & \text{ with probability } \half(1+(-1/3)^{|S|}) \\
				-1 & \text{ with probability } \half(1-(-1/3)^{|S|}).
			\end{cases}
		\]
		Thus
		\[ \EE \left( \prod_{s \in S} x_sy_s \right) = \left( -\frac13 \right)^{|S|}.  \]
	\end{itemize}
	Piecing this altogether, we now have that
	\[
		\EE D(f(x_\bullet), g(y_\bullet), h(z_\bullet))
		=
		\left( \wh f(S) \wh g(T) + \wh g(S) \wh h(T) + \wh h(S) \wh f(T) \right)
		\left( -\frac13 \right)^{|S|}.
	\]
	Then, we obtain that
	\begin{align*}
		&\EE \frac14 \left( 1 + D(f(x_\bullet), g(y_\bullet), h(z_\bullet)) \right) \\
		=& \frac14 + \frac14\sum_S
		\left( \wh f(S) \wh g(T) + \wh g(S) \wh h(T) + \wh h(S) \wh f(T) \right)
		\wh f(S)^2 \left( -\frac13 \right)^{|S|}.
	\end{align*}
	Comparing this with the definition of $D$ gives the desired result.
	\end{sol}
\end{problem}