\section{A catalogue of constraint functions\label{constraintAppendix}}

Let us consider some examples of constraints to clarify the procedure.

\subsection{Fixed point in space (`Nail')\label{constrNail}}

This simple constraint `nails' a particular point in a rigid body to a fixed point
in world space. (It's a very flexible nail, because despite fixing the position, it allows all
three modes of rotation.) Let \ve{p} be the position of the centre of mass of our rigid
body, \ve{s} the vector from the centre of mass to the point in the body we want to attach,
$\ve{\omega}$ the angular velocity of the rigid body, and \ve{t} the coordinates in world
space that we want to nail the point to. Then we can set up a simple constraint function,
\begin{equation}
\ve{c} = \ve{p} + \ve{s} - \ve{t}
\end{equation}
which equals the null vector when $\ve{p}+\ve{s}$ and $\ve{t}$ coincide, as required.
Since this is a three-dimensional vector equation, we are actually defining three constraints
at once. \ve{t} does not change over time, so we obtain
\begin{eqnarray}
\dot{\ve{c}} &=& \dot{\ve{p}} + \ve{\omega}\times\ve{s} \nonumber\\
&=& \dot{\ve{p}} - \dual{\ve{s}}\,\ve{\omega} \\
\ddot{\ve{c}} &=& \ddot{\ve{p}} + \dot{\ve{\omega}}\times\ve{s} +
    \ve{\omega}\times(\ve{\omega}\times\ve{s}) \nonumber\\
&=& \ddot{\ve{p}} - \dual{\ve{s}}\,\dot{\ve{\omega}} -
    \dual{(\ve{\omega}\times\ve{s})}\,\ve{\omega}
\end{eqnarray}
(cf.\ similar derivations in~\cite{Kalra:95}). We have already moved the `chosen variables' to
the rightmost position of each product. We will now factor $\dot{\ve{c}}$ and write out the
components of $\m{J}$ in terms of the vector components:

\begin{equation}
\label{constrEx1J}
\dot{\ve{c}} = \left[\begin{array}{ccc} 1&0&0\\0&1&0\\0&0&1 \end{array}\right]
    \dot{\ve{p}} + \left[\begin{array}{ccc}
    0 & s_3 & -s_2 \\ -s_3 & 0 & s_1 \\ s_2 & -s_1 & 0
    \end{array}\right] \ve{\omega}
\end{equation}

The two matrices in equation~\ref{constrEx1J} thus form two slices of $\m{J}$ at
the locations appropriate for $\dot{\ve{p}}$ and $\ve{\omega}$. We now continue to the
next step of the procedure:

\begin{eqnarray}
\dot{\m{J}}\dot{\ve{x}} =
\ddot{\ve{c}} - \m{J}\ddot{\ve{x}} &=&
    \big(\ddot{\ve{p}} - \dual{\ve{s}}\,\dot{\ve{\omega}} -
    \dual{(\ve{\omega}\times\ve{s})}\,\ve{\omega}\big) -
    (\ddot{\ve{p}} - \dual{\ve{s}}\,\dot{\ve{\omega}}) \nonumber\\
& = & -\dual{(\ve{\omega}\times\ve{s})}\,\ve{\omega} \nonumber\\
& = & \left[\begin{array}{ccc} 0 &
    \omega_1 s_2 - \omega_2 s_1 &
    \omega_1 s_3 - \omega_3 s_1 \\
    \omega_2 s_1 - \omega_1 s_2 & 0 &
    \omega_2 s_3 - \omega_3 s_2 \\
    \omega_3 s_1 - \omega_1 s_3 &
    \omega_3 s_2 - \omega_2 s_3 & 0
    \end{array}\right] \ve{\omega} \nonumber\\\label{constrEx1JDot}
\end{eqnarray}

We see that here there is only one slice for $\dot{\m{J}}$; the slice belonging to $\ddot{\ve{p}}$
is zero. Since $\ve{\omega}$ also occurs inside the matrix, there are actually several
alternative representations of this matrix which are equally valid.

\subsection{Ball-and-socket joint\label{constrJoint}}

Two rigid bodies are attached together at a particular point in each of the bodies.
They may not separate, but all three rotational degrees of freedom are permitted. This is
a good representation e.g.\ of a human shoulder joint.

Let $\ve{a}$ and $\ve{b}$ be the positions of the centres of mass in the first and
second rigid body respectively. Let $\ve{s}$ be the vector from $\ve{a}$ to the
attachment point, and $\ve{t}$ the vector from $\ve{b}$ to the attachment point.
Also let $\ve{\omega}$ be the angular velocity of the first body, and $\ve{\phi}$ that
of the second. Then our constraint function and its derivatives are:

\begin{eqnarray}
\ve{c} &=& \ve{a} + \ve{s} - \ve{b} - \ve{t} \\
\dot{\ve{c}} &=& \dot{\ve{a}} + \ve{\omega}\times\ve{s} -
    \dot{\ve{b}} - \ve{\phi}\times\ve{t} \\
\ddot{\ve{c}} &=& \ddot{\ve{a}} + \dot{\ve{\omega}}\times\ve{s} +
    \ve{\omega}\times(\ve{\omega}\times\ve{s}) -
    \ddot{\ve{b}} - \dot{\ve{\phi}}\times\ve{t} -
    \ve{\phi}\times(\ve{\phi}\times\ve{t})
\end{eqnarray}

The rest of the derivation is very similar to the previous example. We obtain four
matrix slices for \m{J} and two slices for $\dot{\m{J}}$.

\subsection{Rotation axis restriction (`Joystick')\label{constrJoystick}}

We now have formulae to define a ball-and-socket joint. How can we express other types
of joints? A good way of doing this is by augmenting the ball-and-socket constraint with
additional constraints which restrict the set of valid rotations. In this section I
derive expressions for a constraint which prohibits rotation about one particular axis~--
or, in other words, confines the axis of any valid rotation to a plane. In engineering terms,
this is called a universal joint~\cite{Shabana:01}. Let us define a unit vector \ve{n} which
points in the direction of the axis we want to prohibit; equivalently, this is the normal of
the confinement plane.

It is not completely easy to visualize what this type of constraint means. One good way to look
at it is to consider a standard two-axis joystick. If it is placed on a table, the two axes of
rotation lie in a plane parallel to the surface of this table. But you cannot turn the stick about
its own axis. Hence the normal of the constraint plane is orthogonal to the table surface.
Don't be confused by the fact that the joystick handle happens to point in the direction of the
normal~-- any sort of obscure shape may be substituted in its place without changing the nature
of the constraint!

A more common sort of joint is the \emph{hinge} or \emph{revolute joint}, which we find in most
doors, in our knees and elbows. It allows rotation only about one particular axis. We can
conveniently express it by employing two `joystick' constraints on the same body, each of which
confines the axis to a plane. Provided the two planes are not parallel, the axis about which
rotation may occur is just the line of intersection of these two planes. In summary, to make a
revolute joint, we first add a ball-and-socket joint. Then we find two non-collinear vectors
which are both orthogonal to the hinge axis, and use them as normal vectors for two `joystick'
constraints. This reduces the original number of six degrees of freedom to one~-- the angle of
the hinge.

For this derivation I will use an alternative notation for quaternions, which is used by
Shoemake~\cite{Shoemake:85}, amongst others. Instead of using the complex constants \qi{}, \qj{}
and \qk{}, a quaternion is written as a pair consisting of a scalar (the real part) and a 3D
vector (the three imaginary parts):
\begin{equation}
\q{q} = q_w + q_x\qi + q_y\qj + q_z\qk = [q_w, (q_x, q_y, q_z)^T] = [q_w, \ve{q}_v]
\end{equation}
Using this notation, we can write the quaternion product in terms of vector dot and cross products:
\begin{equation}\label{quatProduct2}
\q{p}\q{q} = [p_w,\; \ve{p}_v]\;[q_w,\; \ve{q}_v] =
    [p_w q_w - \ve{p}_v \cdot \ve{q}_v,\;
    p_w \ve{q}_v + q_w \ve{p}_v + \ve{p}_v \times \ve{q}_v]
\end{equation}

We shall now consider the relative rotation of two rigid bodies. Say the first body has an
orientation quaternion \q{p} and angular velocity \ve{\phi}, and the second body orientation
\q{q} and angular velocity \ve{\omega}. Assume that each quaternion expresses the rotation
required to transform from the body's frame to the world frame. Then the quaternion product
$\q{p}^{-1}\q{q}$ is the rotation required to transform from the second body's frame to the first
one's~-- that is, the relative rotation of the two bodies.

To confine the axis of rotation, we use the fact that the axis is contained in the imaginary
parts of a quaternion (equation~\ref{quatRotation}, page~\pageref{quatRotation}). We want the dot
product of this axis and the normal vector \ve{n} to be zero. Conveniently, the dot product
happens to be implicitly present in the real part of the quaternion product
(see equation~\ref{quatProduct2}). Hence we can define our constraint
function as follows:
\begin{equation}\label{rotConstrEqn}
c = \Re(\tilde{\ve{n}}\, \q{p}^{-1}\q{q})
\end{equation}

As with complex numbers, the function $\Re$ returns only the real part of its argument.
Since the real part of $\tilde{\ve{n}}$ is zero by definition, this is just the dot product of
\ve{n} and the axis of $\q{p}^{-1}\q{q}$, with an extra minus sign in
front~(cf.\ equation~\ref{quatProduct2}). The constraint function is a scalar because
we are only losing one degree of freedom.

The derivative of \q{q} with respect to time is
$\dot{\q{q}} = \frac{1}{2}\tilde{\ve{\omega}}\q{q}$. Pushing the differential operator
onto the inside of a quaternion inverse produces a minus sign and reverses the order, provided
we are dealing with a unit quaternion:
\begin{equation}
\frac{\diff}{\diff t} \q{p} = \frac{1}{2}\tilde{\ve{\phi}}\,\q{p} \iff
    \frac{\diff}{\diff t} (\q{p}^{-1}) = -\frac{1}{2}\q{p}\,\tilde{\ve{\phi}}
\end{equation}

We now have everything in place to calculate the constraint function derivatives:
\begin{eqnarray}
\dot{c} & = & \label{constrJoystickC}
    \frac{1}{2} \Re(\tilde{\ve{n}}\, \q{p}^{-1}\, \tilde{\ve{\omega}}\,\q{q}) -
    \frac{1}{2} \Re(\tilde{\ve{n}}\, \q{p}^{-1}\, \tilde{\ve{\phi}}\, \q{q}) \\
\ddot{c} & = & \label{constrJoystickCDot}
    \frac{1}{2} \Re(\tilde{\ve{n}}\, \q{p}^{-1}\, \tilde{\dot{\ve{\omega}}}\,\q{q})
  - \frac{1}{2} \Re(\tilde{\ve{n}}\, \q{p}^{-1}\, \tilde{\dot{\ve{\phi}}}\, \q{q}) \\*
&&+ \frac{1}{4} \Re(\tilde{\ve{n}}\, \q{p}^{-1}\, \tilde{\ve{\omega}}\,\tilde{\ve{\omega}}\, \q{q})
  - \frac{1}{2} \Re(\tilde{\ve{n}}\, \q{p}^{-1}\, \tilde{\ve{\phi}}\,\tilde{\ve{\omega}}\, \q{q})
  + \frac{1}{4} \Re(\tilde{\ve{n}}\, \q{p}^{-1}\, \tilde{\ve{\phi}}\,\tilde{\ve{\phi}}\, \q{q})
    \nonumber
\end{eqnarray}

We manipulate these equations into the form required to find \m{J}:
\begin{eqnarray*}
\Re(\tilde{\ve{n}}\, \q{p}^{-1}\, \tilde{\ve{\omega}}\,\q{q}) & = &
    \Re\big( [0,\:\ve{n}]\: [p_w,\:-\ve{p}_v]\: [0,\:\ve{\omega}]\: [q_w,\:\ve{q}_v] \big) \\*
&=& \Re\big( [\ve{n}\cdot\ve{p}_v,\: p_w\ve{n} - \ve{n}\times\ve{p}_v]\:
             [-\ve{\omega}\cdot\ve{q}_v,\: q_w\ve{\omega} + \ve{\omega}\times\ve{q}_v] \big) \\*
&=& -(\ve{n}\cdot\ve{p}_v) (\ve{\omega}\cdot\ve{q}_v) -
    ( p_w\ve{n} - \ve{n}\times\ve{p}_v ) \cdot ( q_w\ve{\omega} + \ve{\omega}\times\ve{q}_v ) \\*
&=& -\ve{n}^T \ve{p}_v \ve{q}_v^T \ve{\omega} - ( p_w\ve{n} + \ve{p}_v\times\ve{n} )^T
    ( q_w\ve{\omega} - \ve{q}_v\times\ve{\omega} ) \\*
&=& -\ve{n}^T \ve{p}_v \ve{q}_v^T \ve{\omega} - ( p_w\ve{n}^T - \ve{n}^T\dual{\ve{p}_v} )
    ( q_w\ve{\omega} - \dual{\ve{q}_v}\,\ve{\omega} ) \\*
&=& -\ve{n}^T \big( \ve{p}_v \ve{q}_v^T + ( p_w\m{1} - \dual{\ve{p}_v} )
    ( q_w\m{1} - \dual{\ve{q}_v} ) \big)\, \ve{\omega}
\end{eqnarray*}

Here \m{1} denotes the $3\times3$ identity matrix. The same derivation is valid if we substitute
\ve{\phi} for \ve{\omega}, hence we obtain the Jacobian
\begin{eqnarray}
\m{J}\dot{\ve{x}} & = &
    -\frac{1}{2} \ve{n}^T \big( \ve{p}_v \ve{q}_v^T + ( p_w\m{1} - \dual{\ve{p}_v} )
    ( q_w\m{1} - \dual{\ve{q}_v} ) \big)\, \ve{\omega} \nonumber\\*
&&  +\frac{1}{2} \ve{n}^T \big( \ve{p}_v \ve{q}_v^T + ( p_w\m{1} - \dual{\ve{p}_v} )
    ( q_w\m{1} - \dual{\ve{q}_v} ) \big)\, \ve{\phi}
\end{eqnarray}

Now the first two terms of equation~\ref{constrJoystickCDot} are generated by $\m{J}\ddot{x}$, so
for finding $\dot{\m{J}}$ we need only consider the last three terms. Let us evaluate the
penultimate term:
\begin{eqnarray*}
\Re(\tilde{\ve{n}}\, \q{p}^{-1}\, \tilde{\ve{\phi}}\,\tilde{\ve{\omega}}\, \q{q}) & = &
    \Re\big( [0,\:\ve{n}]\: [p_w,\:-\ve{p}_v]\: [0,\:\ve{\phi}]\: [0,\:\ve{\omega}]\:
    [q_w,\:\ve{q}_v] \big) \\*
&=& \Re\big( [0,\:\ve{n}]\: [\ve{p}_v \cdot \ve{\phi},\: p_w\ve{\phi} - \ve{p}_v\times\ve{\phi}]\:
    [-\ve{\omega}\cdot\ve{q}_v,\: q_w\ve{\omega} + \ve{\omega}\times\ve{q}_v] \big) \\*
&=& \Re\Big( \begin{array}[t]{lll} \big[ &
    \ve{n}\cdot(\ve{p}_v\times\ve{\phi}) - p_w \ve{\phi}\cdot\ve{n}, &\\ &
    (\ve{p}_v\cdot\ve{\phi}) \ve{n} + p_w\ve{n}\times\ve{\phi} -
        \ve{n}\times(\ve{p}_v\times\ve{\phi}) \quad\big] &\\
    \big[ & -\ve{\omega}\cdot\ve{q}_v,\: q_w\ve{\omega} +
        \ve{\omega}\times\ve{q}_v \quad\big] & \Big) \end{array} \\
&=& -\big( \ve{n}\cdot(\ve{p}_v\times\ve{\phi}) - p_w \ve{\phi}\cdot\ve{n} \big)
     \big( \ve{\omega}\cdot\ve{q}_v \big) \\* &&
    -\big( (\ve{p}_v\cdot\ve{\phi}) \ve{n} + p_w\ve{n}\times\ve{\phi} -
        \ve{n}\times(\ve{p}_v\times\ve{\phi}) \big) \cdot
     \big( q_w\ve{\omega} - \ve{q}_v\times\ve{\omega} \big) \\
&=& -\big( \ve{n}\cdot(\ve{p}_v\times\ve{\phi}) \big) \big( \ve{q}_v\cdot\ve{\omega} \big)
    + p_w (\ve{n}\cdot\ve{\phi}) (\ve{q}_v\cdot\ve{\omega}) \\* &&
    +\big( \ve{n}\cdot(\ve{q}_v\times\ve{\omega}) \big) \big( \ve{p}_v\cdot\ve{\phi} \big)
    - q_w (\ve{n}\cdot\ve{\omega}) (\ve{p}_v\cdot\ve{\phi}) \\* &&
    -\big( \ve{n}\times(p_w\ve{\phi} - \ve{p}_v\times\ve{\phi}) \big) \cdot
     \big( q_w\ve{\omega} - \ve{q}_v\times\ve{\omega} \big) \\
&=&  \ve{n}^T (p_w\m{1} - \dual{\ve{p}_v})\, \ve{\phi}\,\ve{q}_v^T \ve{\omega}
    -\ve{n}^T (q_w\m{1} - \dual{\ve{q}_v})\, \ve{\omega}\,\ve{p}_v^T \ve{\phi} \\* &&
    +(p_w\ve{\phi} - \ve{p}_v\times\ve{\phi})^T \dual{\ve{n}}
    (q_w\m{1} - \dual{\ve{q}_v})\, \ve{\omega}
\end{eqnarray*}

Fortunately, the other two terms of equation~\ref{constrJoystickCDot} we are interested in are
similar, so we can obtain them by substitution in the last expression. This gives us the
following expression for $\dot{\m{J}}$:

\begin{eqnarray}
\dot{\m{J}}\dot{\ve{x}} &=&
    \frac{1}{4}\Big( \ve{n}^T (p_w\m{1} - \dual{\ve{p}_v})\, \ve{\omega}\,\ve{q}_v^T
    -\ve{n}^T (q_w\m{1} - \dual{\ve{q}_v})\, \ve{\omega}\,\ve{p}_v^T \nonumber\\*&&\quad
    +(p_w\ve{\omega} - \ve{p}_v\times\ve{\omega})^T \dual{\ve{n}} (q_w\m{1} - \dual{\ve{q}_v})
    \nonumber\\*&&\quad
    -2\ve{n}^T (p_w\m{1} - \dual{\ve{p}_v})\, \ve{\phi}\,\ve{q}_v^T
    -2(p_w\ve{\phi} - \ve{p}_v\times\ve{\phi})^T \dual{\ve{n}} (q_w\m{1} - \dual{\ve{q}_v})
    \Big)\, \ve{\omega} + \nonumber\\*&&
    \frac{1}{4}\Big( \ve{n}^T (p_w\m{1} - \dual{\ve{p}_v})\, \ve{\phi}\,\ve{q}_v^T
    -\ve{n}^T (q_w\m{1} - \dual{\ve{q}_v})\, \ve{\phi}\,\ve{p}_v^T \nonumber\\*&&\quad
    +(p_w\ve{\phi} - \ve{p}_v\times\ve{\phi})^T \dual{\ve{n}} (q_w\m{1} - \dual{\ve{q}_v})
    \nonumber\\*&&\quad
    +2\ve{n}^T (q_w\m{1} - \dual{\ve{q}_v})\, \ve{\omega}\,\ve{p}_v^T \Big)\,\ve{\phi}
\end{eqnarray}

\subsection{Rotation angle limitation}

As mentioned in section~\ref{generalizedCollisions}, the constraint function of the last section
can be used in an inequality context to limit the angle of rotation rather than just the axes.
Adding a constant value to the constraint function makes no difference to its derivatives and
therefore leaves the Jacobians derived in the last section unchanged.

So the implementation is easy, but the challenge is to understand what effect such an inequality
actually has. Since quaternions are rather hard to visualize, let us translate the meaning of
equation~\ref{rotConstrEqn} into Euler angles. Choose three orthogonal axes with basis vectors
\ve{a}, \ve{b} and \ve{g} such that $\ve{a}\times\ve{b}=\ve{g}$ and $\ve{b}\times\ve{g}=\ve{a}$
and $\ve{g}\times\ve{a}=\ve{b}$ and $\norm{\ve{a}} = \norm{\ve{b}} = \norm{\ve{g}} = 1$.
Consider the rotation $\q{p}^{-1}\q{q}$ (the relative rotation between the two bodies), and
decompose it into a sequence of three rotations: first a rotation of $\alpha$ about the \ve{a}
axis, then a rotation of $\beta$ about the \ve{b} axis, and finally a rotation of $\gamma$ about
the \ve{g} axis. Thus we have
\begin{eqnarray}
\q{p}^{-1}\q{q} &=& \q{g}\q{b}\q{a} \\
\q{a} &=& \cos\big(\frac{\alpha}{2}\big) + \tilde{\ve{a}}\sin\big(\frac{\alpha}{2}\big) \\
\q{b} &=& \cos\big(\frac{\beta }{2}\big) + \tilde{\ve{b}}\sin\big(\frac{\beta }{2}\big) \\
\q{g} &=& \cos\big(\frac{\gamma}{2}\big) + \tilde{\ve{g}}\sin\big(\frac{\gamma}{2}\big)
\end{eqnarray}

Now if we evaluate the constraint function about each of the axes \ve{a}, \ve{b} and \ve{g}, we
get (skipping some boring algebra):
\begin{eqnarray}
\Re(\tilde{\ve{a}}\,\q{g}\q{b}\q{a}) &=& -\sin\big(\frac{\alpha}{2}\big)
     \cos\big(\frac{\beta }{2}\big)   \cos\big(\frac{\gamma}{2}\big)
    -\cos\big(\frac{\alpha}{2}\big)   \sin\big(\frac{\beta }{2}\big)
     \sin\big(\frac{\gamma}{2}\big)   \label{angleLimit1}\\
\Re(\tilde{\ve{b}}\,\q{g}\q{b}\q{a}) &=& -\cos\big(\frac{\alpha}{2}\big)
     \sin\big(\frac{\beta }{2}\big)   \cos\big(\frac{\gamma}{2}\big)
    -\sin\big(\frac{\alpha}{2}\big)   \cos\big(\frac{\beta }{2}\big)
     \sin\big(\frac{\gamma}{2}\big)   \label{angleLimit2}\\
\Re(\tilde{\ve{g}}\,\q{g}\q{b}\q{a}) &=& -\cos\big(\frac{\alpha}{2}\big)
     \cos\big(\frac{\beta }{2}\big)   \sin\big(\frac{\gamma}{2}\big)
    -\sin\big(\frac{\alpha}{2}\big)   \sin\big(\frac{\beta }{2}\big)
     \cos\big(\frac{\gamma}{2}\big)   \label{angleLimit3}
\end{eqnarray}

These expressions make clear how interdependent the three axes are~-- it is generally not
possible to change a constraint on one axis without affecting the others. The best way to proceed
from here is to enter inequalities using formulae~\ref{angleLimit1} to~\ref{angleLimit3} into
a program for graphical visualization, and to experimentally determine the values such that the
desired behaviour is achieved.

\subsection{Confinement to a plane (vertex/face collision) \label{vertexFaceConstraint}}

We want to define a constraint function whose value is the distance between a point
and a plane, where the point is attached to one rigid body, and the plane to another. The plane
is defined by a point in the plane and a normal vector. The distance should be positive if the
point is on the side of the plane pointed to by the normal, and negative if it is on the opposite
side. If we put this constraint directly into the Lagrange multiplier equation, it will enforce
the condition that the distance be zero~-- the point is confined to move only within the plane.
But if we use the same constraint function in an inequality, we have a handler for the
vertex/face collision case.

Say \ve{a} is the centre of mass position of the body to which the plane is attached, and \ve{s}
is the vector from the centre of mass to an arbitrary point in the plane. The angular velocity
of this body is \ve{\omega}. The plane has a unit normal vector $\hat{\ve{n}}$
($\norm{\hat{\ve{n}}} = 1$). The point we are interested in is $\ve{b} + \ve{t}$, where \ve{b} is
the centre of mass position of the body to which the point belongs. This body has angular
velocity \ve{\phi}.

Then the constraint function and its derivatives are given by
\begin{eqnarray}
c &=& (\ve{b} + \ve{t} - \ve{a} - \ve{s})\cdot\hat{\ve{n}} \\
\dot{c} &=& (\dot{\ve{b}} + \ve{\phi}\times\ve{t} - \dot{\ve{a}} - \ve{\omega}\times\ve{s})
    \cdot\hat{\ve{n}} + (\ve{b} + \ve{t} - \ve{a} - \ve{s})\cdot(\ve{\omega}\times\hat{\ve{n}})
    \nonumber\\
&=& \hat{\ve{n}}^T\dot{\ve{b}} - \hat{\ve{n}}^T\dot{\ve{a}} - \hat{\ve{n}}^T\dual{\ve{t}}\ve{\phi}
    + (\ve{a} - \ve{b} - \ve{t})^T \dual{\hat{\ve{n}}} \ve{\omega} \\
\ddot{c} &=& \big( \ddot{\ve{b}} + \dot{\ve{\phi}}\times\ve{t} +
    \ve{\phi}\times(\ve{\phi}\times\ve{t}) - \ddot{\ve{a}} \big) \cdot \hat{\ve{n}}+\nonumber\\*&&
    2\big(\dot{\ve{b}} + \ve{\phi}\times\ve{t} - \dot{\ve{a}}\big)
    \cdot \big(\ve{\omega}\times\hat{\ve{n}}\big)
    + \big(\ve{b} + \ve{t} - \ve{a}\big) \cdot \big( \dot{\ve{\omega}}\times\hat{\ve{n}} +
    \ve{\omega}\times(\ve{\omega}\times\hat{\ve{n}}) \big) \nonumber\\
&=&  \hat{\ve{n}}^T \ddot{\ve{b}}
    -\hat{\ve{n}}^T \ddot{\ve{a}}
    -\hat{\ve{n}}^T\dual{\ve{t}} \dot{\ve{\phi}}
    +(\ve{a} - \ve{b} - \ve{t})^T \dual{\hat{\ve{n}}} \dot{\ve{\omega}} + \\*&&
    \big( 2(\dot{\ve{a}} - \dot{\ve{b}} - \ve{\phi}\times\ve{t})^T \dual{\hat{\ve{n}}}
    +(\ve{a} - \ve{b} - \ve{t})^T \dual{(\ve{\omega}\times\hat{\ve{n}})}\big) \ve{\omega}
    -\hat{\ve{n}}^T\dual{(\ve{\phi}\times\ve{t})} \ve{\phi} \nonumber
\end{eqnarray}
from which \m{J} and $\dot{\m{J}}$ can be read off as usual.

\subsection{Edge/edge collision\label{edgeEdgeConstraint}}

In this section we require a constraint function whose value is the shortest distance between
two straight lines. The problem is closely related to the one in
section~\ref{vertexFaceConstraint}. If used as an equality constraint, it could be used to
simulate two metal rods which are joined together such that the join can move up or down either
of the rods, but the rods always have to touch in one point~-- a kind of combination of a
ball-and-socket joint with two one-dimensional sliding rails. However, the practical use of such
a system is rather limited. Much more important is the use of this constraint as an inequality,
where it can handle the collision situation in which two edges collide.

We assume that each straight line is connected to a rigid body. The first body's centre of mass
is located at \ve{a}, its angular velocity is \ve{\omega}, \ve{s} is a vector from the centre
of mass to an arbitrary point on the line, and \ve{u} is a vector pointing along the line (unit
magnitude is not required). Similarly, the second body's CoM is \ve{b}, its angular velocity is
\ve{\phi}, \ve{t} points at the line and \ve{v} points in the line's direction. In this
derivation we shall assume that the lines are not parallel
($\norm{\ve{u}\times\ve{v}} \ne \ve{0}$); the parallel case is special and needs to be handled
separately.

If \ve{u} and \ve{v} are not parallel, we can find a unique plane which contains one line and is
parallel to the other. This plane has a normal vector $\ve{n} = \ve{u}\times\ve{v}$ (or
$\ve{n} = \ve{v}\times\ve{u}$). Interestingly this normal is the direction in which the force or
impulse acts in the event of a collision. It is not obvious that this is the case, but by playing
around with two books or similar it is possible to convince oneself. Then the closest distance
between the two lines is given by
\begin{equation}
c = (\ve{b} + \ve{t} - \ve{a} - \ve{s})\cdot\frac{\ve{n}}{\norm{\ve{n}}}
\end{equation}

The derivation of the Jacobian matrices involves some of the most messy linear algebra in this
project, so hold on tight. Let us first define a few auxiliary variables:
\begin{eqnarray}
\ve{n} & = & \ve{u}\times\ve{v} \\
h &=& \frac{1}{\norm{\ve{n}}} = (\ve{n}\cdot\ve{n})^{-\frac{1}{2}} \\
\ve{z} &=& \dot{\ve{n}} = \dot{\ve{u}}\times\ve{v} + \ve{u}\times\dot{\ve{v}} \nonumber\\*
    &=& (\ve{\omega}\times\ve{u})\times\ve{v} + \ve{u}\times(\ve{\phi}\times\ve{v}) \nonumber\\*
    &=& \dual{\ve{v}}\dual{\ve{u}}\ve{\omega} - \dual{\ve{u}}\dual{\ve{v}}\ve{\phi} \\
\ve{y} &=& \dot{\ve{z}} = \ddot{\ve{u}}\times\ve{v} + 2\,\dot{\ve{u}}\times\dot{\ve{v}} +
        \ve{u}\times\ddot{\ve{v}} \nonumber\\*
    &=& (\dot{\ve{\omega}}\times\ve{u})\times\ve{v} +
        (\ve{\omega}\times(\ve{\omega}\times\ve{u}))\times\ve{v} +
        2\,(\ve{\omega}\times\ve{u})\times(\ve{\phi}\times\ve{v}) + \nonumber\\*&&
        \ve{u}\times(\dot{\ve{\phi}}\times\ve{v}) +
        \ve{u}\times(\ve{\phi}\times(\ve{\phi}\times\ve{v})) \\*
    &=& \dual{\ve{v}}\dual{\ve{u}}\dot{\ve{\omega}} - \dual{\ve{u}}\dual{\ve{v}}\dot{\ve{\phi}} +
        \dual{\ve{v}}\dual{(\ve{\omega}\times\ve{u})}\ve{\omega} -
        \dual{\ve{u}}\dual{(\ve{\phi}\times\ve{v})}\ve{\phi} +
        2\,\dual{(\ve{\phi}\times\ve{v})}\dual{\ve{u}}\ve{\omega} \nonumber\\
\hat{\ve{n}} & = & h\ve{n} \\
\dot{\hat{\ve{n}}} & = & h\ve{z} - \frac{1}{2}\ve{n}(\ve{n}\cdot\ve{n})^{-\frac{3}{2}}
        (\ve{z}\cdot\ve{n} + \ve{n}\cdot\ve{z}) \nonumber\\*
    &=& h\ve{z} - h^3\,(\ve{n}\cdot\ve{z})\,\ve{n} \\
\ddot{\hat{\ve{n}}} & = & h\ve{y} - 2h^3 (\ve{n}\cdot\ve{z})\,\ve{z} -
        h^3 (\ve{n}\cdot\ve{y})\,\ve{n}
        -h^3 (\ve{z}\cdot\ve{z})\,\ve{n} + 3h^5 (\ve{n}\cdot\ve{z})^2\,\ve{n}
\end{eqnarray}

Now we can turn to calculating the derivatives of $c$:
\begin{eqnarray}
\dot{c} &=& \hat{\ve{n}}\cdot(\dot{\ve{b}} + \ve{\phi}\times\ve{t} - \dot{\ve{a}} -
        \ve{\omega}\times\ve{s}) + (\ve{b}+\ve{t}-\ve{a}-\ve{s})\cdot\dot{\hat{\ve{n}}} \\*
    &=& h\,(\ve{u}\times\ve{v})^T (\dot{\ve{b}} + \ve{\phi}\times\ve{t} - \dot{\ve{a}} -
        \ve{\omega}\times\ve{s}) + \nonumber\\*&&
        (\ve{b}+\ve{t}-\ve{a}-\ve{s})^T \big( h\ve{z} - h^3\,(\ve{n}\cdot\ve{z})\,\ve{n} \big)
        \nonumber\\
    &=& h\ve{u}^T\dual{\ve{v}}\dot{\ve{b}} - h\ve{u}^T\dual{\ve{v}}\dot{\ve{a}} -
        h\ve{u}^T\dual{\ve{v}}\dual{\ve{t}}\ve{\phi} +
        h\ve{u}^T\dual{\ve{v}}\dual{\ve{s}}\ve{\omega} + \nonumber\\*&&
        (\ve{b}+\ve{t}-\ve{a}-\ve{s})^T
        \big(h\dual{\ve{v}}\dual{\ve{u}}\ve{\omega} - h\dual{\ve{u}}\dual{\ve{v}}\ve{\phi} -
        h^3\dual{\ve{u}}\ve{v}\ve{u}^T\dual{\ve{v}}
        (\dual{\ve{v}}\dual{\ve{u}}\ve{\omega} - \dual{\ve{u}}\dual{\ve{v}}\ve{\phi})\big)
        \nonumber\\
    &=& h\ve{u}^T\dual{\ve{v}}\dot{\ve{b}} - h\ve{u}^T\dual{\ve{v}}\dot{\ve{a}} \nonumber\\*&&
        +\Big( h\ve{u}^T\dual{\ve{v}}\dual{\ve{s}} + h\,(\ve{b}+\ve{t}-\ve{a}-\ve{s})^T
        (\m{1} - h^2 \dual{\ve{u}}\ve{v}\ve{u}^T\dual{\ve{v}}) \dual{\ve{v}}\dual{\ve{u}}
        \Big)\:\ve{\omega} \nonumber\\*&&
        -\Big( h\ve{u}^T\dual{\ve{v}}\dual{\ve{t}} + h\,(\ve{b}+\ve{t}-\ve{a}-\ve{s})^T
        (\m{1} - h^2 \dual{\ve{u}}\ve{v}\ve{u}^T\dual{\ve{v}}) \dual{\ve{u}}\dual{\ve{v}}
        \Big)\:\ve{\phi} \\*
    &=& \m{J}\dot{\ve{x}} \nonumber\\
\ddot{c} &=& \hat{\ve{n}}\cdot\Big(\ddot{\ve{b}} + \dot{\ve{\phi}}\times\ve{t} +
        \ve{\phi}\times(\ve{\phi}\times\ve{t}) - \ddot{\ve{a}} - \dot{\ve{\omega}}\times\ve{s} -
        \ve{\omega}\times(\ve{\omega}\times\ve{s})\Big) + \nonumber\\*&&
        2\,\Big(\dot{\ve{b}} + \ve{\phi}\times\ve{t} - \dot{\ve{a}} - \ve{\omega}\times\ve{s}\Big)
        \cdot\dot{\hat{\ve{n}}} +
        \Big(\ve{b} + \ve{t} - \ve{a} - \ve{s}\Big)\cdot\ddot{\hat{\ve{n}}} \\
    &=& h\ve{u}^T\dual{\ve{v}}\ddot{\ve{b}} - h\ve{u}^T\dual{\ve{v}}\ddot{\ve{a}} -
        h\ve{u}^T\dual{\ve{v}}\dual{\ve{t}}\dot{\ve{\phi}} +
        h\ve{u}^T\dual{\ve{v}}\dual{\ve{s}}\dot{\ve{\omega}} + \nonumber\\*&&
        h\ve{u}^T\dual{\ve{v}}\dual{(\ve{\omega}\times\ve{s})}\ve{\omega} -
        h\ve{u}^T\dual{\ve{v}}\dual{(\ve{\phi}\times\ve{t})}\ve{\phi} \nonumber\\*&&
        +2h\,(\dot{\ve{b}} + \ve{\phi}\times\ve{t} - \dot{\ve{a}} - \ve{\omega}\times\ve{s})^T
        \Big(\dual{\ve{v}}\dual{\ve{u}}\ve{\omega} - \dual{\ve{u}}\dual{\ve{v}}\ve{\phi}
        \nonumber\\*&&\quad\quad -h^2\dual{\ve{u}}\ve{v}\ve{u}^T\dual{\ve{v}}
        (\dual{\ve{v}}\dual{\ve{u}}\ve{\omega} - \dual{\ve{u}}\dual{\ve{v}}\ve{\phi})\Big)
        \nonumber\\*&& +(\ve{b} + \ve{t} - \ve{a} - \ve{s})^T \Big(
        \nonumber\\*&&\quad\quad h\,\big(
        \dual{\ve{v}}\dual{\ve{u}}\dot{\ve{\omega}} - \dual{\ve{u}}\dual{\ve{v}}\dot{\ve{\phi}} +
        \dual{\ve{v}}\dual{(\ve{\omega}\times\ve{u})}\ve{\omega} \nonumber\\*&&\quad\quad\quad\quad
        -\dual{\ve{u}}\dual{(\ve{\phi}\times\ve{v})}\ve{\phi} +
        2\,\dual{(\ve{\phi}\times\ve{v})}\dual{\ve{u}}\ve{\omega} \big)
        \nonumber\\*&&\quad\quad
        -2h^3 (\dual{\ve{v}}\dual{\ve{u}}\ve{\omega} - \dual{\ve{u}}\dual{\ve{v}}\ve{\phi})
        \ve{u}^T\dual{\ve{v}}
        (\dual{\ve{v}}\dual{\ve{u}}\ve{\omega} - \dual{\ve{u}}\dual{\ve{v}}\ve{\phi})
        \nonumber\\*&&\quad\quad
        -h^3\dual{\ve{u}}\ve{v}\ve{u}^T\dual{\ve{v}} \big(
        \dual{\ve{v}}\dual{\ve{u}}\dot{\ve{\omega}} - \dual{\ve{u}}\dual{\ve{v}}\dot{\ve{\phi}} +
        \dual{\ve{v}}\dual{(\ve{\omega}\times\ve{u})}\ve{\omega} \nonumber\\*&&\quad\quad\quad\quad
        -\dual{\ve{u}}\dual{(\ve{\phi}\times\ve{v})}\ve{\phi} +
        2\,\dual{(\ve{\phi}\times\ve{v})}\dual{\ve{u}}\ve{\omega} \big)
        \nonumber\\*&&\quad\quad
        -h^3 \dual{\ve{u}}\ve{v}
        (\ve{\omega}^T\dual{\ve{u}}\dual{\ve{v}} - \ve{\phi}^T\dual{\ve{v}}\dual{\ve{u}})
        (\dual{\ve{v}}\dual{\ve{u}}\ve{\omega} - \dual{\ve{u}}\dual{\ve{v}}\ve{\phi})
        \nonumber\\*&&\quad\quad
        +3h^5 \dual{\ve{u}}\ve{v} \ve{u}^T\dual{\ve{v}}
        (\dual{\ve{v}}\dual{\ve{u}}\ve{\omega} - \dual{\ve{u}}\dual{\ve{v}}\ve{\phi})
        \ve{u}^T\dual{\ve{v}}
        (\dual{\ve{v}}\dual{\ve{u}}\ve{\omega} - \dual{\ve{u}}\dual{\ve{v}}\ve{\phi}) \Big) \nonumber
\end{eqnarray}
\newpage

Finally we subtract the terms generated by $\m{J}\ddot{\ve{x}}$ from $\ddot{c}$ and separate
into factors of \ve{\omega} and \ve{\phi} as usual:
\begin{eqnarray}
\dot{\m{J}}\dot{\ve{x}} &=& \bigg(
        h\ve{u}^T\dual{\ve{v}}\dual{(\ve{\omega}\times\ve{s})} \\*&&\quad\quad +
        2h\,(\dot{\ve{b}} + \ve{\phi}\times\ve{t} - \dot{\ve{a}} - \ve{\omega}\times\ve{s})^T
        (\m{1} - h^2\dual{\ve{u}}\ve{v}\ve{u}^T\dual{\ve{v}}) \dual{\ve{v}}\dual{\ve{u}}
        \nonumber\\*&&\quad\quad
        +(\ve{b} + \ve{t} - \ve{a} - \ve{s})^T \Big(
        h\dual{\ve{v}}\dual{(\ve{\omega}\times\ve{u})}
        +2h\,\dual{(\ve{\phi}\times\ve{v})}\dual{\ve{u}} \nonumber\\*&&\quad\quad\quad\quad
        -h^3\dual{\ve{u}}\ve{v}\ve{u}^T\dual{\ve{v}}\dual{\ve{v}}\dual{(\ve{\omega}\times\ve{u})}
        -2h^3\dual{\ve{u}}\ve{v}\ve{u}^T\dual{\ve{v}}\dual{(\ve{\phi}\times\ve{v})}\dual{\ve{u}}
        \nonumber\\*&&\quad\quad\quad\quad
        -2h^3\,(\dual{\ve{v}}\dual{\ve{u}}\ve{\omega} - \dual{\ve{u}}\dual{\ve{v}}\ve{\phi})
        \ve{u}^T\dual{\ve{v}}\dual{\ve{v}}\dual{\ve{u}}
        \nonumber\\*&&\quad\quad\quad\quad
        -h^3 \dual{\ve{u}}\ve{v} \big(
        \ve{\omega}^T\dual{\ve{u}}\dual{\ve{v}} - \ve{\phi}^T\dual{\ve{v}}\dual{\ve{u}}
        \nonumber\\*&&\quad\quad\quad\quad\quad\quad
        -3h^2 \ve{u}^T\dual{\ve{v}}
        (\dual{\ve{v}}\dual{\ve{u}}\ve{\omega} - \dual{\ve{u}}\dual{\ve{v}}\ve{\phi})
        \ve{u}^T\dual{\ve{v}} \big) \dual{\ve{v}}\dual{\ve{u}} \Big) \bigg)\: \ve{\omega}
        \nonumber\\*&&
        -\bigg(
        h\ve{u}^T\dual{\ve{v}}\dual{(\ve{\phi}\times\ve{t})} \nonumber\\*&&\quad\quad
        +2h\,(\dot{\ve{b}} + \ve{\phi}\times\ve{t} - \dot{\ve{a}} - \ve{\omega}\times\ve{s})^T
        (\m{1} - h^2\dual{\ve{u}}\ve{v}\ve{u}^T\dual{\ve{v}}) \dual{\ve{u}}\dual{\ve{v}}
        \nonumber\\*&&\quad\quad
        +(\ve{b} + \ve{t} - \ve{a} - \ve{s})^T \Big(
        h\dual{\ve{u}}\dual{(\ve{\phi}\times\ve{v})}
        -h^3\dual{\ve{u}}\ve{v}\ve{u}^T\dual{\ve{v}}\dual{\ve{u}}\dual{(\ve{\phi}\times\ve{v})}
        \nonumber\\*&&\quad\quad\quad\quad
        -2h^3\,(\dual{\ve{v}}\dual{\ve{u}}\ve{\omega} - \dual{\ve{u}}\dual{\ve{v}}\ve{\phi})
        \ve{u}^T\dual{\ve{v}} \dual{\ve{u}}\dual{\ve{v}}
        \nonumber\\*&&\quad\quad\quad\quad
        -h^3 \dual{\ve{u}}\ve{v} \big(
        \ve{\omega}^T\dual{\ve{u}}\dual{\ve{v}} - \ve{\phi}^T\dual{\ve{v}}\dual{\ve{u}}
        \nonumber\\*&&\quad\quad\quad\quad\quad\quad
        -3h^2 \ve{u}^T\dual{\ve{v}}
        (\dual{\ve{v}}\dual{\ve{u}}\ve{\omega} - \dual{\ve{u}}\dual{\ve{v}}\ve{\phi})
        \ve{u}^T\dual{\ve{v}}
        \big) \dual{\ve{u}}\dual{\ve{v}} \Big) \bigg)\: \ve{\phi} \nonumber
\end{eqnarray}


\chapter{Applications and Case Studies}
This chapter contains a number of case studies designed to deepen our understanding of \textsl{Python}.

\section{Solving Equations via Fixed-Point Algorithms}
\href{https://en.wikipedia.org/wiki/Fixed-point_iteration}{Fixed-Point iterations} are very important, both in
computer science and in mathematics.  As a first example, we show how to solve a given equation numerically via
a fixed point iteration. \index{fixed point iteration} Suppose we want to solve the equation
\\[0.2cm]
\hspace*{1.3cm} $x = \cos(x)$. \\[0.2cm]
Here, $x$ is a real number that we seek to compute.  Figure \ref{fig:xEqualsCosX.pdf} on page
\pageref{fig:xEqualsCosX.pdf} shows the graphs of the two functions
\\[0.2cm]
\hspace*{1.3cm}
$y = x$  \quad and \quad $y = \cos(x)$.
\\[0.2cm]
Since the graphs of these functions intersect, it is obvious that there exists a value $x$ such that
\\[0.2cm]
\hspace*{1.3cm}
$x = \cos(x)$.
\\[0.2cm]
Furthermore, from Figure \ref{fig:xEqualsCosX.pdf} it is obvious that this value of $x$ is somewhat bigger than $0.6$
but less than $0.8$.

\begin{figure}[!ht]
  \hspace*{-3.0cm}
  \epsfig{file=Figures/xEqualsCosX.pdf,scale=0.6}

  \caption{The functions $y = x$ and $y = cos(x)$.}
  \label{fig:xEqualsCosX.pdf}
\end{figure}

A simple approach that enables us to solve the equation $x = \cos(x)$ is to conduct a
\href{https://en.wikipedia.org/wiki/Fixed-point_iteration}{fixed-point iteration}.  To this end, we
define the sequence $\bigl(x_n\bigr)_{n\in\mathbb{N}}$ inductively as follows:
\\[0.2cm]
\hspace*{1.3cm}
$x_0 = 0$ \quad and \quad $x_{n+1} = \mathtt{cos}(x_n)$ \quad for all $n \in \mathbb{N}$.
\\[0.2cm]
With the help of the
\href{https://en.wikipedia.org/wiki/Banach_fixed-point_theorem}{Banach fixed-point theorem}\footnote{
  The Banach fixed-point theorem is discussed in the lecture on
  \href{https://en.wikipedia.org/wiki/Differential_calculus}{differential calculus}.  This lecture is part of the
  second semester.
}
\index{Banach fixed-point theorem}
it can be shown that this sequence converges to a solution of the equation $x = \cos(x)$, i.e.~if we define
\\[0.2cm]
\hspace*{1.3cm}
$\bar{x} = \lim\limits_{n\rightarrow\infty} x_n$,
\\[0.2cm]
then we have
\\[0.2cm]
\hspace*{1.3cm}
$\cos\bigl(\bar{x}\bigr) = \bar{x}$.
\\[0.2cm]
Figure \ref{fig:solve.py} on page \pageref{fig:solve.py} shows the program
\href{https://github.com/karlstroetmann/Logic/blob/master/Python/solve.py}{\texttt{solve.py}}
that uses this approach to solve the equation $x = \cos(x)$.

\begin{figure}[!ht]
  \centering
\begin{minted}[ frame         = lines,
                framesep      = 0.3cm,
                numbers       = left,
                numbersep     = -0.2cm,
                bgcolor       = sepia,
                xleftmargin   = 0.8cm,
                xrightmargin  = 0.8cm,
              ]{python3}
    import math

    x     = 1.0
    old_x = 0.0
    i     = 1
    while abs(x - old_x) >= 4.0E-16:
        old_x = x
        x = math.cos(x)
        print(f'{i} : {x}')
        i += 1
\end{minted}
\vspace*{-0.3cm}
\caption{Solving the equation $x = \cos(x)$ via fixed-point iteration.}  \label{fig:solve.py}
\end{figure} %\$

In this program, the iteration stops as soon as the difference between the variables \texttt{x} and
\texttt{old\_x} is less that $4 \cdot 10^{-16}$.  Here, \texttt{x} corresponds to $x_{n+1}$, while \texttt{old\_x}
corresponds to $x_n$.  Once the values of $x_{n+1}$ and $x_n$ are sufficiently close, the execution of the \texttt{while} loop
terminates.
\href{https://github.com/karlstroetmann/Logic/blob/master/Python/Fixed-Point-Iteration.ipynb}{Fixed-Point-Iteration.ipynb}
shows a \textsl{Jupyter} notebook that implements fixed point iteration.

\begin{figure}[!ht]
\centering
\begin{minted}[ frame         = lines,
                framesep      = 0.3cm,
                firstnumber   = 1,
                numbers       = left,
                numbersep     = -0.2cm,
                bgcolor       = sepia,
                xleftmargin   = 0.8cm,
                xrightmargin  = 0.8cm,
              ]{python3}
    from math import cos

    def solve(f, x0):
        """
        Solve the equation f(x) = x using a fixed point iteration.
        x0 is the start value.
        """
        x = x0
        for n in range(10000):  # at most 10000 iterations
            oldX = x;
            x    = f(x);
            if abs(x - oldX) < 1.0e-15:
                return x;

    print("solution to x = cos(x): ", solve(cos, 0));
    print("solution to x = 1/(1+x):", solve(lambda x: 1/(1+x), 0));
\end{minted}
\vspace*{-0.3cm}
\caption{A generic implementation of the fixed-point algorithm.}
\label{fig:fixed-point.py}
\end{figure}

Figure \ref{fig:fixed-point.py} on page \pageref{fig:fixed-point.py} shows the program
\href{https://github.com/karlstroetmann/Logic/blob/master/Python/fixed-point.py}{\texttt{fixed-point.py}}.
In this program we have implemented a function \texttt{solve} that takes two arguments.
\begin{enumerate}
\item \texttt{f} is a unary function.  The purpose of the \texttt{solve} is to compute the solution of the equation
      \\[0.2cm]
      \hspace*{1.3cm}
      $f(x) = x$.
      \\[0.2cm]
      This equation is solved with the help of a fixed-point algorithm.
\item \texttt{x0} is used as the initial value for the fixed-point iteration.
\end{enumerate}
Line 11 calls \texttt{solve} to compute the solution of the equation $x = \cos(x)$.
Line 12 solves the equation
\\[0.2cm]
\hspace*{1.3cm}
$\ds x = \bruch{1}{1+x}$.
\\[0.2cm]
This equation is equivalent to the quadratic equation $x^2 + x = 1$.  Note that we have defined the function
 $\ds x \mapsto \frac{1}{1+x}$ via the expression
 \\[0.2cm]
\hspace*{1.3cm}
\texttt{lambda x: 1/(1+x)}.
\\[0.2cm]
This expression is called an \blue{anonymous function}
\index{lambda expression, \texttt{lambda x: f(x)}}
since we haven't given a name to the function.

\remarkEng
The function \texttt{solve} is only able to solve the equation $f(x) = x$ if the function $f$ is a
\href{https://en.wikipedia.org/wiki/Contraction_mapping}{contraction mapping} (Deutsch: \blue{kontrahierende Abbildung}).
\index{contraction mapping}
  A function
$f:\mathbb{R} \rightarrow \mathbb{R}$
is called a \blue{contraction mapping} iff
\\[0.2cm]
\hspace*{1.3cm}
$|f(x) - f(y)| < |x - y|$ \quad for all $x,y \in \mathbb{R}$.
\\[0.2cm]
This notion will be discussed in more detail in the lecture on
\href{https://github.com/karlstroetmann/Analysis/blob/master/Skript/analysis.pdf}{analysis} in the second
semester. \eox

\section{Case Study: Computation of Poker Probabilities}
\index{poker}
In this short section we are going to show how to compute probabilities for the
\href{https://en.wikipedia.org/wiki/Texas_hold_%27em}{\textsl{Texas Hold'em}} variation of
\href{https://en.wikipedia.org/wiki/Poker}{poker}.   Texas Hold'em poker is played with a deck of 52
cards.  Every card has a \blue{value}.  This value is an element of the set
\\[0.2cm]
\hspace*{1.3cm}
$\textsl{Values} = \{ 2, 3, 4, 5, 6, 7, 8, 9, 10, \textsl{Jack}, \textsl{Queen}, \textsl{King}, \textsl{Ace} \}$.
\\[0.2cm]
Furthermore, every card has a \blue{suit}.  This suit is an element of the set
\\[0.2cm]
\hspace*{1.3cm}
$\textsl{Suits} = \{ \club, \mbox{$\color{red}{\heart}$}, \mbox{$\color{red}{\diamondsuit}$}, \spade \}$.
\\[0.2cm]
These suits are pronounced \blue{club}, \blue{heart}, \blue{diamond}, and \blue{spade}.
As a card is determined by its value and its suit, a card can be represented as a pair $\pair(v,s)$, where $v$
denotes the value while $s$ is the suit of the card.  Hence, the set of all cards can be represented as the set
\\[0.2cm]
\hspace*{1.3cm}
$\textsl{Deck} = \bigl\{ \pair(v,s) \mid v \in \textsl{Values} \wedge \textsl{s} \in \textsl{Suits} \bigr\}$.
\\[0.2cm]
At the start of a game of Texas Hold'em, every player receives two cards.  These two cards are known
as the \blue{preflop} or the \blue{hole}.  Next, there is a \blue{bidding phase} where players can bet on their
cards.   After this bidding phase, the dealer puts three cards open on the table.  These three cards are
known as \blue{flop}.  Let us assume that a player has been dealt the set of cards
\\[0.2cm]
\hspace*{1.3cm}
$\{ \pair(3, \club), \pair(3, \spade) \}$.
\\[0.2cm]
This set of cards is known as a \blue{pocket pair}.  Then the player would like to know the probability
that the flop will contain another card with value $3$, as this would greatly increase her chance of
winning the game.  In order to compute this probability we have to compute the number of possible
flops that contain a card with the value $3$ and we have to divide this number by the number of all
possible flops:
\\[0.2cm]
\hspace*{1.3cm}
$\ds \frac{\;\mbox{number of flops containing a card with value $3$}\;}{\mbox{number of all possible flops}}$
\\[0.2cm]
The program
\href{https://github.com/karlstroetmann/Logic/blob/master/Python/Poker.ipynb}{Poker.iypnb}
shown in Figure \ref{fig:poker-triple.py} performs this computation.  We proceed to discuss this
program line by line.

\begin{figure}[!ht]
\centering
\begin{minted}[ frame         = lines,
                framesep      = 0.3cm,
                numbers       = left,
                numbersep     = -0.2cm,
                bgcolor       = sepia,
                xleftmargin   = 0.0cm,
                xrightmargin  = 0.0cm,
              ]{python3}
    Values = { "2", "3", "4", "5", "6", "7", "8", "9", "T", "J", "Q", "K", "A" }
    Suits  = { "c", "h", "d", "s" }
    Deck   = { (v, s) for v in Values for s in Suits }
    Hole   = { ("3", "c"), ("3", "s") }
    Rest   = Deck - Hole
    Flops  = { (k1, k2, k3) for k1 in Rest for k2 in Rest for k3 in Rest
                            if  len({ k1, k2, k3 }) == 3
             }
    Trips  = { f for f in Flops if ("3", "d") in f or ("3", "h") in f }
    print(len(Trips) / len(Flops))
\end{minted}
\vspace*{-0.3cm}
\caption{Computing a probability in poker.}
\label{fig:poker-triple.py}
\end{figure}

\begin{enumerate}
\item In line 1 the set \texttt{Values} is defined to be the set of all possible values that a card
      can take.  In defining this set we have made use of the following abbreviations:
      \begin{enumerate}
      \item ``\texttt{T}'' is short for ``\blue{Ten}'',
      \item ``\texttt{J}'' is short for ``\blue{Jack}'',
      \item ``\texttt{Q}'' is short for ``\blue{Queen}'',
      \item ``\texttt{K}'' is short for ``\blue{King}'', and
      \item ``\texttt{A}'' is short for ``\blue{Ace}''.
      \end{enumerate}
\item In line 2 the set \texttt{Suits} represents the possible suits of a card.  Here, we have used
      the following abbreviations:
      \begin{enumerate}
      \item ``\texttt{c}'' is short for $\club$ (\underline{c}lub),
      \item ``\texttt{h}'' is short for \mbox{\color{red}{$\heart$}} (\underline{h}earts),
      \item ``\texttt{d}'' is short for \mbox{\color{red}{$\diamondsuit$}} (\underline{d}iamonds), and
      \item ``\texttt{s}'' is short for $\spade$ (\underline{s}pades).
      \end{enumerate}
\item Line 3 defines the set of all cards.  This set is stored as the variable \texttt{Deck}.  Every
      card is represented as a pair of the form $(v,s)$. Here, $v$ is the value of the card, while $s$ is its suit.
\item Line 4 defines the set \texttt{Hole}.  This set represents the two cards that have been given to our player.
\item The remaining cards are defined as the variable  \texttt{Rest} in line 5.
\item Line 6 computes the set of all possible flops.  Since the order of the cards in the flop does
      not matter, we use sets to represent these flops.  However, we have to take care that the flop
      does contain three \colorbox{amethyst}{different} cards.  Hence, we have to ensure that the three
      cards \texttt{k1}, \texttt{k2}, and \texttt{k3} that make up the flop satisfy the inequalities
      \\[0.2cm]
      \hspace*{1.3cm}
      $\mathtt{k1} \not= \mathtt{k2}$, \quad $\mathtt{k1} \not= \mathtt{k3}$,  \quad and \quad $\mathtt{k2} \not= \mathtt{k3}$.
      \\[0.2cm]
      These inequalities are satisfied if and only if the set
      $\{ \mathtt{k1}, \mathtt{k2}, \mathtt{k3} \}$ contains exactly three elements.  Hence, when
      choosing \texttt{k1}, \texttt{k2}, and \texttt{k3} we have to make sure that the condition
      \\[0.2cm]
      \hspace*{1.3cm}
      $\texttt{len}\bigl({\{ \mathtt{k1}, \mathtt{k2}, \mathtt{k3} \} \;\mathtt{==}\; 3 }\bigr)$
      \\[0.2cm]
      holds.
\item Line 9 computes the subset \texttt{Trips} of those flops that contain at least one card with a value of 3.
      As the 3 of clubs and the 3 of spades have already been dealt to our player, the only cards
      with value 3 that are left in the deck are the 3 of diamonds and the 3 of hearts.  Therefore, we are looking for
      those flops that contain one of these two cards.
\item Finally, the probability for obtaining another card with a value of 3 in the flop is computed as
      the ratio of the number of flops containing a card with a value of 3 to the number of all possible flops.
\end{enumerate}
When we run the program we see that the probability of improving a \blue{pocket pair} on the flop to \blue{trips} or better
is about  $11.8\%$.

\remarkEng
The method to compute probabilities that has been sketched above only works if the sets that have to
be computed are small enough to be retained in memory.  If this condition is
not satisfied we can use the \href{https://en.wikipedia.org/wiki/Monte_Carlo_method}{\emph{Monte Carlo method}}
\index{Monte Carlo method}
to compute the probabilities instead.  This method will be discussed in the lecture on
\href{https://github.com/karlstroetmann/Algorithms/blob/master/Lecture-Notes/algorithms.pdf}{algorithms}.

\section{Finding a Path in a Graph}
We will now discuss the problem of finding a \blue{path} \index{path} in a
\href{https://en.wikipedia.org/wiki/Directed_graph}{directed graph}.
\index{directed graph}
Abstractly, a \emph{directed graph} consists of \blue{vertices} and \blue{edges} that connect these vertices.  In an application, the
vertices could be towns and villages, while the edges would be interpreted as one-way streets connecting these
villages.  To simplify matters, let us assume for now that the vertices are given as natural numbers.  As the
edges represent connections between vertices,  the edges are represented as pairs of natural numbers.  Then,
the graph can be represented as the set of its edges, as the set of vertices is implicitly given once the edges
are known.  To make things concrete, let us consider an example.  In this case, the set of edges is called
\texttt{R} and is defined as follows:
\\[0.2cm]
\hspace*{1.3cm}
$\texttt{R}\; \mathtt{=}\; \bigl\{ \pair(1,2), \pair(2,3), \pair(1,3), \pair(2,4), \pair(4,5) \bigr\}$.
\\[0.2cm]
In this graph, the set of vertices is given as
\\[0.2cm]
\hspace*{1.3cm}
$\{ 1, 2, 3, 4, 5 \}$.
\\[0.2cm]
This graph is shown in Figure \ref{fig:graph0} on page \pageref{fig:graph0}.  You should note that the
connections between vertices that are given in this graph are \blue{unidirectional}:  While there is a connection from
vertex $1$ to vertex $2$, there is no connection from vertex $2$ to vertex $1$.

\begin{figure}[!ht]
  \centering
  \epsfig{file=Figures/graph0,scale=0.6}

  \caption{A simple graph.}
  \label{fig:graph0}
\end{figure}

\noindent
The graph given by the relation \texttt{R} contains only the direct connections of vertices.  For example, in
the graph shown in Figure \ref{fig:graph0}, there is a direct connection from vertex $1$ to vertex $2$ and
another direct connection from vertex $2$ to vertex $4$.  Intuitively, vertex $4$ is reachable from vertex $1$,
since from vertex $1$ we can first reach vertex $2$ and from vertex $2$ we can then reach vertex $4$.  However,
there is is no direct connection between the vertices $1$ and $4$.  To make this more formal, define
a \blue{path} of a graph $R$ as a list of vertices
\\[0.2cm]
\hspace*{1.3cm}
$[x_1, x_2, \cdots, x_n]$ \quad such that \quad $\pair(x_i,x_{i+1}) \in R$ \quad for all $i=1,\cdots,n-1$.
\\[0.2cm]
In this case, the path $[x_1, x_2, \cdots, x_n]$ is written as
\\[0.2cm]
\hspace*{1.3cm}
$x_1 \mapsto x_2 \mapsto \cdots \mapsto x_n$
\\[0.2cm]
and has the \blue{length} $n-1$, since there are $n-1$ direct connections of the form $\pair(x_i,x_{i+1})$ that
make up this path.
To put it differently,  the length of a path
$[x_1,x_2,\cdots,x_n]$ is defined as the number of edges connecting the vertices and not as the
number of vertices appearing on the path.

Furthermore,  two vertices $a$ and $b$ of a graph are said to be \blue{connected} \index{connected} iff there exists a path
\\[0.2cm]
\hspace*{1.3cm}
$[x_1,\cdots,x_n]$ \quad such that \quad $a = x_1$ \quad and \quad $b = x_n$.
\\[0.2cm]
The goal of this section is to develop an algorithm that checks whether two vertices $a$ and $b$ are connected.
Furthermore, we want to be able to compute the corresponding path connecting the vertices $a$ and $b$.

\subsection{Breadth-First Search}
We are now ready to present \blue{breadth-first search} \index{breadth-first search}.  This is an algorithm
for finding a path in a given graph.  Figure \ref{fig:Breadth-First-Search.ipynb} on page
\pageref{fig:Breadth-First-Search.ipynb} shows the function \texttt{search}.  This function uses three
arguments:
\begin{enumerate}[(a)]
\item $R$ is a binary relation that is interpreted as a directed graph.
\item \texttt{start} and \texttt{goal} are nodes in this graph.
\end{enumerate}
The function search returns a path leading from \texttt{start} to  \texttt{goal} if such a path exists.
Otherwise it returns \texttt{None}.  We discuss the implementation of the function \texttt{search} line by line.

\begin{figure}[!ht]
  \centering
\begin{minted}[ frame         = lines,
                framesep      = 0.3cm,
                numbers       = left,
                numbersep     = -0.2cm,
                bgcolor       = sepia,
                xleftmargin   = 0.8cm,
                xrightmargin  = 0.8cm,
              ]{python3}
    def search(R, start, goal):
        Paths   = { (start,) }
        Visited = { start }
        while True:
            NewPaths = set()
            for Path in Paths:
                for (y, z) in R:
                    if Path[-1] == y and z not in Visited:
                        LongerPath = Path + (z,)
                        if z == goal:
                            return LongerPath
                        NewPaths.add(LongerPath)
                        Visited .add(z)
            if NewPaths == set():
                return
            Paths = NewPaths
\end{minted}
\vspace*{-0.3cm}
\caption{Breadth-first search.}  \label{fig:Breadth-First-Search.ipynb}
\end{figure} %\$

\begin{enumerate}
\item In line 2 we initialize the variable \texttt{Paths} to contain the path that starts in the node
      \texttt{start} and has the length $0$.  After the $n$th iteration of the \texttt{while} loop in line 4,
      the set \texttt{Paths} will contain all paths in the graph that start in node \texttt{start} and have a
      length of $n$.
\item In line 3 we initialize the set \texttt{Visited} to contain the state \texttt{start}.
      Every time we discover a new node, it is added to the set \texttt{Visited}.
      The reason is that we want all the paths stored in the set \texttt{Paths} to be shortest paths.
      Therefore, we only add a new path to this set if the end point of this path has not yet been visited.
\item Before the $n$th iteration of the \texttt{while} loop in line 4 all paths stored in
      the set \texttt{Paths} have a length of $n-1$.  The purpose of the next iteration is to extend the paths to
      a length of $n$.  Those paths that can not be extended are discarded.
\item \texttt{NewPaths} is the set of those paths that have been extended.  In the $n$th iteration,
      all paths in this set will have a length of $n$.
\item In line 6 to 8 we iterate over all paths discovered so far that end in the node $y$, where $\langle y, z\rangle$
      is a pair in the relation $R$ and the node $z$ has not yet been visited.
\item If we find such a path \texttt{Path} we extend it by appending the node $z$.
\item If $z$ is equal to \texttt{goal} we have found a path from \texttt{start} to \texttt{goal} and return
      this path.
\item Otherwise, this new path is added to the set \texttt{NewPaths} and the node $z$ is added to the set
      \texttt{Visited} to record the fact that we now have found a path leading to $z$.
\item If we do not find any new path in a given iteration, then there is no path from \texttt{start} to
      \texttt{goal} and the function returns.
\item Otherwise, \texttt{Paths} is set to \texttt{NewPaths} and the loop repeats.
\end{enumerate}

\subsection{The Wolf, the Goat, and the Cabbage}
\index{wolf, goat, and cabbage}
Next, we present an application of the theory developed so far.  We solve a problem that has puzzled
the greatest agricultural economists for centuries.  The puzzle we want to solve is known as the
\href{http://jeux.lulu.pagesperso-orange.fr/html/anglais/loupChe/loupChe1.htm}{wolf-goat-cabbage puzzle}:
\vspace*{0.3cm}

\begin{minipage}[c]{16cm}
{\sl
An agricultural economist has to sell a wolf, a goat, and a cabbage on a market place.  In order to
reach the market place, she has to cross a river.  The boat that she can use is so small that it can
only accommodate either the goat, the wolf, or the cabbage in addition to the agricultural economist.
Now if the agricultural economist leaves the wolf alone with the goat, the wolf will eat the goat.
If, instead, the agricultural economist leaves the goat with the cabbage, the goat will eat the cabbage.
Is it possible for the agricultural economist to develop a schedule that allows her to cross the river
without either the goat or the cabbage being eaten?
}
\end{minipage}
\vspace*{0.3cm}

\noindent
In order to compute a schedule, we first have to model the problem.  The various \blue{states} of the problem will
be regarded as \blue{vertices} of a graph and this graph will be represented as a binary relation.
To this end we define the set
\begin{verbatim}
  All = {'farmer', 'wolf, 'goat', 'cabbage'}.
\end{verbatim}
Every node will be represented as a subset \texttt{S} of the set \texttt{All}.  The idea is that the set \texttt{S}
specifies those objects that are on the left side of the river.  We assume that initially the farmer and his goods
are on the left side of the river.
Therefore, the set of all states that are \blue{allowed} according to the specification of the problem can be defined
as the set
\begin{verbatim}
  States = { S for S in power(All) if not problem(S) and not problem(All-S) }
\end{verbatim}
Here, we have used the procedure \texttt{problem} to check whether a given set \texttt{S} has a problem,
where a problem is any situation where either the goat eats the cabbage or the wolf eats the goat.
Note that since \texttt{S} is the set of objects on the left side, the expression $\texttt{All-S}$
computes the set of objects on the right side of the river.

Formally, a set \texttt{S} of objects has a problem if both of the following conditions
are satisfied:
\begin{enumerate}
\item The farmer is not an element of \texttt{S} and
\item either \texttt{S} contains both the goat and the cabbage or \texttt{S} contains both the wolf and the goat.
\end{enumerate}
Therefore, we can implement the function \texttt{problem} as follows:
\begin{verbatim}
  def problem(S):
      return ('farmer' not in S) and             \
             (('goat' in S and 'cabbage' in S) or   # goat eats cabbage
              ('wolf' in S and 'goat'    in S)   )  # wolf eats goat
\end{verbatim}
Note that we have to use a \blue{line continuation backslash} ``\texttt{$\backslash$}''
at the end of the first line of the return statement.
We do not need a continuation backslash at the end of the second line of the return statement since
the opening parenthesis at the beginning of the second line has not yet been closed when the second line
finishes and therefore \textsl{Python} is able to figure out that the expression defined in this line is
continued in the third line.

We proceed to compute the relation \texttt{R} that contains all possible transitions between
different states.  We will compute \texttt{R} using the formula:
\\[0.2cm]
\hspace*{0.75cm}
\texttt{R = R1 + R2;}
\\[0.2cm]
Here \texttt{R1} describes the transitions that result from the farmer crossing the river from left
to right, while \texttt{R2} describes the transitions that result from the farmer crossing the river
from right to left.  We can define the relation \texttt{R1} as follows:
\begin{verbatim}
  R1 = { (S, S-B) for S in States
                  for B in power(S)
                  if S-B in States and 'farmer' in B and len(B) <= 2
       }
\end{verbatim}
Let us explain this definition in detail:
\begin{enumerate}
\item Initially, \texttt{S} is the set of objects on the left side of the river.  Hence, \texttt{S}
      is an element of the set of all states that we have defined as \texttt{States}.
\item \texttt{B} is the set of objects that are put into the boat and that do cross the river.  Of
      course, for an object to go into the boat is has to be on the left side of the river to begin
      with.  Therefore, \texttt{B} is a subset of \texttt{S} and hence \texttt{B} is an element of the power set
      of \texttt{S}.
\item Therefore  \texttt{S-B} is the set of objects that are left on the left side of the river after
      the boat has crossed.  Of course, the new state \texttt{S-B} has to be a state that does not
      have a problem.  Therefore, we check that the set \texttt{S-B} is an element of the set \texttt{States}.
\item Furthermore, the farmer has to be inside the boat.  This explains the condition
      \\[0.2cm]
      \hspace*{1.3cm}
      \texttt{\symbol{39}farmer\symbol{39} in B}.
\item Finally, the boat can only have two passengers.  Therefore, we have added the condition
      \\[0.2cm]
      \hspace*{1.3cm}
      \texttt{len(B) <= 2}.
\end{enumerate}
Next, we have to define the relation \texttt{R2}.  However, as crossing the river from right to left
is just the reverse of crossing the river from left to right, \texttt{R2} is just the \blue{inverse} of
\texttt{R1}.   Hence we define:
\begin{verbatim}
  R2 = { (S2, S1) for (S1, S2) in R1 }.
\end{verbatim}
Next, the relation \texttt{R} is the union of \texttt{R1} and \texttt{R2}:
\begin{verbatim}
  R = R1 | R2.
\end{verbatim}
Finally, the start state has all objects on the left side.  Therefore, we have
\begin{verbatim}
  start = All.
\end{verbatim}
In the end, all objects have to be on the right side of the river.  That means that nothing is left
on the left side.  Therefore, we define
\begin{verbatim}
  goal = {}.
\end{verbatim}

\begin{figure}[h]
  \centering

  \epsfig{file=Figures/wolf-goat-cabbage, scale=0.4}

  \caption{The relation \texttt{R} shown as a directed graph.}
  \label{fig:wolf-goat-cabbage.pdf}
\end{figure}

\noindent
Figure \ref{fig:wolf-goat-cabbage.pdf} on page \pageref{fig:wolf-goat-cabbage.pdf} displays the relation $R$ graphically.
Figure \ref{fig:wolf-ziege} on page \pageref{fig:wolf-ziege} shows the program
\href{https://github.com/karlstroetmann/Logic/blob/master/Python/wolf-goat-cabbage.py}{\texttt{wolf-goat-cabbage.py}}
that combines the statements shown so far.  The solution computed by this program is shown in Figure
 \ref{fig:wolf-ziege-solution}.

\begin{figure}[!ht]
  \centering
\begin{minted}[ frame         = lines,
                framesep      = 0.3cm,
                numbers       = left,
                numbersep     = -0.2cm,
                bgcolor       = sepia,
                xleftmargin   = 0.3cm,
                xrightmargin  = 0.3cm,
              ]{python3}
    def problem(S):
        return ('farmer' not in S) and             \
               (('goat' in S and 'cabbage' in S) or   # goat eats cabbage
                ('wolf' in S and 'goat'    in S)   )  # wolf eats goat

    All   = frozenset({ 'farmer', 'wolf', 'goat', 'cabbage' })
    R1    = { (S, S - B) for S in States for B in power(S)
                         if S - B in States and 'farmer' in B and len(B) <= 2
            }
    R2    = { (S2, S1) for (S1, S2) in R1 }
    R     = R1 | R2
    start = All
    goal  = frozenset()
    Path  = findPath(start, goal, R)
\end{minted}
\vspace*{-0.3cm}
\caption{Solving the wolf-goat-cabbage problem.}
\label{fig:wolf-ziege}
\end{figure}

\begin{figure}[!ht]
  \centering
\begin{minted}[ frame         = lines,
                framesep      = 0.3cm,
                numbers       = left,
                numbersep     = -0.2cm,
                bgcolor       = sepia,
                xleftmargin   = 0.8cm,
                xrightmargin  = 0.8cm,
              ]{python3}
    {'cabbage', 'farmer', 'goat', 'wolf'}                                 {}
                             >>>> {'farmer', 'goat'} >>>>
    {'cabbage', 'wolf'}                                   {'farmer', 'goat'}
                             <<<< {'farmer'} <<<<
    {'cabbage', 'farmer', 'wolf'}                                   {'goat'}
                             >>>> {'farmer', 'wolf'} >>>>
    {'cabbage'}                                   {'farmer', 'goat', 'wolf'}
                             <<<< {'farmer', 'goat'} <<<<
    {'cabbage', 'farmer', 'goat'}                                   {'wolf'}
                             >>>> {'cabbage', 'farmer'} >>>>
    {'goat'}                                   {'cabbage', 'farmer', 'wolf'}
                             <<<< {'farmer'} <<<<
    {'farmer', 'goat'}                                   {'cabbage', 'wolf'}
                             >>>> {'farmer', 'goat'} >>>>
    {}                                 {'cabbage', 'farmer', 'goat', 'wolf'}
\end{minted}
\vspace*{-0.3cm}
\caption{A schedule for the agricultural economist.}
\label{fig:wolf-ziege-solution}
\end{figure}
\pagebreak
\vspace*{\fill}

\section{Symbolic Differentiation}
\index{symbolic differentiation}
In this section we will develop a program that reads an arithmetic expression like the string
\\[0.2cm]
\hspace*{1.3cm}
\texttt{\symbol{34}x * exp(x)\symbol{34}},
\\[0.2cm]
interprets this string as describing the real valued function
\\[0.2cm]
\hspace*{1.3cm}
$x \mapsto x \cdot \exp(x)$,
\\[0.2cm]
and then takes the derivative of this function with respect to the variable $x$.  In order to specify the input
of this program more clearly, we first define the notion of an \blue{arithmetic expression} inductively.
\begin{enumerate}[(a)]
\item Every number $c \in \mathbb{R}$ is an arithmetic expression.
\item Every variable $v$ is an arithmetic expression.
\item If $s$ and $t$ are arithmetic expressions, then
      \\[0.2cm]
      \hspace*{1.3cm}
      $s + t$, \quad $s - t$, \quad $s * t$, \quad $s / t$, \quad and \quad $s \,\mathtt{**}\, t$
      \\[0.2cm]
      are arithmetic expressions.  Here $s \,\mathtt{**}\, t$ is interpreted as $s^t$.

\item If $e$ is an arithmetic expression, then both
      \\[0.2cm]
      \hspace*{1.3cm}
      $\exp(e)$ \quad and \quad $\ln(e)$
      \\[0.2cm]
      are arithmetic expressions.
\end{enumerate}
We want do implement a function \texttt{diff} that takes two arguments:
\begin{enumerate}
\item The first argument \texttt{expr} is an arithmetic expression.
\item The second argument \texttt{var} is the name of a variable.
\end{enumerate}
The function call \texttt{diff(expr, var)} will then compute the derivative of \texttt{expr} with respect to the variable \texttt{var}.  For example, the function call \texttt{diff(\symbol{34}x*exp(x)\symbol{34}, \symbol{34}x\symbol{34})} will compute the output
\\[0.2cm]
\hspace*{1.3cm}
\symbol{34}\texttt{1*exp(x) + x*exp(x)}\symbol{34}
\\[0.2cm]
because we have:
$$ \frac{\mathrm{d}\;}{\mathrm{d}x} \bigl( x \cdot \mathrm{e}^x \bigr) = 1 \cdot x + x \cdot \mathrm{e}^x $$
It would be very tedious to \blue{represent} arithmetic expressions as strings.  Instead, we will represent
arithmetic expressions as \blue{nested tuples}.  \index{nested tuple}
The notion of a \emph{nested tuple} is defined inductively:
\begin{itemize}
\item $\langle x_1, x_2, \cdots, x_n \rangle$ is a nested tuple if each of the components $x_i$ is either a
      number, a string, or is itself a nested tuple.
\end{itemize}
For example, the arithmetic expression ``\texttt{x*exp(x)}'' is represented as the nested tuple
\\[0.2cm]
\hspace*{1.3cm}
$\bigl\langle\texttt{\symbol{34}}*\texttt{\symbol{34}}, \texttt{\symbol{34}}x\texttt{\symbol{34}}, \langle \texttt{\symbol{34}}\mathtt{exp}\texttt{\symbol{34}}, \texttt{\symbol{34}}x\texttt{\symbol{34}} \rangle\bigr\rangle$.
\\[0.2cm]
In order to be able to convert string into nested tuples, we need a \blue{parser}.
\index{parser}
  A parser is a program that
takes a string as input and transforms this string into a nested tuple, which is then returned as a result.
I have implemented a parser in the file ``\texttt{exprParser.py}''.  The details of the implementation of this
parser will be discussed in the lecture on
\href{https://github.com/karlstroetmann/Algorithms/blob/master/Lecture-Notes/algorithms.pdf}{algorithms} in the
second semester..

\noindent
The function \texttt{diff} that is shown in Figure \ref{fig:diff.py} on page \pageref{fig:diff.py} is part
of the program
\href{https://github.com/karlstroetmann/Logic/blob/master/Python/Symbolic-Differentiation.ipynb}{\texttt{Symbolic-Differentiation.ipynb}}.
This function is called with one argument:
The argument \texttt{e} is an arithmetic expression.
The function \texttt{diff} interprets its argument \texttt{e} as a function of the variable
\texttt{x}.  We take the \href{https://en.wikipedia.org/wiki/Derivative}{derivative} of this
function with respect to the variable \texttt{x}.  For example, in order to compute the derivative of
the function
\\[0.2cm]
\hspace*{1.3cm}
$x \mapsto x^x$,
\\[0.2cm]
we can call the function  \texttt{diff} as follows:
\\[0.2cm]
\hspace*{1.3cm}
\texttt{diff(\symbol{34}x ** x\symbol{34})}.
\\[0.2cm]
Let us now discuss the implementation of the function \texttt{diff} in more detail.
\begin{enumerate}
\item The lines 3 - 6 implement the rule:
      $$\frac{\mathrm{d}\;}{\mathrm{d}x}\bigl(f(x) + g(x)\bigr) = \frac{\mathrm{d}\;}{\mathrm{d}x} f(x) + \frac{\mathrm{d}\;}{\mathrm{d}x} g(x)$$
\item Line 7 - 10 implement the rule:
      $$\frac{\mathrm{d}\;}{\mathrm{d}x}\bigl(f(x) - g(x)\bigr) = \frac{\mathrm{d}\;}{\mathrm{d}x} f(x) - \frac{\mathrm{d}\;}{\mathrm{d}x} g(x)$$
\item Line 11 - 14 deals with the case where \texttt{e} is a product.  The
      \href{https://en.wikipedia.org/wiki/Product\_rule}{product rule} is
      $$ \frac{\mathrm{d}\;}{\mathrm{d}x}\bigl(f(x) \cdot g(x)\bigr) = \left(\frac{\mathrm{d}\;}{\mathrm{d}x} f(x)\right)\cdot g(x) + f(x) \cdot \left(\frac{\mathrm{d}\;}{\mathrm{d}x} g(x)\right)
      $$
\item Line 15 - 17 deals with the case where \texttt{e} is a quotient.  The
      \href{https://en.wikipedia.org/wiki/Quotient\_rule}{quotient rule} is
      $$ \frac{\mathrm{d}\;}{\mathrm{d}x}\left(\frac{f(x)}{g(x)}\right) =
         \frac{\displaystyle\left(\frac{\mathrm{d}\;}{\mathrm{d}x} f(x)\right)\cdot g(x) -
         f(x) \cdot \left(\frac{\mathrm{d}\;}{\mathrm{d}x} g(x)\right)}{g(x) \cdot g(x)}
      $$
\item Line 19 - 21 deals with the case where \texttt{e} is a power.  Now in order to take the derivative of an
      expression of the form
      $$  f(x)^{g(x)} $$
      we first need to rewrite this expression using the following trick:
      $$ f(x)^{g(x)} = \exp\bigl(\ln\bigl(f(x)^{g(x)}\bigr)\bigr) = \exp\bigl(g(x) \cdot \ln(f(x))\bigr) $$
      Then, we can recursively call \texttt{diff} for this expression.  This works, because the function
      \texttt{diff} can deal with both the exponential function $x \mapsto \exp(x)$ and with the natural
      logarithm $x \mapsto \ln(x)$.  This rewriting is done in line 21.
\item Line 22-25 deals with the case where \texttt{e} has the form
      $$\ln\bigl(f(x)\bigr)$$
      In order to take the derivative of this expression, we first need to know the derivative of the natural
      logarithm.  This derivative is given as
      $$ \frac{\mathrm{d}\;}{\mathrm{d}x} \ln(x) = \frac{1}{x}$$
      Then, using the \href{https://en.wikipedia.org/wiki/Chain\_rule}{chain rule} we have that
      $$ \frac{\mathrm{d}\;}{\mathrm{d}x} \ln\bigl(f(x)\bigr) = \frac{\frac{\mathrm{d}\;}{\mathrm{d}x} f(x)}{f(x)}$$
\item Line 26 - 29 deals with the case where \texttt{e} has the form $\exp\bigl(f(x)\bigr)$.
      In order to take the derivative of this expression, we first need to know the derivative of the
      \href{https://en.wikipedia.org/wiki/Exponential\_function}{exponential function}.
      This derivative is given as
      $$ \frac{\mathrm{d}\;}{\mathrm{d}x} \exp(x) = \exp(x)$$
      Then, using the \href{https://en.wikipedia.org/wiki/Chain\_rule}{chain rule} we have that
      $$\frac{\mathrm{d}\;}{\mathrm{d}x} \exp\bigl(f(x)\bigr) = \left(\frac{\mathrm{d}\;}{\mathrm{d}x} f(x)\right) \cdot \exp\bigl(f(x)\bigr) $$
\item Line 30-31 deals with the case where \texttt{e} is a variable and happens to be the same variable as
      \texttt{x}.  This is checked using the condition
      \texttt{e == x}.  As we have
      $$\frac{\mathrm{d}x}{\mathrm{d}x} = 1,$$
      the function \texttt{diff} returns \texttt{1} in this case.
\item Otherwise, the expression is assumed to be a constant and hence we return 0.
\end{enumerate}

\begin{figure}[!ht]
\centering
\begin{minted}[ frame         = lines,
                framesep      = 0.3cm,
                firstnumber   = 1,
                numbers       = left,
                numbersep     = -0.2cm,
                bgcolor       = sepia,
                xleftmargin   = 0.8cm,
                xrightmargin  = 0.8cm,
              ]{python3}
    def diff(e):
        'differentiate the expressions e with respect to the variable x'
        if e[0] == '+':
            f , g  = e[1:]
            fs, gs = diff(f), diff(g)
            return ('+', fs, gs)
        if e[0] == '-':
            f , g  = e[1:]
            fs, gs = diff(f), diff(g)
            return ('-', fs, gs)
        if e[0] == '*':
            f , g  = e[1:]
            fs, gs = diff(f), diff(g)
            return ('+', ('*', fs, g), ('*', f, gs))
        if e[0] == '/':
            f , g  = e[1:]
            fs, gs = diff(f), diff(g)
            return ('/', ('-', ('*', fs, g), ('*', f, gs)), ('*', g, g))
        if e[0] == '**':
            f , g  = e[1:]
            return diff(('exp', ('*', g, ('ln', f))))
        if e[0] == 'ln':
            f  = e[1]
            fs = diff(f)
            return ('/', fs, f)
        if e[0] == 'exp':
            f  = e[1]
            fs = diff(f)
            return ('*', fs, e)
        if e == 'x':
            return '1'
        return 0
\end{minted}
\vspace*{-0.3cm}
\caption{A function for symbolic differentiation}
\label{fig:diff.py}
\end{figure}

In order to test this function we can implement a function \texttt{test} as shown in Figure \ref{fig:test-diff.py}.
Then the expression
\\[0.2cm]
\hspace*{1.3cm}
\texttt{diff(\symbol{34}x ** x\symbol{34})}
\\[0.2cm]
yields the result:
\\[0.2cm]
\hspace*{1.3cm}
d/dx x ** x = (1*ln(x) + x*1/x)*exp(x*ln(x))
\\[0.2cm]
This shows that
\\[0.2cm]
\hspace*{1.3cm}
$\ds \frac{\mathrm{d}\;}{\mathrm{d}x} x^x = \bigl(\ln(x) + 1\bigr) \cdot \exp\bigl(x \cdot \ln(x)\bigr) =
 \bigl(\ln(x) + 1\bigr) \cdot x^x
$.

\begin{figure}[!ht]
\centering
\begin{minted}[ frame         = lines,
                framesep      = 0.3cm,
                firstnumber   = 1,
                numbers       = left,
                numbersep     = -0.2cm,
                bgcolor       = sepia,
                xleftmargin   = 0.8cm,
                xrightmargin  = 0.8cm,
              ]{python3}
    import exprParser as ep

    def test(s):
        t = ep.ExprParser(s).parse()
        d = diff(t)
        print(f'd/dx {s} = {ep.toString(d)}')
\end{minted}
\vspace*{-0.3cm}
\caption{Testing symbolic differentiation.}
\label{fig:test-diff.py}
\end{figure}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "logic"
%%% End:


\section{``Cartesian closed'', Hausdorff, Basepoints}\label{basepoints}
Pushouts are colimits, so the quotient space $X/A=X\cup_A \ast$ is an example of a colimit.
Let $Y$ be a topological space, and consider the functor $Y\times -:\Top\to\Top$.
Applying this to the pushout square, we find that $(Y\times X)\cup_{Y\times A}\ast\simeq (Y\times X)/(Y\times A)$.
As we discussed in \S \ref{CGWHspaces}, this product is \emph{not} the same as $Y\times (X/A)$!
There is a bijective map $Y\times X/Y\times A\to Y\times(X/A)$,
but it is not, in general, a homeomorphism.
From a categorical point of view (see Theorem \ref{adjointslimits}), the reason for this failure
stems from $Y\times -$ not being a left adjoint.

The discussion in \S \ref{mappingspaces} implies that, when working with $k$-spaces,
that functor is indeed a left adjoint
(in fancy language, the category $k\Top$ is Cartesian closed),
which means that --- in $k\Top$ --- there is a homeomorphism $Y\times X/Y\times A\to Y\times(X/A)$.
This addresses the issues raised in \S \ref{CGWHspaces}.
The ancients had come up with a good definition of a topology ---
but $k$-spaces are better!
Sometimes, though, we can be greedy and ask for even more:
for instance, we can demand that points be closed.
This leads to a further refinement of $k$-spaces.

\todo[inline]{I don't like point-set topology, so I'll return to editing this lecture at the end.}
\subsection{``Hausdorff''}
\begin{definition}
    A space is ``weakly Hausdorff'' if the image of every map $K\to X$ from a compact Hausdorff space $K$ is closed.
\end{definition}
Another way to say this is that the map itself if closed. Clearly Hausdorff implies weakly Hausdorff. Another thing this means is that every point in $X$ is closed (eg $K=\ast$).
\begin{prop}
    Let $X$ be a $k$-space.
    \begin{enumerate}
	\item $X$ is weakly Hausdorff iff $\Delta:X\to X\times^k X$ is closed. In algebraic geometry such a condition is called separated.
	\item Let $R\subseteq X\times X$ be an equivalence relation. If $R$ is closed, then $X/R$ is weakly Hausdorff.
    \end{enumerate}
\end{prop}
\begin{definition}
    A space is compactly generated if it's a weakly Hausdorff $k$-space. The category of such spaces is called $\CG$.
\end{definition}
We have a pair of adjoint functors $(i,k):\Top\to k\Top$. It's possible to define a functor $k\Top\to \CG$ given by $X\mapsto X/\bigcap\text{all closed equivalence relations}$. It is easy to check that if $Z$ is weakly Hausdorff, then $Z^X$ is weakly Hausdorff (where $X$ is a $k$-space). What this implies is that $\CG$ is also Cartesian closed!

I'm getting a little tired of point set stuff. Let's start talking about homotopy and all that stuff today for a bit. You know what a homotopy is. I will not worry about point-set topology anymore. So when I say $\Top$, I probably mean $\CG$. A homotopy between $f,g:X\to Y$ is a map $h:I\times X\to Y$ such that the following diagram commutes:
\begin{equation*}
    \xymatrix{
	X\ar[dr]_{i_0}\ar[drr]^f & &\\
	& I\times X\ar[r]^h & Y\\
	X\ar[ur]^{i_1}\ar[urr]_g & &
    }
\end{equation*}
We write $f\sim g$. We define $[X,Y]=\Top(X,Y)/\sim$. Well, a map $I\times X\to Y$ is the same as a map $X\to Y^I$ but also $I\to Y^X$. The latter is my favorite! It's a path of maps from $f$ to $g$. So $[X,Y]=\pi_0Y^X$.

To talk about higher homotopy groups and induct etc. we need to talk about basepoints.
\subsection{Basepoints}
A pointed space is $(X,\ast)$ with $\ast\in X$. This gives a category $\Top_\ast$ where the morphisms respect the basepoint. This has products because $(X,\ast)\times (Y,\ast)=(X\times Y,(\ast,\ast))$. How about coproducts? It has coproducts as well. This is the wedge product, defined as $X\sqcup Y/\ast_X\sim \ast_Y=:X\vee Y$. This is \verb|\vee|, not \verb|\wedge|. Is this category also Cartesian closed?

Define the space of pointed maps $Z^X_\ast\subseteq Z^X$ topologized as a subspace. Does the functor $Z\mapsto Z^X_\ast$ have a left adjoint? Well $\Top(W,Z^X)=\Top(X\times W,Z)$. What about $\Top(W,Z^X_\ast)$? This is $\{f:X\times W\to Z:f(\ast,w)=\ast\forall w\in W\}$. That's not quite what I wanted either! Thus $\Top_\ast(W,Z^X_\ast)=\{f:X\times W\to Z:f(\ast,w)=\ast=f(x,\ast)\forall x\in X, w\in W\}$. These send both ``axes'' to the basepoint. Thus, $\Top_\ast(W,Z^X_\ast)=\Top_\ast(X\wedge W,Z)$ where $X\wedge W=X\times W/X\vee W$ because $X\vee W$ are the ``axes''.

So $\Top_\ast$ is not Cartesian closed, but admits something called the smash product\footnote{Remark by Sanath: this is like the tensor product.}. What properties would you like? Here's a good property: $(X\wedge Y)\wedge Z$ and $X\wedge(Y\wedge Z)$ are bijective in pointed spaces. If you work in $k\Top$ or $\CG$, then they are homeomorphic! It also has a unit.

Oh yeah, some more things about basepoints! So there's a canonical forgetful functor $i:\Top_\ast\to \Top$. Let's see. If I have $\Top(X,iY)=\Top_\ast(??,Y)$? This is $X_+=X\sqcup \ast$. Thus we have a left adjoint $(-)_+$. It is clear that $(X\sqcup Y)_\ast = X_+ \vee Y_+$. The unit for the smash product is $\ast_+ = S^0$.

On Friday I'll talk about fibrations and fiber bundles.


\section{Relative homotopy groups}
%Pset 2 has question 8; there's still one more to go. Hood has office hours today 12 -- 1 in 2-390, and I have office hours today tomorrow 4-5 in 2-478.
\subsection{Spheres and homotopy groups}
The functor $\Omega$ (sending a space to its based loop space) admits a left adjoint.
To see this, recall that $\Omega X = X^{S^1}_\ast$, so that
$$\Top_\ast(W,\Omega X) = \Top_\ast(S^1\wedge W,X).$$
\begin{definition}
    The \emph{reduced suspension} $\Sigma W$ is $S^1\wedge W$.
\end{definition}
If $A\subseteq X$, then
$$X/A\wedge Y/B = (X\times Y)/((A\times Y)\cup_{A\times B}(X\times B)).$$
Since $S^1 = I/\partial I$, this tells us that $\Sigma X = S^1\wedge X$ can be identified with
$I\times X/(\partial I \times X\cup I\times \ast)$: in other words, we collapse the top and bottom of a cylinder to a point,
as well as the line along a basepoint.

The same argument says that $\Sigma^n X$ (defined inductively as $\Sigma(\Sigma^{n-1} X)$)
is the left adjoint of the $n$-fold loop space functor $X\mapsto \Omega^n X$.
In other words, $\Sigma^n X = (S^1)^{\wedge n}\wedge X$.
We claim that $S^1\wedge S^n \simeq S^{n+1}$.
To see this, note that
$$S^1\wedge S^n = I/\partial I\wedge I^n\wedge \partial I^n = (I\times I^n)/(\partial I\times I^n\cup I\times \partial I^n).$$
The denominator is exactly $\partial I^{n+1}$, so $S^1\wedge S^n\simeq S^{n+k}$.
It's now easy to see that $S^k\wedge S^n\simeq S^{k+n}$.
\begin{definition}
    The \emph{$n$th homotopy group} of $X$ is $\pi_n X = \pi_0(\Omega^n X)$.
\end{definition}
This is, as we noted in the previous section, $[S^0,\Omega^n X]_\ast = [S^n, X]_\ast = [(I^n,\partial I^n),(X,\ast)]$.

\subsection{The homotopy category}
Define the \emph{homotopy category of spaces} $\Ho(\Top)$ to be the category
whose objects are spaces, and whose hom-sets are given by taking $\pi_0$ of the mapping space.
To check that this is indeed a category, we need to check that if $f_0,f_1:X\to Y$ and $g:Y\to Z$, then $gf_0\simeq gf_1$ ---
but this is clear.
Similarly, we'd need to check that $f_0h\simeq f_1h$ for any $h:W\to X$.
We can also think about the homotopy category of pointed spaces (and pointed homotopies) $\Ho(\Top_\ast)$; this is the category
we have been spending most of our time in.
Both $\Ho(\Top)$ and $\Ho(\Top_\ast)$ have products and coproducts, but very few other limits or colimits.
From a category-theoretic standpoint, these are absolutely terrible.

Let $W$ be a pointed space.
We would like the assignment $X\mapsto X^W_\ast$ to be a homotopy functor.
It clearly defines a functor $\Top_\ast\to\Top_\ast$, so this desire is equivalent to providing a dotted arrow in the
following diagram:
\begin{equation*}
    \xymatrix{
	\Top_\ast\ar[d]\ar[r]^{X\mapsto X^W_\ast} & \Top_\ast\ar[d]\\
	\Ho(\Top_\ast)\ar@{-->}[r] & \Ho(\Top_\ast).
    }
\end{equation*}
Before we can prove this, we will check that a homotopy $f_0\sim f_1:X\to Y$ is the same as a map $I_+\wedge X\to Y$.
There is a nullhomotopy if the basepoint of $I$ is one of the endpoints, so a homotopy is the same as a map
$I\times X/I\times\ast \to Y$. The source is just $I_+\wedge X$, as desired.

A homotopy $f_0 \simeq f_1:X\to Y$ begets a map $(I_+\wedge X)^W\to Y^W_\ast$.
For the assignment $X\mapsto X^W_\ast$ to be a homotopy functor, we need a natural transformation $I_+\wedge X^W_\ast\to Y^W_\ast$, so this map is not quite what's necessary.
Instead, we can attempt to construct a map $I_+\wedge X^W_\ast\to (I_+\wedge X)^W_\ast$.

We can construct a general map $A\wedge X^W_\ast\to (A\wedge X)^W_\ast$:
there is a map $A\wedge X^W_\ast\to A^W_\ast\wedge X^W_\ast$, given by sending $a\mapsto c_a$;
then the exponential law gives a homotopy $A^W_\ast\wedge X^W_\ast\to (A\wedge X)^W_\ast$.
This, in turn, gives a map $I_+\wedge X^W_\ast\to (I_+\wedge X)^W_\ast\to Y^W_\ast$,
thus making $X\mapsto X^W_\ast$ a homotopy functor.

Motivated by our discussion of homotopy fibers, we can study composites which ``behave'' like short exact sequences.
\begin{definition}
    A \emph{fiber sequence} in $\Ho(\Top_\ast)$ is a composite $X\to Y\to Z$ that is
    isomorphic, in $\Ho(\Top_\ast)$, to some composite $Ff\xrightarrow{p} E\xrightarrow{f}B$;
    in other words, there exist (possibly zig-zags of) maps that are homotopy equivalences, that make the following
    diagram commute:
    \begin{equation*}
	\xymatrix{
	    X\ar[r]\ar[d] & Y\ar[r]\ar[d] & Z\ar[d]\\
	    Ff\ar[r]_p & E\ar[r]_f & B.
	    }
    \end{equation*}
\end{definition}
Let us remark here that if $A^\prime \xar{\sim} A$ is a homotopy equivalence, and $A\to B \to C$ is a fiber sequence, so
is the composite $A^\prime\xar{\sim} A \to B\to C$.

\begin{exercise}\label{loopslimit}
    Prove the following statements.
    \begin{itemize}
	\item $\Omega$ takes fiber sequences to fiber sequences.
	\item $\Omega Ff\simeq F\Omega f$. Check this!
    \end{itemize}
\end{exercise}

We've seen examples of fiber sequences in our elaborate study of the Barratt-Puppe sequence.
\begin{example}
Recall our diagram:
\begin{equation*}
    \xymatrix{
	\cdots\ar[r] & Fp_4\ar[r] & Fp_3 \ar[r] & Fp_2\ar[r] & Fp_1\ar[r]^{p_2} & Ff\ar[r]^{p_1} & X\ar[r]^{f} & Y\\
    \cdots\ar[r] & \Omega Fp_1\ar[r]|{\overline{\Omega p_2}}\ar[u]_{\simeq} & \Omega Ff\ar[u]_{\simeq}\ar[ur]|{i(p_2)}\ar[r]|{\overline{\Omega p}} & \Omega X\ar[r]|{\overline{\Omega f}}\ar[u]_{\simeq}\ar[ur]|{i(p_1)} & \Omega Y\ar[u]_\simeq \ar[ur]|{i(f)} & &\\
	\Omega^2 X\ar[u]_{\simeq}\ar[r]_{\Omega f} & \Omega Y\ar[u]_{\simeq}\ar[ur]_{\overline{\Omega i(f)}} & & &
    }
\end{equation*}
The composite $Ff\to X\xar{f} Y$ is canonically a fiber sequence.
The above diagram shows that $\Omega Y\to F\xrightarrow{p}X$ is another fiber sequence: it is isomorphic to
$Fp\to F\to X$ in $\Ho(\Top_\ast)$.
Similarly, the composite $\Omega X\xrightarrow{\overline{\Omega f}}\Omega Y\to F$ is another fiber sequence;
this implies that $\Omega X\xrightarrow{\Omega f}\Omega Y\to F$ is also an example of a fiber sequence
(because these two fiber sequences differ by an automorphism of $\Omega X$)

Applying $\Omega$ again, we get $\Omega F\xrightarrow{\Omega p} \Omega X\xrightarrow{\Omega f} \Omega Y$.
Since this is a looping of a fiber sequence, and taking loops takes fiber sequences to fiber sequences (Exercise \ref{loopslimit}), this is another fiber sequence.
Looping again gives another fiber sequence $\Omega^2 Y\xrightarrow{\Omega i} \Omega F\xrightarrow{\Omega p}\Omega X$.
(For the category-theoretically--minded folks, this is an unstable version of a triangulated category.)
\end{example}

\subsection{The long exact sequence of a fiber sequence}
As discussed at the end of \S \ref{secbarrattpuppe}, applying $\pi_0 = [S^0,-]_\ast$ to the
Barratt-Puppe sequence associated to a map $f:X\to Y$ gives a long exact sequence:
\begin{equation*}
    \xymatrix{
	& \cdots\ar[r] & \pi_2 Y\ar[dll]\\
	\pi_1 F\ar[r] & \pi_1 X\ar[r] & \pi_1 Y\ar[dll]\\
	\pi_0 F\ar[r] & \pi_0 X. &
    }
\end{equation*}
of pointed sets.
The space $\Omega^2 X$ is an \emph{abelian} group object in $\Ho(\Top)$
(in other words, the multiplication on $\Omega^2 X$ is commutative up to homotopy).
This implies $\pi_1(X)$ is a group, and that $\pi_k(X)$ is abelian for $k\geq 2$;
hence, in our diagram above, all maps (except on $\pi_0$) are group homomorphisms.

Consider the case when $X\to Y$ is the inclusion $i:A\hookrightarrow X$ of a subspace.
In this case,
$$Fi=\{(a,\omega)\in A\times X^I_\ast|\omega(1) = a\};$$
this is just the collection of all paths that begin at $\ast\in A$ and end in $A$.
This motivates the definition of \emph{relative homotopy groups}:
\begin{definition}
    Define:
    $$\pi_n(X,A,\ast) = \pi_n(X,A) := \pi_{n-1}Fi = [(I^n,\partial I^n,(\partial I^n\times I)\cup (I^{n-1}\times 0)),(X,A,\ast)].$$
\end{definition}
We have a sequence of inclusions
$$\partial I^n\times I\cup I^{n-1}\times 0 \subset \partial I^n \subset I^n.$$
One can check that
$$\pi_{n-1}Fi = [(I^n,\partial I^n,(\partial I^n\times I)\cup (I^{n-1}\times 0)),(X,A,\ast)].$$
This gives a long exact sequence on homotopy, analogous to the long exact sequence in relative homology:
\begin{equation}\label{lexseqhomotopy}
    \xymatrix{
	& \cdots\ar[r] & \pi_2 (X,A)\ar[dll]\\
	\pi_1 A\ar[r] & \pi_1 X\ar[r] & \pi_1 (X,A)\ar[dll]\\
	\pi_0 A\ar[r] & \pi_0 X &
    }
\end{equation}


\documentclass[../../main.tex]{subfiles}
\begin{document}
\subsubsection{Iso To Unix Epoch}
The $isoToUnix$ operation converts an ISO 8601 Timestamp (see the \href{https://github.com/adlnet/xAPI-Spec/blob/master/xAPI-Data.md#timestamps}{xAPI Specification})
to the number of seconds that have elapsed since January 1, 1970
\begin{schema}{IsoToUnix}
  Timestamp \\
  seconds! : \nat \\
  isoToUnix~\_ : \finset_1 \fun \nat \\
  \where
  seconds! = isoToUnix(timestamp)
\end{schema}
\begin{argue}
  ts = 2015-11-18T12:17:00+00:00 \equiv 2015-11-18T12:17:00Z \\
  isoToUnixEpoch(ts) =  1447849020 & ISO 8601 $\to$ Epoch time
\end{argue}
\end{document}