\section{Pascal characterization}

In this section we present some results about Pascal array and
its inverse, taken modulo some prime $p$.

Associating different colours to elements belonging to different remainder
classes, we show formally that for $p=2$, elements in the same
position get the same colour, hence both triangles are coloured in the same
way, ignoring signs for elements in the inverse array, which can happen to be
negative. On the other hand, for $p$ odd there is not such a simple
correspondence and we attempt to observe some repeating pattern in the mapping
among elements in those arrays.

\subsection{$\equiv_{p}$ over $\mathcal{P}$ and $\mathcal{P}^{-1}$,
    where $p$ \emph{is} a prime}

Let $\mathcal{P}$ be the Riordan array for the Pascal triangle,
defined as:
\begin{displaymath}
    \mathcal{P} = \left(\frac{1}{1-t}, \frac{t}{1-t}  \right)
\end{displaymath}
and let $\mathcal{P}^{-1}$ be the inverse of $\mathcal{P}$:
\begin{displaymath}
    \mathcal{P}^{-1} = \left(\frac{1}{1+t}, \frac{t}{1+t}  \right)
\end{displaymath}

Let $d_{nk}$ and $\hat{d}_{nk}$ be the generic elements of $\mathcal{P}$ and of
$\mathcal{P}^{-1}$, respectively. Since both arrays are in the Riordan group,
$d_{nk}$ can be written as:
\begin{displaymath}
    \begin{split}
        d_{nk} &= [t^n]\frac{1}{1-t}\left(\frac{t}{1-t}\right)^k = [t^{n-k}](1-t)^{-(k+1)} \\
            &= {{-(k+1)} \choose {n-k}}(-1)^{n-k} = {{k+1 +n-k -1} \choose {n-k}} = {{n} \choose {n-k}} \\
    \end{split}
\end{displaymath}
using the same approach $\hat{d}_{nk}$ can be written as:
\begin{displaymath}
  \begin{split}
    \hat{d}_{nk} &= [t^n]\frac{1}{1+t}\left(\frac{t}{1+t}\right)^k = [t^{n-k}](1+t)^{-(k+1)} =
    {{-(k+1)} \choose {n-k}} \\
    &= {{k+1 +n-k -1} \choose {n-k}} (-1)^{n-k} = {{n} \choose {n-k}} (-1)^{n-k}\\
  \end{split}
\end{displaymath}
Hence, equating binomial coefficients yields:
\begin{displaymath}
  [t^n]\frac{1}{1-t}\left(\frac{t}{1-t}\right)^k = (-1)^{k-n}[t^n]\frac{1}{1+t}\left(\frac{t}{1+t}\right)^k
\end{displaymath}
Choose a prime $p$ and apply the modulo operator on both members:
\begin{displaymath}
  [t^n]\frac{1}{1-t}\left(\frac{t}{1-t}\right)^k \equiv_{p} (-1)^{k-n}[t^n]\frac{1}{1+t}\left(\frac{t}{1+t}\right)^k
\end{displaymath}

From now on we have \marginpar{$b^{-1}\mod p$ exists if and only if $(b,p)=1$}
to reason according to modular arithmetic rules. In particular, we have to keep in
mind that multiplying a term $b$ by a term $a$ in order to simplify it, namely
to get $1$, we are requested  to show that $a$ is the
\emph{multiplicative inverse} of $b$ modulo $p$, denoted by:
\begin{displaymath}
    a = b^{-1}\mod p
\end{displaymath}

First of all, observe that $-1 \equiv_{p} p-1$ and since $p$ is a prime by
hypothesis, it follows that $(p, p-1)=1$ (this result holds in general, not
just for $p$ prime), which proofs the existence of both $-1$ and $p-1$
inverses, denoted by $(-1)^{-1}\mod p$ and $(p-1)^{-1}\mod p$ respectively.

In order to find $(p-1)^{-1}\mod p$ we have to satisfy the congruence equation
$(p-1) \cdot (p-1)^{-1} \equiv_{p} 1$. Choose $(p-1)^{-1}\mod p = p-1$ and
verify $(p-1) \cdot (p-1) \equiv_{p} p^{2} -2\,p +1 \equiv_{p} 1$ as required.

Another useful observation concerns raising to negative powers:
\begin{displaymath}
    (-1)^{-k} \equiv_{p} \left((-1)^{-1}\right)^{k} \equiv_{p} (p-1)^k
\end{displaymath}
where $k \geq 0$ and, since $-1 \equiv_{p} p-1$, it follows that
$(-1)^{-k} \equiv_{p} (-1)^k$.

Now we can use previous observation on the main congruence:
\begin{displaymath}
    \begin{split}
        [t^n]\frac{1}{1-t}\left(\frac{t}{1-t}\right)^k
            &\equiv_{p} (-1)^{k-n}[t^n]\frac{1}{1+t}\left(\frac{t}{1+t}\right)^k \\
            &\equiv_{p} (-1)^{k+n}[t^n]\frac{1}{1+t}\left(\frac{t}{1+t}\right)^k \\
            &\equiv_{p} (p-1)^{k+n}[t^n]\frac{1}{1+t}\left(\frac{t}{1+t}\right)^k \\
    \end{split}
\end{displaymath}
Hence, multiplying by $(p-1)^{-1}\mod p$ both members $k+n$ times:
\begin{displaymath}
    (p-1)^{k+n}[t^n]\frac{1}{1-t}\left(\frac{t}{1-t}\right)^k
        \equiv_{p} [t^n]\frac{1}{1+t}\left(\frac{t}{1+t}\right)^k
\end{displaymath}
which is the same as:
\begin{displaymath}
    (-1)^{k+n}[t^n]\frac{1}{1-t}\left(\frac{t}{1-t}\right)^k \equiv_{p} [t^n]\frac{1}{1+t}\left(\frac{t}{1+t}\right)^k
\end{displaymath}
and relating generic elements $d_{nk}$ and $\hat{d}_{nk}$:
\marginpar{a general congruence $\equiv_{p}$ over $\mathcal{P}$ and $\mathcal{P}^{-1}$, where $p$
    is an arbitrary prime}
\begin{equation}
    \label{eq:general:congruence:over:pascal:arrays}
    (p-1)^{k+n}d_{nk}\equiv_{p}(-1)^{k+n}d_{nk} \equiv_{p} \hat{d}_{nk}
\end{equation}

\subsection{$\mathcal{P}_{\equiv_{2}}$}

The case where $p$ is the even prime produces the colouring reported in
\autoref{fig:pascal-standard-ignore-negatives-centered-colouring-127-rows-mod2-partitioning-triangle} for $\mathcal{P}$,
and the colouring reported in
\autoref{fig:pascal-inverse-ignore-negatives-centered-colouring-127-rows-mod2-partitioning-triangle} for $\mathcal{P}^{-1}$,
where negatives entries are ignored, while in
\autoref{fig:pascal-inverse-handle-negatives-centered-colouring-127-rows-mod2-partitioning-triangle} such entries are handled.
Formally, $\mathcal{P}$ and $\mathcal{P}^{-1}$ look the same because if we fix $p=2$,
\autoref{eq:general:congruence:over:pascal:arrays} yields:
\begin{displaymath}
    d_{nk} \equiv_{2} \hat{d}_{nk}
\end{displaymath}
for any choice of $n$ and $k$ in $\mathbb{N}$.

\input{Chapters/mod-p-characterization/pascal/pascal-standard-ignore-negatives-centered-colouring-127-rows-mod2-partitioning-include-matrix.tex}
\input{Chapters/mod-p-characterization/pascal/pascal-standard-ignore-negatives-centered-colouring-127-rows-mod2-partitioning-include-figure.tex}
\input{Chapters/mod-p-characterization/pascal/pascal-inverse-ignore-negatives-centered-colouring-127-rows-mod2-partitioning-include-figure.tex}
\input{Chapters/mod-p-characterization/pascal/pascal-inverse-handle-negatives-centered-colouring-127-rows-mod2-partitioning-include-figure.tex}

\subsection{$\mathcal{P}_{\equiv_{3}}$}

Here there's a more interesting pattern to study, and in general,
for any odd prime $p$, colourings of $\mathcal{P}$ and its inverse
mismatch. In this section we tackle the case for $p=3$,
instantiating \autoref{eq:general:congruence:over:pascal:arrays} yields:
\begin{displaymath}
  2^{k+n}d_{nk}\equiv_{3}(-1)^{k+n}d_{nk} \equiv_{3} \hat{d}_{nk}
\end{displaymath}

For the sake of clarity, let us consider row $4$ of both triangles:
\begin{itemize}
\item $\mathcal{P}_{[3,\,:]} = (1 \quad 3 \quad 3 \quad 1) \equiv_{3}(1 \quad 0 \quad 0 \quad 1)$
\item $\mathcal{P}^{-1}_{[3,\,:]} = (-1 \quad 3 \quad -3 \quad 1) \equiv_{3}(2 \quad 0 \quad 0 \quad 1)$
\end{itemize}
\marginpar{$[\alpha,\beta]$
 in $\mathcal{R}_{[\alpha,\beta]}$ is the \emph{slice operator},
 where $\alpha,\beta\in\mathbb{Z}\cup\lbrace:\rbrace$, as defined in NumPy or in Octave}
Hence element $d_{30}$ gets a color $c$ while $\hat{d}_{30}$ gets
a color $c'$ different from $c$: no general mapping about this relationship appears, so we
study the two coloured triangles carefully.

Our exercise here is as follow: first fix the order $\alpha$ of a principal
cluster $\mathcal{P}^{(\alpha)}$, say $\alpha=4$; then move upwards from the
last row of $\mathcal{P}^{(4)}$ to the top, one row by one, repeatedly. For
each considered row, move toward the right over columns, trying to understand a
possible congruence over coefficients in the two arrays, see
\autoref{fig:pascal:standard:mod3:congruence:zero:row:above}
for some highlights:
\input{Chapters/mod-p-characterization/pascal/pascal-standard-mod3-zero-row-above-end-include-figure}
\begin{displaymath}
    \begin{split}
        d_{3^4 -1,0} &\equiv_{3} \hat{d}_{3^4 -1,0} \\
        d_{3^4 -1,1} &\equiv_{3} \hat{d}_{3^4 -2,0} \\
        d_{3^4 -1,2} &\equiv_{3} \hat{d}_{3^4 -3,0} \\
        d_{3^4 -1,3} &\equiv_{3} \hat{d}_{3^4 -4,0} \\
        &\vdots
    \end{split}
\end{displaymath}
next, a congruence going up one row:
\marginpar{remember that the last row of $\mathcal{P}^{(\alpha)}$ is $p^{\alpha}-1$,
    for $p$ given }
\begin{displaymath}
    \begin{split}
        d_{3^4 -2,0} &\equiv_{3} \hat{d}_{3^4 -1,1} \\
        d_{3^4 -2,1} &\equiv_{3} \hat{d}_{3^4 -2,1} \\
        d_{3^4 -2,2} &\equiv_{3} \hat{d}_{3^4 -3,1} \\
        d_{3^4 -2,3} &\equiv_{3} \hat{d}_{3^4 -4,1} \\
        &\vdots
    \end{split}
\end{displaymath}
next, a congruence going up two rows:
\begin{displaymath}
    \begin{split}
        d_{3^4 -3,0} &\equiv_{3} \hat{d}_{3^4 -1,2} \\
        d_{3^4 -3,1} &\equiv_{3} \hat{d}_{3^4 -2,2} \\
        d_{3^4 -3,2} &\equiv_{3} \hat{d}_{3^4 -3,2} \\
        d_{3^4 -3,3} &\equiv_{3} \hat{d}_{3^4 -4,2} \\
        &\vdots
    \end{split}
\end{displaymath}

A structure over indices can be caught, so let us introduce variable
$b\in\lbrace1,\ldots,3^{4}\rbrace$ running over rows, and variable
$a\in\lbrace 0,\ldots,3^{4}-b\rbrace$, running over columns. So we can state the following
congruence:
\begin{equation}
    \label{eq:pascal:arrays:congruence:rows:cols}
    d_{3^4 -b,a} \equiv_{3} \hat{d}_{3^4 -1-a,b-1}
\end{equation}
\marginpar{column segments of $\mathcal{P}$ are congruent to row segment of
    $\mathcal{P}^{-1}$ and viceversa}

Let's say, assume the colouring for $\mathcal{P}^{-1}$ triangle is
given and the colouring for row $26$ of $\mathcal{P}$ is desired. It
is necessary to find $b$: by $3^4 -b=26$ get $b=55$, so the
coefficients lying on the required row satisfy the following congruence:
\begin{displaymath}
        d_{26,a} \equiv_{3} \hat{d}_{80-a,54}
\end{displaymath}
\input{Chapters/mod-p-characterization/pascal/pascal-standard-mod3-zero-row-26-include-figure}
an highlighting is shown in \autoref{fig:pascal:standard:mod3:congruence:row:26}.
On the other hand, we could have choosed a column, compute the value
for $a$ and then state a congruence with $b$ as free variable.

% the following chunk of statements is not clearly written and
% I believe it is not so important, so for now skip it.
\iffalse
\\\\
A question is still open: why do we choose row $3^4$ as reference
row?

It is useful to recall a theorem due to Fine:
\begin{theorem}
  A necessary and sufficient condition for a binomial coefficient
  ${{n} \choose {m}}$ to be divisible by a prime $p$ is that $n$
  be a power of $p$.
\end{theorem}
Consider the colouring for triangle $\mathcal{P}$, we can use the
given theorem to point out ``interesting'' rows, namely those rows
affected by the theorem, they correspond to powers
$3^1, 3^2, 3^3, 3^4, \ldots$, each one of them can be easily
recognized since dots lying on it have all the same colour. In the
triangle $127$ former rows are drawn and in order to have ``more
space'' to find a modular relationship between $d_{nk}$ and
$\hat{d}_{nk}$ we choose as \emph{reference} row the one with
index $3^4$.  From here we start moving backwards by rows toward
the root: observe that the entire row $3^4 -1$, containing
$3^4 -1$ remainders, of triangle $\mathcal{P}$ is the first
segment of the first column of $\mathcal{P}^{-1}$, in other words
$d_{3^4 -1,a} \equiv_{3} \hat{d}_{3^4 -1 -a, 0}$ for
$a \in \lbrace 0, \ldots, 3^4 -1\rbrace$.

\fi

% again the following paragraph is a technical detail that can
% be understood only by talking.
\iffalse
It seems that coefficient $d_{3^4-1,0}$ acts as a pivot on which
the triangle ``flips'': the root moves toward the reader while the
bottom edge moves toward opposite the reader. This rigid motion is
captured by the following modular relationships among three
important points:
\begin{displaymath}
    \begin{split}
        d_{3^4 -1,0} &\equiv_{3} \hat{d}_{3^4 -1,0} \\
        d_{3^4 -1,3^4 -1} &\equiv_{3} \hat{d}_{0,0} \\
        d_{3^4 -3^3,3^3-1} &\equiv_{3} \hat{d}_{3^4 -3^3,3^3-1} \\
    \end{split}
\end{displaymath}
\fi

\subsection{$\mathcal{P}_{\equiv_{p}}$, where $p$ is a prime greater than $3$}

It is natural to ask if \autoref{eq:pascal:arrays:congruence:rows:cols}
holds for an arbitrary prime $p$.
\begin{conjecture}
    Let $p$ be a prime and let $\alpha\in\mathbb{N}$.
    If $d_{nk}\in\mathcal{P}^{(\alpha)}$ and
    $\hat{d}_{nk}\in\left(\mathcal{P}^{-1}\right)^{(\alpha)}$, then:
    \marginpar{a general congruence over rows of $\mathcal{P}$
        and columns of $\mathcal{P}^{-1}$, for any prime $p$?}
    \begin{equation}
        d_{p^\alpha -b,a} \equiv_{p} \hat{d}_{p^\alpha -1-a,b-1}
    \end{equation}
    where $b\in\lbrace1,\ldots,p^{\alpha}\rbrace$ and
    $a\in\lbrace 0,\ldots,p^{\alpha}-b\rbrace$.
\end{conjecture}
We provide a proof sketch:
\begin{proof}
    Using closed formula for coefficients $d_{p^\alpha -b,a}$ and $\hat{d}_{p^\alpha -1-a,b-1}$
    we can rewrite:
    \begin{displaymath}
        {{p^\alpha -b} \choose {a}} \equiv_{p} {{p^\alpha -1-a}\choose{b-1} }\left(-1\right)^{p^{\alpha}-a-b}
    \end{displaymath}
    and simple manipulation of binomial coefficients yield:
    \begin{displaymath}
        {{p^\alpha -b} \choose {p^\alpha -a-b}} \equiv_{p}
            {{p^\alpha -1-a}\choose{p^\alpha -a-b} }\left(-1\right)^{p^{\alpha}-a-b}
    \end{displaymath}
    rewrite the right hand side using the identity ${{-n}\choose{k}}={{n+k-1}\choose{k}}(-1)^{k}$:
    \begin{displaymath}
        {{p^\alpha -b} \choose {p^\alpha -a-b}} \equiv_{p}
            {{-b}\choose{p^\alpha -a-b} }
    \end{displaymath}
    to close the proof we could represent $-b$ in base $p$ and,
    since $b\in\lbrace1,\ldots,p^{\alpha}\rbrace$, the application of
    Lucas theorem to both members produces the same product of binomial
    coefficients, as required.

\end{proof}

\subsection{A proof of \emph{Sierpinski}'s gasket}

In this section we show that in a Pascal array $\mathcal{P}$ it is possible to
recognize a structure similar to the one observed by Sierpinski during his
studies on fractals. Interesting references are \cite{sokolov},
\cite{stewart:four:encounters:sierpinski} and \cite{barnsley}.

\begin{theorem}
    Let $\mathcal{P}^{(\alpha)}$ be the principal cluster of order $\alpha$.
    $\mathcal{P}^{(\alpha)}$ repeats itself $3$ times within a
    \emph{chunk} of the principal cluster of one order bigger: such repetitions
    happens in the first $2\,p^{\alpha}$ rows of $\mathcal{P}^{(\alpha+1)}$.
    Moreover, those copies of $\mathcal{P}^{(\alpha)}$ surround a zero-hole of
    order $\alpha$.
\end{theorem}

Note that for even prime $p$, $\mathcal{P}^{(\alpha)}$ repeats
itself $3$ times in $\mathcal{P}^{(\alpha+1)}$, exactly as in the
\emph{Sierpinski's gasket}, having a pure recursive structure.

\input{Chapters/mod-p-characterization/pascal/pascal-standard-handle-negatives-centered-colouring-127-rows-mod5-partitioning-include-figure.tex}
On the other hand, for $p$ odd prime, $\mathcal{P}^{(\alpha)}$ repeats itself
in $\mathcal{P}^{(\alpha+1)}$ somewhere, without a complete recursive
structure, due to the oddness of $p$, ie. other coloured subtriangles appears
in $\mathcal{P}^{(\alpha+1)}$, see the cluster in the \emph{top left} corner of
\autoref{fig:pascal:mod5:principal:cluster:for:sierpinski:proof}.

However if we study the repetition up to row
$2\,p^{\alpha}$, the proof still hold for arbitrary prime $p$.

% the following paragraph should be put in a dedicated section, after this one.
\iffalse
, while in general we can say that maximal
triangles of coefficients multiples of $p$, appear with regularity in
$\mathcal{P}_{n+1}$ a number of times equals to:
\begin{displaymath}
    \frac{(p-1)p}{2}
\end{displaymath}
in other words, considering triangles $\mathcal{P}_{n}$ and
$\mathcal{P}_{n+1}$, there are $\frac{(p-1)p}{2}$ upside-down
maximal triangles, with all coefficients multiple of $p$, from row $p^n$
to row $p^{n+1}-1$; this result will be prove at the end of this
section.
\fi
% ----------------------------------------------------------------------

\begin{proof}

  Let $\mathcal{P}^{(\alpha)}$ be the principal cluster of order $\alpha$ of
  Pascal array $\mathcal{P}$ and let $r$ be a row index for
  $\mathcal{P}^{(\alpha)}$, so $r \in \lbrace 0, \ldots, p^{\alpha} -1
  \rbrace$.

  Let $\mathcal{P}_{k;\alpha}$ be a \emph{chunk} of $\mathcal{P}^{(\alpha +1)}$
  that starts at the root and extends downward $k\,p^{\alpha}$ rows, therefore
  including $\mathcal{P}^{(\alpha)}$ as its first rows.

  It is requested to
  prove that coefficients at \emph{equivalent positions} in the bottom left and
  bottom right triangles of $\mathcal{P}_{2;\alpha}$ are congruent, modulo $p$,
  to coefficients at \emph{equivalent positions} in $\mathcal{P}^{\alpha}$.

  In order to formalize \marginpar{proving congruences over coefficients at
  equivalent positions} the concept of \emph{equivalent positions} we
  introduce the following objects:
  \begin{itemize}
    \item the pair $(r,c)$ of indices is an \emph{equivalent position}
        for coefficient $d_{rc}\in\mathcal{P}^{(\alpha)}$ and coefficient
            $d_{rc}^{\swarrow}$ in the \emph{bottom left} triangle of $\mathcal{P}_{2;\alpha}$ if and only if
                $d_{rc}^{\swarrow} = d_{p^{\alpha}+r,c}$
    \item the pair $(r,c)$ of indices is an \emph{equivalent position}
        for coefficient $d_{rc}\in\mathcal{P}^{(\alpha)}$ and coefficient
            $d_{rc}^{\searrow}$ in the \emph{bottom right} triangle of $\mathcal{P}_{2;\alpha}$ if and only if
                $d_{rc}^{\searrow} = d_{p^{\alpha}+r,p^{\alpha}+c}$
  \end{itemize}
  in the \emph{top right} corner of \autoref{fig:pascal:mod5:principal:cluster:for:sierpinski:proof},
  equivalent position $(2,1)$ for $\mathcal{P}_{\equiv_{5}}^{(2)}$ are highlighted
  for the sake of clarity.  After these definitions, we've to prove:
  \begin{displaymath}
    d_{rc} \equiv_p d_{rc}^{\swarrow} \equiv_p d_{rc}^{\searrow}
  \end{displaymath}
  or, in other words:
  \begin{displaymath}
    {{r} \choose {c}} \equiv_p {{p^{\alpha}+r} \choose {c}} \equiv_p {{p^{\alpha}+r} \choose {p^{\alpha}+c}}
  \end{displaymath}

  In order to prove such congruences we'll use Lucas theorem:
  first of all, observe that $c \leq r$ because $\mathcal{P}^{(\alpha)}$
  is a triangle, therefore $c \in \lbrace 0, \ldots, p^{\alpha} -1 \rbrace$,
  as $r$ satisfies.  By basis representation theorem, there exists
  sequences $\lbrace r_i\rbrace$ and $\lbrace c_i\rbrace$,
  with both $r_i< p$ and $c_i < p$, for $i \in \lbrace 0, \ldots, p^{\alpha} -1 \rbrace$,
  such that:
  \begin{displaymath}
    \begin{split}
      r &= r_0 + r_1 p + r_2 p^2 + \ldots + r_{{\alpha}-1}p^{{\alpha}-1} \\
      c &= c_0 + c_1 p + c_2 p^2 + \ldots + c_{{\alpha}-1}p^{{\alpha}-1} \\
    \end{split}
  \end{displaymath}
  Settings for Lucas theorem are ready, hence apply it:
  \begin{displaymath}
    \begin{split}
      {{p^{\alpha}+r} \choose {c}} &\equiv_{p} {{r_0} \choose {c_0}} {{r_1} \choose {c_1}}{{r_2} \choose {c_2}} \ldots
      {{r_{{\alpha}-1}} \choose {c_{{\alpha}-1}}}{{1} \choose {0}} \equiv_{p} {{r} \choose {c}}\\
      {{p^{\alpha}+r} \choose {p^{\alpha}+c}} &\equiv_{p} {{r_0} \choose {c_0}} {{r_1} \choose {c_1}}{{r_2} \choose {c_2}} \ldots
      {{r_{{\alpha}-1}} \choose {c_{{\alpha}-1}}}{{1} \choose {1}} \equiv_{p} {{r} \choose {c}}\\
    \end{split}
  \end{displaymath}
  therefore coefficients located at \emph{equivalent position}
  $(r,c)$ belong to the same remainder class of congruence relation
  modulo $p$, as required.
  \\\\

  For the second part \marginpar{proving that a zero-hole of order $\alpha$
  is surrounded within $\mathcal{P}_{2;\alpha}$} of the statement, we've to show that in
  $\mathcal{P}_{2;\alpha}$ a zero-hole of order $\alpha$ is
  surrounded by the three ``congruent'' subtriangles composed of
  coefficients $d_{rc}, d_{rc}^{\swarrow}$ and $d_{rc}^{\searrow} $, respectively.
  In the \emph{bottom left} corner of \autoref{fig:pascal:mod5:principal:cluster:for:sierpinski:proof},
  a zero-hole of order $1$ in $\mathcal{P}_{2;1}$ is highlighted for the sake of clarity.

  Observe that the very first coefficient $d_{p^{\alpha}, 0}$ and very last $d_{p^{\alpha}, p^{\alpha}}$ of row
  with index $p^{\alpha}$ are congruent to the unit, modulo $p$:
  \begin{displaymath}
    {{p^{\alpha}} \choose {0}} \equiv_{p}{{p^{\alpha}} \choose {p^{\alpha}}} \equiv_{p} 1
  \end{displaymath}
  while $p$ divides every coefficient between them, let
  $c\in\lbrace1,\ldots, p^{\alpha}-1 \rbrace$:
  \begin{displaymath}
    {{p^{\alpha}} \choose {c}} \equiv_{p} {{0} \choose {c_0}} {{0} \choose {c_1}}{{0} \choose {c_2}} \ldots
    {{0} \choose {c_{{\alpha}-1}}}{{1} \choose {0}} \equiv_{p} 0
  \end{displaymath}
  By the recurrence rule ${{n+1}\choose {k+1}} = {{n} \choose {k}} + {{n}\choose{ k+1}}$
  characterizing $\mathcal{P}$, observe that:
  \begin{displaymath}
    \begin{split}
      d_{p^{\alpha}+1, 1} &\equiv_{p} d_{p^{\alpha}, 0} + d_{p^{\alpha}, 1}\equiv_{p} 1 \\
      d_{p^{\alpha}+1, p^{\alpha}} &\equiv_{p} d_{p^{\alpha}, p^{\alpha}-1} + d_{p^{\alpha}, p^{\alpha}}\equiv_{p} 1 \\
      d_{p^{\alpha}+1, i} &\equiv_{p} d_{p^{\alpha}, i-1} + d_{p^{\alpha}, i}\equiv_{p} 0 \quad \forall i \in \lbrace 2, \ldots, p^{\alpha} -1\rbrace \\
    \end{split}
  \end{displaymath}
  therefore row $p^{\alpha} + 1$ has one less coefficient multiple of $p$
  than row $p^{\alpha}$: $\left|\lbrace2,\ldots, p^{\alpha}-1 \rbrace\right|=
    \left|\lbrace1,\ldots, p^{\alpha}-1 \rbrace\right|-1$.
    Since in row $p^{\alpha}$ there are $p^{\alpha}+1$ coefficients, where
  $p^{\alpha}-1$ of them are multiples of $p$, it follows that after $p^{\alpha}-1$
  rows there are no such coefficients at all, that happens at row $2\,p^{\alpha} -1$.
  Formally, for any column index $c$:
  \begin{displaymath}
    \begin{split}
      {{2p^{\alpha} - 1} \choose {c}} &\equiv_{p} {{p^{\alpha} +(p^{\alpha}- 1)} \choose {c}} \\
      &\equiv_{p} {{p-1} \choose {c_0}} {{p-1} \choose {c_1}}{{p-1} \choose {c_2}} \ldots
      {{p-1} \choose {c_{{\alpha}-1}}}{{1} \choose {0}} \\
      &\not\equiv_{p} 0
    \end{split}
  \end{displaymath}
  by representing $c=(c_{0}, \ldots,c_{\alpha-1})_{p}$ in base $p$, where
  $c_i \in \lbrace 0, \ldots, p-1 \rbrace$, for any $i\in\lbrace0,\ldots,\alpha-1\rbrace$.

  % the following derivation is simply a proof of coefficient extraction
  % using the definition of Riordan array for Pascal triangle, redundant.
  % Recall $\mathcal{P}$ is defined as the Riordan array :
  % \begin{displaymath}
  %   \mathcal{P} = \left(\frac{1}{1-t}, \frac{t}{1-t}  \right)
  % \end{displaymath}
  % hence the following derivation holds:
  % \begin{displaymath}
  %   \begin{split}
  %     {{p^n+r} \choose {c}} &\equiv_p [t^{p^n +r}]\frac{1}{1-t} \left(\frac{t}{1-t}\right)^c \\
  %     &\equiv_p [t^{p^n +r-c}](1-t)^{-(c+1)} \\
  %     &\equiv_p [t^{p^n +r-c}]\mathcal{G}\left\lbrace {{-(c+1)} \choose {k}}(-1)^k \right\rbrace_{k\in\mathbb{N}} \\
  %     &\equiv_p  {{-(c+1)} \choose {p^n +r-c}}(-1)^{p^n +r-c}  \\
  %     &\equiv_p  {{ p^n +r} \choose {p^n +r-c}} \left((-1)^{p^n +r-c}\right)^2  \\
  %     &\equiv_p  {{ p^n +r} \choose {c}}  \\
  %   \end{split}
  % \end{displaymath}

\end{proof}

The \marginpar{for every positive $k\in\mathbb{N}$, no coefficient multiple of $p$ lies on row $k\,p^{\alpha}-1$}
last argument can be generalized: consider $\mathcal{P}^{(\alpha)}$, then every row denoted by index
$k\,p^{\alpha} -1$, for each $k\in\mathbb{N}\setminus\lbrace0\rbrace$, doesn't contain any
coefficient multiple of $p$. Here's a short proof:
\begin{displaymath}
    \begin{split}
      {{kp^{\alpha}-1} \choose {c}} &\equiv_{p}{{(k-1)p^{\alpha}+ (p^{\alpha}-1)} \choose {c}}\\
        &\equiv_{p} {{p-1} \choose {c_0}} {{p-1} \choose {c_1}} \ldots
      {{p-1} \choose {c_{{\alpha}-1}}}{{k-1} \choose {0}} \not\equiv_{p} 0
    \end{split}
\end{displaymath}
in the \emph{bottom right} corner of \autoref{fig:pascal:mod5:principal:cluster:for:sierpinski:proof}
such rows are highlighted, where $p=5$ and $k\in\lbrace1,\ldots,5\rbrace$.

% about the last proof it should be useful to build two images, where
% some coefficients at equivalent positions are highlighted and a picture
% that shows a triangle with a different colouring than the principal cluster
% of one order less.

\subsection{On the number of \emph{zero-holes} within
    $\mathcal{P}^{(\alpha+1)}$}

Let us finish this section proving an observation about the number
of \emph{maximal} zero-holes of order $\alpha$ within a principal
cluster $\mathcal{P}^{(\alpha+1)}$ of one order greater. We start with
an important lemma that ``slices'' $\mathcal{P}^{(\alpha+1)}$ in strips,
each one of them composed of $p^{\alpha}$ rows.

\begin{lemma}
    Let $\mathcal{P}^{(\alpha+1)}$ be a principal cluster of order $\alpha+1$.
    Denote with $\Omega_{j}$ the range that starts at row index $j\,p^{\alpha}$ and stops at
    row index $(j+1)p^{\alpha} -1$, and with $\mathcal{H}_{j}^{\bigtriangledown}$ the number
    of \emph{maximal} zero-holes of order $\alpha$ within $\Omega_{j}$.
    Then:
    \begin{displaymath}
        \mathcal{H}_{j}^{\bigtriangledown}=j \quad\text{where}\quad j\in \lbrace 1, \ldots, p-1 \rbrace
    \end{displaymath}
\end{lemma}

\begin{proof}
    By absurd, choose any $j\in \lbrace 1, \ldots, p-1 \rbrace$ and show
    that in range $\Omega_{j}$ cannot be one zero-hole more \emph{or} one zero-hole less,
    both of order $\alpha$: formally, $\mathcal{H}_{j}^{\bigtriangledown}=j\pm1$ yields a contraddiction,
    for any choice of $j$.

    Proceed by cases:
    \begin{itemize}
        \item assume a zero-hole $h^{\bigtriangledown}$ is missing, so there exists a coefficient
                $\tilde{d}_{nk}\in h^{\bigtriangledown}$ such that $\tilde{d}_{nk}\not\equiv_{p}0$, while all
                other coefficient $d_{nk}\in h^{\bigtriangledown}$ satisfies $d_{nk} \equiv_{p}0$.
                Proceed by cases on parity of $j$:
            \begin{itemize}
                \item $j=2k+1$ for some $k\in\mathbb{N}$. Without loss of generality, suppose $h^{\bigtriangledown}$ is the one in the
                very middle, due to symmetric shape of $\mathcal{P}$ (the cases where it is on the left or
                on the right are less interesting) and
                suppose that $p$ does not divide the coefficient in the very bottom corner of $h^{\bigtriangledown}$,
                formally:
                \begin{displaymath}
                    \tilde{d}_{(j+1) p^{\alpha} -2, (k+1)p^{\alpha} -1}\not\equiv_{p}0
                \end{displaymath}
                But this is impossible because if $d_{nk}\in \mathcal{P}$
                then $d_{nk} = d_{n-1,k-1} + d_{n-1, k}$, so:
                \begin{displaymath}
                    \begin{split}
                        0&\not\equiv_{p}\tilde{d}_{(j+1) p^{\alpha} -2, (k+1)p^{\alpha} -1} \\
                        &\equiv_{p} d_{(j+1) p^{\alpha} -3, (k+1)p^{\alpha} -1} + d_{(j+1) p^{\alpha} -3, (k+1)p^{\alpha}-2 }\\
                        &\equiv_{p}0
                    \end{split}
                \end{displaymath}
                In the \emph{top} of \autoref{fig:pascal-mod7-zero-holes} there's a highlighted strip of rows
                within $\mathcal{P}_{\stackrel{\circ}{\equiv_{7}}}^{(2)}$,where $j=5=2\cdot2+1$ and $p=7$, so $k=2$
                and $\alpha=1$.
                The highlighted coefficient on the very bottom is the one handled in the proof, it is
                $\tilde{d}_{6\cdot 7-2, 3\cdot 7 -1}=\tilde{d}_{40, 20}$: to finish the proof over
                those highlighted coefficients it necessary to use recurrence rule for binomial coefficients,
                otherwise we cannot prove the argument using the symmetry property only.

                \item $j=2k$ for some $k\in\mathbb{N}$. Without loss of generality,
                suppose  $ h^{\bigtriangledown}$ is the one on the very left.
                Choose $r \in\lbrace 0,\ldots,p^n-2\rbrace$, so there exists $c\in\lbrace r+1,\ldots,p^n-1\rbrace$ such that:
                \begin{displaymath}
                    \tilde{d}_{j\,p^{\alpha} +r, c}\not\equiv_{p}0
                \end{displaymath}
                Since $ h^{\bigtriangledown}$ is the only one missing, by symmetry of $\mathcal{P}$ we get another contradiction:
                \begin{displaymath}
                    0\not\equiv_{p}\tilde{d}_{j\, p^{\alpha} +r, c}
                        \equiv_{p} d_{j\, p^{\alpha} +r, j\, p^{\alpha} +r-c}\equiv_{p}0
                \end{displaymath}
            \end{itemize}
        \item assume triangle $ h^{\bigtriangledown}$ is over, therefore the very first coefficient
            $\tilde{d}$ in the \emph{top-left} corner of $ h^{\bigtriangledown}$ has to satisfy:
            \begin{displaymath}
                 \tilde{d} \equiv_{p} d_{j\,p^{n},j\,p^{n}+1}
            \end{displaymath}
            by structure of $\mathcal{P}$, coefficient $d_{j\,p^{n},j\,p^{n}+1}$ has no
            meaning so $\tilde{d}$ cannot satisfy the requested congruence.
    \end{itemize}
\end{proof}

\input{Chapters/mod-p-characterization/pascal/pascal-mod7-abstracting-zero-holes-include-figure}

\begin{theorem}
    Let $\mathcal{P}^{(\alpha+1)}$ be a principal cluster of order $\alpha+1$.
    Then, within $\mathcal{P}^{(\alpha+1)}$ there are ${{p}\choose{2}}$
    \emph{maximal} zero-holes of order $\alpha$.
\end{theorem}

\begin{proof}
    Let $j\in\lbrace 1, \ldots, p-1 \rbrace$, by previous lemma between row index $j\,p^n$ and
    row index $(j+1)p^n -1$, there are $j$ \emph{maximal} zero-holes of order $\alpha$,
    therefore consider the sum:
    \begin{displaymath}
        \sum_{i=1}^{p-1}{i} = \frac{(p-1)p}{2}
    \end{displaymath}
    as required.

\end{proof}

In the \emph{bottom} of \autoref{fig:pascal-mod7-zero-holes}
$\mathcal{P}_{\stackrel{\circ}{\equiv_{7}}}^{(2)}$ is reported: within it there are
${{7}\choose{2}}=21$ zero-holes of order $1$.

\subsection{Congruences over antidiagonals of $\mathcal{P}$ and $\mathcal{P}^{-1}$}

In this section we show two results: a first one about congruences over coefficients
lying on the same \emph{antidiagonal}, spaced by a fixed distance; a second one
about a colour mapping among coefficients, in $\mathcal{P}$ and in the inverse,
at the same location.

% PAY ATTENTION: the following corollary is not correct since congruences can be
% used only in the ring of polynomials.
\iffalse
\begin{corollary}
Let $\mathcal{P}$ be the Pascal array and let $p$ be a prime. Choose $k\in\mathbb{N}$,
than $k$-th column is congruent to $k$-antidiagonal, modulo $p$.
\end{corollary}
\begin{proof}
    \begin{displaymath}
        \begin{split}
            { {n} \choose {k} } &\equiv_{p} { {n} \choose {n-k} } \\
            d_{nk} &\equiv_{p} d_{n,n-k}\\
            \sum_{n\geq 0}{d_{nk} t^n} &\equiv_{p}\sum_{n\geq 0}{d_{n,n-k} t^n} \\
            d(t)h(t)^k &\equiv_{p}\sum_{n\geq 0}{d_{n,n-k} t^n} \\
        \end{split}
    \end{displaymath}
no a standard form for antidiagonal exists, so on the right we leave the explicit
expression for the $k$-th antidiagonal.
\end{proof}
\fi

In order to state the following lemma we define a subset of $\mathcal{P}$
composed of coefficients lying on \emph{antidiagonals}. In the following,
given $\mathcal{P}^{(\alpha)}$ a principal cluster of order $\alpha$,
denote with $\Theta_{\diagdown}^{(\alpha)}$ a set of positions, encoding
each one of them as a pair of integers:
\begin{displaymath}
    \Theta_{\diagdown}^{(\alpha)}=\left\lbrace(n,k) : n\in\lbrace p^{\alpha},\ldots,p^{\alpha+1}-1\rbrace
        \,,k\in\lbrace0,\ldots,n-p^{\alpha}\rbrace    \right\rbrace
\end{displaymath}

\begin{lemma}
    Let $p$ be a prime and $\mathcal{P}^{(\alpha)}$ a principal cluster of order $\alpha$.
    Consider a position $(n,k)\in \Theta_{\diagdown}^{(\alpha)}$, then for any $\gamma\in\mathbb{N}$:
    \begin{displaymath}
        d_{n,k} \equiv_{p} d_{n+\gamma p^{\alpha+1}, k+\gamma p^{\alpha+1}}
    \end{displaymath}
\end{lemma}
\begin{proof} % $n+\gamma p^{m+1}$
    By the basis representation theorem, write $n$ in base $p$ as:
    \begin{displaymath}
        n = n_{0} + n_{1}p + n_{2}p^2 + \ldots + n_{\alpha}p^\alpha %+ 0p^{\alpha+1} + \ldots + 0p^k
    \end{displaymath}
    since $n<p^{\alpha+1}$. For $k$, on the other hand, consider extrema cases:
    \begin{itemize}
        \item $n=p^{\alpha}\rightarrow k\in\lbrace0\rbrace$
        \item $n=p^{\alpha+1}-1 \rightarrow k\in\lbrace 0,\ldots,p^{\alpha}(p-1)-1\rbrace$
    \end{itemize}
    so, for any choice of $n$, we can write $k$ in base $p$ as:
    \begin{displaymath}
        k = k_{0} + k_{1}p + k_{2}p^2 + \ldots + k_{\alpha}p^\alpha %+ 0p^{\alpha+1} + \ldots + 0p^k
    \end{displaymath}
    an application of Lucas theorem finishes the proof:
    \begin{displaymath}
        {{n+\gamma p^{\alpha+1}} \choose { k+\gamma p^{\alpha+1}}} \equiv_{p}
            {{n_{0}} \choose {k_{0}}}
            {{n_{1}} \choose {k_{1}}}
            {{n_{2}} \choose {k_{2}}}
            \ldots
            {{n_{\alpha-1}} \choose {k_{\alpha-1}} }
            {{n_{\alpha}} \choose {k_{\alpha}} }
            {{\gamma} \choose {\gamma}}
            \equiv_{p}{{n} \choose {k}}
    \end{displaymath}
    which is the requested relation.
\end{proof}

Previous lemma is important because it does hold for the inverse array $\mathcal{P}^{-1}$ too:
\begin{displaymath}
    \begin{split}
        \hat{d}_{n,k} &\equiv_{p} \hat{d}_{n+\gamma p^{\alpha+1}, k+\gamma p^{\alpha+1}} \\
        (-1)^{n-k}d_{n,k} &\equiv_{p} (-1)^{n+\gamma p^{\alpha+1}-(k+\gamma p^{\alpha+1})}d_{n+\gamma p^{\alpha+1}, k+\gamma p^{\alpha+1}} \\
        (-1)^{n-k}d_{n,k} &\equiv_{p} (-1)^{n-k}d_{n+\gamma p^{\alpha+1}, k+\gamma p^{\alpha+1}} \\
    \end{split}
\end{displaymath}
Since there exists $(-1)^{-1}\mod p$ we can multiply by it both member $n-k$ times,
obtaining the congruence proved in the previous lemma.
In \autoref{fig:pascal-multiples-over-antidiagonal} are shown both the set
$\Theta_{\diagdown}^{(1)}$ and the highlighting of congruent coefficients
according to previous result, within $\mathcal{P}$ and in its inverse.

Pay attention: this doesn't relate coefficients
of $\mathcal{P}$ and $\mathcal{P}^{-1}$ (from the colouring point of view
doesn't imply any relation about colours assignment: generally it is not the case,
except for even $p$), it merely says that
congruent coefficients on the chosen antidiagonal, \emph{in the same triangle
either $\mathcal{P}$ or $\mathcal{P}^{-1}$}, repeat with structure.
\\\\
The following theorem tackle what the above argument leaves out: it
shows how to colour an antidiagonal $p^{\alpha}$ of $\mathcal{P}$
having coloured the same antidiagonal of $\mathcal{P}^{-1}$, and viceversa.

\input{Chapters/mod-p-characterization/pascal/multiples-over-antidiagonal-include-figure}

\begin{theorem}
    Let $\mathcal{P}^{(\alpha)}$ a principal cluster of order $\alpha$ and denote with
    $d_{nk}, \hat{d}_{nk}$ the generic elements in arrays $\mathcal{P}$ and $\mathcal{P}^{-1}$,
    respectively.
    Choose an odd prime $p$ and let $c\in \lbrace 0, \ldots, p-1 \rbrace$ be a remainder class
    witness, then:
    \begin{displaymath}
        \begin{split}
            d_{n,n-p^{\alpha}} \equiv_{p} c &\leftrightarrow \hat{d}_{n,n-p^{\alpha}} \equiv_{p} p-c
        \end{split}
    \end{displaymath}
    for any $n\geq p^{\alpha}$.
\end{theorem}
\begin{proof}
We show both directions using a set of congruences: reading them from top to bottom provides a
proof for $\rightarrow$ direction, while reading them from bottom to top provides a proof for
$\leftarrow$ direction. Recall that $p^\alpha$ is odd because $p$ is odd by hp, therefore $(-1)^{p^\alpha} = -1$:
\begin{displaymath}
    \begin{split}
        d_{n,n-p^{\alpha}} &\equiv_{p} c \\
        (-1)^{p^\alpha}\hat{d}_{n,n-p^{\alpha}} &\equiv_{p} c \\
        (-1)^{p^\alpha }(-1)\hat{d}_{n,n-p^{\alpha}} &\equiv_{p} -c \\
        (-1)^{p^\alpha }(-1)\hat{d}_{n,n-p^{\alpha}} &\equiv_{p} p -c \\
        \hat{d}_{n,n-p^{\alpha}} &\equiv_{p} p -c \\
    \end{split}
\end{displaymath}
\end{proof}

\subsection{$\equiv_{p}$ over $\mathcal{P}$ and $\mathcal{P}^{-1}$,
    where $p$ \emph{is not} a prime}

\label{subsection:pascal:congruence:characterization:p:not:prime}

\input{Chapters/mod-p-characterization/pascal/pascal-standard-ignore-negatives-centered-colouring-127-rows-multiples-of-4-partitioning-include-figure.tex}

The case when $p$ is not a prime yield coloured triangles that, nonetheless some
repetitive patterns appears, their shape isn't so sharp as the case when $p$ is prime.
In particular, $p$ can take the following forms:
\begin{itemize}
    \item $p=k^{\alpha}$, where $k$ is a prime and $\alpha\in\mathbb{N}$. This case
        is the first step in the generalization toward congruences where the modulo
        isn't a prime, and it is explored by \citeauthor{mclean:1974} in \cite{mclean:1974}.
        In \autoref{fig:pascal-standard-ignore-negatives-centered-colouring-127-rows-multiples-of-4-partitioning-triangle}
        $\mathcal{P}_{\stackrel{\circ}{\equiv_{4}}}$ is shown;
    \item $p=p_{1}^{\alpha_{1}}p_{2}^{\alpha_{2}}\ldots p_{n}^{\alpha_{n}}$, where $p_{1},p_{2},\ldots,p_{n}$
        are prime numbers and $\alpha_{1},\alpha_{2},\ldots,\alpha_{n}\in\mathbb{N}$. This is the most general case
        which yield colourings that seems quite random.
\end{itemize}


\section*{Exercises}

% ----------------------------------------------------------------------

\begin{ex}
  Find vector and parametric equations for the plane through the
  points $P = (0,1,1)$, $Q = (-1,2,1)$, and $R = (1,1,2)$.
\end{ex}

\begin{ex}
  Consider the following vector equation for a plane in $\R^4$:
  \begin{equation*}
    \begin{mymatrix}{c} x\\y\\z\\w \end{mymatrix}
    = \begin{mymatrix}{r} 1\\2\\0\\0 \end{mymatrix}
    + t\,\begin{mymatrix}{r} 1\\0\\0\\1 \end{mymatrix}
    + s\,\begin{mymatrix}{r} -1\\-1\\1\\0 \end{mymatrix}.
  \end{equation*}
  Find a new vector equation for the same plane by doing the change of
  parameters%
  \index{plane!change of parameters}%
  \index{change of parameters!plane}
  $t=1-r_1$, $s=r_1+r_2$.
  \begin{sol}
    We have
    \begin{equation*}
      \begin{mymatrix}{c} x\\y\\z\\w \end{mymatrix}
      = \begin{mymatrix}{r} 1\\2\\0\\0 \end{mymatrix}
      + (1-r_1)\,\begin{mymatrix}{r} 1\\0\\0\\1 \end{mymatrix}
      + (r_1+r_2)\,\begin{mymatrix}{r} -1\\-1\\1\\0 \end{mymatrix}
      = \begin{mymatrix}{r} 2\\2\\0\\1 \end{mymatrix}
      + r_1\,\begin{mymatrix}{r} -2\\-1\\1\\-1 \end{mymatrix}
      + r_2\,\begin{mymatrix}{r} -1\\-1\\1\\0 \end{mymatrix}.
    \end{equation*}
  \end{sol}
\end{ex}

\begin{ex}
  Determine which of the following points lie on the plane through the
  points $P = (2,6,1)$, $R = (1,4,1)$, and $Q = (1,2,-1)$.
  \begin{enumerate}
  \item $S_1=(1,2,4)$.
  \item $S_2=(1,5,2)$.
  \item $S_3=(0,0,0)$.
  \end{enumerate}
\end{ex}

\begin{ex}
  Use cross products to find the normal vector to the plane going
  through the points $P=(1,2,3)$, $Q=(-2,1,8)$ and $R=(2,2,2)$.
\end{ex}

\begin{ex}
  Find normal and standard equations of the plane through the
  point $P=(1,1,2)$ and orthogonal to $\vect{n}=\mat{1,0,-1}^T$.
\end{ex}

\begin{ex}
  Find normal and standard equations for the plane through the points
  $P = (2,1,0)$, $Q=(1,-1,0)$, and $R=(1,1,-1)$.
\end{ex}

\begin{ex}
  Find a vector equation for the plane $2x+y-z=1$.
\end{ex}

\begin{ex}
  The chapter mentions that the normal equation and standard equation
  of a plane only work in $\R^3$, and not in general $\R^n$. Why does
  the equation $ax+by+cz+dw=e$ not describe a plane in $\R^4$?
  \begin{sol}
    The general solution of $ax+by+cz+dw=e$ involves at least three
    parameters.  But the vector equation of a plane only has two
    parameters, therefore $ax+by+cz+dw=e$ does not describe a
    plane. (It describes a $3$-dimensional so-called {\em
      hyperplane}%
    \index{hyperplane}%
    \index{plane!hyperplane} inside
    $\R^4$).
  \end{sol}
\end{ex}

\begin{ex}
  Find the intersection between the planes $x+3y+4z=3$ and $2x+5y-z=2$.
  Is the intersection a line, a plane, or empty?
\end{ex}

\begin{ex}
  Find the intersection of the line
  \begin{equation*}
    \begin{mymatrix}{r} x \\ y \\ z \end{mymatrix}
    = \begin{mymatrix}{r} 1 \\ 1 \\ 1 \end{mymatrix}
    + t \begin{mymatrix}{r} 2 \\ -1 \\ 2 \end{mymatrix}
  \end{equation*}
  and the plane $x+3y+z = 6$.
  Is the intersection a point, a line, or empty?
\end{ex}

\begin{ex}
  Find the angle between the planes $x+y=5$ and $2x+y-z=4$.
\end{ex}

\begin{ex}
  Find the angle between the line
  \begin{equation*}
    \begin{mymatrix}{r} x \\ y \\ z \end{mymatrix}
    = \begin{mymatrix}{r} 0 \\ 3 \\ 7 \end{mymatrix}
    + t \begin{mymatrix}{r} 1 \\ 1 \\ 4 \end{mymatrix}
  \end{equation*}
  and the plane $4x+7y+4z = 6$.
\end{ex}

\begin{ex}
  In Example~\ref{exa:angle-line-plane}, we calculated the angle
  $\theta$ between a line and a plane by calculating the angle $\phi$
  between the direction vector of the line and the normal vector of
  the plane according to the formula
  \begin{equation*}
    \cos\phi =
    \frac{\vect{n}\dotprod\vect{d}}{\norm{\vect{n}}\norm{\vect{d}}}
  \end{equation*}
  and then taking $\theta = \frac{\pi}{2}-\phi$.
  \begin{enumerate}
  \item Explain what happens when the dot product is negative. How
    should we adjust the formula to ensure that $\theta$ is always
    between $0$ and $\frac{\pi}{2}$?
  \item Show that one can get the answer in a single step with the
    formula
    \begin{equation*}
      \sin\theta =
      \frac{\abs{\vect{n}\dotprod\vect{d}}}{\norm{\vect{n}}\norm{\vect{d}}}.
    \end{equation*}
  \end{enumerate}
  \begin{sol}
    (a) If the dot product is negative, $\phi$ will be greater than
    $\frac{\pi}{2}$, and therefore $\theta$ will end up being
    negative. We can fix this, similarly to
    Exercise~\ref{ex:angle-lines}, by taking the absolute value of the
    dot product, i.e., by solving
    \begin{equation}\label{eqn:ex-angle-line-plane}
      \cos\phi =
      \frac{\abs{\vect{n}\dotprod\vect{d}}}{\norm{\vect{n}}\norm{\vect{d}}}
    \end{equation}
    (b) From trigonometry, we know that $\sin\theta =
    \sin(\frac{\pi}{2}-\phi) = \cos\phi$. Together with
    {\eqref{eqn:ex-angle-line-plane}}, this gives the desired formula.
  \end{sol}
\end{ex}

\begin{ex}
  Find the shortest distance from the point $P = (1,1,-1)$ to the plane
  given by $x + 2y + 2z = 6$, and find the point $Q$ on the plane
  that is closest to $P$.
\end{ex}

\begin{ex}
  Use Exercise~\ref{exer-box-product-zero} to find an equation of a
  plane containing the two vectors $\vect{p}$ and $\vect{q}$ and the
  point $0$. \textbf{Hint:} If $(x,y,z)$ is a point in this
  plane, the volume of the parallelepiped determined by $(x,y,z)$
  and the vectors $\vect{p}$, $\vect{q}$ equals 0.
  \begin{sol}
    $\vect{x}\dotprod (\vect{a}\times \vect{b}) =0$.
  \end{sol}
\end{ex}


% !TeX root = ../main.tex
% Add the above to each chapter to make compiling the PDF easier in some editors.

\chapter{Normal Subgroups and Quotient Groups}
Let $(G,\cdot)$ be a group. In this chapter we will see that the coset structure of a group can itself be represented as a group.

\begin{defn}[Normal Subgroup]
A subgroup $N \subgroup G$ is called \emph{normal}\index{normal subgroup} (denoted $N \normal G$) if \begin{align}
    \forall a \in G.\quad a N \inv{a} \subseteq N. \label{eq:normal_subgroup}
\end{align}
\end{defn}

\begin{rmk}\label{rmk:normal_subgroup}
The condition from \cref{eq:normal_subgroup} is equivalent to, \begin{align}
    \forall a \in G.\quad a N \inv{a} = N.
\end{align}
\end{rmk} \begin{proof}[Proof (sketch)]
This follows directly from using \cref{eq:normal_subgroup} for $a \in G$ and $\inv{a} \in G$. This yields $\inv{a} N a \subseteq N$. By multiplying from the left with $a$ and from the right with $\inv{a}$, we obtain $N \subseteq a N \inv{a}$.
\end{proof}

\begin{ex}{Normal subgroups}{}
\begin{itemize}
    \item the trivial subgroups $\{e\}$ and $G$ are also normal, $\{e\}, G \normal G$
    \item the \emph{center}\index{center}, \begin{align}
        Z(G) \defeq \{a \in G \mid \forall x \in G.\ ax = xa\} \normal G,
    \end{align} is a normal subgroup of $G$.
\end{itemize}
\end{ex}

\begin{lem}[Sufficient Conditions for Normal Subgroups]
\leavevmode\begin{lemlist}
    \item If $\varphi : G \to H$ is a homomorphism, then $\ker{\varphi} \normal G$.
    \item Every subgroup $U \subgroup G$ with index $\Index{G}{U} = 2$ is a normal subgroup of $G$.
    \item\label{lem:normal_subgroup_c} If $U$ is the only subgroup of order $m < \infty$ of $G$, then $U \normal G$.
    \item If $G$ is abelian, then all subgroups of $G$ are normal.
\end{lemlist}
\end{lem} \begin{proof}[Proof (sketches)]
\leavevmode\begin{lemlist}
    \item We have already seen in \cref{lem:homomorphism_kernel_subgroup} that $\ker{\varphi} \subgroup G$. We have, \begin{align*}
        \forall a \in G, x \in \ker{\varphi}.\quad \varphi(a x \inv{a}) = \varphi(a) \cdot \underbrace{\varphi(x)}_{= e_H} \cdot \inv{\varphi(a)} = e_H.
    \end{align*} Thus, $a \cdot \ker{\varphi} \cdot \inv{a} \subseteq \ker{\varphi}$ and $\ker{\varphi} \normal G$.

    \addtocounter{lemlisti}{1}\item For all $a \in G$, we have $a U \inv{a} = i_a(U)$ where $i_a$ is the inner automorphism. As automorphisms are bijective, they map a subgroup to a subgroup of the same order. Therefore, ${i_a(U) = U}$, assuming that $U$ is the only subgroup of order $m < \infty$. Using \cref{rmk:normal_subgroup} completes the proof.

    \item Follows immediately from the definition of normal subgroups by applying commutativity. \qedhere
\end{lemlist}
\end{proof}

\begin{marginfigure}
    \includegraphics[width=\textwidth]{s3_subgroup_graph_index.png}
    \caption{Subgroup graph of the symmetric group $S_3$. The index of the subgroups is shown in orange.}
\end{marginfigure}

\begin{ex}{Normal subgroups (continued)}{}
\begin{itemize}
    \item $A_n \normal S_n$ as $A_n = \ker{\sgn}$, see \cref{eq:alternating_group}
    \item $\SL{n}{\R} \normal \GL{n}{\R}$ as $\SL{n}{\R} = \ker{\det}$, see \cref{eq:det_kernel}
    \item ${\Inn{G} \defeq \{i_g \mid g \in G\}}$ known as the \emph{inner automorphism group}\index{inner automorphism group} is a normal subgroup of the automorphism group, $\Inn{G} \normal \Aut{G}$
    \item For the symmetric group $S_3$, we have the normal subgroups \begin{itemize}
        \item $\{e\}, G \normal G$
        \item $A_3 = \gen{(1\ 2\ 3)} \normal G$
    \end{itemize} by \cref{lem:normal_subgroup_c}. Simple calculations confirm that the subgroups of order 2 are not normal.
\end{itemize}
\end{ex}

\begin{thm}[Quotient Group]\label{thm:quotient_group}
Let $N \normal G$. Then the set \begin{align}
    \Quot{G}{N} \defeq \{a N \mid a \in G\}
\end{align} is a group under the operation, \begin{align}
    a N \cdot b N \defeq (a \cdot b) N \quad \forall a, b \in G. \label{eq:quotient_group_op}
\end{align} $\Quot{G}{N}$ is called the \emph{quotient group}\index{quotient group} $G$ modulo $N$.
\end{thm} Thus, the quotient group is the group of (left) cosets of a normal subgroup. In particular, if $G$ is finite, we have, \begin{align}
    |\Quot{G}{N}| = \Index{G}{N} = \frac{|G|}{|N|},
\end{align} due to the \hyperref[defn:index]{definition of the index} and \hyperref[thm"lagrange]{Lagrange's theorem}. \begin{proof}
In proving that $\Quot{G}{N}$ is a group, we will see why we need the restriction of normal subgroups.

\begin{itemize}
    \item First, we need to show that the group operation \eqref{eq:quotient_group_op} is well-defined.\footnote{By \emph{well-defined}\index{well-defined function}, we mean that a function maps the same input to the same output.} Let us fix $a_1, a_2, b_1, b_2 \in G$ such that $a_1 N = a_2 N$ and $b_1 N = b_2 N$. We need to show $a_1 b_1 N = a_2 b_2 N$.

    As $N$ contains the neutral element $e$, we know $a_1 \in a_1 N$ and $a_2 \in a_2 N$. Therefore, $\exists n \in N.\ a_1 = a_2 n$ and, analogously, $\exists \Tilde{n} \in N.\ b_1 = b_2 \Tilde{n}$. We have, \begin{align*}
        a_1 b_1 = a_2 n b_2 \Tilde{n} = a_2 b_2 (\inv{b_2} n b_2 \Tilde{n}).
    \end{align*} Using that $N$ is normal, $\inv{b_2} n b_2 \in N$. Then, as $N$ is a subgroup, we also have $\inv{b_2} n b_2 \Tilde{n} \in N$. This shows that $a_1 b_1 N = a_2 b_2 N$.
    \item $e N = N$ is the neutral element.
    \item The group operation is closed under $\Quot{G}{N}$ by definition.
    \item $\inv{(a N)} = \inv{a} N$ clearly is the inverse of $a N$.
\end{itemize} $\implies \Quot{G}{N}$ is a group.
\end{proof}

\begin{ex}{Residue classes}{}
We will consider the quotient group $\Quot{\Z}{n\Z}$ of the group $(\Z,+)$ for any fixed $n \in \NZ$. We write, \begin{align}
    n\Z \defeq \gen{n} = \{n \cdot k \mid k \in \Z\}.
\end{align}

Observe that the left cosets are of the form \begin{align}
    a + n\Z = \{a + n \cdot k \mid k \in \Z\}.
\end{align} They are also called \emph{residue classes}\index{residue classes} modulo $n$.

To specify $\Quot{\Z}{n\Z}$, we are interested in finding when ${a + n\Z = b + n\Z}$ holds. We have, \begin{align}
    a + n\Z = b + n\Z \overset{\ref{lem:cosets_eq}}{\iff} a - b \in n\Z \overset{\ref{eq:left_coset}}{\iff} \divides{n}{a - b}.
\end{align} Equivalently to ${\divides{n}{a - b}}$, we say that $a$ is \emph{congruent}\index{congruent} $b$ modulo $n$ (denoted ${a \equiv b \mod n}$). For ${n > 0}$ this is equivalent to $a$ and $b$ having the same residue ${r \in \{0, 1, \dots, n - 1\}}$ when dividing by $n$.

From now on, we will assume ${n > 0}$. We denote elements by \begin{align}
    \rep{a} \defeq a + n\Z,
\end{align} where $a$ is referred to as the \emph{representative}\index{representative} of $\rep{a}$. It follows that, \begin{align}
    \Quot{\Z}{n\Z} = \{\rep{0}, \rep{1}, \dots, \rep{n-1}\}.
\end{align} By \cref{thm:quotient_group}, $\Quot{\Z}{n\Z}$ with the mapping ``$+$'' is a cyclic group of order $n$. It is often denoted by \begin{align}
    Z_n \defeq \Quot{\Z}{n\Z} = \gen{\rep{1}}.
\end{align}

As an example, consider $Z_8 = \{\rep{0}, \rep{1}, \rep{2}, \rep{3}, \rep{4}, \rep{5}, \rep{6}, \rep{7}\}$. We have, \vspace{-10pt}\begin{itemize}
    \item $\rep{2} + \rep{5} = \rep{7}$,
    \item $\rep{2} + \rep{6} = \rep{8} = \rep{0}$, and
    \item $\rep{-6} = \rep{2}$.
\end{itemize}
\end{ex}

\begin{defn}[Cokernel]
The \emph{cokernel}\index{cokernel} of a homomorphism ${\varphi : G \to H}$ is the quotient group $\Quot{H}{\im{\varphi}}$.
\end{defn}

\begin{ex}{Outer automorphism group}{}
Automorphisms that are not inner automorphisms are called \emph{outer automorphisms}\index{outer automorphism}. The \emph{outer automorphism group}\index{outer automorphism group} is the group of cosets of the inner automorphism group with respect to outer automorphisms, \begin{align}
    \Out{G} \defeq \Quot{\Aut{G}}{\Inn{G}}.
\end{align}

Let us define the homomorphism ${\sigma : G \to \Aut{G}, g \mapsto i_g}$. It can be shown that \vspace{-10pt}\begin{itemize}
    \item $\ker{\sigma} = Z(G)$,
    \item $\im{\sigma} = \Inn{G}$, and
    \item the cokernel of $\sigma$ is $\Out{G} = \Quot{\Aut{G}}{\Inn{G}}$.
\end{itemize}
\end{ex}


\chapter{Combination of Events}

\section{Important/Useful Theorems}

\subsection{}

\begin{equation}
	P(A|B) = \frac{P(AB)}{P(B)}
\end{equation}

\subsection{}
If $A = \bigcup{}_k A_k$ where $A_k$ are all mutually exclusive
\begin{equation}
	P(A|B) = \sum_k P(A_k|B)
\end{equation}

\subsection{}
The events $A_1, A_2, A_3, ..., A_n$ are said to be statistically independent if
\begin{eqnarray*}
	P(A_i A_j) = P(A_i)P(A_j) \\
	P(A_i A_j A_k) = P(A_i)P(A_j)P(A_k) \\
	... \\
	P(A_1 A_2 A_3...A_n) = P(A_1)P(A_2)P(A_3)...P(A_n)
\end{eqnarray*}
For all possible combinations of the indeces.

\subsection{Second Borel-Cantelli lemma}
If the events $A_1, A_2, A_3, ...$ are all statistically independent and $p_k = P(A_k$
\begin{equation}
	\sum_{k=1}^\infty p_k = \infty
\end{equation}

\section{Answers to Problems}

%answer template
\subsection{}
%problem 3.1

We need to examine $A(\bar{A}B)$ , $A \overline{A \cup B}$ and $(\bar{A}B) \overline{A \cup B}$.

\begin{equation}
	A(\bar{A}B) = A\bar{A}B = 0
\end{equation}
since $A$ will not overlap with its complement at all
\begin{equation}
	A \overline{A \cup B} = A \bar{A}\bar{B} = 0
\end{equation}
for the same reason
\begin{equation}
	(\bar{A}B) \overline{A \cup B} = \bar{A}B \bar{A}\bar{B} = 0
\end{equation}
again for the same reason.

Clearly, since none of the three proposed events intersect at all with any of the others, these events are mutually exclusive and cannot happen in concert.
\textbf{Answer not verified}

\subsection{}
%problem 3.2
To get $C$ to be mutually exclusive, its overlap with both $B$ and $A$ must be zero, giving a logical conclusion of:

\begin{equation}
	C = \bar{A}\bar{B}
\label{answer3.2}

To put it in English, the mutually exclusive outcomes of a chess match are, white wins, black wins and neither wins (a draw).
\end{equation}
\textbf{Answer not verified}

\subsection{}
%problem 3.3

\begin{equation}
	P(A|B) = \frac{P(AB)}{P(B)} = \frac{P(BA)}{P(B)} \frac{P(A)}{P(A)} = P(B|A) \frac{P(A)}{P(B)} > P(A)
\end{equation}

\begin{equation}
	P(B|A)  >  P(B)
\label{answer3.3}
\end{equation}

\textbf{Answer not verified}

\subsection{}
%problem 3.4
\begin{equation}
	P(A|B) = \frac{P(AB)}{P(B)} = \frac{3}{2} P(AB)
\end{equation}
We can come up for an expression for $P(AB)$ in terms of $P(A\cup B)$ with a little algebra.
\begin{equation}
	P(A \cup B) = P(A) +P(B) - P(AB) = \frac{4}{3} - P(AB)
\end{equation}
clearly $P(A \cup B)$ must be between 0 and 1.
\begin{equation}
	 1 \geq \frac{4}{3} - P(AB) \geq 0
\end{equation}
\begin{equation}
	 -\frac{1}{3} \geq - P(AB) \geq -\frac{4}{3}
\end{equation}
\begin{equation}
	 \frac{1}{3} \leq  P(AB) \leq \frac{4}{3}
\end{equation}
\begin{equation}
	 \frac{1}{3} \leq  \frac{2}{3} P(A|B) \leq \frac{4}{3}
\end{equation}
\begin{equation}
	 \frac{1}{2} \leq  P(A|B) \leq 2
\end{equation}
\begin{equation}
	\frac{1}{2} \leq  P(A|B)
\label{answer3.4}
\end{equation}
\textbf{Answer not verified}

\subsection{}
%problem 3.5

\begin{equation}
	P(A)P(B|A)P(C|AB) = P(A)\frac{P(BA)}{P(A)} \frac{P(ABC)}{P(AB)} = P(ABC)
\end{equation}
Simple enough, but now we need a general theorem
\begin{equation}
	P\left(  \bigcap_{k=1}^n A_k  \right) = P(A_1)P(A_2|A_1)P(A_3|A_1A_2)P(A_4|A_1A_2A_3) ... P\left(A_n| \bigcap_{k=1}^{n-1} A_k  \right)
\end{equation}

\begin{equation}
	P\left(  \bigcap_{k=1}^n A_k  \right) = P(A_1) \prod_{z=2}^n} P\left(A_z| \bigcap_{k=1}^{z-1} A_k  \right)
\label{answer3.5}
\end{equation}
\textbf{Answer not verified}

\subsection{}
%problem 3.6
\begin{equation}
	P(A) = P(A|B) +  P(A|\bar{B}) = \frac{P(AB)}{P(B)} + \frac{P(A\bar{B})}{P(\bar{B})}
\end{equation}

\subsubsection{$A=0$}
\begin{equation}
	P(0) = \frac{P(0)}{P(B)} + \frac{P(0)}{P(\bar{B})} = 0
\end{equation}
Because the intersection with nothing is nothing.

\subsubsection{$B=0$}
\begin{equation}
	P(A) = \frac{P(0)}{P(0)} + \frac{P(A\Omega)}{P(\Omega)} = P(A)
\end{equation}

\subsubsection{$B=\Omega$}
\begin{equation}
	P(A) = \frac{P(A\Omega)}{P(\Omega)} + \frac{P(0)}{P(0)} = P(A)
\end{equation}

\subsubsection{$B=A$}
\begin{equation}
	P(A) = \frac{P(AA)}{P(A)} + \frac{P(A\bar{A})}{P(\bar{A})} = 1 ???
\end{equation}
I cannot prove this to be true

\subsubsection{$B=\bar{A}$}
\begin{equation}
	P(A) = \frac{P(A\bar{A})}{P(\bar{A})} + \frac{P(AA)}{P(A)} = 1 ???
\end{equation}
I cannot prove this to be true

\textbf{Answer not verified}

\subsection{}
%problem 3.7
We start knowing:
\begin{equation}
	P(AB) = P(A)P(B)
\end{equation}
But we want to prove:
\begin{equation}
	P(\bar{A}\bar{B}) = P(\bar{A})P(\bar{B})
\end{equation}
as the problem suggests
\begin{equation}
	P(B|A) + P(\bar{B}|A) = 1
\end{equation}
\begin{equation}
	\frac{P(BA)}{P(A)} + \frac{P(A\bar{B})}{P(A)} = 1
\end{equation}
\begin{equation}
	P(B) + \frac{P(A\bar{B})}{P(A)} = 1
\end{equation}
\begin{equation}
	\frac{P(A\bar{B})}{P(A)} = 1 - P(B)
\end{equation}
\begin{equation}
	P(A\bar{B}) = \left(1 - P(B)\right)P(A)
\end{equation}
\begin{equation}
	P(A\bar{B}) = P(\bar{B})P(A)
\end{equation}
so clearly $A \text{ and } \bar{B}$ are independent.  Since we have just proved that for two events that are independent, we can show that one of the events is independent of the others complement, without loss of generality, we can apply this logic recursively, thus proving that if two events are independent, so are their complements.

\textbf{Answer not verified}

\subsection{}
%problem 3.8
Given two mutually exclusive events, we wonder if they are dependent.
\begin{equation}
	P(AB) = 0
\end{equation}
but in order for them to be independent, we need to be able to say
\begin{equation}
	P(AB) = P(A)P(B)
\end{equation}
but we were told that $P(A)$ and $P(B)$ are positive, therefore they are \textbf{NOT} independent!
\textbf{Answer not verified}

\subsection{}
%problem 3.9
Let $A_i$ be the event of getting a white ball at the $i^{th}$ urn.   Since there are 2 possibilities for each step, there are $2^n$ different ways to do this ``experiment''.  Clearly
\begin{equation}
	P(A_1) = \frac{w}{w+b}
\end{equation}
but things get more complicated as $i>1$.  Here there are two mutually exclusive possibilities: we got a white one or we got a black one from the first urn.
\begin{equation}
	P(A_2) = P(A_2|A_1)P(A_1) + P(A_2|\bar{A_1})P(\bar{A_1})
\end{equation}
\begin{equation}
	P(A_2) = P(A_2A_1) + P(A_2\bar{A_1})
\end{equation}
For both denominators, there were $w+b$ possibilities to begin with and $w+b+1$ for the second.   Then, there were $w$ ways to get white first and $b$ ways to get black first.  Then, respectively, there would be $w+1$ ways and $w$ ways to get white for the second urn.

\begin{equation}
	P(A_2) = \frac{1}{(w+b)(w+b+1)} \left( w(w+1) + bw \right)
\end{equation}
\begin{equation}
	P(A_2) = \frac{w(w+b+1)}{(w+b)(w+b+1)} = \frac{w}{w+b}
\end{equation}

Since we started off with $\frac{w}{w+b}$ and then got the same answer for the second step, if we were to do it for a third step, a fourth step, etc. we would be starting with the same initial conditions and would get the same answer therefore this is true in general for $n$ urns.

\textbf{Answer not verified}

\subsection{}
%problem 3.10
Labelling the points starting from the left, we know that
\begin{eqnarray}
	P(C_1|B_1) = \frac{1}{3} = P(C_2|B_1) \\
	P(C_3|B_2) = \frac{1}{2} \\
	P(C_4|B_4) = \frac{1}{5} = P(C_5|B_4) = P(C_6|B_4)
\end{eqnarray}
Since there's only one way to get to each of these specified end-points, the total probabilities are just the conditional probabilities times the probabilities of the conditions.
\begin{eqnarray}
	P(C_1) = P(C_1|B_1)P(B_1) = \frac{1}{12} = P(C_2) \\
	P(C_3) = P(C_3|B_2)P(B_2) = \frac{1}{8} \\
	P(C_4) = P(C_4|B_4)P(B_4) = \frac{1}{20} = P(C_5) = P(C_6)
\end{eqnarray}
So when we combine all these possibilities with the probability of getting to $A$:
\begin{equation}
	\frac{67}{120} + \frac{2}{12} + \frac{1}{8} + \frac{3}{20} = 1
\label{answer3.10}
\end{equation}
\textbf{Answer not verified}

\subsection{}
%problem 3.11
Let's go from 1 dollar stakes to $q$ dollar stakes; all the other variables stay the same.

\begin{equation}
	p(x) = \frac{1}{2} \left[  p(x+q) + p(x-q)  \right], q \leq x \leq m - q
\end{equation}
but the boundary conditions do not change.  Therefore, there is no change here between the original linear equation and the new linear equation.  Thus:
\begin{equation}
	p(x) = 1 - \frac{x}{m}
\label{answer3.11}
\end{equation}
\textbf{Answer not verified}

\subsection{}
%problem 3.12
\begin{equation}
	P(B|A) = P(B | \bar{A})
\end{equation}
but let's work on the assumption they are not independent
\begin{eqnarray*}
	\frac{P(AB)}{P(A)} = \frac{P(\bar{A}B)}{\bar{A}} \\
	\frac{P(AB)}{P(\bar{A}B)} = \frac{P(A)}{P(\bar{A})}
\end{eqnarray*}
but now let's factor out the expression for the intersection, leaving some error behind
\begin{eqnarray*}
	\frac{P(A)P(B)\epsilon}{P(\bar{A})P(B)\epsilon{}'} = \frac{P(A)}{P(\bar{A})} \\
	\frac{P(A)\epsilon}{P(\bar{A})\epsilon{}'} = \frac{P(A)}{P(\bar{A})}
\end{eqnarray*}
The only way for this to always be true is for both error terms to be the same and the only way for it to be true for arbitrary events is for them to both be one such that $A$ and $B$ are independent.
\textbf{Answer not verified}

\subsection{}
%problem 3.13
There are four mutually exclusive outcomes for the first step: getting two whites, one white and zero whites($A_2, A_1, A_0$).  We'll call the last event we want to turn out white, $B$.
\begin{equation}
	P(B) = P(B|A_2)P(A_2) + P(B|A_1)P(A_1) + P(B|A_0)P(A_0)
\end{equation}

\begin{eqnarray}
	P(B|A_2)P(A_2) =  1 \frac{w_1 w_2}{(w_1 + b_1)(w_2 + b_2)}\\
	P(B|A_1)P(A_1) = \frac{1}{2} \frac{w_1 b_2 + w_2 b_1}{(w_1 + b_1)(w_2 + b_2)} \\
	P(B|A_0)P(A_0) = 0
\end{eqnarray}
putting it together
\begin{eqnarray}
	P(B) = \frac{1}{2} \frac{w_1 b_2 + w_2 b_1 + 2 w_1 w_2}{(w_1 + b_1)(w_2 + b_2)}  \\
	P(B) = \frac{1}{2} \frac{(w_1 +b_1 )w_2 + (w_2 + b_2) w_1 }{(w_1 + b_1)(w_2 + b_2)}  \\
	P(B) = \frac{1}{2} \left( \frac{w_1}{w_1+b_1} + \frac{w_2}{w_2+b_2} \right)
\end{eqnarray}

\textbf{Answer not verified}

\subsection{}
%problem 3.14
As the hint suggests, we'll use Bayes' Rule.  Our set of mutually exclusive events $\{ B_i\}$ are the getting of the ball from the $i^{th}$ urn and $A$ is the event of getting a white ball.  So we want the probability of choosing the odd urn out (we'll call it $B_k$) given that we got a white ball.

\begin{equation}
	P(B_k|A) = \frac{P(B_k)P(A|B_k)}{\sum_{i=1}^{10} P(B_i)P(A|B_i}
\end{equation}
For all $i$, $P(B_i) = \frac{1}{10}$ and for $i \neq k$ $P(A|B_i) = \frac{1}{2}$ but for $i=k$,  $P(A|B_k) = \frac{5}{6}$
\begin{equation}
	P(B_k|A) = \frac{\frac{5}{6 \cdot 10}}{9\frac{1}{20} + \frac{5}{6 \cdot 10}} = \frac{5}{32}
\end{equation}

\textbf{Answer verified}

\subsection{}
%problem 2.15
To do this problem, let's figure out what the chance is that we picked the all-white urn, the event $B_1$. $A$ is the event we pick a white ball and $B_@$ is the event of picking the other urn with $\frac{3}{4}$ white balls.

\begin{equation}
	P(B_1|A) = \frac{P(B_1)P(A|B_1)}{P(B_1)P(A|B_1) + P(B_2)P(A|B_2)} = \frac{\frac{1}{2}}{\frac{1}{2} + \frac{1}{2} \frac{3}{4}} = \frac{4}{7}
\end{equation}
Thus, there is a $\frac{3}{7}$ chance we chose the urn that actually has black balls in it and a $\frac{1}{4}$ chance that from that urn once chooses a black ball giving an overall probability of picking a black ball given the information in the problem of $\frac{3}{28}$.
\textbf{Answer verified}

\subsection{}
%problem 2.16
\textbf{SKIPPED}: unsure of meaning of problem.

\subsection{}
%problem 2.17
Clearly $P(A) = \frac{1}{2} = P(B) = P(C)$ and $P(AB) = \frac{1}{4} = P(BC) = P(AC)$ because there is only one place out of four for the die to hit both letters.  There is, however, still only one way to hit all three letters at once such that $P(ABC)=\frac{1}{4}$ therefore  the events in question are pairwise independent since for all sets of letters $P(AB)=P(A)P(B)$ but not completely independent since $P(ABC) \neq P(A)P(B)P(C)$.
\textbf{Answer not verified}

\subsection{}
%problem 2.18

\textbf{SKIPPED}: didn't feel like doing the problem... so there.

%%answer template
%\subsection{}
%%problem n.n
%
%
%\begin{equation}
%
%\label{answern.n}
%\end{equation}
%\textbf{Answer [not] verified}


\documentclass[twoside]{MATH77}
\usepackage{multicol}
\usepackage[fleqn,reqno,centertags]{amsmath}
\begin{document}
\hyphenation{SWPWRI}
\begmath 17.1 Computation Using Derivative Arrays or
\hbox{Univariate Taylor Series}

\silentfootnote{$^\copyright$1997 Calif. Inst. of Technology, \thisyear \ Math \`a la Carte, Inc.}

\subsection{Purpose}

This set of subroutines performs computations using arrays of length N+1,
where each array contains the value of a function, $f(t)$, and its first N
derivatives with respect to $t$, evaluated at some point, $t_0$. Such an
array can alternatively be regarded as a scaled representation of the first
N + 1 coefficients of the Taylor series of $f(t)$ expanded at $t_0$, since
the coefficient of $(t-t_0)^k$ in such a Taylor series is $f^{(k)}(t)/k!$
evaluated at $t_0.$

This package provides a way of computing values of the first N derivatives
of a univariate function that is defined by a sequence of computational
steps involving arithmetic and elementary functions, without the need to
derive and code expressions for the derivatives.

\subsection{Usage}

We shall use $t$ as the generic name of the single independent variable with
respect to which all derivatives are defined. We shall use the term
W-variable to denote an (N+1)-tuple, consisting of a function value and
values of the function's first N derivatives with respect to $t$, evaluated
at some point. Note that the representation of $t$ as a W-variable evaluated
at $t_0$ is the (N+1)-tuple, $(t_0$, 1, 0, 0, ...,~0).

This package consists of one subroutine, SWSET, for assigning a value to a
W-variable, 22~subroutines for doing arithmetic operations and computing
elementary functions using W-variables, and three supplementary subroutines,
SWCHN, SWRCHN, and SPASCL. These will be described in the following sections:

\begin{tabular*}{3.3in}{@{}l@{~~}l}
B.1 & SWSET\dotfill \pageref{B1}\\
B.2 & Arithmetic and elementary functions\dotfill \pageref{B2}\\
B.3 & SWCHN\dotfill \pageref{B3}\\
B.4 & SWRCHN\dotfill \pageref{B4}\\
B.5 & SPASCL\dotfill \pageref{B5}\\
B.6 & Modifications for double-precision usage\dotfill \pageref{B6}\\
 & \rule{3in}{0pt}~
\end{tabular*}\vspace{-5pt}

\subsubsection{SWSET, Assigning a value to a W-variable\label{B1}}

\paragraph{Program Prototype, Single Precision}

\begin{description}
\item[INTEGER]  \ {\bf N}

\item[REAL]  \ {\bf VAL, DERIV, W}($\geq $N+1)
\end{description}

Assign values to N, VAL, and DERIV
$$
\fbox{{\bf CALL SWSET(N, VAL, DERIV, W)}}
$$
Computed quantities are returned in W().

\paragraph{Argument Definitions}

\begin{description}
\item[N]  \ [in] Highest order derivative to be considered. Require 0 $\leq $
N $\leq $ NMAX. See Section E for the definition of NMAX.

\item[VAL]  \ [in] Value to be assigned to the W-variable, W().

\item[DERIV]  \ [in] Value to be assigned as the first derivative of the
W-variable, W().

\item[W()]  \ [out] Array of length at least N + 1 in which this subroutine
will place N + 1 values to represent a W-variable as follows: (VAL, DERIV,
0.0, ..., 0.0). The user should set DERIV = 1.0 to define this variable as
the variable, $t$, with respect to which all differentiation is done. The
user can set DERIV = 0.0 to define this W-variable to be a constant, $i.e$.
a variable not depending on $t.$
\end{description}

\subsubsection{Arithmetic and elementary functions using W-variables\label{B2}}

In describing the following subroutines, X() and Y() denote input
W-variables and Z() denotes an output W-variable. The variables A and I are
input variables that are constant relative to $t.$

In most of these subroutines the output array Z() must occupy storage
locations distinct from any of the input data. Exceptions to this rule are
SWSUM, SWDIF, SWPRO, SWSUM1, SWDIF1, and SWPRO1. The subroutine SWQUO, which
computes $x/y \rightarrow z$, permits $x$ and $z$ to occupy the same
storage, but $y$ and $z$ must occupy distinct storage.

\paragraph{Program Prototype, Single Precision}

\begin{description}
\item[INTEGER]  \ {\bf N, I}

\item[REAL]  \ {\bf A, X}($\geq $N+1){\bf , Y}($\geq $N+1){\bf , Z}($\geq $%
N+1)
\end{description}

Assign values to N, I, A, X(), and Y(), as appropriate.

\begin{center}
{\bf Two-argument operations with both arguments depending on $t.$}

\fbox{\begin{tabular}{@{\bf \ }lr}
CALL SWSUM(N, X, Y, Z) & $x+y \rightarrow z$\\
CALL SWDIF(N, X, Y, Z) & $x-y \rightarrow z$\\
CALL SWPRO(N, X, Y, Z) & $x\times y \rightarrow z$\\
CALL SWQUO(N, X, Y, Z) & $ x/y \rightarrow z$\\
CALL SWATN2(N, X, Y, Z) &  atan2($x,y) \rightarrow z$\\
\multicolumn{2}{c}{where for atan2: \ $-\pi < z \leq \pi $ and
$\tan (z) = x/y$}\\
\end{tabular}}
\end{center}

\pagebreak

\begin{center}
{\bf Two-argument operations with only one argument depending on $t.$}

\fbox{\begin{tabular}{@{\bf \ }lr}
CALL SWSUM1(N, A, Y, Z) & $ a+y \rightarrow z$\\
CALL SWDIF1(N, A, Y, Z) & $ a-y \rightarrow z$\\
CALL SWPRO1(N, A, Y, Z) & $ a\times y \rightarrow z$\\
CALL SWQUO1(N, A, Y, Z) & $ a/y \rightarrow z$\\
CALL SWPWRI(N, I, Y, Z) & $ y^i \rightarrow z$\\
\multicolumn{2}{c}{(See following note.)}\\
\end{tabular}}
\end{center}

Note: I may be positive, negative, or zero. If I = 0, SWPWRI sets Z(1) = 1.0
and all derivative values of $z$ to~0.0, regardless of the given value of $y$%
. It is an error to have $y = 0.0$ when I $< 0.$

\begin{center}
{\bf One-argument operations with the argument depending on $t.$}

\fbox{\begin{tabular}{@{\bf \ }lr}
CALL SWSQRT(N, X, Z) &  $\sqrt x \rightarrow z$\\
CALL SWEXP(N, X, Z) & $\exp (x) \rightarrow z$\\
CALL SWLOG(N, X, Z) & $\log (x) \rightarrow z$\\
CALL SWSIN(N, X, Z) & $\sin (x) \rightarrow z$\\
CALL SWCOS(N, X, Z) & $\cos (x) \rightarrow z$\\
CALL SWTAN(N, X, Z) & $\tan (x) \rightarrow z$\\
CALL SWASIN(N, X, Z) & $\sin^{-1}(x) \rightarrow z$\\
CALL SWACOS(N, X, Z) & $\cos^{-1}(x) \rightarrow z$\\
CALL SWATAN(N, X, Z) & $\tan^{-1}(x) \rightarrow z$\\
CALL SWSINH(N, X, Z) & $\sinh(x) \rightarrow z$\\
CALL SWCOSH(N, X, Z) & $\cosh(x) \rightarrow z$\\
CALL SWTANH(N, X, Z) & $\tanh(x) \rightarrow z$\\
\end{tabular}}
\end{center}

\paragraph{Argument Definitions}

\begin{description}
\item[N]  \ [in] Highest order derivative to be considered. Require 0 $\leq $
N $\leq $ NMAX. See Section E for the definition of NMAX.

\item[A]  \ [in] Floating point value that is independent of $t.$

\item[I]  \ [in] Integer value that is independent of $t.$

\item[X()]  \ [in] An input W-variable.

\item[Y()]  \ [in] An input W-variable.

\item[Z()]  \ [out] An output W-variable.
\end{description}

\subsubsection{SWCHN, application of the chain rule\label{B3}}

This subroutine is called by all of the elementary function subroutines. The
user will not need to call it directly unless the user is developing a new
function subroutine that is not conveniently representable in terms of the
available functions.

\paragraph{Program Prototype, Single Precision}

\begin{description}
\item[INTEGER]  \ {\bf N}

\item[REAL]  \ {\bf X}($\geq $N+1){\bf , F}($\geq $N+1)
\end{description}

Assign values to N, X(), and F().
$$
\fbox{{\bf CALL SWCHN(N, X, F)}}
$$
Computed quantities are returned in F().

\paragraph{Argument Definitions}

\begin{description}
\item[N]  \ [in] Highest order derivative to be considered. Require 0 $\leq $
N $\leq $ NMAX. See Section E for the definition of NMAX.

\item[X()]  \ [in] On entry, X() must contain $x(t_0)$ and derivatives
through order N of $x(t)$ with respect to $t$. X() will be unchanged by this
subroutine.

\item[F()]  \ [inout] On entry, F() must contain $f(x_0)$ and derivatives
through order N of $f(x)$ with respect to $x$, evaluated at $x_0=x(t_0)$. On
return F() will contain $f(x(t_0))$ and derivatives through order N of $%
f(x(t))$ with respect to $t$, evaluated at $t_0$. Note that the contents of
F(1) will remain unchanged.
\end{description}

\subsubsection{SWRCHN, application of the reverse chain rule\label{B4}}

This subroutine reverses the action of SWCHN, in the sense that if one were
to make the two calls

CALL SWCHN(N, X, F)

CALL SWRCHN(N, X, F)

the first would transform the contents of the array F(), and the second
would (if X(2) is nonzero) transform the contents of F() back to the same
values (to within computational errors) as before the first call. See
Section C for application of SWRCHN to series reversion.

\paragraph{Program Prototype, Single Precision}

\begin{description}
\item[INTEGER]  \ {\bf N}

\item[REAL]  \ {\bf X}($\geq $N+1){\bf , F}($\geq $N+1)
\end{description}

Assign values to N, X(), and F().
$$
\fbox{{\bf CALL SWRCHN(N, X, F)}}
$$
Computed quantities are returned in F().

\paragraph{Argument Definitions}

\begin{description}
\item[N]  \ [in] Highest order derivative to be considered. Require 0 $\leq $
N $\leq $ NMAX. See Section E for the definition of NMAX.

\item[X()]  \ [in] On entry, X() must contain $x(t_0)$ and derivatives
through order N of $x(t)$ with respect to $t$. X(2), representing $dx/dt$
evaluated at $t_0$, must be nonzero. The contents of X(1) will not be used
by this subroutine. X() will be unchanged by this subroutine.

\item[F()]  \ [inout] On entry, F() must contain $f(x(t_0))$ and derivatives
through order N of $f(x(t))$ with respect to $t$, evaluated at $t=t_0$. On
return F() will contain $f(x_0)$ and derivatives through order N of $f(x)$
with respect to $x$, evaluated at $x_0=x(t_0)$. Note that the contents of
F(1) will remain unchanged.
\end{description}

\subsubsection{SPASCL, computation of the ``Pascal triangle" of binomial
coefficients\label{B5}}

\paragraph{Program Prototype, Single Precision}

\begin{description}
\item[INTEGER]  \ {\bf N}

\item[REAL]  \ {\bf C}($\geq $ 1 + (N*(N+1))/2)
\end{description}

Assign a value to N.
$$
\fbox{{\bf CALL SPASCL(N, C)}}
$$
Computed quantities are returned in C().

\paragraph{Argument Definitions}

\begin{description}
\item[N]  \ [in] Highest order derivative to be considered. Require 0 $\leq $
N $\leq $ NMAX. See Section E for the definition of NMAX.

\item[C()]  \ [out] On return will contain values constituting the ``Pascal
triangle'' of binomial coefficients. For example, if N = 4, the Pascal
triangle is
\end{description}

\begin{center}
\begin{tabular}{c}
1\\
1\ \ \ 1\\
1\ \ \ 2\ \ \ 1\\
1\ \ \ 3\ \ \ 3\ \ \ 1\\
1\ \ \ 4\ \ \ 6\ \ \ 4\ \ \ 1
\end{tabular}
\end{center}

SPASCL omits the first element of each row except the first, and thus
returns the 11~values: 1, 1, 2, 1, 3, 3, 1, 4, 6, 4,~1.

\subsubsection{Modifications for Double Precision\label{B6}}

For double-precision usage, replace the initial S in the name of each
subroutine with D, and replace the REAL declarations by DOUBLE PRECISION.

\subsection{Examples and Remarks}

As a demonstration problem, consider the following assignments:

\begin{tabbing}
\hspace{.2in}\=$t$ = 2\\
\>$z_1$ = log(sqrt($t$))\\
\>$z_2$ = exp(2 * $z_1$)\\
\>$d$ = $z_2$ - $t$
\end{tabbing}

This should result in $z_2 = t$ and $d = 0$. These quantities and their
derivatives through order~3 with respect to $t$ are computed by the program
DRDCOMP. The results are shown in ODDCOMP. Note that $d$ and all of its
derivatives are zero to machine accuracy, as they should be.

The running time for this package is approximately proportional to $\text{N}%
^3$. To save time when developing new code using this package, one may run
with N = 0 until one is satisfied that the function evaluation is as
desired, and then increase N to activate the derivative computation.

\subparagraph{Implicit functions and series reversion}

Let $[x;t]$ denote a W-variable containing $x$ and all of its derivatives
through order N with respect to $t$, evaluated at $t_0$. Let $x_0 = x(t_0)$.
Suppose one would like to have $[t;x]$. This would represent the inverse
function that is defined implicitly by $[x;t]$. When the components of these
arrays are interpreted as scaled coefficients of Taylor series the
transformation from $[x;t]$ to $[t;x]$ is known as series reversion.

The subroutine SWRCHN can be used to produce $[t;x]$. Note, from the
specifications in Section B.4, that given $[x;t]$ and $[f;t]$, SWRCHN
transforms $[f;t]$ to $[f;x]$. Formally replacing the symbol $f$ by $t$, we
see that given $[x;t]$ and $[t;t]$, SWRCHN transforms $[t;t]$ to $[t;x]$,
thus producing the desired object. Note that the W-variable, $[t;t]$, needed
as part of the input, is just $(t_0$, 1, 0, ..., 0).

\subsection{Functional Description}

The ``W" in the names of subroutines in this package refers to R.  E.
Wengert, who, in \cite{Wengert:1964:ASA}, presented the ideas on which the
package is based.  Subsequent authors have incorporated this approach into
more automated systems \cite{Griewank:1991:ADA} that would probably be
easier to use than the present package, but such systems present various
hurdles regarding acquisition and installation.  The present
implementation provides the basic capabilities in a highly portable form.

The fundamental idea is simply to provide, for each basic mathematical
operation, code that not only performs the operation but propagates
derivative values up to the desired order. For example, to support the sine
function with derivatives through second degree, we note that if

\begin{tabbing}
\hspace{.4in}\=$z = \sin y,$\\
then\\
\>$z'= y' \cos y,$\\
and\\
\>$z''= y'' cos y - {y'}^2 \sin y,$
\end{tabbing}

where the primes denote derivatives with respect to an independent variable,
$t$. Clearly these formulas permit one to write a subroutine that can accept $%
y$, $y^{\prime}$, and $y^{\prime \prime}$ as input, and
produce $z$, $z^{\prime}$, and $z^{\prime \prime}$
as output.

This approach has been systematized for arbitrary N by appropriate use of
the chain rule of differentiation. Our subroutine SWCHN implements the chain
rule as

\begin{tabbing}
\hspace{.2in}\=do L = 0, N $-$ 1\\
\>\ \ \ \ call SWPRO(L, F(N+1$-$L), X(2), F(N+1$-$L))\\
\>enddo
\end{tabbing}

Since SWCHN is called by all of the elementary function subroutines, this
loop essentially determines the dependence on N of the running time of the
whole package. SWPRO is an $O(\text{N}^2)$ process and SWCHN is an $O(\text{N%
}^3)$ process. To improve efficiency we have coded the cases of N = 0, 1, 2,
3, and~4 as special cases in SWPRO. Multiplication counts in SWPRO and SWCHN
are as follows:

\begin{tabular}{@{\ \ }r@{\ \ }r@{\ \ }r@{\ \ }r@{\ \ }r@{\ \ }r@{\ \ }r@{\ \ }r@{\ \ }r@{\ \ }r@{\ \ }r@{\ \ }r@{\ \ }}
N $\Rightarrow$ & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 & 10\\
SWPRO & 1 & 3 & 7 & 12 & 19 & 27 & 37 & 48 & 61 & 75 & 91\\
SWCHN & 0 & 1 & 4 & 11 & 23 & 42 & 69 & 106 & 154 & 215 & 290
\end{tabular}
\nocite{Lawson:1971:CDU}

\bibliography{math77}
\bibliographystyle{math77}

\subsection{Error Procedures and Restrictions}

\subsubsection{Upper limit on the derivative order, N}

Subroutines SWATAN, SWSUM, and SWPRO in this package contain internal arrays
whose dimensions depend on NMAX. NMAX is a PARAMETER in each of these
subroutines, nominally set to~10. This limits the derivative order argument,
N, in all of the subroutines to~10. The user would need to increase NMAX in
these three subroutines if higher order derivatives are needed.

\subsubsection{Invalid arguments for derivative computation}

The user will likely be accustomed to avoiding sending invalid arguments to
the elementary functions, such as a negative argument to the square root. In
computing derivatives there are some additional singularities to avoid. Note
that the derivative is infinite at zero for the square root, and at $\pm 1$
for arcsin and arccosine.

\subsubsection{Error handling}

Following is a list of error conditions the package detects and for which
error messages are issued. These errors are fatal in the sense that the
requested operation cannot be done; however, the default action is to return
after issuing an error message. The user can use the MATH77 library
subroutine, ERMSET of Chapter~19.2, to alter this action to cause a STOP
if desired. Error conditions not on this list, $e.g.$, negative argument
in log, will be handled by the usual host system error handler.

\begin{tabular}{@{}l@{\ \ }l}
\bf Error No.\\
\bf \& Program & \multicolumn{1}{c}{\bf Explanation}\\
\bf 1 SWASIN & Infinite derivative when arg = $-$1 or +1\\
\bf 1 SWACOS & Infinite derivative when arg = $-$1 or +1\\
\bf 2 SWSQRT & Infinite derivative when arg = 0\\
\bf 3 SWQUO1 & Zero divisor\\
\bf 4 SWPWRI & Y**I is infinite when Y = 0 and I $< 0$\\
\bf 5 SWPRO & Require dimension NMAX $\geq $ N\\
\bf 6 SWQUO & Require dimension NMAX $\geq $ N\\
\bf 7 SWQUO & Zero divisor.\\
\bf 8 SPASCL & Require N $\geq 0$\\
\bf 9 SWRCHN & Require X($2) \neq 0.$
\end{tabular}

\subsection{Supporting Information}

The source language is ANSI Fortran~77.

All of the double precision entry points require files:

DWCOMP, ERFIN, ERMSG, IERM1, IERV1.

For single precision, file SWCOMP is required in place of file DWCOMP.
\vspace{-5pt}
\begin{center}
\begin{tabular}{llll}
\multicolumn{4}{c}{\bf Entries} \\
DPASCL & DWACOS & DWASIN & DWATAN\\
DWATN2 & DWCHN & DWCOS & DWCOSH\\
DWDIF & DWDIF1 & DWEXP & DWLOG\\
DWPRO & DWPRO1 & DWPWRI & DWQUO\\
DWQUO1 & DWRCHN & DWSET & DWSIN\\
DWSINH & DWSQRT & DWSUM & DWSUM1\\
DWTAN & DWTANH & SPASCL & SWACOS\\
SWASIN & SWATAN & SWATN2 & SWCHN\\
SWCOS & SWCOSH & SWDIF & SWDIF1\\
SWEXP & SWLOG & SWPRO & SWPRO1\\
SWPWRI & SWQUO & SWQUO1 & SWRCHN\\
SWSET & SWSIN & SWSINH & SWSQRT\\
SWSUM & SWSUM1 & SWTAN & SWTANH\\
\end{tabular}
\end{center}

Designed by C. L. Lawson, JPL, 1971. Adapted to Fortran~77 for the JPL
MATH77 library, Aug.~1987.

\begcodenp
\lstset{language=[77]Fortran,showstringspaces=false}
\lstset{xleftmargin=.8in}

\centerline{\bf \large DRSWCOMP}\vspace{10pt}
\lstinputlisting{\codeloc{swcomp}}

\vspace{30pt}\centerline{\bf \large ODSWCOMP}\vspace{10pt}
\lstset{language={}}
\lstinputlisting{\outputloc{swcomp}}
\end{document}


\documentclass[a4paper]{article}

\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage[colorinlistoftodos]{todonotes}

\title{MATH 542 Homework 2}
\author{Saket Choudhary\\skchoudh@usc.edu}

\begin{document}
\maketitle
\section*{Problem 1}
$$ A=\begin{pmatrix} A_{11} & A_{12}\\
A_{21} & A_{22}
\end{pmatrix} $$Consider $C=A^{-1}$ so that $CA=I$

$$
C = \begin{pmatrix} C_{11} & C_{12}\\ C_{21} & C_{22}\end{pmatrix}
$$
\begin{align*}
CA &= \begin{pmatrix} A_{11}C_{11}+A_{12}C_{21} & A_{11}C_{12}+A_{12}C_{22}\\
A_{21}C_{11}+A_{22}C_{21} & A_{21}C_{12}+A_{22}C_{22}
\end{pmatrix}\\
&= \begin{pmatrix} I_k &  O_{n-k} \\ O_{n-k} & I_{k} \end{pmatrix}
\end{align*}
Thus,
\begin{align*}
A_{11}C_{11}+A_{12}C_{21} &=  I_k\\
A_{11}C_{12}+A_{12}C_{22} &= O_{n-k}\\
A_{21}C_{11}+A_{22}C_{21} &= O_{n-k}\\
A_{21}C_{12}+A_{22}C_{22}&=I_k
\end{align*}

Thus,

\begin{align*}
C_{11}&= -A_{11}^{-1}(I-A_{12}C_{21})\\
C_{12} &= -A_{11}^{-1}A_{12}C_{22}\\
C_{21} &= -A_{22}^{-1}A_{21}C_{11}\\
C_{22}&=A_{22}^{-1}(I-A_{21}C_{12})
\end{align*}

Substituting for $C_{12}$ and $C_{21}$ :

\begin{align*}
A_{11}C_{11}-A_{12}A_{22}^{-1}A_{21}C_{11} &= I_k\\
\implies C_{11} &= (A_{11}-A_{12}A_{22}^{-1}A_{21})^{-1}
\end{align*}

Similarly,
\begin{align*}
-A_{21}A_{11}^{-1}A_{12}C_{22}+A_{22}C_{22} &= I_k\\
C_{22} &=  (A_{22}-A_{21}A_{11}^{-1}A_{12})^{-1}\\
&=B
\end{align*}
Thus,
\begin{align*}
B &= (A_{22}-A_{21}A_{11}^{-1}A_{12})\\
C_{11} &=(A_{11}-A_{12}A_{22}^{-1}A_{21})^{-1} \\
C_{21}&= -A_{22}^{-1}A_{21}(A_{11}-A_{12}A_{22}^{-1}A_{21})^{-1}\\
C_{22}&= (A_{22}-A_{21}A_{11}^{-1}A_{12})^{-1} = B^{-1}\\
C_{12} &= -A_{11}^{-1}A_{12}C_{22}\\
\end{align*}

Now we use Woodbury matrix identity $(A+UCV )^{-1}= A^{-1}-A^{-1}U(C^{-1}+VA^{-1}U)^{-1}VA^{-1}$

Using which, \begin{align*}
C_{11} &= (A_{11}-A_{12}A_{22}^{-1}A_{21})^{-1}\\
&= A_{11}^{-1}-A_{11}^{-1}(-A_{12})(A_{22}^{-1} + -A_{21}A_{11}^{-1}A_{12} )^{-1}A_{21}A_{11}^{-1}\\
&= A_{11}+A_{11}^{-1}A_{12}B^{-1}A_{21}A_{11}^{-1}
\end{align*}

\begin{align*}C_{12} &= -A_{11}^{-1}A_{12}C_{22}\\
&= -A_{12}^{-1}A_{12}B^{-1} \end{align*}

\subsection*{Problem 1.b}
For this part we substitute $k=1$
\begin{align*}
B_{1 \times 1} &= a_{22}-a_{12}'A_{11}^{-1}a_{12}\\
C_{11} &= A_{11}+A_{11}a_{12}B^{-1}a_{21}'A_{11}^{-1}\\
C_{12}&=-A_{11}^{-1}a_{12}B^{-1}\\
C_{21}&= -B^{-1}a_{12}'A_{11}^{-1}\\
C_{22}&=B^{-1}\\
\end{align*}

\section*{Problem 2}

\subsection*{Problem 2.a}

$X'X = \begin{pmatrix}2J & J  & J \\ J & J & 0 \\ J & 0 & J  \end{pmatrix} = J \begin{pmatrix} 2 & 1 &1\\ 1 & 1 & 0 \\ 1 & 0 & 1 \end{pmatrix}$

\subsection*{Problem 2.b}
$X'X$ is not invertible( Perform $C1-C3$)

$(X'X) \times (X'X)_1^{-} =J \begin{pmatrix} 2 & 1 &1\\ 1 & 1 & 0 \\ 1 & 0 & 1 \end{pmatrix} \times XX_1^- = \begin{pmatrix}1 & 0 & 0\\ 0 & 1 & 0\\ 1& -1 & 0 \end{pmatrix}$

Also, $(X'X) \times (X'X)_2^{-} =J \begin{pmatrix} 2 & 1 &1\\ 1 & 1 & 0 \\ 1 & 0 & 1 \end{pmatrix} \times XX_2^- = \begin{pmatrix} 0 & 0 & 1\\ 0 & 1 & 0\\ 0 & 0 & 1 \end{pmatrix}$
\subsection*{Problem 2.c}

$P = X(X'X)_1^-X' = \begin{pmatrix}2 & 1 & 1\\ 1 & 1 & 0\\ 1 & 0 & 1 \end{pmatrix}=X(X'X)_2^-X'= X'X$
and thus P is unique

\section*{Problem 3}

$$
X = \begin{pmatrix}
1 & -1 & 1\\
-1 & 0 & 2\\
1 & 1 & 1\\
\end{pmatrix}
$$Consider $X^T = \begin{pmatrix}
1 & -1 & 1\\
-1 & 0 & 1\\
1 & 2 & 1\\
\end{pmatrix}$

Finding the inverse gives $X^{-1} =   \begin{pmatrix}
1 & -1 & 1\\
-1 & 0 & 2\\
1 & 1 & 1\\
\end{pmatrix}$
In fact $X^TX = \begin{pmatrix}3 & 0 & 0\\
0 & 2 & 0\\
0 & 0 & 6\\
\end{pmatrix}$and $XX^T = \begin{pmatrix}3 & 1 & 1\\ 1 & 5 & 1\\ 1 & 1 &3
\end{pmatrix}$

Clearly $X^T \neq X^{-1}$ and this leads us to conclude that $X$ is \textbf{not} an orthonormal matrix.

$$
C = \begin{pmatrix} 1/\sqrt{3} & -1/\sqrt{2} & 1/\sqrt{6}\\
-1/\sqrt{3} & 0/\sqrt{2} & 2/\sqrt{6}\\
1/\sqrt{3} & 1/\sqrt{2} & 1/\sqrt{6}
\end{pmatrix}
$$
$$
C^T = \begin{pmatrix} 1/\sqrt{3} & -1/\sqrt{3} & 1/\sqrt{3}\\
-1/\sqrt{2} & 0/\sqrt{2} & 1/\sqrt{2}\\
1/\sqrt{6} & 2/\sqrt{6} & 1/\sqrt{6}
\end{pmatrix}
$$
Thus, $$CC^T = C^TC = I_3 \text{ (Skipped calculations, did in R)}$$

\section*{Problem 4}

\subsection*{Problem 4.a}

Using the property $\det(XY) = \det(X) \times \det(Y)$  we have $\det(A^2) =  \det(A)\times \det(A) = (\det(A))^2$

\subsection{Problem 4.b}

$A = P\Lambda P^T$ such that $\Lambda$ is a diagonal matrix with its diagonals as the eigen values and $P$ is an orthonormal matrix thus, $PP^T=I$ and $\det(P) = +1\ or\-1$

$\det(A) = \det(P\Lambda P^T)= \det(P) \det(\Lambda) \det(P^T) = \det(\Lambda) \det(P)^2 = \det(\Lambda) = \prod \lambda_i$
Using $tr(AB) = tr(BA)$
$tr(A) = tr(P\Lambda P^T) = tr(\lambda P^T P) = tr(\Lambda \times I) = tr(\Lambda) = \sum \lambda_i$

\end{document}


% $Id$
%
% Author: David Fournier
% Copyright (c) 2008 Regents of the University of California
%

\section{Truncated linear regression}

The linear regression model we consider here has the form
$$Y_i=\sum_{j=1}^m a_j x_{ij} +\epsilon_i$$
where the $Y_i$ for $i=1,\ldots,n$ are the $n$ observations
and the $a_j$ are $m$ parameters to be estimated.
The $\epsilon_i$ are assumed to be normally distributed random
variables with mean $0$ and variance $v$.

Let $r_i=Y_i-\sum_{j=1}^m a_j x_{ij}$. The log-likelihood function
for the standard regression model is given by
$$ -.5n\log(v) - \sum_{i=1}^n  \frac{r_i^2 }{ 2v}$$
Now assume that we only consider the $Y_i$ for $Y_i \ge 0$,
i.e., the left truncated situation.
The probability that $Y_i\ge 0$ is equal to the probability that
$\epsilon_I>-\sum_{j=1}^m a_j x_{ij} $. This is equal to
$1-\Phi(-\sum_{j=1}^m a_j x_{ij} /v)$, where
$$\Phi(u)=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^u \exp \left(-t^2/2 \right)\,\textrm{d}t$$
For this truncated regression, the log-likelihood function
has the logarithm of this quantity subtracted from it,
so it becomes
$$ -.5n\log(v) - \sum_{i=1}^n  \frac{r_i^2 }{ 2v}
   -\log\Bigg(1-\Phi \bigg(-\sum_{j=1}^m a_j x_{ij} /v \bigg)\Bigg)$$
If instead we consider the right truncated case, where only the $Y_i<0$
are considered, the log-likelihood function becomes
$$ -.5n\log(v) - \sum_{i=1}^n    \frac{r_i^2 }{ 2v}
   -\log\Bigg(\Phi \bigg(-\sum_{j=1}^m a_j x_{ij} /v \bigg)\Bigg)$$

To parameterize $v$, we introduce a new parameter $a$ satisfying
the condition $v=a\hat v$, where
$\hat v=\frac{1}{ n}\sum_{i=1}^n  r_i^2$ is the usual maximum likelihood
estimate for $v$. This leads to more
numerically stable behavior. In terms of $a$, the
expression for the log-likelihood simplifies to
$$ -.5n\log(a)-.5n\log(\hat v) - \frac{n}{ 2a}
   -\log\Bigg(1-\Phi \bigg(-\sum_{j=1}^m a_j x_{ij}/(a \hat v)\bigg)\Bigg)$$

\section{The \ADM\ truncated\br regression program}

Here are the contents of the file \texttt{truncreg.tpl}:
\begin{lstlisting}
DATA_SECTION
  init_int nobs
  init_int m
  init_int trunc_flag
  init_matrix data(1,nobs,1,m+1)
  vector Y(1,nobs)
  matrix X(1,nobs,1,m)
 LOC_CALCS
  Y=column(data,1);
  for (int i=1;i<=nobs;i++)
  {
    X(i)=data(i)(2,m+1).shift(1);
  }
PARAMETER_SECTION
  sdreport_number sigma
  number vhat
  init_bounded_number log_a(-5.0,5.0);
  sdreport_number a
  init_vector u(1,m)
  objective_function_value f
PROCEDURE_SECTION
  a=exp(log_a);
  dvar_vector pred=X*u;
  dvar_vector res=Y-pred;
  dvariable r2=norm2(res);
  vhat=r2/nobs;
  dvariable v=a*vhat;
  sigma=sqrt(v);

  dvar_vector spred=pred/sigma;
  f=0.0;
  switch (trunc_flag)
  {
  case -1:  // left_truncated
    {
      for (int i=1;i<=nobs;i++)
      {
        f+=log(1.00001-cumd_norm(-spred(i)));
      }
    }
    break;
  case 1:   // right truncated
    {
      for (int i=1;i<=nobs;i++)
      {
        f+=log(0.99999*cumd_norm(-spred(i)));
      }
    }
    break;
  case 0:   // no truncation
    break;
  default:
    cerr << "Illegal value for truncation flag" << endl;
    ad_exit(1);
  }
  f+=0.5*nobs*log(v)+0.5*r2/v;

REPORT_SECTION
  report << "#u " << endl << u << endl;
  report << "#sigma " << endl << sigma << endl;
  report << "#a " << endl << a << endl;
  report << "#vhat " << endl << vhat << endl;
  report << "#shat " << endl << sqrt(vhat) << endl;

\end{lstlisting}


\section{Matrix arithmetic modulo \texorpdfstring{$p$}{p}}

\begin{outcome}
  \begin{enumerate}
  \item Perform matrix operations over the field $\Z_p$.
  \end{enumerate}
\end{outcome}

In Section~\ref{sec:fields}, you learned that most of linear algebra
can be done over scalars from any field $K$, and not just the real
numbers. You also learned that $\Z_p$, the set of integers modulo $p$,
is a field whenever $p$ is a prime number.

Indeed, all of the operations on matrices that we covered in this
chapter make sense over any field: addition, scalar multiplication,
matrix multiplication, inverses, elementary matrices, and the
transpose.

\begin{example}{A matrix product over $\Z_5$}{matrix-product-z5}
  Compute the matrix product $AB$ over the field $\Z_5$, where
  \begin{equation*}
    A = \begin{mymatrix}{ccc}
      1 & 0 & 4 \\
      2 & 3 & 1 \\
    \end{mymatrix}
    \quad\mbox{and}\quad
    B = \begin{mymatrix}{cc}
      3 & 1 \\
      4 & 0 \\
      2 & 2 \\
    \end{mymatrix}.
  \end{equation*}
\end{example}

\begin{solution}
  For example, the $(1,1)$-entry of $AB$ is calculated by multiplying
  the first row of $A$ by the first column of $B$, i.e.,
  \begin{equation*}
    c_{11} ~=~ 1\cdot 3+0\cdot 4 + 4\cdot 2 ~=~ 3 + 0 + 3 ~=~ 1,
  \end{equation*}
  keeping in mind that all arithmetic operations are done in $\Z_5$.
  We repeat the same for the other entries and obtain
  \begin{equation*}
    AB ~=~
    \begin{mymatrix}{ccc}
      1 & 0 & 4 \\
      2 & 3 & 1 \\
    \end{mymatrix}
    \begin{mymatrix}{cc}
      3 & 1 \\
      4 & 0 \\
      2 & 2 \\
    \end{mymatrix}
    ~=~
    \begin{mymatrix}{cc}
      1 & 4 \\
      0 & 4 \\
    \end{mymatrix}.
  \end{equation*}
\end{solution}

\begin{example}{An inverse over $\Z_7$}{matrix-inverse-z7}
  Compute the inverse of the matrix
  \begin{equation*}
    A = \begin{mymatrix}{ccc}
      1 & 4 & 2 \\
      0 & 6 & 2 \\
      5 & 0 & 3 \\
    \end{mymatrix}
  \end{equation*}
  with scalars in the field $\Z_7$.
\end{example}

\begin{solution}
  We use exactly the method of
  Algorithm~\ref{algo:matrix-inversion-algorithm}, i.e., we set up the
  augmented matrix $\mat{A\mid I}$ and reduce it to {\rref}. The only
  thing we have to keep in mind is that all operations are done modulo
  $7$. Also, as usual, instead of dividing by a scalar, we must
  multiply by its inverse.
  \begin{eqnarray*}
    \mat{A\mid I}
    &=&
        \begin{mymatrix}{ccc|ccc}
          1 & 4 & 2  &  1 & 0 & 0 \\
          0 & 6 & 2  &  0 & 1 & 0 \\
          5 & 0 & 3  &  0 & 0 & 1 \\
        \end{mymatrix}
    \\
    &\stackrel{R_3\rowop R_3+2R_1}{\roweq}
      &
        \begin{mymatrix}{ccc|ccc}
          1 & 4 & 2  &  1 & 0 & 0 \\
          0 & 6 & 2  &  0 & 1 & 0 \\
          0 & 1 & 0  &  2 & 0 & 1 \\
        \end{mymatrix}
    \\
    &\stackrel{R_2\rowswap R_3}{\roweq}
      &
        \begin{mymatrix}{ccc|ccc}
          1 & 4 & 2  &  1 & 0 & 0 \\
          0 & 1 & 0  &  2 & 0 & 1 \\
          0 & 6 & 2  &  0 & 1 & 0 \\
        \end{mymatrix}
    \\
    &\stackrel{R_3\rowop R_3+R_2}{\roweq}
      &
        \begin{mymatrix}{ccc|ccc}
          1 & 4 & 2  &  1 & 0 & 0 \\
          0 & 1 & 0  &  2 & 0 & 1 \\
          0 & 0 & 2  &  2 & 1 & 1 \\
        \end{mymatrix}
    \\
    &\stackrel{R_3\rowop 2^{-1}R_3 ~=~ 4R_3}{\roweq}
      &
        \begin{mymatrix}{ccc|ccc}
          1 & 4 & 2  &  1 & 0 & 0 \\
          0 & 1 & 0  &  2 & 0 & 1 \\
          0 & 0 & 1  &  1 & 4 & 4 \\
        \end{mymatrix}
    \\
    &\stackrel{R_1\rowop R_1-4R_2}{\roweq}
      &
        \begin{mymatrix}{ccc|ccc}
          1 & 0 & 2  &  0 & 0 & 3 \\
          0 & 1 & 0  &  2 & 0 & 1 \\
          0 & 0 & 1  &  1 & 4 & 4 \\
        \end{mymatrix}
    \\
    &\stackrel{R_1\rowop R_1-2R_3}{\roweq}
      &
        \begin{mymatrix}{ccc|ccc}
          1 & 0 & 0  &  5 & 6 & 2 \\
          0 & 1 & 0  &  2 & 0 & 1 \\
          0 & 0 & 1  &  1 & 4 & 4 \\
        \end{mymatrix}.
  \end{eqnarray*}
  Therefore, the inverse is
  \begin{equation*}
    A^{-1} =
    \begin{mymatrix}{ccc}
      5 & 6 & 2 \\
      2 & 0 & 1 \\
      1 & 4 & 4 \\
    \end{mymatrix}.
  \end{equation*}
  As usual, we can double-check that we didn't make any mistakes by
  calculating
  \begin{equation*}
    AA^{-1} ~=~
    \begin{mymatrix}{ccc}
      1 & 4 & 2 \\
      0 & 6 & 2 \\
      5 & 0 & 3 \\
    \end{mymatrix}
    \begin{mymatrix}{ccc}
      5 & 6 & 2 \\
      2 & 0 & 1 \\
      1 & 4 & 4 \\
    \end{mymatrix}
    ~=~
    \begin{mymatrix}{ccc}
      1 & 0 & 0 \\
      0 & 1 & 0 \\
      0 & 0 & 1 \\
    \end{mymatrix}
    ~=~ I.
  \end{equation*}
  So indeed, we have calculated the inverse correctly.
\end{solution}