\documentclass{article}

\title{\sc\LARGE CSCA67 Tutorial, Week 2\\
{\Large Sept. 21st-25th, 2015}}
\date{}
\author{\sc Compiled by {\em G. Singh Cadieux}\\[1ex]
\sc Adapted from\\
A. Bretscher, \href{http://www.utsc.utoronto.ca/~bretscher/a67/lectures/w2.pdf}{\em CSCA67 Week 2 Lecture Notes},\\
\href{http://www.intmath.com/counting-probability/3-permutations.php}{\em Interactive Mathematics: Permutations (Ordered Arrangements)},\\
\href{http://www.intmath.com/counting-probability/4-combinations.php}{\em Interactive Mathematics: Combinations (Unordered Selections)},\\
Hunter, David J. \textit{Essentials of Discrete Mathematics.} Jones \& Bartlett, 2012. \&\\
Lov\'{a}sz, et al. \textit{Discrete Mathematics: Elementary and Beyond.} Springer, 2003.}

\usepackage{fullpage}
\usepackage{amsmath,amssymb}
%\usepackage{color}
%\usepackage{multicol}
\usepackage{tikz}
\usepackage{hyperref}

\setlength{\parindent}{0pt}

\begin{document}
\maketitle

\section{\sc Review of week 2's lecture}
\subsection{\em Sum rule vs. product rule}
\begin{tabular}{p{0.5\textwidth}|p{0.5\textwidth}}
{\bf Sum Rule}&{\bf Product Rule}\\[0.5ex]\hline\hline\noalign{\smallskip}
Used for counting the \textbf{total \# of possible outcomes} of an operation which can be performed $n$ ways, each with some number of outcomes.&
Used for counting the \textbf{total \# of possible ways} an operation can be performed when the operation takes $k$ steps, each of which can be performed in some number of ways.\\\hline
\begin{equation*}
\sum\limits_{i=1}^n x_i
\end{equation*}
where way $i$ has $x_i$ possible outcomes.&
\begin{equation*}
\prod\limits_{i=1}^k x_i
\end{equation*}
where step $k$ can be performed in $x_i$ ways.\\\hline\noalign{\smallskip}
\multicolumn{2}{c}{\em From Week 1's pizza example:}\\\hline\noalign{\smallskip}
Used for counting the number of combinations of up to 5 pizza toppings.\newline
Recall:\newline
\# of combinations of up to 5 toppings =
\begin{flushright}
\# of combinations of no toppings +\\
\# of combinations of 1 topping +\\
\ldots +\\
\# of combinations of 5 toppings
\end{flushright}
&
Used for counting the number of combinations of 5 pizza toppings (and 4, 3, etc.).\newline
Recall:\newline
\# of (non-unique) combinations of 5 toppings =
\begin{flushright}
\# of ways to choose first topping $\times$\\
\# of ways to choose second topping $\times$\\
\ldots $\times$\\
\# of ways to choose fifth topping
\end{flushright}
\end{tabular}

\subsection{\em Permutation vs. combination}
\begin{tabular}{c|c}
{\bf Permutation}&{\bf Combination}\\[0.5ex]\hline\hline\noalign{\smallskip}
\textit{Ordered} arrangement of a group of objects.& \textit{Unordered} arrangement of a group of objects.\\[0.5ex]\hline\noalign{\smallskip}
Total \# of permutations of $n$ objects: $n!$&Total \# of combinations of $n$ objects: 1\\[0.5ex]\hline\noalign{\smallskip}
{\bf $r$-permutation:}&{\bf $r$-combination:}\\
permutation of $r$ objects from a group of $n$ objects&combination of $r$ objects from a group of $n$ objects\\[1ex]
Total \# of $r$-permutations: $\dfrac{n!}{(n-r)!}$&Total \# of $r$-combinations: $\dfrac{n!}{(n-r)!\,r!}$\\[0.5ex]\noalign{\smallskip}\hline\noalign{\smallskip}
Represented $P(n,r)$&Represented $C(n,r)$ or $\binom{n}{r}$
\end{tabular}\\[2ex]
\textsc{Consider first} the number of permutations of $n$ objects.\\[1ex]
This is equivalent to considering the number of ways we can arrange $n$ objects. We can see that this will require $n$ steps: the first step is selecting the first object in the arrangement; the second step is selecting the second object, and so on.\\[1ex]
Using the product rule, we determine that
\begin{multline*}
\text{\bf\# of ways of permuting }n\text{ \bf objects}=\text{\# of ways of selecting the first object }\times\\
\text{\# of ways of selecting the second object }\times\ldots\times\text{\# of ways of selecting the }n\text{th object}
\end{multline*}
When we select the first object, there are $n$ objects from which to choose, meaning that there are $n$ ways to select an object. When we select the second object, we select from the reduced group of $(n-1)$ objects, since we have already selected one; this means that there are $(n-1)$ ways to choose. Eventually, when we select the final, $n$th object, we select from a group of only 1 object, which can only be done in 1 way.\\[1ex]
Overall, we can see that
\begin{equation*}
\text{\# of ways of permuting }n\text{ objects}=n\times(n-1)\times\ldots\times 1=n!
\end{equation*}
\textsc{Using this result,} we can determine the number of $r$-permutations of $n$ objects.\\[1ex]
If we select a subset of $r$ objects from an overall group of $n$ objects, we ignore the remaining $(n-r)$ objects.\\[1em]
Consider the 2-permutations of \{A, B, C, D, E\}.\\[1ex]
If we select \{A, B\}, we have 2 possible 2-permutations: AB and BA. We can permute all 5 objects to obtain (among others) ABCDE, ABCED, and ABDEC. However, because we have selected only \{A, B\} to permute, and ABCDE, ABCED, and ABDEC contain the same permutation AB of \{A, B\}, these 3 are equivalent here.\\[1ex]
To determine the number of 2-permutations, we can diminish the number of 5-permutations (permutations of all 5 objects) by the number of 5-permutations we consider to be equivalent.\\[1ex]
Having selected 2 ordered objects, the remaining 3 objects can be permuted $3!$ times: by the same reasoning as above, we can select the third object from 3 in 3 ways; then we can select the fourth object from 2 in 2 ways; and finally, we can select the fifth object from the remaining 1 object in only 1 way.\\
Since we are interested only in the order of the first 2 objects we selected, there are $3!$ ways that the remaining 3 objects can be selected to produce permutations we consider equivalent. So of the $5!$ total 5-permutations, we consider only 1 out of $3!$ to be a unique 2-permutation.\\
And thus, there are $\dfrac{5!}{3!}=20$ 2-permutations of \{A, B, C, D, E\}.\\[1em]
We can generalize this to say that
\begin{equation*}
\text{total \# of }r\text{-perms.}=\dfrac{\text{total \# of perms. of }n\text{ objects}}{\text{total \# of perms. of the remaining }(n-r)\text{ objects}}=\dfrac{n!}{(n-r)!}
\end{equation*}\\[1em]
\textsc{We can use this result} to determine the number of $r$-combinations of $n$ objects.\\[1ex]
Unlike an $r$-permutation, an $r$-combination is unordered. From all possible $r$-permutations of $n$ objects, we have multiple equivalent $r$-combinations.\\[1em]
Consider again the 2-permutations of \{A, B, C, D, E\}.\\[1ex]
If we select \{A, B\}, we have 2 possible 2-permutations: AB and BA. However, these are equivalent $r$-combinations, since both are composed of the same objects. For every possible 2-permutation \{object1, object2\}, there is another 2-permutation \{object2, object1\} that contains the same objects in the reverse order.\\[1ex]
To determine the number of 2-combinations, we can diminish the total number of 2-permutations by the number of 2-permutations which contain the same group of 2 objects.\\
Here, there are $\dfrac{5!}{3!}\dfrac{1}{2}=\dfrac{20}{2}=10$ 2-combinations of \{A, B, C, D, E\}.\\[1em]
We can generalize this to say that, in a group of $n$ objects, the number of $r$-permutations containing the same group of $r$ objects is $r!$, the number of ways that $r$ objects can be ordered. So of the $\dfrac{n!}{(n-r)!}$ total $r$-permutations, only 1 out of $r!$ is a unique $r$-combination.
\begin{equation*}
\text{total \# of }r\text{-combs.}=\dfrac{\text{total \# of }r\text{-perms. of }n\text{ objects}}{\text{total \# of perms. of the selected }r\text{ objects}}=\dfrac{n!}{(n-r)!\,r!}
\end{equation*}

\subsection{\em Permutation with repetition}
Consider the group of letters that form the word ``banana": \{b, a, n, a, n, a\}.\\[1ex]
\textsc{Q: How many ways can we ``arrange" (here, permute) these letters?}\\[1em]
If all 6 letters were distinct, we know that there would be $6!$ arrangements. We come to this total by multiplying together the 6 ways to choose the first letter from 6 possibilities, and the 5 ways to choose the second letter from 5 possibilities, and so on.\\[1ex]
But because the letters here are not unique, there are fewer than 6 different ways to choose the first letter, and fewer than 5 different ways to choose the second, and so on.\\[1em]
\textsc{Suppose} that we convert this to a problem in which all the letters are distinct: \{b, a$_1$, n$_1$, a$_2$, n$_2$, a$_3$\}.\\[1ex]
Then the arrangements ``ba$_1$a$_2$a$_3$n$_1$n$_2$", ``ba$_1$a$_2$a$_3$n$_2$n$_1$", and ``ba$_1$a$_3$a$_2$n$_1$n$_2$" are distinct. But without the distinguishing subscripts - that is, where all `a's are considered equivalent and all `n's are considered equivalent - these arrangements are not distinct.\\[1ex]
We can re-order equivalent letters to form equivalent arrangements.\\
For example, we can form the same arrangement ``baanna" by ordering the `a's and `n's as ``ba$_1$a$_2$n$_1$n$_2$a$_3$", ``ba$_1$a$_2$n$_2$n$_1$a$_3$", ``ba$_1$a$_3$n$_1$n$_2$a$_2$", and ``ba$_2$a$_3$n$_1$n$_2$a$_1$", among other ways. What differs between these is the order of the equivalent letters, but not their overall position in the arrangement.\\[1ex]
Thus, the total number of equivalent arrangements is the total number of ways we can order the equivalent letters: we can order 3 `a's in $3!$ ways, and 2 `n's in $2!$ ways. Then, by the product rule, we can order both `a's and `n's in $3!\times 2!$ ways.
\begin{equation*}
\text{total \# of distinct arrangements}=\dfrac{\text{total \# of arrangements}}{\text{\# of equivalent arrangements}}=\dfrac{6!}{3!\,2!}=60
\end{equation*}

\textsc{Alternatively,} rather than constructing an arrangement by selecting the first letter of the arrangement, followed by the second letter, and so on, let us select one letter out of the group and assign it a place within the arrangement. Then, we select another letter from the group and assign it one of the remaining places in the arrangement. We continue this until all the letters from the group have been placed into the arrangement. For example,
\begin{center}
\begin{tikzpicture}
\draw (0,0) -- (0.5,0) node[below,align=center] {letter$_1$} -- (1,0);
\draw (1.25,0) -- (1.75,0) node[below,align=center] {letter$_2$} -- (2.25,0);
\draw (2.5,0) -- (3,0) node[below,align=center] {letter$_3$} -- (3.5,0);
\draw (3.75,0) -- (4.25,0) node[below,align=center] {letter$_4$} -- (4.75,0);
\draw (5,0) -- (5.5,0) node[below,align=center] {letter$_5$} -- (6,0);
\draw (6.25,0) -- (6.75,0) node[below,align=center] {letter$_6$} -- (7.25,0);
\node[above,align=center] at (1.75,0) {a};
\node[above,align=center] at (3,0) {b};
\node[above,align=center] at (5.5,0) {a};
\node[font=\large] at (8.5,0.5) {a};
\node[font=\large] at (9.5,0.4) {n};
\node[font=\large] at (8.75,0) {n};
\draw (8.25,-0.25) rectangle (9.75,0.75);
\end{tikzpicture}
\end{center}
Then, the total number of possible arrangements is the product of the number of places possible for the first letter selected, and the number of places possible for the second letter selected, and so on.\\[1ex]
If all 6 letters are distinct, then this gives us 6 possible places for the first letter, 5 possible places for the second, and so on, which is $6!$ in total, as expected.\\[1ex]
However, because the letters here are not all distinct, if we assign one letter to letter$_1$ and another instance of that letter to letter$_3$, this is equivalent to having first assigned that letter to letter$_3$ and then, another instance of it to letter$_1$. Yet we counted fewer possible places for the first assignment than the second assignment.\\[1ex]
To deal with these equivalences, we assign places to all identical letters simultaneously; that is, we assign places to all 3 `a's simultaneously, and to both `n's simultaneously.\\[1ex]
For example, this means that the number of possible places for `n's is $\dbinom{\text{\# of unassigned places}}{2}$.\\
Note that this is no different if the letters are all distinct: $\binom{6}{1}\binom{5}{1}\binom{4}{1}\binom{3}{1}\binom{2}{1}\binom{1}{1}=6\times 5\times 4\times 3\times 2\times 1=6!$.\\[1ex]
Thus, by this reasoning,
\begin{align*}
\text{total \# of distinct arrangements}& =\dbinom{\text{\# of unass. places}}{\text{\# of `a's}}\dbinom{\text{\# of unass. places}}{{\text{\# of `n's}}}\dbinom{\text{\# of unass. places}}{{\text{\# of `b's}}}\\
& =\dbinom{6}{3}\dbinom{6-3}{2}\dbinom{6-3-2}{1}\\
& =\dfrac{6!}{(6-3)!\,3!}\dfrac{3!}{(3-2)!\,2!}\dfrac{1!}{(1-1)!\,1!}\\
& =\dfrac{6!}{3!\,2!\,1!}=60
\end{align*}

\textsc{In general,} there are
\begin{equation*}
\dbinom{n}{r_1}\dbinom{n-r_1}{r_2}\ldots\dbinom{n-r_1-r_2-\ldots-r_{m-1}}{r_m}=\dfrac{n!}{r_1!\,r_2!\,\ldots\,r_m!}
\end{equation*}
distinct arrangements/permutations of $n$ objects divided into $m$ types, where $r_i$ is the number of objects of type $i=1,2,\ldots,m$.\\[1ex]
Note that the $n$ objects are \textit{partitioned} into $m$ types; that is, $r_1+r_2+\ldots+r_m=n$.

\subsection{\em Combination/selection with repetition}

Consider a situation in which a supermarket has 3 kinds of bagels: poppy seed, sesame seed, and plain.\\[1ex]
\textsc{Q: If you want to buy 6 bagels, how many combinations of different kinds can you get?}\\[1ex]
If we were selecting 6 bagels from a group of $n$ unique bagels, we know that there would be $\dbinom{n}{6}=\dfrac{n!}{(n-6)!\,6!}$ possible combinations.\\[1ex]
But because we are selecting 6 bagels \textit{with repetition} (we have only 3 types from which to choose, so we must have more than 1 of at least 1 type of bagel), some of these $\binom{n}{6}$ combinations are equivalent.\\[1ex]
For example, suppose that we convert this to a problem in which all the bagels are unique. Then the combinations
\begin{center}
\begin{tikzpicture}
\draw (0,0) rectangle (1,1);
\draw (1.25,0) rectangle (2.25,1);
\draw (2.5,0) rectangle (3.5,1);
\draw (3.75,0) rectangle (4.75,1);
\draw (5,0) rectangle (6,1);
\draw (6.25,0) rectangle (7.25,1);

\filldraw[red] (0.5,0.5) circle (0.35);
\filldraw[white] (0.5,0.5) circle (0.15);

\filldraw[red] (1.75,0.5) circle (0.35);
\filldraw[white] (1.75,0.5) circle (0.15);

\filldraw[red] (3,0.5) circle (0.35);
\filldraw[white] (3,0.5) circle (0.15);

\filldraw[cyan] (4.25,0.5) circle (0.35);
\filldraw[white] (4.25,0.5) circle (0.15);

\filldraw[green] (5.5,0.5) circle (0.35);
\filldraw[white] (5.5,0.5) circle (0.15);

\filldraw[green] (6.75,0.5) circle (0.35);
\filldraw[white] (6.75,0.5) circle (0.15);

\node [font=\small] at (0.5,0.5) {1};
\node [font=\small] at (1.75,0.5) {2};
\node [font=\small] at (3,0.5) {3};

\node [font=\small] at (5.5,0.5) {1};
\node [font=\small] at (6.75,0.5) {2};
\end{tikzpicture}\\
and\\[1ex]
\begin{tikzpicture}
\draw (0,0) rectangle (1,1);
\draw (1.25,0) rectangle (2.25,1);
\draw (2.5,0) rectangle (3.5,1);
\draw (3.75,0) rectangle (4.75,1);
\draw (5,0) rectangle (6,1);
\draw (6.25,0) rectangle (7.25,1);

\filldraw[red] (0.5,0.5) circle (0.35);
\filldraw[white] (0.5,0.5) circle (0.15);

\filldraw[red] (1.75,0.5) circle (0.35);
\filldraw[white] (1.75,0.5) circle (0.15);

\filldraw[red] (3,0.5) circle (0.35);
\filldraw[white] (3,0.5) circle (0.15);

\filldraw[cyan] (4.25,0.5) circle (0.35);
\filldraw[white] (4.25,0.5) circle (0.15);

\filldraw[green] (5.5,0.5) circle (0.35);
\filldraw[white] (5.5,0.5) circle (0.15);

\filldraw[green] (6.75,0.5) circle (0.35);
\filldraw[white] (6.75,0.5) circle (0.15);

\node [font=\small] at (0.5,0.5) {2};
\node [font=\small] at (1.75,0.5) {3};
\node [font=\small] at (3,0.5) {5};

\node [font=\small] at (5.5,0.5) {3};
\node [font=\small] at (6.75,0.5) {4};
\end{tikzpicture}
\end{center}
are distinct. But without the distinguishing identifiers - that is, where all bagels of the same type are considered equivalent - these combinations are not distinct.\\[1ex]
\textsc{Since all} bagels of the same type are equivalent, rather than considering how to select 6 bagels one by one for the arrangement, let us consider how to divide our arrangement into 3 groups of types of bagels.\\
For example, we can divide our 6 possibilities into a group of 1 poppy seed bagel, a group of 4 sesame seed bagels, and a group of 1 plain bagel:
\begin{center}
\begin{tikzpicture}
\draw (0,0) rectangle (1,1);
\draw (1.25,0) rectangle (2.25,1);
\draw (2.5,0) rectangle (3.5,1);
\draw (3.75,0) rectangle (4.75,1);
\draw (5,0) rectangle (6,1);
\draw (6.25,0) rectangle (7.25,1);
\draw (7.5,0) rectangle (8.5,1);
\draw (8.75,0) rectangle (9.75,1);

\filldraw[red] (0.5,0.5) circle (0.35);
\filldraw[white] (0.5,0.5) circle (0.15);

\filldraw[cyan] (3,0.5) circle (0.35);
\filldraw[white] (3,0.5) circle (0.15);

\filldraw[cyan] (4.25,0.5) circle (0.35);
\filldraw[white] (4.25,0.5) circle (0.15);

\filldraw[cyan] (5.5,0.5) circle (0.35);
\filldraw[white] (5.5,0.5) circle (0.15);

\filldraw[cyan] (6.75,0.5) circle (0.35);
\filldraw[white] (6.75,0.5) circle (0.15);

\filldraw[green] (9.25,0.5) circle (0.35);
\filldraw[white] (9.25,0.5) circle (0.15);

\draw (1.25,0) -- (2.25,1);
\draw (7.5,0) -- (8.5,1);
\end{tikzpicture}
\end{center}
To mark the separation between groups, we use an empty pigeonhole. To divide our bagels into 3 groups, we require 2 pigeonholes to act as separators, and 6 filled pigeonholes + 2 empty pigeonholes = 8 piegeonholes in total.\\[1ex]
\textsc{In general,} to partition any number of objects into $n$ groups, we require $n-1$ empty pigeonholes.\\[1ex]
We can divide our 6 possibilities into 3 groups by selecting which pigeonholes will be filled and which will be empty. For example, if we select all but these pigeonholes to be filled
\begin{center}
\begin{tikzpicture}
\draw (0,0) rectangle (1,1);
\draw (1.25,0) rectangle (2.25,1);
\draw (2.5,0) rectangle (3.5,1);
\draw (3.75,0) rectangle (4.75,1);
\draw (5,0) rectangle (6,1);
\draw (6.25,0) rectangle (7.25,1);
\draw (7.5,0) rectangle (8.5,1);
\draw (8.75,0) rectangle (9.75,1);

\draw (2.5,0) -- (3.5,1);
\draw (3.75,0) -- (4.75,1);
\end{tikzpicture}
\end{center}
then our groups are 2 poppy seed bagels, 0 sesame seed bagels, and 4 plain bagels:
\begin{center}
\begin{tikzpicture}
\draw (0,0) rectangle (1,1);
\draw (1.25,0) rectangle (2.25,1);
\draw (2.5,0) rectangle (3.5,1);
\draw (3.75,0) rectangle (4.75,1);
\draw (5,0) rectangle (6,1);
\draw (6.25,0) rectangle (7.25,1);
\draw (7.5,0) rectangle (8.5,1);
\draw (8.75,0) rectangle (9.75,1);

\filldraw[red] (0.5,0.5) circle (0.35);
\filldraw[white] (0.5,0.5) circle (0.15);

\filldraw[red] (1.75,0.5) circle (0.35);
\filldraw[white] (1.75,0.5) circle (0.15);

\filldraw[green] (5.5,0.5) circle (0.35);
\filldraw[white] (5.5,0.5) circle (0.15);

\filldraw[green] (6.75,0.5) circle (0.35);
\filldraw[white] (6.75,0.5) circle (0.15);

\filldraw[green] (8,0.5) circle (0.35);
\filldraw[white] (8,0.5) circle (0.15);

\filldraw[green] (9.25,0.5) circle (0.35);
\filldraw[white] (9.25,0.5) circle (0.15);

\draw (2.5,0) -- (3.5,1);
\draw (3.75,0) -- (4.75,1);
\end{tikzpicture}
\end{center}
\textsc{The question} then becomes: in how many ways can we choose our 2 empty pigeonholes, or our 6 filled pigeonholes?\\[1ex]
Since these pigeonholes are unordered, the answer is simply $\dbinom{8}{6}=28$.\\[1ex]
\textsc{In general,} there are
\begin{equation*}
\dbinom{\text{total \# of pigeonholes}}{\text{\# of objects being selected}}=\dbinom{r+(n-1)}{r}
\end{equation*}
distinct combinations of $r$ objects from a group of $n$ types of objects.

\section{\sc Counting problems: Permutations and combinations}

\subsection*{Q: {\em How many ways can the letters A, B, and C be arranged?}}
We are being asked how many different ways 3 distinct objects, the letters A, B, and C, can be permuted.\\[1ex]
There are $n!\Rightarrow 3!=6$ permutations of 3 objects.

\subsection*{Q: {\em How many ways can 4 different resistors be arranged in series?}}
Again, we are being asked how many different ways 4 distinct objects - 4 resistors - can be permuted.\\[1ex]
There are $n!\Rightarrow 4!=24$ permutations of 4 objects.

\subsection*{Q: {\em How many ways can a supermarket manager display 5 brands of cereal in 3 spaces on a shelf?}}
We are again being asked about an arrangement, or permutation, of multiple objects. However, here we must determine the number of $r$-permutations: permutations of 3 brands out of the 5 possible brands.\\[1ex]
There are $\dfrac{n!}{(n-r)!}\Rightarrow\dfrac{5!}{(5-3)!}=60$ permutations of 3 objects from 5.

\subsection*{Q: {\em How many different license plates for cars can be made if each contains 4 of the digits 0-9, followed by a letter A-Z, assuming}}
\subsubsection*{a) {\em no repetition of digits?}}
\begin{tikzpicture}
\draw[thick,rounded corners](0,0)rectangle(5.25,1.45);
\draw[thick,rounded corners](-.1,-.1)rectangle(5.35,1.55);
\node[color=blue] at (2.62,1.15) {\sc Ontario};
\draw(0.25,0.2) -- (0.625,0.2) node [align=center,above,font={\Large\bf},color=blue] {D$_1$} -- (1,0.2);
\draw(1.25,0.2) -- (1.625,0.2) node [align=center,above,font={\Large\bf},color=blue] {D$_2$} --(2,0.2);
\draw(2.25,0.2) -- (2.625,0.2) node [align=center,above,font={\Large\bf},color=blue] {D$_3$} --(3,0.2);
\draw(3.25,0.2) -- (3.625,0.2) node [align=center,above,font={\Large\bf},color=blue] {D$_4$} --(4,0.2);
\draw(4.25,0.2) -- (4.625,0.2) node [align=center,above,font={\Large\bf},color=blue] {L} --(5,0.2);
\end{tikzpicture}\\[1ex]
Let us consider all 4 digits as a single entity \textbf{D}. Then, using the multiplication principle,
\begin{align*}
\text{\# of combinations of \textbf{D} and \textbf{L}}& =\text{\# of ways of choosing \textbf{D}}\times\text{\# of ways of choosing \textbf{L}}\\
& =\text{\# of perms. of 4 digits}\times\text{\# of letters}
\end{align*}
The number of permutations of 4 digits, with no repetition of digits, is the number of 4-permutations of 10 objects, since there are 10 digits from which we select. So
\begin{align*}
\text{\# of combinations of \textbf{D} and \textbf{L}}& =\left(\dfrac{10!}{(10-4)!}\right)\times 26\\
& =5040\times 26\\
& =131\,040
\end{align*}

\subsubsection*{b) {\em possible repetition of digits?}}
If repetition of digits (using the same digit more than once) is possible, then we are no longer dealing with permutations. Instead, using the product rule, we determine that the number of ways of choosing \textbf{D} is the product of the number of ways of choosing the first digit D$_1$, and then the second digit D$_2$, and so on.\\[1ex]
Since repetition is possible, at each step, we are choosing from the same set of 10 digits, meaning that there are 10 ways to choose each digit.
\begin{align*}
\text{\# of combinations of \textbf{D} and \textbf{L}}& =\text{\# of ways of choosing \textbf{D}}\times\text{\# of ways of choosing \textbf{L}}\\
& =\left(\text{\# of ways of choosing D}_1\times\ldots\times\text{\# of ways of choosing D}_4\right)\times\text{\# of letters}\\
& =\left(10\times 10\times 10\times 10\right)\times 26\\
& =260\,000
\end{align*}

%\subsection*{Q: {\em How many different sets of 4 letters can be selected from the alphabet?}\\
%{\normalsize Eg., \{A, E, R, T\} and \{E, A, T, R\} are the same set, but are different from \{Q, D, P, T\}.}}
%We are being asked how many different ways 4 letters from the alphabet can be combined. We know that these are combinations rather than permutations because sets are, by definition, unordered.\\[1ex]
%There are $\dfrac{n!}{(n-r)!\,r!}\Rightarrow \dfrac{26!}{(26-4)!\,4!}=14\,950$ combinations of 4 objects from 26.
%
%\subsection*{Q: {\em In how many ways can 3 components be selected from a batch of 20 different components?}}
%Again, we are being asked how many different ways 3 objects from a group of 20 objects can be combined.\\[1ex]
%There are $\dfrac{n!}{(n-r)!\,r!}\Rightarrow \dfrac{20!}{(20-3)!\,3!}=1140$ combinations of 3 objects from 20.

\subsection*{Q: {\em In how many ways can a group of 4 boys be selected from 10 if}}
\subsubsection*{a) {\em the eldest boy is included in each group?}}
If the eldest boy is included in each group, we select only the remaining 3 boys in the group; and we select these other 3 group members from 9 rather than from 10, since we have already selected 1 boy.\\[1ex]
So we are asked how many different ways 3 objects can be combined from a group of 9 objects.\\[1ex]
There are $\dfrac{n!}{(n-r)!\,r!}\Rightarrow \dfrac{9!}{(9-3)!\,3!}=84$ possible groups that include the eldest boy.

\subsubsection*{b) {\em the eldest boy is excluded from all groups?}}
If the eldest boy is excluded from all groups, we select 4 boys to form a group, but we select from 9 rather than from 10, since we have eliminated 1 possible boy.\\[1ex]
There are $\dfrac{n!}{(n-r)!\,r!}\Rightarrow \dfrac{9!}{(9-4)!\,4!}=126$ possible groups that exclude the eldest boy.

\subsubsection*{c) {\em What proportion of all possible groups contain the eldest boy?}}
Since there are only two possibilities - either a group includes the eldest boy or it does not - and both possibilities cannot be true simultaneously - a group cannot simultaneously include \textit{and} exclude the eldest boy - the total number of possible groups must be the sum of the number of groups that include the eldest boy and the number of groups that do not.
\begin{align*}
\text{total \# of possible groups}& =\text{\# of groups incl. the eldest boy}+\text{\# of groups excl. the eldest boy}\\
& =84+126\\
& =210
\end{align*}
Alternatively, we can calculate the total number of possible groups as the total number of combinations of 4 boys from 10, with no restrictions:
\begin{equation*}
\text{total \# of possible groups}=\dfrac{10!}{(10-4)!\,4!}=210
\end{equation*}
The proportion of groups containing the eldest boy is then
\begin{equation*}
\dfrac{\text{\# of groups incl. the eldest boy}}{\text{total \# of possible groups}}=\dfrac{84}{210}=40\%
\end{equation*}

\section{\sc Counting problems with repetition}

\subsection*{Q: {\em How many different ways can the letters of the word ``mammal" be arranged?}}
This question is directly analogous to the ``banana" example above: we want to determine the number of arrangements of a group of objects, where some objects are of the same type.\\[1ex]
Specifically, we have $n=6$ letters, $r_1=3$ of which are `m's, $r_2=2$ of which are `a's, and $r_3=1$ of which are `l's.
\begin{equation*}
\dfrac{n!}{r_1!\,r_2!\,r_3!}\Rightarrow\dfrac{6!}{3!\,2!\,1!}=60\text{ ways to arrange ``mammal"}
\end{equation*}

\subsection*{Q: {\em How many different ways can 3 red, 4 yellow, and 2 blue bulbs be arranged in a string of Christmas lights with 9 sockets?}}
Again, we are asked to determine the number of arrangements of a group of objects with repetition.\\[1ex]
We have $n=9$ bulbs, $r_1=3$ of which are red, $r_2=4$ of which are yellow, and $r_3=2$ of which are blue.
\begin{equation*}
\dfrac{n!}{r_1!\,r_2!\,r_3!}\Rightarrow\dfrac{9!}{3!\,4!\,2!}=1260\text{ ways to arrange bulbs in 9 sockets}
\end{equation*}
\textsc{Note} that we are able to use the formula for calculating the number of arrangements with repetition, derived above, because we are asked to arrange 9 objects (in 9 sockets) and have $3+4+2=9$ objects in total.\\
That is,
\begin{equation*}
r_1+r_2+\ldots+r_m=3+4+2=9=n
\end{equation*}

%\subsection*{{\normalsize You want to send postcards to 12 friends. In the shop, there are only 3 kinds of postcards.}\\
%Q: {\em How many ways can you send the postcards if}}
%\subsubsection*{a) {\em there is a large number of each kind, and you want to send 1 card to each friend?}}
%If we are sending 1 postcard to 12 friends, we must select 12 postcards in total. However, because there are only $n=3$ kinds of postcards, some friends will receive the same kind of card.\\
%So we are being asked how many ways we can select $r=12$ postcards, with repetition. We can determine this using the formula for selection with repetition, derived above:
%\begin{equation*}
%\binom{r+(n-1)}{r}\Rightarrow\binom{12+(3-1)}{12}=\dfrac{14!}{12!\,(14-12)!}=91\text{ ways to select 12 postcards}
%\end{equation*}
%
%\subsubsection*{b) {\em there is a large number of each kind, and you are willing to send 1 or more cards to each friend, but no one should get more than 1 of the same kind of card?}}
%We are again being asked how many ways we can select \textit{at least} 12 postcards, with repetition. We cannot send more than 3 to any friend because there are only 3 types, and so, we would be sending at least more than 1 of the same type.\\[1ex]
%The number of cards we want to select can be as few as 12 (if we only send 1 to each friend) or as many as 36 (if we send 3 to each friend). This makes it difficult to use the same method as in \textbf{(a)}. Instead, we can use the product rule:
%\begin{equation*}
%\text{\# of ways to send 1-3 cards 12 times}=\underbrace{\text{\# of ways to send 1-3 cards}}_\text{friend1}\times\ldots\times\underbrace{\text{\# of ways to send 1-3 cards}}_\text{friend12}
%\end{equation*}
%Then, using the sum rule, we determine that, for each friend,
%\begin{multline*}
%\text{\# of ways to send 1-3 cards}=\\
%\text{\# of ways to send 1 card }+\text{ \# of ways to send 2 cards }+\text{ \# of ways to send 3 cards}
%\end{multline*}
%Since we cannot have repetition when selecting 2 or 3 postcards, and there are only 3 unique postcards from which to choose, we find the number of ways to choose $r=1,2,3$ postcards from $n=3$:
%\begin{align*}
%\text{\# of ways to send 1-3 postcards}& =\binom{3}{1}+\binom{3}{2}+\binom{3}{3}\\
%& =3+3+1\\
%& =7
%\end{align*}
%And finally, we determine that
%\begin{equation*}
%\text{\bf\# of ways to send 1-3 postcards 12 times}=7^{12}=13\,841\,287\,201
%\end{equation*}
%
%\subsubsection*{c) {\em there are only 4 of each kind, and you want to send 1 card to each friend?}}
%There are 4 of 3 kinds of postcards, meaning that there are $n=12$ cards in total from which to choose.\\[1ex]
%If we suppose the order in which we select the postcards to be meaningful - that is, we send the first card we select to friend1, the second to friend2, etc. - then we are being asked to determine the number of ways in which we can select 12 cards, where some cards are of the same type.
%\begin{equation*}
%\dfrac{n!}{r_1!\,r_2!\,r_3!}\Rightarrow\dfrac{12!}{4!\,4!\,4!}=34\,650\text{ ways to send 12 postcards}
%\end{equation*}

\subsection*{Q: {\em How many solutions are there to the equation $x_1+x_2+x_3+x_4+x_5=13$ if $x_1,\ldots,x_5$ must be non-negative integers?}}
We can consider this question to be asking how many ways we can distribute 13 units among the five variables $x_1,\ldots,x_5$. (We can make the question more intuitive by imagining that, as in the example above, we have five types of bagels and want to select 13 bagels in total.)\\[1ex]
For example, the solution $x_1=4,\,x_2=0,\,x_3=5,\,x_4=1,\,x_5=3$ distributes 13 1's into the five groups:
\begin{equation*}
\underbrace{\;1\;1\;1\;1\;}_{x_1}|\underbrace{\;}_{x_2}|\underbrace{\;1\;1\;1\;1\;1\;}_{x_3}|\underbrace{\;1\;}_{x_4}|\underbrace{\;1\;1\;1\;}_{x_5}
\end{equation*}
Note that a unique solution is a set of assignments to $x_1,\ldots,x_5$: thus, $x_1=4,\,x_2=0,\,x_3=5,\,x_4=1,\,x_5=3$ is a different solution than $x_1=5,\,x_2=1,\,x_3=3,\,x_4=4,\,x_5=0$.
\begin{equation*}
\binom{r+(n-1)}{r}\Rightarrow\binom{13+(5-1)}{13}=\dfrac{17!}{13!\,(17-13)!}=2380\text{ unique solutions}
\end{equation*}

\section{\sc Additional practice problems}

At a competition of 100 athletes, only the order of the first 10 is recorded.\\
{\bf Q: How many different outcomes does the competition have?}\\[1ex]
Suppose we record the order of all 100 athletes.\\
{\bf Q: How many different outcomes can we have then?}\hfill(1)\\[1ex]
{\bf Q: How many of the outcomes in {\normalfont(1)} give the same result for the first 10 places?}\hfill(2)\\[1ex]
{\bf Q: Show that the number of outcomes for the first 10 places can also be obtained using {\normalfont(1)} and {\normalfont(2)}.}\\[1em]
Possible grades for a class are A, B, C, D, and F. (No +/-'s).\\
{\bf Q: How many ways are there to assign grades to a class of seven students?}\\[1ex]
{\bf Q: How many ways are there to assign grades to a class of seven students, if nobody receives an F and exactly one person receives an A?}\\[1em]
{\bf Q: How many ways are there to arrange the letters in the word ``inaneness"?}\\[1em]
Two teams, A and B, play a best-of-seven match. The match ends when one team wins four games.\\
{\bf Q: How many different win or loss scenarios are possible?}

\end{document}


\chapter{Basic Statistics in Spreadsheets}

When you completed the problems in the last section, you probably noticed
how long it took to compute statistics like the mean, the median,
and variance by hand. Luckliy, computers were designed to free us from these
sorts of tedious tasks. The most basic tool for automating
calculations is the spreadsheet program.\index{spreadsheet}

There are lots of spreadsheet programs including Microsoft's Excel and
Apple's Numbers. Any spreadsheet program will work; they are all very
similar. The instructions and screenshots here will be from Google
Sheets -- a free spreadsheet program you use through your web browser.

\section{Your First Spreadsheet}

In whatever spreadsheet program you are using, create a new spreadsheet document.

A spreadsheet is essentially a grid of cells. In each cell you can put data (like numbers or text) and a formulas.

\includegraphics[width=0.6\textwidth]{BlankSheet.png}

Let's put some labels in the column:
\begin{itemize}
\item Select the first cell (A1) and type ``A number''.
\item Select the cell below it (A2) and type ``Another number''.
\item Select the cell below that one (A3) and type ``Their product''.
\item In the next column, type the number 5 in B1 and 7 in B2.
\end{itemize}

It should look like this:

\includegraphics[width=0.5\textwidth]{NoFormulas.png}

Now put a formula in cell B3. Select B3, and type ``= B1 + B2''. The spreadsheet knows this is a formula because it starts with `=`. It will look like this as you type:\index{Spreadsheet!Entering formula}

\includegraphics[width=0.5\textwidth]{TypingFirstFormula.png}

When you press Return or Tab, the spreadsheet will remember the formula, but display its value:

\includegraphics[width=0.5\textwidth]{FirstCalc.png}

If you change the values of cell B1 or B2, the cell B3 will automatically be recalculated. Try it.

\section{Formatting}

Every spreadsheet lets you change the formatting of your columns and cells. They are all a little different, so play with your spreadsheet a little now. Try to do the following:
\begin{itemize}
\item Set the background of the first column to light gray.
\item Right-justify the text in the first column.
\item Make the text in the first column bold.
\item Make the numbers in the second column have one digit after the decimal point.
\end{itemize}

It should look something like this:

\includegraphics[width=0.6\textwidth]{FirstFormatting.png}

That's a spreadsheet. You have grid of cells. Each cell can hold a
value or a formula that uses values from other cells. The cells with
formulas automatically update as you edit the values in the other
cells.

\section{Comma-Separated Values}

A lot of data is exchanged in a file format called
\textit{Comma-Separated Values} or just CSV. Each CSV file holds one
table of data. It is a text file, and each line of text corresponds to
one row of data in the table. The data in each column is separated by
a comma. The first line of a CSV is usually the names of the
columns. A CSV might look like this:

\begin{Verbatim}
studentID,firstName,lastName,height,weight
1,Marvin,Sumner,260,45.3
2,Lucy,Harris,242,42.2
3,James,Boyd,261,44.2
\end{Verbatim}

In your digital resources for this module, you should have a file
called \path{1000cars.csv}. It is a CSV with only one column called
``speed''. The first few lines look like this:

\begin{Verbatim}
speed
33.8000
29.9920
34.8699
27.9936
\end{Verbatim}

There is a title line and 1000 data lines.

Import this CSV into your spreadsheet program. In Google Sheets, it looks like this:

\includegraphics[width=0.5\textwidth]{ImportingCSV.png}

You should see a long, long column of data appear. (Mine goes from cell A2 through A1001.)

\includegraphics[width=0.5\textwidth]{ImportedCSV.png}

\section{Statistics in Spreadsheets}

Let's take the mean all 1000 numbers.  In cell B2, type in a label:
``Mean''. (Feel free to format your labels as you wish. Bolding is recommended.)

In cell C2, enter the formula ``=AVERAGE(A2:A1001)''. When
you press return, the cell will show the mean: 31.70441, if done correctly .

\includegraphics[width=0.4\textwidth]{Spread_mean.png}

Notice that by specifing that the function \pyfunction{AVERAGE} was to
be performed on a range of cells: cells A2 through A1001.

Do the calculations for variance, standard deviation, and median.

\begin{itemize}
\item The function for variance is \pyfunction{VAR}.
\item The function for standard deviation is \pyfunction{STDEV}.
\item The function for median is \pyfunction{MEDIAN}.
\end{itemize}

\includegraphics[width=0.4\textwidth]{var_stdev_median.png}

\section{Histogram}

Most spreadsheets have the ability to create a histogram. In Google
Sheets, you select the entire range A2:A1001 by selecting the first
cell and then shift-clicking the last. Then you choose
Insert$\rightarrow$Chart. In the inspector, change the type of the
chart to histogram. This will get you a basic histogram.
% Add: Define histogram or give example, defined in basic statistics, must come previous

\includegraphics[width=0.7\textwidth]{default_histogram.png}

Play with the formatting to see how unquie you can make data. Here is an example:

\includegraphics[width=0.8\textwidth]{final_histogram.png}

\begin{Exercise}[title={RMS}, label=rms_spreadsheet]

  In your spreadsheet, calculate the quadratic mean (the root-mean-squared) of the speeds.

  You will need the following three functions:
  \begin{itemize}
  \item \pyfunction{SUMSQ} returns the sum of the squares of a range of cells.
  \item \pyfunction{COUNT} returns the number of cells in a range that contain numbers.
  \item \pyfunction{SQRT} returns the square root of a number.
  \end{itemize}

\end{Exercise}
\begin{Answer}[ref=rms_spreadsheet]

The formula for the RMS is ``=SQRT(SUMSQ(A2:A1001)/COUNT(A2:A1001))''.
% KA: https://www.khanacademy.org/computing/ap-computer-science-principles/data-analysis-101/data-tools/a/learning-from-data-sets

\end{Answer}


\section{Proof of Theorem~\ref{th:main}}\label{se:proof}
\begin{proof}
To simplify our proof we introduce the notion of \emph{canonical scenario} for the the ledger $\ledger_2$. In a canonical scenario the ledger
$\ledger_2$ is executed in the standard way. More precisely,
we assume the existence of a genesis block and that $\asmp_2[\tau]$=1 for all $\tau \geq 0$. Let $\parties$ be the set of parties that is running $\ledger_2$. Also,
let $t_j$ be the smallest time slot in which $B_j$ appears in  $\state_2^{P_i}[t_j]$ for each $P_i\in\parties$ and let $t_{j+k}$ be smallest time slot in which $B_{j+k}$ appears in %\nnote{$\state_2^{P_i}[t_{j+k}]$}
$\state_2^{P_i}[t_{j+k}]$ for each $P_i\in\parties$.
We are now ready to prove the security of $\Pi$.

In the protocol $\Pi$, by assumption, we have that $\asmp_2[\tau]=1$ for all $\tau \geq t$.
%From the description of $\Pi$ we can claim that $t \leq t_{P_1}+k\cg^{-1}=t_{P_1}+\Delta_1$.
%That is, when an honest party is activated she waits to be sure that also the other honest parties are activated. %Hence, from the chain-growth and the common-prefix parameters
%each party needs to wait at most $\Delta_1=k\cg^{-1}$ time slots.
From the moment when $\asmp_2$ becomes true the activation process takes $\Delta\leq k \cg^{-1} + k \cg^{-1}$  time slots to be completed.
This is because the parties need to wait that the $(j+k)$-th block of $\state_1$ is part of $\check \state_1^{P_i}[t]$  for all $P_i$ and that $k$ blocks are generated in $\state_2$. Note that when $k$ blocks are generated in $\state_2$ at least
$k$ blocks are generated in $\state_1$ since $\ledger_2$ and $\ledger_1$ have the same parameters and that the honest parties that maintain $\ledger_1$ are greater or equal than the parties that maintain $\ledger_2$. Therefore, the parties need to wait the \emph{special genesis block} of $\ledger_2$ to appear
in $\state_2^P$ for each honest $P\in\activep$. Given that a block in $\ledger_1$ ($\ledger_2$) takes at most $\cg^{-1}$ time slots then we have that $\Delta \leq k \cg^{-1} + k \cg^{-1}$.
In the moment that a \emph{candidate} block $B^i_{j+k}$ becomes available to an honest party $P_i\in\activep$ (i.e., $B^i_{j+k}$ is part of $\check \state_1^{P_i}$) then she starts running $\ledger_2$
using ${B^i}'$ which is computed from $B^i_{j+k}$ as described earlier (we recall that at this time slot the assumption $\asmp_2$ holds).
Let $t'$ be the smallest time slot in which $B_{j+k}$ appears in $ \state_2^{P_i}[t']$
%\nnote{$\state_2^{P_i}[t']$}
 for each $P_i\in\parties$.
If we take the execution of the protocol from time $t$ and $t'$ this can be seen as a canonical execution of $\ledger_2$ given
that the parameters of $\ledger_1$ and $\ledger_2$ are the same. The only difference between this and the canonical scenario is
that the blocks $B_{j}, B_{j+1},\dots, B_{j+k}$  are generated using $\ledger_1$, but this does not represent an issue since we are assuming that
any block of $\state_1$ can be turned into a block of $\state_2$. \end{proof}


\section{genetic Personalized Community Detection}

%
My proposed gPCD model contains an offline and an online step. Figure \ref{fig:pipeline} shows the pipeline of the whole framework. The offline step first encodes the user-independent binary community tree (\textit{Section \ref{sc:c3_offline}}), and subsequently learns embedding representations for both user need and nodes on the binary community tree (\textit{Section \ref{sc:c3_representation}}). The online step introduces the genetic personalized community detection approach (\textit{Section \ref{sc:c3_online}}). To accelerate the running speed, a distributed version of gPCD model is also deployed on HDFS (\textit{Section \ref{sc:c3_distributed}}).  To disambiguate the notations mentioned in this section to better explain the gPCD model, some commonly used notations can be found in Table \ref{tab:notation}.

\subsection{Offline Community Tree Index} \label{sc:c3_offline}

One challenge of solving personalized community detection problem is the computational cost due to the complexity of personalization and graph structure. In order to reduce the online workload, most of computation cost is put into the one-time offline step whose time cost can be excluded from the online personalized community detection step. Thus, I first convert the graph into a binary community tree offline to retain user-independent community information.

Infomap algorithm \cite{rosvall2011multilevel} is employed to generate the user-independent communities solely based on the graph \textit{G(V, E)}. Infomap algorithm simulates a random walker wandering on the graph and indexes the  description length of his / her random walk path via multilevel codebooks. By minimizing the description length based on the map equation below, community structures are formed for the graph.
%\begin{equation} \small
%\textit{$L(\mathcal{M})$} = q_{\curvearrowright}H(\mathcal{Q})+\sum_{i=1}^{m} \textit{$p_{\circlearrowright}^{i}$}H(\textit{$\mathcal{P}^{i}$})
%\end{equation}
%where $L(\mathcal{M})$ is the description length for a random walker in the current community $\mathcal{M}$. $q_{\curvearrowright}$ and $p_{\circlearrowright}^{i}$ are the jumping rates between communities and within the $i_{th}$ community. $H(\mathcal{Q})$ is the frequency-weighted average length of codewords in the global index codebook and $H(\mathcal{P}^{i})$ is frequency-weighted average length of codewords in the $i_{th}$ community codebook.

\begin{equation}
\textit{$L(\pi)$} = \sum_{i}^{m}q_{\curvearrowright}^{i}H(\mathcal{Q})+\sum_{i=1}^{m} \textit{$p_{\circlearrowright}^{i}$}H(\textit{$\mathcal{P}^{i}$})
\end{equation}
where \textit{$L(\pi)$} is the description length for a random walker under current community partition $\pi$. $q_{\curvearrowright}^{i}$ and $p_{\circlearrowright}^{i}$ are the jumping rates between communities and within the $i_{th}$ community in each step. $H(\mathcal{Q})$ is the frequency-weighted average length of codewords in the global index codebook and $H(\mathcal{P}^{i})$ is frequency-weighted average length of codewords in the $i_{th}$ community codebook. Followed by this equation to partition communities into sub-communities , a hierarchical community tree \textit{$T_{c}(N^{c},L^{c})$} is constructed from the original graph $G(V,E)$.
\begin{figure}
	% \advance\leftskip-1cm
	\center
	\includegraphics[width=1.0\columnwidth]{img/chapter3/4.pdf}
	%  \vspace{-3em}
	\caption{The framework of gPCD model. (a) refers to the offline construction step on the original graph and (b) refers to the online genetic pruning step to generate personalized communities.}
	\label{fig:pipeline}
	%  \vspace{-1em}
\end{figure}
In \textit{$T_{c}(N^{c},L^{c})$}, each parent node can have multiple child nodes which can be regarded as a community partition on the parent node. For instance, a node $N_{k}^{c} \in N^{c}$ from $T_{c}(N^{c},L^{c})$ represents a community of vertices.  Its $m$ child nodes $\{N_{k_{1}}^{c},N_{k_{2}}^{c},...,N_{k_{m}}^{c}\}$ represent $m$ sub-communities of vertices from $G(V,E)$ where we have $	\bigcap_{i=1}^{m} N_{k_{i}}^{c} = \varnothing$ and  $	\bigcup_{i=1}^{m} N_{k_{i}}^{c} =  N_{k}^{c}$.

In order to achieve an efficient personalized community detection in the following online step, I convert the hierarchical community tree \textit{$T_{c}(N^{c},L^{c})$} to a binary community tree \textit{$T_{b}(N^{b},L^{b})$} for index. Specifically, for $m$ child nodes of a parent node $N_{k}^{c}$, a bottom-up approach is proposed to merge a selected pair of sibling nodes as a new node in an iterative manner. The approach runs until all $m$ child nodes merged together to form the parent node $N_{k}^{c}$. To avoid an unbalanced tree where small communities are always left to merge with huge communities in the end, I first select the node with the smallest community size among all sibling nodes in each merging step. It is merged with its sibling node with the largest normalized linked weight (Please refer to Figure \ref{fig:pipeline}(a)). The normalized linked weight function $w(\cdot)$ between  two nodes $N_{i}^{c}$ and $N_{j}^{c}$ is defined as:
\begin{equation}
\textit{$w(N_{i}^{c},N_{j}^{c})$} =\frac{\textit{$N_{i}^{c}\odot N_{j}^{c}$}-\frac{\mathcal{D}(N_{i}^{c})\cdot \mathcal{D}(N_{j}^{c})}{2|E|} }{\textit{$| N_{i}^{c}||N_{j}^{c}|$}}
\end{equation}
where \textit{$N_{i}^{c}\odot N_{j}^{c}$} denotes the number of edges linked between vertices in node \textit{$N_{i}^{c}$} and \textit{$N_{j}^{c}$}, which can be interpreted as the linkage strength between them; \textit{$|N_{i}^{c}|$} is the number of vertices inside node \textit{$N_{i}^{c}$}; \textit{$\mathcal{D}(N_{i}^{c})$} is the out-degree of node \textit{$N_{i}^{c}$} (the total number of edges linked to other nodes) and \textit{$|E|$} is the total number of edges in the original graph $G(V,E)$ . $\frac{\mathcal{D}(N_{i}^{c})\cdot \mathcal{D}(N_{j}^{c})}{2|E|} $ denotes the random linkage strength between node \textit{$N_{i}^{c}$} and \textit{$N_{j}^{c}$}. The \textit{$w(\cdot)$} function calculates how much that two nodes are better connected beyond random connection and is normalized by node size. Given the node $N_{i}^{c}$ with the smallest community size and all its sibling node set $S$, The merging step can be formulated as:
\begin{equation}
\begin{aligned}
& N_{j}^{c} \Leftarrow \argmax_{N_{j}^{c} \in S}w(N_{i}^{c},N_{j}^{c})\\
& N_{*}^{c} = N_{i}^{c}\bigcup N_{j}^{c}
\end{aligned}
\end{equation}
The bottom-up process will stop until all child nodes are merged together to form the parent node. In the end, the hierarchical community tree \textit{$T_{c}(N^{c},L^{c})$} is fully converted to a binary community tree \textit{$T_{b}(N^{b},L^{b})$} with user-independent community information.  The node size $|N^{b}|$ as well as the link size $|L^{b}|$  in \textit{$T_{b}(N^{b},L^{b})$} is at most $2|V|$ which is smaller than the size of original graph $G(V,E)$. If I consider to form the binary community tree with only $k$ levels, the size of \textit{$T_{b}(N^{b},L^{b})$} can be even smaller.

\begin{table}[h]
	%\scriptsize
	%	\small
	\centering

	\begin{tabular}{|p{3cm}|p{11cm}|}
		\hline
		\textbf{Notations} & \textbf{Descriptions} \\ \hline
		$G(V,E)$ & Original graph $G$ with vertex set $V$ and edge set $E$ \\ \hline
		\textit{$T_{c}(N^{c},L^{c})$} & The hierarchical community tree generated from graph $G(V,E)$ with node set $N^{c}$ and link set $L^{c}$. Each node  $N^{c}_{k} \in N^{c}$ denotes a group of vertices belonging to $V$. \\\hline
		\textit{$T_{b}(N^{b},L^{b})$} & The binary community tree reconstructed from \textit{$T_{c}(N^{c},L^{c})$}. Each node $N^{b}_{k} \in N^{b}$ denotes a group of vertices belonging to $V$. \\ \hline
		$B$ & The binary codebook for \textit{$T_{b}(N^{b},L^{b})$}. Particularly, $B_{k} \in B$ denotes the binary code of both $N_{k}^{b} \in N^{b}$ and $L_{k}^{b}\in L^{b}$ where $L_{k}^{b}$ is the link points to node $N_{k}^{b}$.
		\\ \hline
	\end{tabular}
	\caption{Commonly used notations in gPCD model}
	\label{tab:notation}
	\vspace{-1em}
\end{table}

For running time analysis, calculating normalized linked weight takes constant time. In each merging step, node pair selection takes linear time. Therefore, in the worst case, the time complexity of binary community tree construction is $O(|V|^2)$ where the depth of the hierarchical community tree $T_{c}(N^{c},L^{c})$ is 1 and each vertex in $G(V,E)$ forms a single-vertex community.

To encode the nodes and links on $T_{b}(N^{b},L^{b})$ as binary code, the root node is encoded as `null' first. For a parent node \textit{$N_{k}^{b}$} with its left child node \textit{$N^{b}_{k_{l}}$} and right child node \textit{$N^{b}_{k_{r}}$}, the binary code of a child node and the related link defined in the Notation Table \ref{tab:notation} is calculated as:

\begin{eqnarray}\text{$B_{k_{i}}$}=
\begin{cases}
\text{$B_{k}$}+``0", & i = ``l"\cr
\text{$B_{k}$}+``1", & i = ``r"
\end{cases}
\end{eqnarray}

For instance, if the node \textit{$N_{k}^{b}$} is with binary code ``$00$," its left child node's binary code is ``$000$" while the right child node's binary code is ``$001$." The link \textit{$L_{k}^{b}$} that points to \textit{$N_{k}^{b}$} also has the binary code ``$00$".

\subsection{Community and User Need Representation} \label{sc:c3_representation}

Node2vec \cite{grover2016node2vec} helps to learn fixed-length embeddings for both user need and communities. It simulates random walks on the graph $G(V,E)$ and learns the vertex embedding by optimizing the sequential relationships from random walk paths. In the end, each vertex $V_{k}$ in graph $G(V,E)$ has a vector representation as $\vec{V_k}$. Each node $N_{k}^{b}$ on the binary community tree \textit{$T_{b}(N^{b},L^{b})$} refers to a vertex community $C_{k}$ in the graph $G(V,E)$. Its representation $\vec{C_{k}}$ is calculated as the averaged embedding of all vertices inside the community. In the end, the binary community tree \textit{$T_{b}(N^{b},L^{b})$} represents the hierarchical community partition of Graph $G(V,E)$. Each node \textit{$N_{k}^{b}$} on the tree is indexed with three attributes: a group of vertices from graph $G(V,E)$, a binary code \textit{$B_{k}$}, and an embedding representation \textit{$\vec{C}_{k}$}.

On the other hand, User need (query) $I$ can also be represented by a combination of $t$ different vertices $\{V_1, V_2... V_t\}$ in the graph $G(V,E)$. In this study, two different scenarios for user need representation are offered:

\textbf{Vertex-based Query}. User need can be directly represented by the vertices based on the generation probability $P(V_k|I)$ between them. Hence the user need representation $\vec{I}$ is calculated as:

\begin{equation}
\vec{I} = \sum_{k=1}^{t} P(V_k|I) \cdot \vec{V_k}
\end{equation}
For instance, in a music sharing network, each vertex $V_k$ denotes a music and a user listing history can be used to reflect the user need $I$. $P(V_k|I)$ therefore can be regarded as the probability that a music being listened by the user.

\textbf{Text-based Query}. Under this scenario, user need $I$ is represented as a text query, and each vertex $V_{k}$ in the graph $G(V,E)$ also contains textual content. From language model viewpoint, each vertex importance weight is the query likelihood $P(I|V_k)$, and the user need can is the weighted average of vertex embedding:

\begin{equation}
\vec{I} = \frac{\sum_{k=1}^{t} P(I|V_k) \cdot \vec{V_k}}{\sum_{k=1}^{t} P(I|V_k)}
\end{equation}

In either case, user need is conceptualized as an embedding with the same dimension as the node embeddings on the binary community tree. It enables very efficient online personalized community detection in later steps. And running Node2vec takes most of the time in this step.

\subsection{Online Genetic Pruning} \label{sc:c3_online}

The whole process, as the Figure \ref{fig:pipeline} shows, is to generate communities by pruning the constructed binary community tree. After each cut on a link, the original tree  will be separated into two sub-trees. After a specific number of cuts to the links on the tree, a fixed number of communities with different resolutions are detected.  By applying genetic selection, crossover, and mutation steps, the model converges to the optimized solution efficiently with a clear-defined fitness function. The details are shown in the following paragraphs.

\subsubsection{Genetic Representation}

A chromosome is formed by a set of genes \{$g_{1},g_{2},...,g_{K-1}$\}, and each gene $g_{i}$ holds a cut link $L_{i}^{b}$ in the binary community tree \text{$T_{b}(N^{b},L^{b})$}. Since communities can be created by cutting links on the offline tree, a chromosome can be represented as a generated community partition of the original graph $G(V,E)$ in this way.
To constrain a chromosome so that it can be decoded to a fixed number of communities, four \textbf{Cutting Rules} are necessarily to be applied:
\begin{itemize} %\itemsep0em
	\item \textbf{ Rule 1}: If a link $L_{i}^{b}$ is picked to cut on the binary community tree \text{$T_{b}(N^{b},L^{b})$}, its pointing node $N_{i}^{b}$ will be retrieved and all the vertices within it form a community.
	\item \textbf{ Rule 2}: If a link {$L_{i}^{b}$} and its ancestor link {$L_{j}^{b}$} are stored in the same chromosome, all vertices in {$L_{i}^{b}$}'s related node {$N_{i}^{b}$} are a subset of vertices in {$L_{j}^{b}$}'s related node {$N_{j}^{b}$}. In this case, the two cut links generate two communities where community $C_{i}$ is all vertices in $ N_{i}^{b}$ and community $C_{j}$ is the remaining vertices in $ N_{j}^{b}$ but not in $ N_{i}^{b}$. It can be formulated as   $C_{i} = \bigcup_{k}\{V_{k}|(V_{k}\in N_{i}^{b})\}$ and community $C_{j} = \bigcup_{k}\{V_{k}|(V_{k}\in N_{j}^{b}) \cap (V_{k}\notin N_{i}^{b})\}$.
	\item \textbf{ Rule 3}: Sibling links can't be stored in the same chromosome, and it is not allowed to store duplicated links in a chromosome.
	\item \textbf{ Rule 4}: The depth's upper bound is set to be $d$, which means all eligible cut links should be located in the first $d$ depth on the binary community tree. It avoids to generate super tiny communities and hugely reduces the genetic searching scope on cut links.
\end{itemize}

By applying the cutting rules to the online pruning process, I ensure a $K$ community partition can be retrieved from a chromosome with $K-1$ cut links.

\subsubsection{Initialization}

Initially, the model generates a given number $P$  chromosomes as the seed ``chromosome population". And each iteration in the genetic approach breeds a new ``generation'' of chromosome population. In order to ensure a chromosome is an encoder of a $K$ community partition, $K-1$ links will be randomly picked (on the binary community tree) following the cutting rules and stored in the related genes of a chromosome.

\subsubsection{Fitness Function}

As each chromosome can be decoded as a community partition, it is important to measure the quality of each generated chromosome (how well the generated communities can satisfy user need). The measurement is hosted in a fitness function.

In the proposed model, the fitness function simulates the user searching behavior on the graph given the community partition. For instance, a user can be more likely to pick the most relevant communities while avoiding the redundant information already selected. With the help of the offline step, the relevance score of node $N_{i}^{b}$ (community $C_{i}$) towards user need $I$ can be calculated with the cosine similarity $cos(\vec{I},\vec{C_{i}})$, and the information redundancy can be $\sum_{C_{j} \in S_c}cos(\vec{C_{j}},\vec{C_{i}})$ where $S_{c}$ is the set of communities that the user have already picked from the communities decoded from the target chromosome. Following this, I use a greedy selection approach to iteratively rank and pick communities given a chromosome (community partition) until all communities are picked:

\begin{equation}  \argmax_{C_{i}}\lambda \cdot cos(\vec{I},\vec{C_{i}})-(1-\lambda)\cdot \frac{\sum_{C_{j} \in S_c}^{ }cos( \vec{C_j},\vec{C_{i}})}{|S_c|}
\end{equation}

where $C_i$ is the candidate community to be picked and $|S_c|$ is the number of communities already been picked. $\lambda$ is a parameter controls  whether user prefers to obtain new useful information or to avoid redundant information.

For chromosome quality evaluation, a query-generated vertex ranking list $l_{q}$ is first created by retrieving top $n$ vertices relevant to the query (user need) with the largest cosine similarities on embeddings of graph $G(V,E)$. I store the top $n$ vertex ranking label $R(l_{q})=\{1,2,...,n\}$ as the pseudo ground truth. On the other hand, given the $k_{th}$ chromosome $ch_k$ in the current chromosome generation, I can also retrieve the community-generated ranking of each vertex $V_{k} \in l_{q}$ from the sequentially selected communities decoded by the chromosome. I assign the ranking label on each vertex $V_{k}$ based on the following formula:
\begin{equation}
\sum_{V_{j} \in l_{q}}\Phi(\delta(V_{j}) < \delta(V_{k})) + 1
\end{equation}

$V_{j}$ refers to all vertices in $l_{q}$. $\delta(V_{j})$ shows the ranking (selection sequence) of the community which $V_{j}$ belongs to. $\Phi$ is a binary operator to determine whether $V_{j}$ satisfy the condition $\delta(V_{j}) < \delta(V_{q})$. This formula helps to construct the community-generated ranking label $R(l_c)$. For instance, when $n=3$, I have a query-generated ranking list $l_q = \{V_1,V_2,V_3\}$ and its related ranking label $R(l_q) = \{1,2,3\}$. Given a chromosome where the community of $V_1$ and $V_2$ is the same and selected before $V_3$, I can generate the related community ranking label $R(l_c) = \{1,1,3\}$ with the same vertex sequence of $l_q$.

Then, I define the fitness function $f(\cdot)$ to evaluate the chromosome $ch_{k}$. As I have the query-generated ranking label $R(l_{q})$ (ground truth) and community-generated ranking label $R(l_{c})$ from $ch_{k}$, I calculate their Kendall's $\tau$  correlation coefficient  as the fitness score $f(ch_k)$ of chromosome $ch_k$ where higher score means the chromosome $ch_{k}$ can  generate better personalized communities to meet with user need.

\begin{equation}
\textit{$f(ch_{k})$} = 1-\frac{\sum_{i=1}^{n} R(l_{ci})\cdot R(l_{qi})}{\sum_{i=1}^{n} R(l_{ci})^2\cdot \sum_{i=1}^{n}R(l_{qi})^2}
\end{equation}
where $R(l_{ci})$ is the $i_{th}$ vertex ranking in community-generated ranking label $R(l_{c})$ and $R(l_{qi})$ is the $i_{th}$ vertex ranking in query-generated ranking label $R(l_{q})$. Kendall's $\tau$ is a widely used metric to evaluate the correlation between two lists where higher score means stronger correlation. Thus, higher fitness score reflects that the generated community ranking $R(l_{c})$ can better meet with user need $R(l_{q})$.

Moreover, it is clear that the fitness function aims to separate all top $n$ vertices in different communities to get the optimal case. It matches the research goal to generate high resolution communities on vertices which are more relevant to user need. As the number of community is a given number $K$, it also leads to a coarser manner partition on the remaining less relevant vertices. On the other hand, the binary community tree $T_{b}(N^{b},L^{b})$ and the Cutting Rule 4 naturally preserve the community structure and unite the most relevant vertices in the same community. Hence the whole genetic approach is a gambling process. The final chromosome result is the equilibrium case to detect communities both contain graph topological structure and meet with user need.

\subsubsection{Selection}

I select the superior chromosomes from current chromosome population based on their fitness scores. The probability that the $k_{th}$ chromosome $ch_k$ is picked can be calculated via the Softmax normalization function $p(ch_{i}) = \frac{exp(f(ch_{k}))}{\sum_{i=1}^{P}exp(f(ch_{i}))}$. Then, the Fitness Proportionate Selection method is applied to randomly select $P$ chromosomes into chromosome pairs based on probability distribution. In order to enhance optimization efficiency, I also use elitism selection to ensure the best chromosome in the current generation will always be selected to the next generation.

\subsubsection{Crossover}

To reach global optimum community partition efficiently, given a pair of chromosomes, the crossover operation can randomly exchange part of the genes in both chromosomes to produce a new pair of chromosomes with a certain crossover rate.

In order to make sure that the newly generated chromosomes meet the cutting rules, an \textbf{Exchange Rule} is defined to restrict gene exchange: If gene \textit{$g$} contains link $L^{b}_{g}$, \textit{$g$} can't do crossover process with genes that contain either link $L^{b}_{g}$ or its sibling link $L^{b'}_{g}$. This rule can help avoid having duplicated links or sibling links stored together in the newly generated chromosome (To satisfy Cutting rule 3).

After \textit{$m$} random numbers are selected from \{$1,2,...,K-1$\} as exchanged gene position indexes, genes located in the chosen positions of two chromosomes will exchange the stored link restricted by the Exchange Rule.
\subsubsection{Mutation}
Mutation operation is applied to avoid local optimization. If a chromosome is chosen to mutate, a gene within the chromosome will be randomly picked, and its stored link will be changed to another link restricted by the Exchange Rules. An example is illustrated in Figure \ref{fig:pipeline}(b) where the link stored in the second gene is changed from ``001'' to ``01''.

\subsubsection{Termination}

After $T$ iterations, the whole process stops and the current best chromosome is retrieved as the final result. Choosing the number of $T$ is dependent on the task. In order to decode the final chromosome to the related community partition, all genes in the chromosome are sorted in an ascending order based on the binary code of their stored cut link. Vertices whose binary codes start with the same cut link's binary code will be assigned to the same community label. And its later assigned community label can overwrite the previous assigned community label. For instance, if there are a vertex with binary code ``0011'' and two cut links with binary code ``00'' and ``001'', the vertex will be assigned to a community label ``00'' first, and its community label is overwritten by ``001'' afterwards. The Termination step in Figure \ref{fig:pipeline}(b) also illustrates a vivid example. In this way, the binary code of the binary community tree can help to decode the final chromosome into communities in an efficient way.

\subsection{Distributed gPCD} \label{sc:c3_distributed}

To enhance the online step efficiency, a MapReduce framework is utilized to enable the distributed genetic evolution. Figure \ref{fig:distributed} depicts the personalized community detection under a MapReduce framework. The chromosome collection is either originally initialized from binary community tree or obtained from the last generation. It contains the whole chromosome population in the central depository. In its first ``Splitter'' process, all chromosomes are split into $M$ groups based on their hash values and sent out to related $M$  Mappers to calculate the ``Fitness'' scores. In the same Mapper, after all chromosomes are assigned fitness scores, based on their scores, a Combiner groups all chromosomes together and random select equal number of chromosomes with duplicated as the ``Selection'' step. All the selected chromosomes are sent to $R$ reducers (I set $R = M$ arbitrarily in order to better represent time complexity) to form pairs for the ``Crossover'' and  ``Mutation'' step, calculate new chromosome offsprings for the next generation and store them back to the central repository.

\begin{figure}
	% \advance\leftskip-4cm
	\center
	\includegraphics[width=1\columnwidth]{img/chapter3/parallel.png}
	%  \vspace{-3em}
	\caption{Online parallel computing process on Hadoop Distributed File System (HDFS)}
	%  \vspace{-1em}
	\label{fig:distributed}
\end{figure}

The complexity of the proposed algorithm is $O(2^dKP)$ without parallel computing and $O(\frac{2^dKP}{M})$ with  parallel computing, where $d$ denotes the upper bound where the cut links are restricted in the top $d$ depth of the binary community tree $T_{b}(N^{b},L^{b})$; $K$ denotes the community number; $P$ denotes the initialized population size of the genetic algorithm and $M$ denotes the number of Mappers/Reducers in parallel environment. As all the parameters are considerably small (compared with the node/edge size in the original graph), the whole process runs very fast to retrieve the final community partition.


\documentclass{memoir}
\usepackage{notestemplate}

%\logo{~/School-Work/Auxiliary-Files/resources/png/logo.png}
%\institute{Rice University}
%\faculty{Faculty of Whatever Sciences}
%\department{Department of Mathematics}
%\title{Class Notes}
%\subtitle{Based on MATH xxx}
%\author{\textit{Author}\\Gabriel \textsc{Gress}}
%\supervisor{Linus \textsc{Torvalds}}
%\context{Well, I was bored...}
%\date{\today}

\begin{document}

% \maketitle

% Notes taken on 03/08/21

\section{Extension of Fields}
\label{sec:extension_of_fields}

\begin{defn}
	If \(K\) is a field containing a subfield \(F\), then \(K\) is said to be an \textbf{extension of \(F\)}, denoted by \(K / F\).\\

	The field \(F\) is sometimes called the \textbf{base field} of the extension.
\end{defn}
Note that if \(K\) is an extension of a field \(F\), then \(K\) is a \(F\)-vector space via the typical \(F\) action.

\begin{defn}
	The \textbf{degree} or \textbf{index} of a field extension \(K / F\), denoted \([K:F]\), is defined to be \(\textrm{dim}_F K\), the dimension of \(K\) as an \(F\)-vector space.
\end{defn}
For example, \([\Q(\sqrt{2} ) : \Q] = 2\) and \([\C: \R] = 2\). One can see the latter example by observing that \(\C\cong \R[x] / (x^2+1)\).

% Recall that we denote by \(F(\theta )\) the field \(F\) adjoined by the root \(\theta \), which is spanned by powers of \(\theta \) as an \(F\)-vector space.
\begin{thm}
	Let \(F\) be a field and \(p(x) \in F[x]\) be an irreducible polynomial. Then \(\exists \) a field extension \(K\) of \(F\) in which \(p(x)\) has a root.
\end{thm}
This field is given by \(K := F[x] / (p(x))\), but we will show this more formally later.

\begin{thm}
	Let \(p(x) \in F[x]\) be an irreducible polynomial of degree over \(F\), and let \(K\) be the field \(F[x] / (p(x))\). Take \(\theta := x + (p(x))\) (root of \(p(x)\) ). Then
	\begin{enumerate}
		\item The elements \(\left\{ 1_F, \theta , \theta ^2, \ldots, \theta ^{n-1} \right\} \) are an \(F\)-vector space basis of the \(F\)-vector space \(K\).
		\item \([K:F] = n\)
		\item \(K = \left\{a_0 + a_1\theta + a_2\theta^2 + \ldots + a_{n-1}\theta ^{n-1} \mid a_0,\ldots,a_{n-1} \in F \right\} \) as an \(F\)-vector space.
	\end{enumerate}
\end{thm}
Another nice example to be familiar with is \(K = \mathbb{F}_2[x] / (x^2+x+1)\). This is a field extension of \(\mathbb{F}_2\) as \(x^2+x+1\) is irreducible in \(\mathbb{F}_2\). We can see that \([\mathbb{F}_2[x] / (x^2+x+1) : \mathbb{F}_2[x]] = 2\) simply because the degree of the polynomial is \(2\), but we can also directly count elements in the set and see that it has twice the elements of \(\mathbb{F}_2[x]\).\\

Now let's define fields formed by adjoining roots more formally.

\begin{defn}
	Let \(K / F\) be a field extension, and let \(\alpha_1,\alpha_2,\ldots \in K\) be elements. The smallest subfield of \(K\) containing both \(F\) and the elements \(\alpha_1,\alpha_2,\ldots,\) denoted \(F(\alpha_1,\alpha_2,\ldots)\) is called the \textbf{field generated by \(\alpha_1, \alpha_2,\ldots\) over \(F\)}.
\end{defn}

\begin{defn}
	The field \(F(\alpha )\) generated by a single element \(\alpha \) over \(F\) is called a \textbf{simple extension of \(F\)}, and the element \(\alpha \) in this case is called \textbf{primitive}.
\end{defn}

\begin{thm}
	Let \(F\) be a field and let \(p(x) \in F[x]\) be an irreducible polynomial. Suppose \(K\) is an extension of \(F\) containing a root \(\alpha \) of \(p(x)\). Then \(F[x] / (p(x)) \cong F(\alpha )\).
\end{thm}
It is natural to view field extensions as the base field appended with roots, and as a result a few definitions arise.

\begin{defn}[Algebraic and Transcendental Elements]
	An element \(\alpha \in K\) is called \textbf{algebraic over \(F\)} if \(\alpha \) is a root of some nonzero polynomial \(f(x) \in F[x]\).\\

	If \(\alpha  \in K\) is not algebraic over \(F\), then we say that \(\alpha \) is \textbf{transcendental over \(F\)}.
\end{defn}
The extension \(K / F\) is \textbf{algebraic over \(F\)} if all elements of \(K\) are algebraic over \(F\).

\begin{exmp}[Examples of Algebraic and Transcendental Elements]
	\begin{itemize}
		\item \(\sqrt{2} \) is an algebraic element over \(\Q\) via the polynomial \(x^2-2\). This actually holds for all \(\sqrt[n]{2} \) with \(x^{n}-2\).
		\item \(i\) is algebraic over \(\R\) and \(\Q\) via the polynomial \(x^2+1\)
		\item Transcendental elements are much rarer-- examples include \(\pi \) and \(e\), but it is non-trivial to show an element is transcendental.
	\end{itemize}
\end{exmp}
\end{document}

Abstract Algebra - Introductory/Algebra II/Notes/source/Lecture15 - IntroFieldExtn.tex


% !TEX root = ../zeth-protocol-specification.tex

\section{Instantiating $\mkhash$}\label{instantiation:mkhash}

In this section we describe the instantiation of $\mkhash$ with a compression function based on \mimc{}~\cite{albrecht2016mimc}. We firstly show how the compression function is constructed, and prove that this instantiation complies with the security requirements mentioned in~\cref{zeth-protocol:sec-req}

\subsection{\mimc{} Encryption}\label{instantiation:mkhash:mimc-encryption}

\mimc{} is a block cipher with a simple design, consisting of a number of rounds (denoted $\rounds$). During the $i$-th round, the message $\msg$ is mixed with the encryption key $\key{}$ and a randomly chosen constant $c[i]$, and a permutation function is applied to generate a new value of $\msg$. The permutation function consists of exponentiation with a carefully chosen exponent $\exponent{}$ (see~\cref{instantiation:mkhash:mimc-encryption:security}). Note that \rounds{} depends on the desired security level $\secpar$. We denote the encryption function by \mimcEnc{} and illustrate it in~\cref{instantiation:fig:mimc}.

\begin{figure*}[ht]
    \centering
    \procedure[linenumbering]{$\mimcEnc(\key, \msg, c, \exponent{}, \rounds)$}{
        \pcforeach\ i \in [\rounds]:\\
        \t \msg \gets {(\key\ \mathsf{OP}\ c[i]\ \mathsf{OP}\ \msg)}^\exponent{}\\
        \pcreturn (\msg\ \mathsf{OP}\ \key)
    }
    \caption{\mimc{} Encryption function.}\label{instantiation:fig:mimc}
\end{figure*}

\mimcEnc{} can be defined on both binary and prime fields, and as such the $\mathsf{OP}$ operation corresponds to either $\oplus$ or $+ \pmod{p}$~\cite{albrecht2016mimc, grassi2016mpc}.
For general prime $p$ (resp.~positive integer $n$), we denote by $\mimcPrime{p}$ (resp.~$\mimc{}_{2^n}$) the \mimcEnc{} function defined over \FFx{p} (resp.~\FFx{2^n}).

\subsubsection{Security parameters and analysis}\label{instantiation:mkhash:mimc-encryption:security}

In this document, we only consider \mimc{} defined over prime fields (in particular, the field $\FFx{\rCURVE}$ over which \zksnark~operates).

Since block ciphers are usually defined over the product space of keys and messages, we consider the variables $c$, \rounds{} and \exponent{} as fixed. We thereby consider an instantiation of $\mimc{}$ with signature
\begin{align*}
    \mimcPrime{\rCURVE} &: \FFx{\rCURVE} \times \FFx{\rCURVE} \to \FFx{\rCURVE}
\end{align*}

In the sections below, and as in ~\cite{albrecht2016mimc}, we will consider exponents of the form $\exponent{} = 2^t-1$ and $\exponent{} = 2^t+1$ where $\gcd(\exponent{}, \rCURVE-1) = 1$. We note that the term cancellation happening with exponents of the form $\exponent{} = 2^t+1$ does not immediatelly translate to the context where $\mimc{}$ is carried out over prime fields of large odd characteristic. In fact, in the case of $\FFx{\rCURVE}$, where $\rCURVE > \binom{\exponent}{\lfloor \exponent / 2 \rfloor}$, polynomials $(x + y)^{\exponent}$ are not sparse.
This comes from the \emph{Binomial Theorem}
\[
  (x + y)^\exponent = \sum_{i = 0}^{\exponent} \binom{\exponent}{i} x^{i}y^{\exponent - i}
\]
and the observation that if $\binom{\exponent}{i} < \rCURVE$ then $\binom{\exponent}{i} \bmod \rCURVE = \binom{\exponent}{i}$, hence ensuring that all the polynomial coefficients are greater than $0$, and that the polynomial is dense.

To achieve a security of $\secpar$, we require that $\rounds \geq \secpar \log_{\exponent}(2)$.
Importantly, since we use $\mimc{}$ over prime fields $\FFx{\rCURVE}$ which are large (where $\rCURVE$ is the prime characteristic of the scalar field of an elliptic curve group, such that $\ceil{\log_2(\rCURVE)} > \secpar$\footnote{Longer elements are needed in Elliptic Curve Cryptography (\ecc) to resist algebraic attacks such as Number Field Sieve (\acrnfs)-based attacks on discrete logs~\cite{DBLP:journals/siamdm/Gordon93} for instance}), then, picking $\rounds = \left\lceil \frac{\log_2 \rCURVE}{\log_2 \exponent{}} \right\rceil > \secpar \log_{\exponent}(2)$ provides a margin of safety on the number of rounds selected to instantiate $\mimc{}$ with desired security level $\secpar$.

We refer to the $\mimc{}$ paper~\cite[Section 4.2 and 5.1]{albrecht2016mimc} and to~\cref{appendix:mimc-security} for more details on the security analysis and attacks on the scheme in the different settings. Note that $\mimcPrime{\rCURVE}$ does not suffer from \emph{inversion subfield attacks} as there are no proper subfields of $\FFx{\rCURVE}$.

\subsection{\mimc{}-based compression function}\label{instantiation:mkhash:mimc-compressionf}

There exist two main techniques to construct a hash function from a block-cipher (or permutation): sponge functions~\cite{bertoni2007sponge} and iterated compression functions~\cite{black2002black}.

A Merkle tree is a binary tree of values of fixed size, where the values in each ``layer'' are generated by hashing pairs of values from the previous ``layer''. That is, we require a compression function $\mkhash$, which we construct via the Miyaguchi-Preneel scheme. (Miyaguchi-Preneel is more secure~\cite[$f_5$ function]{black2002black} than the more flexible Davies-Meyer construct~\cite[Section 3]{gazzoni2006maelstrom}, but this flexibility is not required in our case).

\subsubsection{Miyaguchi-Preneel compression construct}

Miyaguchi-Preneel (MP)~\cite[$f_3$ function]{black2002black} is a general scheme for constructing compression functions from block ciphers (see~\cref{preliminaries:definitions:hashcomp}). Given a block cipher \Enc, the corresponding compression function by \fMP{} is given in~\cref{instantiation:fig:mp-constructions}. The original construction is defined over binary fields, however \zeth~operates over prime fields. Hence, in the general discussion here we replace the bitwise addition operator $\oplus$ by modular addition in $\FFx{\rCURVE}$ (see~\cite{mp-security-ethsnarks}).

We denote by \mimcMP{} the compression function defined by the application of the Miyaguchi-Preneel construct over \mimc{}. Similarly, for general prime $p$ we denote by $\mimcMPPrime{p}$ (see~\cref{instantiation:fig:mimc-mp-constructions}) the compression function defined by application of the Miyaguchi-Preneel construct over $\mimcPrime{p}$.

\begin{figure*}[ht]
    \begin{minipage}[t]{0.50\textwidth}
        \procedure[linenumbering]{$\fMP{} (\key, \msg)$}{
            res \gets \Enc_\key(\msg) \\
            \pcreturn (res + \msg + \key) \pmod{\rCURVE}
        }
        \caption{\MP{} construct in $\FFx{\rCURVE}$.}\label{instantiation:fig:mp-constructions}
    \end{minipage}%
    \begin{minipage}[t]{0.50\textwidth}
        \procedure[linenumbering]{$\mimcMPPrime{\rCURVE}(\key, \msg)$}{
            res \gets \mimcPrime{\rCURVE}(\key, \msg) \\
            \pcreturn (res + \key + \msg) \pmod{\rCURVE}
        }
    \caption{$\mimcMPPrime{\rCURVE}$ construction.}\label{instantiation:fig:mimc-mp-constructions}
    \end{minipage}%
\end{figure*}

\subsection{An efficient instantiation of \mimc{} primitives}\label{instantiation:mkhash:efficient-instance}

To select appropriate instances of $\mimcPrime{\rCURVE}$ and $\mimcMPPrime{\rCURVE}$, we consider the cost (in terms of gas consumption and prover efficiency). For given $\exponent{}$ and $\rounds{}$, the final definition of $\mimcMPPrime{\rCURVE}$ is given in \cref{instantiation:fig:mimcp-construction} and \cref{instantiation:fig:mimcp-mp-construction}.

\newcommand{\initRoundConstants}{\algostyle{InitRoundConstants}}

\begin{figure*}[ht]
    \begin{minipage}[t]{0.5\textwidth}
        \centering
        \procedure[linenumbering]{$\mimcPrime{\rCURVE}(\key, \msg)$}{
            c \gets \initRoundConstants() \\
            \pcforeach i \in [\rounds]:\\
            \t \msg \gets {(\key + c[i] + \msg)}^\exponent{} \pmod{\rCURVE}\\
            \pcreturn (\msg + \key) \pmod{\rCURVE}
        }
    \end{minipage}%
    \begin{minipage}[t]{0.5\textwidth}
        \centering
        \procedure{$\initRoundConstants()$}{
            iv \gets \keccak{256} (\text{``clearmatics\_mt\_seed''}) \\
            c[0] \gets 0 \\
            c[1] \gets \keccak{256} (iv) \\
            \pcforeach i \in \range{2}{\rounds}:\\
            \t  c[i] \gets \keccak{256} (c[i-1])\\
            \pcreturn c = (c[0], \ldots, c[\rounds-1])
        }
    \end{minipage}%
    \caption{$\mimcPrime{\rCURVE}$ full construction}\label{instantiation:fig:mimcp-construction}
\end{figure*}

\begin{figure*}[ht]
    \centering
    \begin{minipage}[t]{0.5\textwidth}
        \procedure{$\mimcMPPrime{\rCURVE}(\key, \msg)$}{
            \pcreturn \mimcPrime{\rCURVE}(\key, \msg) + \msg + \key \pmod{\rCURVE}
        }
    \caption{\mimcMPPrime{\rCURVE} full construction}\label{instantiation:fig:mimcp-mp-construction}
    \end{minipage}%
\end{figure*}

\begin{remark}
    Note that \keccak{256} is the $256$-bit digest instance of the \keccak{} family that won the NIST SHA-3 competition~\cite{keccak-submission}. It is supported by the \evm via an opcode (see~\cite[Appendix G]{wood2014ethereum}), making it convenient for use in smart contracts.
\end{remark}

\begin{remark}
    To increase the security of the $\mathsf{\mkhash}$, different round constants for each level of the Merkle tree could be used.
\end{remark}

We define $\mkhash$ to be $\mimcMPPrime{}$ over $\FFx{\rCURVE}$. Thereby, for input values $m_0$ and $m_1$, $\mkhash : \FFx{\rCURVE} \times \FFx{\rCURVE} \to \FFx{\rCURVE}$ is defined by
\begin{equation}\label{instantiation:eq:mkhash-instantiation}
    \mkhash(\msg_0, \msg_1) = \mimcMPPrime{\rCURVE}(m_0, m_1)
\end{equation}

\newcommand{\constraints}{\varstyle{constraints}}

For specific values of $\rCURVE$ (such as $\rBN$ for $\BNCurve$ or $\rBLS$ for $\BLSCurve$), it remains to select concrete values of $\exponent{}$ and rounds, where $\rounds = \lceil \frac{\log_2 \rCURVE}{\log_2 \exponent{}} \rceil$).  These values influence the number of constraints in the arithmetic circuit (see~\cref{zeth-protocol:statement} for details of the statement) and the gas cost of Merkle tree operations on the contract (see~\cref{zeth-protocol:process-tx} for details of the specific operations).

In the arithmetic circuit, an invocation of $\mimcMPPrime{}$ requires $\rounds \cdot \mults$ constraints, where $\mults$ is the number of multiplications required for exponentiation. For exponents of the form $\exponent = 2^t - 1$, we have $\mults = 2 \cdot t - 2$, (using the \emph{square-and-multiply} algorithm~\cite{menezes1996handbook}), and for $\exponent = 2^t + 1$ we have $mults = t + 1$. Thus we expect that exponents of the latter form are more optimal. The implementation in the contract performs a very similar set of arithmetic operations (exponentiation in the field through a series of multiplications and modulo reductions), and so the cost is dominated by the same number $\rounds \cdot \mults$ as for the circuit.  Hence, values of $\exponent$ and $\rounds$ that are optimal for the circuit will also result in gas-efficient implementations in the contract.

For several concrete values of $\exponent{}$, the number of $\rounds$ required to attain the desired security level, along with the number of constraints, are shown in~\cref{table:mimc-exp-analysis}.

\definecolor{Gray}{gray}{0.9}
\begin{table}
  \centering
    \begin{minipage}[t]{0.50\textwidth}
        \centering
        \begin{tabular}{r c c c c}
            \toprule
            \multirow{2}{*}{$\exponent{}$} & \multicolumn{2}{c}{\BNCurve} & \multicolumn{2}{c}{\BLSCurve} \\ [0.5ex]
            & $\rounds$ & $\constraints$ & $\rounds$ & $\constraints$ \\ [0.5ex]
            \midrule
            \rowcolor{Gray}
            5 & 110 & 331 & & \\
            7 & 91 & 365 & & \\
            \rowcolor{Gray}
            17 & 65 & 316 & 62 & 311 \\
            31 & 52 & 417 & 51 & 409 \\
            127 & 37 & 445 & 37 & 445 \\
            \rowcolor{Gray}
            257 & 32 & 289 & 32 & 289 \\
            511 & 29 & 465 & & \\
            2047 & 24 & 481 & 23 & 461 \\
            8191 & 20 & 481 & 20 & 481 \\
            32676 & 17 & 477 & & \\
            \rowcolor{Gray}
            65537 & 16 & 273 & 16 & 273 \\
            131071 & 15 & 481 & 15 & 481 \\
            524287 & 14 & 505 & 14 & 505 \\
            \rowcolor{Gray}
            1048577 & 13 & 274 & 13 & 274 \\
            2097151 & 13 & 521 & & \\
            \bottomrule
        \end{tabular}
    \end{minipage}%
    \caption{Arithmetic constraints required to represent \mimcMP{} as an R1CS program, for different exponents $\exponent{}$ and curves. Grey (resp.~white) lines represent exponents of shape $2^t + 1$ (resp.~$2^t - 1$). Missing entries where $\gcd(\exponent{},\rCURVE - 1) \neq 1$}\label{table:mimc-exp-analysis}
\end{table}

For the case of \BNCurve~we set $\exponent{} = 17$ with $\rounds{} = 65$, to achieve a $254$-bit security level. For \BLSCurve~we set $\exponent{} = 17$ with $\rounds{} = 62$, to achieve $253$-bit security. These values are chosen such that they satisfy the requirement that $\gcd(\exponent{},\rCURVE - 1) = 1$, and give a balance between the number of constraints in the arithmetic circuit and the gas cost of hashing on the contract.

\subsection{Security requirements satisfaction}\label{instantiation:mkhash:security}

After presenting the state of the art of MiMC cryptanalysis, we present the security proof of \mimcMPPrime{} collision resistance.

\subsubsection{Cryptanalysis of $\mimc{}$ block cipher and primitives}\label{instantiation:mkhash:security:cryptanalysis}

\mimc{}'s security is increasingly being analysed since the primitive has gained traction in zero-knowledge and cryptocurrency communities for its succinct algebraic constraint representation. As of today, we do not know of any attacks breaking \mimc{} on prime fields on full rounds.

The first attack on \mimc{} was an interpolation attack~\cite{li2019improved} which targets a reduced-round version for a scenario in which the attacker has only limited memory.
An attack on Feistel-based \mimc{}~\cite{bonnetain2019collisions} was discovered shortly after, by using generic properties of the used Feistel construction (instead of exploiting properties of the primitive itself).
Additionally,~\cite{albrecht2019algebraic} proposes an attack based on Gr\"{o}bner basis. The authors state that by introducing a new intermediate variable in each round, the resulting multivariate system of equations is a Gr\"{o}bner basis. As such, the first step of a Gr\"{o}bner basis attack can be obtained for free. However, the following steps of the attack are so computationally demanding that the attack becomes infeasible in practice.
A recent work~\cite{DBLP:conf/asiacrypt/Eichlseder0LORS20} targets \mimc{} on binary fields, and achieves a full-round break of the scheme. While, the attack presented does not apply to prime fields, the authors note that it ``can be generalized to include ciphers over $\FFx{p}$'', and that only the lack of efficient distinguishers over prime fields precludes this.
Another attack from Beyne et al~\cite{cryptoeprint:2020:188} uses a low complexity distinguisher against full \mimc{} permutation leading to a practical collision attack on reduced round sponge-based \mimc{} hash defined with security of 128 bits.

\subsubsection{Security proof of \mimcMPPrime{} collision resistance}\label{instantiation:mkhash:security:colres-proof}
We now prove that this compression scheme satisfies all the security requirements listed in~\cref{zeth-protocol:sec-req}. To do so, we first assume that the round constants are pseudo-random, i.e.~that $\keccak{256}$ is a \prf{}.

\begin{lemma}
	\keccak{256} is a $\prf$ with $\lambda=128$.
\end{lemma}

The security of \mimcMPPrime{} derives from a more general result, i.e.~from modelling \mimcPrime{} as an ideal cipher (see~\cref{preliminaries:def:ICM}). More specifically, we show a security result for the \MP{} construction on \FFx{\rCURVE} by proving that, in the Ideal Cipher Model, the collision resistance advantage of any adversary is bounded by $\frac{q(q+1)}{\rCURVE}$, where $q$ is the number of different queries that the attacker makes to the oracle. This means that, assuming a maximum $q$ number of possible encryption/decryption queries, parameter $\rCURVE$ can be chosen to make the advantage small as needed and $\fMP$ considered collision resistant. Similar result applies to the ${2^n}$ case.

The instance of \mimc{} we use is modelled as an ideal cipher defined on field elements, for this reason we consider a variant of the ICM model where the keys, inputs and outputs are field elements in $\FFx{\rCURVE}$ and the block cipher scheme, with key $\key$, correspond to a family of $\rCURVE$ independent random permutations $f_{\key}: \FFx{\rCURVE} \times \FFx{\rCURVE} \to \FFx{\rCURVE}$.

In the proof, without loss of generality, we assume the following conventions for an adversary \adv{}:
\begin{itemize}
    \item the adversary asks distinct queries: i.e.~if \adv{} asks a query $\oracleEnc(\key,\msg)$ and this returns $y$, then \adv{} does not ask a subsequent query of $\oracleEnc(\key,\msg)$ or $\oracleDec(\key,y)$, and inversely;
    \item the adversary necessarily obtained the candidate collision from the oracle. This property follows suite from modelling \mimc{} as an ideal cipher.
\end{itemize}

\begin{lemma}\label{lemma:colrescomp}
    Let \fMP{} be the \MP{} compression function built on an ideal block-cipher \Enc{} on \FFx{\rCURVE}, the probability for an adversary \adv{} to find a collision is not greater than $q(q+1)/\rCURVE$ where $q$ is a (positive) number of distinct oracle queries.
\end{lemma}

The following proof has been adapted from~\cite[Lemma 3.3]{black2002black}\footnote{It states the collision resistance of a set of compression functions $f_1,,\ldots,,f_{12}$, denoted as \emph{group-1 compression functions} and showed in~\cite[Figure 3]{black2002black}. As mentioned above, Miyaguchi-Preneel corresponds to $f_3$ of that group. Since the proof of~\cite[Lemma 3.3]{black2002black} shows collision resistance of $f_1$, we slightly modified it to work for $f_3$.}.

\begin{proof}
    Fix $h_0\in\FFx{\rCURVE}$. Let \adv{} be an adversary attacking the compression function \fMP{}.
    Assume that \adv{} asks the oracles \oracleEnc{} and \oracleDec{} a total of \emph{distinct} $q$ queries. Let us denote the result of the $q$ queries and output of the attacker (candidate collision) as $\left ( (\key_1, \msg_1, y_1), \ldots , (\key_q, \msg_q, y_q), \text{out} \right )$.
    If \adv{} is successful it means that it outputs $(\key, \msg)$, $(\key', \msg')$ such that either $(\key, \msg) \neq (\key', \msg')$ and $\fMP(\key, \msg) = \fMP(\key', \msg')$ or $\fMP(\key, \msg) = h_0$.
    By the definition of \fMP, we have that $\Enc_\key(\msg) + \msg + \key = \Enc_{\key'}(\msg') + \msg' + \key'$ for the first case, or $\Enc_\key(\msg) + \msg + \key = h_0$ for the second.
    So either there are distinct $r, s \in [1,\ldots, q]$ such that $(\key_r, \msg_r, y_r) = (\key, \msg, \Enc_\key(\msg))$ and $(\key_s,\msg_s, y_s) = (\key',\msg', \Enc_{\key'}(\msg'))$ and $\Enc_{\key_r}(\msg_r) + \msg_r + \key_r = \Enc_{\key_s}(\msg_s) + \msg_s + \key_s$ or else there is an $r \in [1,\ldots, q]\ \suchthat\ (\key_r, \msg_r, y_r) = (\key, \msg, h_0)$ and $\Enc_{\key_r}(\msg_r) + \msg_r + \key_r = h_0$. We show that this event is unlikely.

    In fact, for each $i \in [1,\ldots, q]$, let $C_i$ be the event that either $y_i + \msg_i + \key_i = h_0$ or does exist $j \in [1,\ldots, i-1]\ \suchthat\ y_i + \msg_i + \key_i = y_j + \msg_j + \key_j$. When carrying out the simulation $y_i$ or $\msg_i$ was randomly selected from a set of at least $\rCURVE - (i-1)$ elements, so $\prob{C_i}\leq i / (\rCURVE-i)$. This means that for the collision advantage of \adv{}, \advCollMP it holds that $\advCollMP \leq \prob{C_1 \lor \cdots \lor C_q} \leq \sum_{i=1}^{q} \prob{C_i}$. For $q \leq \frac{\rCURVE}{2}$ this probability is bounded by $l \cdot \frac{q(q+1)}{\rCURVE}$. However, we allow only a polynomial number of queries, thus for $q = \poly$ this probability becomes $\frac{\poly}{\rCURVE}$, where $\rCURVE \approx 2^\secpar$.
\end{proof}

\begin{notebox}
   \cref{lemma:colrescomp} is applicable to our case by the strong assumption of \mimcPrime{\rCURVE} being an ideal cipher. In other words, the proof does not take into account any structural weakness or knowledge that an attacker is aware of. Any such additional information could make~\cref{lemma:colrescomp} invalid, and consequently could be used to break the collision resistance.
\end{notebox}

\begin{remark}
    Note that from~\cref{lemma:colrescomp} follows that the collision resistance security of the \zeth{} Merkle tree is $\log_2(\rCURVE/2)$ (around $127$ bits for $\rCURVE = \rBN$ or $\rBLS$).
\end{remark}

\begin{notebox}
    $\mimc{}$ has \emph{not} received as much cryptanalytic scrutiny as other ``older'' and more established hash functions. This is important to note since, for these type of primitives which are not provably secure, the amount of attacks received by a scheme is a great indicator of its security and robustness.
    A natural alternative to $\mimc{}$ here consists in using Pedersen hash which is provably collision resistant under the discrete-logarithm assumption.
\end{notebox}


\documentclass[manuscript,screen,nonacm]{acmart}

%% The enumitem package let's us enumerate lists with, e.g. a b c, instead of numbers
\usepackage{enumitem}

%% The clrscode3e package provides pseudocode environments. It is the style pacakge used
%% to generate the type for Introduction to Algorithms 3rd Edition
%% by Cormen, Leiserson, Rivest and Stein
\usepackage{clrscode3e}

%% The hyperref package let's us include hyperlinks in our document
\usepackage{hyperref}

%% The physics package makes it easy to write bras and kets
\usepackage{physics}

%% Declare path to figures directory
\graphicspath{{./figures/}}

\theoremstyle{plain}
\newtheorem{lemma}{Lemma}[section]
\newtheorem{claim}{Claim}[section]
\newtheorem{theorem}{Theorem}[section]

\theoremstyle{remark}
\newtheorem{case}{Case}
\newtheorem{step}{Step}

\theoremstyle{definition}
\newtheorem{definition}{Definition}[section]

%% The card(inality) command puts mathematically acceptable vertical bars around the input
\newcommand{\card}[1]{\lvert #1 \rvert}

%% The dotprod command displays the dot product of two vectors
\newcommand{\dotprod}[2]{#1 \cdot #2}

%% The argmax and argmin commands typeset their name sake functions
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}

% Define probability macro
\renewcommand{\Pr}{\operatorname{\mathsf{P}}}

% Define expected value macro.
\newcommand{\E}{\operatorname{\mathsf{E}}}

% Define set macro, i.e., {1,2,3}
\newcommand{\Set}[1]{\{ #1 \}}

% Define a math hyphen
\mathchardef\mhyphen="2D

%% Uncommon operations that I always forget

%% Matrices
%% \begin{equation}
%% 	lhs =
%% 	\begin{pmatrix}
%% 		(1,1) & (1,2)\\
%% 		(2,1) & (2,2)
%% 	\end{pmatrix}
%% \end{equationW}

%% Splits
%% \begin{equation}
%% \begin{split}
%% 	lhs & = rhs\\
%% 	&= rhs2\\
%% 	&= rhs3\\
%% \end{split}
%% \end{equation}

%% Cases
%% \begin{equation}
%% 	lhs =
%% 	\begin{cases}
%% 		case1 & \text{when this}\\
%% 		case2 & \text{when this}\\
%% 		case3 & \text{otherwise}
%% 	\end{cases}
%% \end{equation}

%% Figures
%% \begin{figure}[t]
%% 	\centering
%% 	\includegraphics[width=0.5\textwidth]{figure.jpg}
%% 	\caption{
%% 		This is a caption.
%% 	}
%% 	\label{fig:figure}
%% \end{figure}

\begin{document}

\title{Homework $\cdot$ Date}
\subtitle{Professor Ringeaux Brimbus $\cdot$ Class Number $\cdot$ Season}

\author{Daniel Salmon}
\email{daniel-salmon@pitt.edu}

\begin{abstract}
	This paper presents a solution to Homework y.  Section
	\ref{sec:problems} restates the problem; Section \ref{sec:solution}
	presents the solution. A summary of results.
\end{abstract}

\maketitle

\section{Problems}
\label{sec:problems}

This is a problem.

\subsection{Problem x (z points)}
\label{subsec:problemx}

This is a problem.

\subsection{Problem y (z points)}
\label{subsec:problemy}

This is a problem.

\section{Solution}
\label{sec:solution}

This is a solution.

\end{document}