\section{Method}
The method is summarized in figure~\ref{fig:workflow}. It consists of three main phases:
\begin{enumerate*}[(i)]
	\item feature extraction,
	\item classification, and
	\item delineation of linear elements.
\end{enumerate*}
%\bigskip
\begin{figure}[!b]
	\centering
	\begin{tikzpicture}[node distance=0.70cm, scale=0.7, every node/.style={transform shape}]

	\node (in1) [io] {Raw Point Cloud};

	\node (param) [title, below=of in1] {Feature extraction};
	\node (pro1) [process, below left=2.5cm and -1.6cm=of param, fill=darkorange!20] {Compute nearest neighbours};
	\node (pro2) [process, below=of pro1, fill=darkorange!20] {Compute neighbourhood features};

	\begin{scope}[on background layer]
	\node (fit1) [fit=(param)(pro1)(pro2), inner sep=4pt, transform shape=false, draw=black!80, fill=darkorange, fill opacity=0.5] {};
	\end{scope}

	\node (out1) [io, above right=-0.55cm and 4.5cm=of fit1] {Point Cloud Features};
	\node (in2) [io, right=of out1] {Manually Classified Point Cloud};

	\node (class) [title, below right=1.5cm and 5.7cm=of out1] {Classification};
	\node (pro4) [process, below left=2.5cm and -6.6cm=of class, fill=lightblue!20] {Remove irrelevant points};
	\node (pro5) [process, below=of pro4, fill=lightblue!20] {Create random forest classifier using manually classified point cloud};
	\node (pro6) [process, below=of pro5, fill=lightblue!20] {Assess performance using cross validation};
	\node (pro7) [process, above right=-3.67cm and 7.5cm=of pro6, fill=lightblue!20] {Classify unknown points};
	\node (pro8) [process, below=of pro7, fill=lightblue!20] {Compute cylindrical neighbourhoods of vegetation points};
	\node (pro9) [process, below=of pro8, fill=lightblue!20] {Seperate trees from low vegetation using height difference};

	\begin{scope}[on background layer]
	\node (fit2) [fit=(class)(pro4)(pro5)(pro6)(pro7)(pro8)(pro9), inner sep=4pt, transform shape=false, draw=black!80, fill=lightblue, fill opacity=0.5] {};
	\end{scope}

	\node (out2) [io, above right= -0.55cm and 13.5cm=of fit2] {Classified Point Cloud};

	\node (lin) [title, below=of out2] {Linear Elements};
	\node (pro10) [process, below left=2.5cm and -15.6cm=of lin, fill=turquoise!20] {Convert to 2D and downsample};
	\node (pro11) [process, below=of pro10, fill=turquoise!20] {Grow regions based on rectangularity};
	\node (pro12) [process, below=of pro11, fill=turquoise!20] {Merge directionally aligned regions};
	\node (pro13) [process, below=of pro12, fill=turquoise!20] {Assess the elongatedness of the regions};

	\begin{scope}[on background layer]
	\node (fit3) [fit=(lin)(pro10)(pro11)(pro12)(pro13), inner sep=4pt, transform shape=false, draw=black!80, fill=turquoise, fill opacity=0.5] {};
	\end{scope}

	\node (out3) [io, below=of fit3] {Linear vegetation elements};

	\draw [arrow] (in1) -- (fit1);
	\draw [arrow] (fit1) -| +(2.6,0) |- (out1.west);
	\draw [arrow] (out1.south) |- +(0,-0.45) -| (fit2.north); %(out1.south|-fit2.north);
	\draw [arrow] (fit2) -| +(4.5,0) |- (out2.west);
	\draw [arrow] (out2) -- (fit3);
	\draw [arrow] (in2.south) |- +(0,-0.195) -| (fit2.north); %(in2.south|-fit2.north);

	\draw [arrow] (pro1) -- (pro2);

	\draw [arrow] (pro4) -- (pro5);
	\draw [arrow] (pro5) -- (pro6);
	\draw [arrow] (pro5) -| +(2,0) |- (pro7);
	\draw [arrow] (pro7) -- (pro8);
	\draw [arrow] (pro8) -- (pro9);

	\draw [arrow] (pro10) -- (pro11);
	\draw [arrow] (pro11) -- (pro12);
	\draw [arrow] (pro12) -- (pro13);

	\draw [arrow] (fit3) -- (out3);

	\end{tikzpicture}
	\caption{Overview of the method in a work flow diagram.}
	\label{fig:workflow}
\end{figure}

\subsection{Feature extraction}
A point cloud \(\mathcal{P}\) is a set of points \(\{p_{1}, p_{2}, \dots, p_{n}\}\) \(\in \mathbb{R}^3\). Each point \(p_{i}\) consists of the X, Y and Z coordinates of the point and additional information on the intensity (\(I\)) of the return signal, the return number, and the number of returns (\(R_{t}\)) at \(p_{i}\) is known (figure~\ref{fig:LiDAR}).
A normalized return number (\(R_{n}\)) was calculated by dividing the return number by the number of returns~\citep{guo2011relevance}.

For each point \(p_{i}\) a neighbourhood set \(\mathcal{N}_{i}\) of points \(\{q_{1}, q_{2}, \dots, q_{k}\}\) is defined, where \(q_{1} = p_{i}\).
%This can be done in three ways:
%\begin{enumerate*}[(i)]
%	\item a \(k\)-nearest neighbours neighbourhood, where the first \(k\ (-1)\) nearest points to \(p_{i}\) are part of the neighbourhood,
%	\item a spherical neighbourhood, where all the points within a certain radius (\(r_{s}\)) of \(p_{i}\) belong to the neighbourhood, and
%	\item a cylindrical neighbourhood, which includes all points within a cylinder with infinite height and a certain radius (\(r_{c}\)) of \(p_{i}\).
%\end{enumerate*}
This neighbourhood was determined using a \(k\)-nearest neighbours method with a \(k\) of 50. This method was used instead of spherical neighbourhood of a certain radius, because it is harder to define a suitable radius which works for fluctuating point densities within and across datasets~\citep{weinmann2014semantic}.

To determine these neighbours a k-d tree data structure was constructed, which is a multidimensional binary search tree that can be used for efficient nearest neighbour searches~\citep{bentley1975kdtree}.

For each neighbourhood some basic geometric properties are computed \citep{weinmann2015semantic}:

{\setlength{\abovedisplayskip}{0pt}
\begin{flalign}
	\label{eq:deltaz}
	&\text{Height difference:}&&& \Delta_{Z_{i}} &= \max_{j:\mathcal{N}_{i}}(q_{Z_{j}}) - \min_{j:\mathcal{N}_{i}}(q_{Z_{j}}) &&&\\
	\label{eq:stdz}
	&\text{Height standard deviation:}&&& \sigma_{Z_{i}} &= \sqrt{\frac{1}{k} \sum_{j=1}^k (q_{Z_{j}} - \overline{q_{Z}})^2} &&&\\
	\label{eq:radius}
	&\text{Local radius:}&&& r_{l_{i}} &= \max_{j: \mathcal{N}_{i}}(|p_{i} - q_{j}|) &&&\\
	\label{eq:density}
	&\text{Local point density:}&&& D_{i} &= \frac{k}{\frac{4}{3} \pi r_{l_{i}}^3} &&&
\end{flalign}

Additionally the neighbourhood of a point can be used to characterize local surface features as first described by~\citet{hoppe1992surface}, and further developed by~\citet{pauly2002efficient}, who used the local structure tensor to estimate the surface normals (the vector perpendicular to the surface (\(\vec{N} = (N_{x}, N_{y}, N_{z})\))) and to define the surface variation (equation~\ref{eq:surfacevariation}). The structure tensor describes the directions of the neighbourhood of a point by determining the covariance matrix of the X, Y and Z coordinates of the points and computing the eigenvalues (\(\lambda_{1}, \lambda_{2}, \lambda_{3}\), where \(\lambda_{1} > \lambda_{2} > \lambda_{3}\)) and eigenvectors of this matrix. The magnitude of the eigenvalues of this covariance matrix describe the spread of points in the direction of the eigenvector. By ranking the eigenvectors based on the respective eigenvalues the spread in the three principle directions of a local point cloud can be quantified. This was expanded upon by~\citet{west2004context} to the following eight structure tensor features:

\begin{flalign}
\label{eq:linearity}
&\text{Linearity: }&&& L_{\lambda} &= \frac{\lambda_{1} - \lambda_{2}}{\lambda_{1}}&&&\\
\label{eq:planarity}
&\text{Planarity: }&&& P_{\lambda} &= \frac{\lambda_{2} - \lambda_{3}}{\lambda_{1}}&&&\\
\label{eq:sphericity}
&\text{Sphericity: }&&& S_{\lambda} &= \frac{\lambda_{3}}{\lambda_{1}}&&&\\
\label{eq:omnivariance}
&\text{Omnivariance: }&&& O_{\lambda} &= \sqrt[3]{\lambda_{1} \lambda_{2} \lambda_{3}}&&&\\
\label{eq:anisotropy}
&\text{Anisotropy: }&&& A_{\lambda} &= \frac{\lambda_{1} - \lambda_{3}}{\lambda_{1}}&&&\\
\label{eq:eigenentropy}
&\text{Eigenentropy: }&&& E_{\lambda} &= -\lambda_{1}\ln(\lambda_{1}) -\lambda_{2}\ln(\lambda_{2}) -\lambda_{3}\ln(\lambda_{3})&&&\\
\label{eq:sumofeigenvalues}
&\text{Sum of eigenvalues: }&&& \sum_{\lambda} &= \lambda_{1} + \lambda_{2} + \lambda_{3}&&&\\
\label{eq:surfacevariation}
&\text{Local surface variation: }&&& C_{\lambda} &= \frac{\lambda_{3}}{\lambda_{1} + \lambda_{2} + \lambda_{3}}&&&
\end{flalign}

To avoid using an unnecessarily large feature set the correlation between all the features was checked by calculating pearson coefficients. Of the pairs of two extremely correlated features (above 0.98), one was removed. This resulted in the removal of anisotropy, which is negatively correlated with sphericity, and eigenentropy, which is correlated with omnivariance.

Consequently the resulting feature set used for the classification was \(\{I, R_{t}, R_{n}, \Delta_{Z}, \sigma_{Z}, r_{l}, D, N_{z}, L_{\lambda}, P_{\lambda}, S_{\lambda}, O_{\lambda}, \sum_{\lambda}, C_{\lambda}\}\).

\subsection{Classification}
\label{sec:class}
Since the linear vegetation elements consist of either trees or low vegetation (shrubs and small trees) the point cloud needs to be classified into \textit{trees}, \textit{low vegetation}, and \textit{irrelevant} points. The difference between trees and low vegetation is ambiguous and is dependent on interpretation. In previous research this boundary has been set at varying height values, ranging from 1.5~\citep{bork2007integrating}, to 3.5~\citep{antonarakis2008object}, to 4~\citep{koukoulas2005spatial}, to 5 meters~\citep{khosravipour2014generating}. We set this boundary at 4m. The irrelevant points are returns of which the linearity does not need to be checked. This may include very low vegetation (herbs), bare ground, water, buildings, and other man made objects.

\subsubsection{Data trimming}
To speed up the classification first points were removed which were certain not relevant. This was done based on the planarity (equation~\ref{eq:planarity}) and sphericity (equation~\ref{eq:sphericity}) features discussed earlier. Since scanning vegetation (which is larger than herbs) always results in a locally very scattered point cloud, with points spread out throughout the 3D space, points with a locally planar neighbourhood can be removed. That is to say points with a high planarity value, \(P_{\lambda} > 0.7\), and a low sphericity value, \(S_{\lambda} < 0.05\), were removed. These values were manually selected in a conservative way to make sure every point removed was certainly not relevant, while still removing a large portion of points.

\subsubsection{Supervised classification}
The remaining point cloud was classified into \textit{vegetation} and \textit{irrelevant} using a random forest classifier. The random forest algorithm creates a collection of decision trees each based on a random subset of the training data~\citep{ho1998random}. These decision trees are computed by a Classification And Regression Tree (CART) algorithm, which creates splits that minimize a gini impurity index~\citep{breiman1984classification}. This impurity index is the probability a randomly picked sample would be misclassified, given it was randomly classified conform the distribution of classes. For each point, each tree in the forest determines a class, and the class which gets selected by the majority of the trees is chosen as the final classification~\citep{breiman2001random}.

Training and testing data was created by analysing the research area and manually segmenting areas of \textit{vegetation} and \textit{irrelevant}. This was done based on the point cloud and aerial photos~\citep{PDOK2015luchtfoto}.

Because the point cloud is trimmed based on planarity/sphericity and it concerns an agricultural landscape it has become very imbalanced, having a lot more \textit{vegetation} than \textit{irrelevant} points, which was also very noticeable in the training and testing data. Imbalanced training data can lead to undesirable classification results~\citep{he2009learning}. Therefore a balanced random forest was used, where instead of the decision trees being made using bootstrap samples of the entire training dataset, a bootstrap sample was taken from only the minority class and a random sample was taken of the majority class based on the size of the minority class sample~\citep{Chen2004using}. Employing enough trees eventually all majority class data gets used, while still maintaining a balance between the two classes. The size of the majority sample compared to the minority sample can be adjusted to find the best balance.

The random forest parameters (i.e. maximum depth, maximum number of features, minimal samples per leaf, minimal samples per split) and the ratio between minority and majority samples were optimized using a cross validated grid search.

\subsubsection{Accuracy Assessment}

To assess the accuracy of the classification a confusion matrix was used. Such a matrix shows the predicted and the actual classes of the tested pixels/points. It gives an overview of the performance of the classification in that it identifies and quantifies the errors. Using this matrix a precision (user's accuracy), recall (producer's accuracy), and overall accuracy can be calculated~\citep{stehman1997selecting}, but these metrics are unable to provide a good picture of the performance of a classifier when the data is very imbalanced.

Instead, three metrics, which provide a useful indication of performance even when dealing with a very imbalanced dataset, were used~\citep{sun2009classification, lopez2013insight}. These are:
\begin{enumerate*}[(i)]
	\item the receiver operating characteristic (ROC) curve~\citep{bradley1997use},
	\item the Matthew's correlation coefficient (MCC)~\citep{matthews1975comparison}, and
	\item the geometric mean~\citep{kubat1998machine}.
\end{enumerate*}

To create a ROC-curve the true positive rate is plotted against the false positive rate at various decision thresholds. The area under a ROC-curve (AUC) is a measure for the performance of the classifier~\citep{bradley1997use}.

The MCC analyses the correlation between the observed and the predicted data and is defined as follows:

\begin{equation}
	\label{eq:MCC}
	{\text{MCC}}={\frac  {TP\times TN-FP\times FN}{{\sqrt  {(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}}}
\end{equation}

where TP are the true positives, TN the true negatives, FP the false positives, and FN the false negatives. It often gives a more balanced evaluation of a classifier than the precision, recall and accuracy~\citep{baldi2000assessing} and it is an effective metric for unbalanced datasets~\citep{kohavi1995study}.

The geometric mean of the recall of both classes evaluates the balanced performance of the classifier for the two classes~\citep{kubat1998machine, sun2009classification}.

These scores were acquired by performing a 10-fold cross validation. This is done by splitting the data into 10 randomly mutually exclusive subsets and using a subset as testing data on a classifier trained on the remaining data~\citep{kohavi1995study}. This method is effective against overfitting and does not further reduce the training data (which a test and validation set would).

To assess which features are most influential when separating vegetation from non-vegetation a feature importance analysis was performed. This is done by summing the decreases in gini impurity due to a feature in all the trees divided by the total amount of trees~\citep{breiman2002manual}.

\subsubsection{Delineating trees and low vegetation}
To divide the vegetation class into \textit{tree} and \textit{low vegetation} for each point a cylindrical neighbourhood was determined, which includes all points within a cylinder with infinite height and a radius of 2 meters. Within this neighbourhood the height difference (equation~\ref{eq:deltaz}) was computed. As mentioned earlier, if this value exceeded 4 meters the point was classified as a \textit{tree} point, otherwise as a \textit{low vegetation} point.

\subsection{Delineating Linear Elements}
To delineate linear from non-linear vegetation elements the two classes were segmented into rectangular objects, since those can be assessed for linearity by their length and width. This is done by growing regions within the relevant points as long as the shape remains rectangular.

\begin{figure}
	\centering
	\includegraphics[scale=0.40]{./img/downsample}
	\caption{An example showing the tree points before and after downsampling.}
	\label{fig:downsample}
\end{figure}
Since the linearity of an object is solely determined by the 2D spatial distribution of points, the point cloud was converted to 2D by removing the Z-coordinate from all the points. To speed up the computation the 2D point cloud was spatially downsampled to 1 meter distance between all points for low vegetation and 2 meter for trees (figure~\ref{fig:downsample}). Low vegetation objects are often smaller than trees and consequently require more points for an accurate delineation. This leads to a slightly less precise delineation, but results in a substantially decreased computation time.

\subsubsection{Rectangularity}
\begin{figure}
	\centering
	\input{./img/hulls.tikz}
	\caption{An example showing the different hulls used for determining the rectangularity.}
	\label{fig:hulls}
\end{figure}
The rectangularity of an object can be described as the ratio between the area of an object and the area of its minimum bounding rectangle~\citep{rosin1999measuring}.

The minimum bounding rectangle (figure~\ref{fig:hulls}) is computed with rotating calipers as described by~\citet{toussaint1983solving}. First a convex hull is determined for the points, which is the smallest convex set of points which contains every point (figure~\ref{fig:hulls}), by using the QuickHull algorithm as defined by~\citet{preparata1985computational}. The minimum bounding rectangle has a side collinear with one of the edges of the convex hull~\citep{freeman1975determining}. Consequently while rotating by the angles of the edges rectangles can be created using the minima and maxima of the coordinates in the rotated system. Keeping track of the rectangle with the minimal area the minimum bounding rectangle can be found.

The area of the object can be found by computing an alpha shape of the set of points belonging to the object~\citep{edelsbrunner1983shape}. An alpha shape is a hull, similar to a convex hull, which describes the shape of a set of points (figure~\ref{fig:hulls}). It is created by computing a Delaunay triangulation of the points~\citep{delaunay1934sphere} and removing the triangles with a circumradius higher than \(1/\alpha\), where \(\alpha\) is a parameter which consequently influences the amount of triangles removed from the triangulation and thus the shape and area of the alpha shape. Higher alphas lead to more complex shapes, while lower ones to more smooth shapes (figure~\ref{fig:hulls}).

\subsubsection{Region Growing}
To segment the point cloud into rectangular objects a region growing segmentation technique was used. This method was introduced by \citet{besl1988segmentation} and has since been an effective approach to segment a point cloud~\citep{tovari2005segmentation, rabbani2006segmentation, nurunnabi2012robust, vosselman2013point, elberink2014user, vo2015octree}. The region growing method consists of two main steps: seed selection and region growing. The seed locations can be selected randomly or based on some properties of the points. The growing is based on a similarity criterion based on the proximity and attributes of the points.

The region growing used in this method is a bit different. Instead of using the attributes to compare the points and grow based on some similarity criteria, the regions are grown based only on proximity and a constraint of rectangularity (figure~\ref{fig:regiongrowing}). The seed selection is done based on the coordinates to minimize the chances of starting at boundary regions.

\begin{figure}
	\centering
	\input{./img/regiongrowing.tikz}
	\caption{An example showing the region growing process.}
	\label{fig:regiongrowing}
\end{figure}

Thus the point with the minimal x-coordinate and its 20 closest neighbours were used as the starting region (given this region is rectangular, otherwise the point is discarded and the process repeated) and subsequently points were added as long as the region’s rectangularity did not drop below a threshold of 0.55. This threshold was determined by experimentation and manual evaluation. Once no more new points could be added to the region the procedure was repeated for the next region until all the points are checked.

\subsubsection{Merging Objects}
The objects can still be quite fragmented after they have been grown, especially for curved regions. To fix this, objects were merged which were
\begin{enumerate*}[(i)]
	\item in proximity of each other,
	\item aligned with each other, and
	\item facing about the same direction.
\end{enumerate*}
The direction was determined by computing the angle between one of the long sides of the minimum bounding box and the x-axis. The alignment was checked by comparing the angle of the line between the two centre points of the objects with the directions. Once merged the lengths of the objects were added and the maximum of the widths taken as the new width.

\subsubsection{Elongatedness}
The resulting rectangular regions were assessed for linearity by determining the elongatedness of an object, which is defined as the ratio between its length and its width~\citep{nagao2013structural}.

The minimum elongatedness of objects was set at 2.5, based on a manual trial and error. To prevent long, but wide patches of forest to be classified as linear elements a maximum width of 60 meters was used.

\subsubsection{Accuracy Assesment}
To assess the accuracy of the delineation of linear objects a manual delineation was done. First a polygon was made both from the tree and low vegetation points using alpha shapes. Subsequently the two polygons were manually further segmented into objects. Finally the polygons belonging to a linear object according to our interpretation were manually marked as being linear. The resulting map was compared with the automated delineation. This was done by a difference map and confusion matrices where we compared the true positive, true negative, false positive and false negative in area. Since this data is, unlike before, not very imbalanced we can use traditional metrics for accuracy. Therefore a user's, producer's and overall accuracy was calculated, as well as an F1, kappa and MCC score~\citep{congalton2008assessing}.

\subsection{Software \& Hardware}
We chose to use solely free and open source software (FOSS), for two main reasons:
\begin{enumerate*}[(i)]
	\item to avoid `black boxes', where it is unclear exactly what calculations or algorithms are being used to produce the results, and
	\item to make the method more easily runnable with cloud- or supercomputing solutions, and thus able to compute for very large areas.
\end{enumerate*}

The large majority was made in Python (2.7.12) with the NumPy~(1.11.2), SciPy~(0.18.1), pandas~(0.18.1), scikit-learn~(0.17.1) and Shapely~(1.5.13) libraries. For the preprocessing of data LASTools~(version~160429) and CloudCompare~(v2.8) were used and Cloudcompare was also used for visualizing and downsampling the point cloud.

For the computation a desktop computer was used with an Intel Xeon E3-1220 3.1 GHz quad-core processor and 16GB of RAM.


\section{Logic}\label{Logic}

This library deals with classical logic and its properties.
The main file is {\tt Classical.v}.

This library also provides some facts on equalities for dependent
types. See the files {\tt Eqdep.v} and {\tt JMeq.v}.


\thispagestyle{empty}
%----------------------------------------------------------------------
\chapter{Control and optimization}
\label{control.chap}
%----------------------------------------------------------------------

\section{Objectives}
%----------------------------------------------------------------------
Once a system has been enough understood, there is still to do the main work: enhance it. This means most of the time optimization, and control can be seen as an online optimization (i. e. continuously making the system better). There are several control techniques as there are many optimization techniques. Since this reader aims at keeping things practical, we will not develop all possible ideas here. Instead we insist onto 2 main ideas:
\begin{enumerate}
	\item explicit a criteria to be optimized: in principle, this is rooted in the system description and what we want to do with the system. There are many ways to enhance a system, and very often the models are too complicated to have an explicit optimum (whenever they have one). But without criteria, there is even no way to know what is better (or worse).
	\item simplify the model so that the optimization (or control) problem can be solved by simple tools. As a rule, it is often better to get a simple but efficient control rather than to try to compute the optimal one. This rules also holds for optimization: the best is the enemy of the good; well enough is enough in lots of practical systems.
\end{enumerate}

Therefore some tools for PDE are applied in the next sections as well as simple tools that can be applied by smartly simplifying the problem.

\section{Tools and examples}

\subsection{Kalman filter}
\input{control-kalman}

\subsection{PID control}

\subsection{Adjoint optimization}


\chapter{Basic Concepts}
\label{Chp:Concepts}

The GraphBLAS C API is used to construct
graph algorithms expressed ``in the language of linear algebra.''
Graphs are expressed as matrices, and the operations over
these matrices are generalized through the use of a
semiring algebraic structure.

In this chapter, we will define the basic concepts used to
define the GraphBLAS C API.  We provide the following elements:
\begin{itemize}
\item Glossary of terms used in this document.

\item Notation

\item Execution model

\item Error model

\end{itemize}

\section{Glossary}

%TGM I'm leaving a few definitions in here just as examples

\subsection{Basic definitions}

\glossBegin

\glossItem{application} A program that calls methods from the GraphBLAS C API to
solve a problem.

\glossItem{GraphBLAS C API} The application programming interface that fully defines the types, objects,
literals, and other elements of the C binding to the GraphBLAS.

\glossEnd

\subsection{Objects and their structure}

\glossBegin
\glossItem{handle}  A variable that uses one of the GraphBLAS opaque data types.
The value of this variable holds a reference to a GraphBLAS object but not the contents of the object itself.
Hence, assigning a value of one handle to another variable copies the reference to the GraphBLAS object
but not the contents of the object.

\glossItem{non-opaque datatype} Any datatype that exposes its internal structure.
This is contrasted
with an \emph{opaque datatype} that hides its internal structure and can
be manipulated only through an API.

\glossEnd

\vfill

\newgeometry{left=2.5cm,top=2cm,bottom=2cm}

\section{Notation}

\begin{tabular}[H]{l|p{5in}}
Notation & Description \\
\hline
$\Dout, \Dinn, \Din1, \Din2$  & Refers to output and input domains of various GraphBLAS operators. \\
$\bDout(*), \bDinn(*),$ & Evaluates to output and input domains of GraphBLAS operators (usually \\
~~~~$\bDin1(*), \bDin2(*)$ & a unary or binary operator, or semiring). \\
$\mathbf{D}(*)$   & Evaluates to the (only) domain of a GraphBLAS object (usually a monoid, vector, or matrix). \\
$f$             & An arbitrary unary function, usually a component of a unary operator. \\
$\mathbf{f}(F_u)$ & Evaluates to the unary function contained in the unary operator given as the argument. \\
$\odot$         & An arbitrary binary function, usually a component of a binary operator. \\
$\mathbf{\bigodot}(*)$ & Evaluates to the binary function contained in the binary operator or monoid given as the argument. \\
$\otimes$       & Multiplicative binary operator of a semiring. \\
$\oplus$        & Additive binary operator of a semiring. \\
$\mathbf{\bigotimes}(S)$ & Evaluates to the multiplicative binary operator of the semiring given as the argument. \\
$\mathbf{\bigoplus}(S)$ & Evaluates to the additive binary operator of the semiring given as the argument. \\
$\mathbf{0}(*)$   & The identity of a monoid, or the additive identity of a GraphBLAS semiring. \\
$\mathbf{L}(*)$   & The contents (all stored values) of the vector or matrix GraphBLAS objects.  For a vector, it is the set of (index, value) pairs, and for a matrix it is the set of (row, col, value) triples. \\
$\mathbf{v}(i)$ or $v_i$   & The $i^{th}$ element of the vector $\vector{v}$.\\
$\mathbf{size}(\vector{v})$ & The size of the vector $\vector{v}$.\\
$\mathbf{ind}(\vector{v})$ & The set of indices corresponding to the stored values of the vector $\vector{v}$.\\
$\mathbf{nrows}(\vector{A})$ & The number of rows in the $\matrix{A}$.\\
$\mathbf{ncols}(\vector{A})$ & The number of columns in the $\matrix{A}$.\\
$\mathbf{indrow}(\vector{A})$ & The set of row indices corresponding to rows in $\matrix{A}$ that have stored values.  \\
$\mathbf{indcol}(\vector{A})$ & The set of column indices corresponding to columns in $\matrix{A}$ that have stored values. \\
$\mathbf{ind}(\vector{A})$ & The set of $(i,j)$ indices corresponding to the stored values of the matrix. \\
$\mathbf{A}(i,j)$ or $A_{ij}$ & The element of $\matrix{A}$ with row index $i$ and column index $j$.\\
$\matrix{A}(:,j)$ & The $j^{th}$ column of the the matrix $\matrix{A}$.\\
$\matrix{A}(i,:)$ & The $i^{th}$ row of the the matrix $\matrix{A}$.\\
$\matrix{A}^T$ &The transpose of the matrix $\matrix{A}$. \\
$\neg\matrix{M}$ & The complement of $\matrix{M}$.\\
$\vector{\widetilde{t}}$ & A temporary object created  by the GraphBLAS implementation. \\
$<type>$ & A method argument type that is {\sf void *} or one of the types from Table~\ref{Tab:PredefinedTypes}. \\
{\sf GrB\_ALL} & A method argument literal to indicate that all indices of an input array should be used.\\
{\sf GrB\_Type} & A method argument type that is either a user defined type or one of the  types from Table~\ref{Tab:PredefinedTypes}.\\
{\sf GrB\_Object} &  A method argument type referencing any of the GraphBLAS object types.\\
{\sf GrB\_NULL} & The GraphBLAS NULL.\\
\end{tabular}

\restoregeometry

\section{Error model}

\section{Execution Model}
\label{Sec:ExecutionModel}

%% TGM: I've left the GraphBLAS model here just as a point of reference.

A program using the GraphBLAS C API constructs GraphBLAS objects,
manipulates them to implement a graph algorithm, and then extracts
values from the GraphBLAS objects as the result of the algorithm.
Functions defined within the GraphBLAS C API that manipulate GraphBLAS
objects are called \emph{methods}.  If the method corresponds to one
of the operations defined in the GraphBLAS mathematical specification,
we refer to the method as an \emph{operation}.

Graph algorithms are expressed as an ordered collection of GraphBLAS
method calls defined by the order they are encountered in a program.
This is called the \emph{program order}.  Each method in the collection
uniquely and unambiguously defines the output GraphBLAS objects based
on the GraphBLAS operation and the input GraphBLAS objects. This is the
case as long as there are no execution errors, which can put objects in
an invalid state (see Section~\ref{Sec:ErrorModel}).

The GraphBLAS method calls in program order are organized into contiguous
and nonoverlapping \emph{sequences}.  A sequence is an ordered collection
of method calls as encountered by an executing thread. (For more on
threads and GraphBLAS, see Section~\ref{Sec:ThreadSafety}.)  A sequence
begins with either (1) the first GraphBLAS method called by a thread,
or (2) the first method called by a thread after the end of the
previous sequence.  A sequence can end (terminate) in a variety of ways.
A call to the GraphBLAS {\sf GrB\_wait()} method (Section~\ref{Sec:GrB_wait})
always ends a sequence.  The GraphBLAS {\sf GrB\_finalize()} method
(Section~\ref{Sec:GrB_finalize}) also implicitly ends a sequence. Finally,
in blocking mode (see below), each GraphBLAS method starts and ends its
own sequence.

The GraphBLAS objects are fully defined at any point in a sequence by
the methods in the sequence as long as there are no execution errors.
In particular, as soon as a GraphBLAS method call returns, its output
can be used in the next GraphBLAS method call.  However, individual
operations in a sequence may not be \emph{complete}. We say that an
operation is complete when all the computations in the operation have
finished and all the values of its output object have been produced and
committed to the address space of the program. Furthermore, no additional
execution time can be charged to a completed operation and no additional
errors can be attributed to a completed operation.

The opaqueness of GraphBLAS objects allows execution to proceed
from one method to the next even when operations are not complete.
Processing of nonopaque objects is never deferred in GraphBLAS. That is,
methods that consume nonopaque objects (\eg, {\sf GrB\_Matrix\_build()},
Section~\ref{Sec:Matrix_build}) and methods that produce nonopaque objects (\eg,
{\sf GrB\_Matrix\_extractTuples()}, Section~\ref{Sec:Matrix_extractTuples})
always finish consuming or producing those nonopaque objects before
returning.

\comment{Furthermore, methods that extract values from opaque GraphBLAS objects
into nonopaque user objects (see Table~\ref{Tab:ExtractMethods})
always force completion of all pending computations on the
corresponding GraphBLAS source object.

\begin{table}[htb]
    \hrule
    \begin{center}
        \caption{Methods that extract values from a GraphBLAS object, thereby
        forcing completion of the operations contributing to that particular object.}
        \label{Tab:ExtractMethods}

        \begin{tabular}{l|l}
            Method    & Section \\ \hline

            {\sf GrB\_Vector\_nvals}        & \ref{Sec:Vector_nvals}        \\
            {\sf GrB\_Vector\_extractElement}     & \ref{Sec:extract_single_element_vec}    \\
            {\sf GrB\_Vector\_extractTuples}    & \ref{Sec:Vector_extractTuples}    \\
            {\sf GrB\_Matrix\_nvals}        & \ref{Sec:Matrix_nvals}        \\
            {\sf GrB\_Matrix\_extractElement}     & \ref{Sec:extract_single_element_mat}    \\
            {\sf GrB\_Matrix\_extractTuples}    & \ref{Sec:Matrix_extractTuples}    \\
            {\sf GrB\_reduce} (vector-scalar variant)        & \ref{Sec:Reduce_vector_scalar}        \\
            {\sf GrB\_reduce} (matrix-scalar variant)        & \ref{Sec:Reduce_matrix_scalar}        \\
        \end{tabular}
    \end{center}
    \hrule
\end{table}
}

\section{Error Model}
\label{Sec:ErrorModel}

%%TGM  I've left the GraphBLAS model here as a point of reference

All GraphBLAS methods return a value of type {\sf GrB\_Info} to provide
information available to the system at the time the method returns. The
returned value can be either {\sf GrB\_SUCCESS} or one of the defined
error values shown in Table~\ref{Tab:ErrorValues}. The errors fall into
two groups: API errors (Table~\ref{Tab:ErrorValues}(a)) and execution
errors (Table~\ref{Tab:ErrorValues}(b)).

\begin{table}[bh]
\hrule
\begin{center}
\caption{Error values returned by GraphBLAS methods.}
\label{Tab:ErrorValues}

\vspace{1\baselineskip}
(a) API errors
\vspace{1\baselineskip}

\begin{tabular}{l|p{3in}}
Error code    & Description \\ \hline
{\sf GrB\_UNINITIALIZED\_OBJECT} & A GraphBLAS object is passed to a method before {\sf new} was called on it.\\
{\sf GrB\_NULL\_POINTER} & A NULL is passed for a pointer parameter. \\
{\sf GrB\_INVALID\_VALUE} & Miscellaneous incorrect values. \\
{\sf GrB\_INVALID\_INDEX} & Indices passed are larger than dimensions of the matrix or vector being accessed. \\
{\sf GrB\_DOMAIN\_MISMATCH} & A mismatch between domains of collections and operations when user-defined domains are in use.\\
{\sf GrB\_DIMENSION\_MISMATCH} & Operations on matrices and vectors with incompatible dimensions. \\
{\sf GrB\_OUTPUT\_NOT\_EMPTY} & An attempt was made to build a matrix or vector using an output object that already contains valid tuples (elements).\\
%{\sf GrB\_NO\_VALUE} & An attempt was made to extract a value from a tuple within a matrix or vector for which there is no stored value.
{\sf GrB\_NO\_VALUE} & A location in a matrix or vector is being accessed that has no stored value at the specified location. \scott{It depends on whether or not the non-opaque scalar is
well-defined on return from {\sf extract}}\\
\end{tabular}

\vspace{1\baselineskip}
(b) Execution errors
\vspace{1\baselineskip}

\begin{tabular}{l|p{3in}}
Error code    & Description \\ \hline
{\sf GrB\_OUT\_OF\_MEMORY}         & Not enough memory for operations. \\
{\sf GrB\_INSUFFICIENT\_SPACE}     & The array provided is not large enough to hold output. \\
{\sf GrB\_INVALID\_OBJECT}         & One of the opaque GraphBLAS objects (input or output) is in an invalid state caused by a previous execution error. \\
{\sf GrB\_INDEX\_OUT\_OF\_BOUNDS}  & Reference to a vector or matrix element that is outside the defined dimensions of the object. \\
{\sf GrB\_PANIC}        & Unknown internal error. \\
\end{tabular}

\end{center}
\hrule
\end{table}

An API error means that a GraphBLAS method was called with parameters that
violate the rules for that method.  These errors are restricted to those
that can be determined by inspecting the types and domains of GraphBLAS
objects, GraphBLAS operators, or the values of scalar parameters fixed at
the time a method is called.  API errors are deterministic and consistent
across platforms and implementations.  API errors are never deferred,
even in nonblocking mode. That is, if a method is called in a manner
that would generate an API error, it always returns with the appropriate
API error value.  If a GraphBLAS method returns with an API error, it
is guaranteed that none of the arguments to the method (or any other
program data) have been modified.

Execution errors indicate that something went wrong during the execution
of a legal GraphBLAS method invocation.  Their occurrence may depend on
specifics of the executing environment and data values being manipulated.
This does not mean that execution errors are the fault of the GraphBLAS
implementation.  For example, a memory leak could arise from an error in
an application's source code (a ``program error''), but it may manifest
itself in different points of a program's execution (or not at all)
depending on the platform, problem size, or what else is running at
that time.  Index-out-of-bounds and insuficient space execution errors
always indicate a program error.

In blocking mode, where each method executes to completion, a returned
execution error value applies to the specific method.  If a GraphBLAS
method, executing in blocking mode, returns with any execution error
from Table~\ref{Tab:ErrorValues}(b) other than {\sf GrB\_PANIC}, it
is guaranteed that no argument used as input-only has been modified.
Output arguments may be left in an invalid state, and their use downstream
in the program flow may cause additional errors.  If a GraphBLAS method
returns with a {\sf GrB\_PANIC} execution error, no guarantees can be
made about the state of any program data.

In nonblocking mode, execution errors can be deferred.  A return value
of {\sf GrB\_SUCCESS} only guarantees that there are no API errors in
the method invocation.  If an execution error value is returned by a
method in nonblocking mode, it indicates that an error was found during
execution of the sequence, up to and including the {\sf GrB\_wait()}
method (Section~\ref{Sec:GrB_wait}) call that ends the sequence. When possible, that return value
will provide information concerning the cause of the error.

As discussed in Section~\ref{Sec:GrB_waitOne}, a {\sf GrB\_wait(obj)} on
a specific GraphBLAS object {\sf obj} does not necessarily end a
sequence. However, no additional errors on the methods of the sequence that
have {\sf obj} as an {\sf OUT} or {\sf INOUT} argument can be reported.
From a GraphBLAS perspective, those methods are {\em complete}.

If a GraphBLAS method, executing in nonblocking mode, returns with
any execution error from Table~\ref{Tab:ErrorValues}(b) other than
{\sf GrB\_PANIC}, it is guaranteed that no argument used as input-only
through the entire sequence has been modified.  Any output argument in
the sequence may be left in an invalid state and its use downstream in the
program flow may cause additional errors.  If a GraphBLAS method returns
with a {\sf GrB\_PANIC}, no guarantees can be made about the state of
any program data.

\begin{figure}[tb]
    \hrule
    \vspace{1\baselineskip}
    \begin{center}
        \begin{minipage}{3in}
            \begin{verbatim}
            const char *GrB_error();
            \end{verbatim}
        \end{minipage}
    \end{center}
    \caption{Signature of {\sf GrB\_error()} function.}
    \label{Fig:GrB_error}
    \hrule
\end{figure}

After a call to any GraphBLAS method, the program can retrieve additional
error information (beyond the error code returned by the method) though a
call to the function {\sf GrB\_error()}. The signature of that function is
shown in Figure~\ref{Fig:GrB_error}.  The function returns a pointer to a
NULL-terminated string, and the contents of that string are implementation
dependent. In particular, a null string (not a {\sf NULL} pointer) is always a valid error string.
The pointer is valid until the next call to any GraphBLAS method by the same thread.
{\sf GrB\_error()} is a thread-safe function, in the sense that multiple threads can
call it simultaneously and each will get its own error string back, referring to the
last GraphBLAS method it called.


% !TEX root = ../../../proposal.tex
\subsection{TLS decryption with special DROWN}
\label{sec:clear_analysis}

Using our oracle $\OracleSSLclear$, we can construct an extremely efficient version of our TLS decryption attack.  The OpenSSL \tOracleSSLclear provides three significant advantages over our export oracle $\OracleSSLexp$: (1) It no longer requires an export cipher suite, and, in fact, we gain efficiency by exploiting regular SSLv2 ciphers; (2) It requires only one handshake per oracle query; and (3) Computation is reduced to one \texttt{ServerVerify} decryption per oracle query, versus $2^{40}$.

\subsubsection{Attack scenario}

As before, we consider a server that accepts TLS connections, and a client that negotiates a secure, state-of-the-art TLS version with a \texttt{TLS\_RSA} cipher suite.  The same RSA key pair used for TLS is also used on a server that is running a vulnerable version of OpenSSL.

\subsubsection{Constructing the attack}

The attacker can exploit the OpenSSL extra clear vulnerability to efficiently decrypt a TLS ciphertext as follows.  We will use the cipher suite \texttt{SSL\_DES\_192\_EDE3\_CBC\_WITH\_MD5} as the cipher suite, allowing the attacker to recover 24 bytes of key at a time from the oracle.
We first present a straightforward adaptation of the general DROWN attack to the \tOracleSSLclear,
before later applying a few additional optimizations made possible by this new oracle.

\begin{enumerate}
 \setcounter{enumi}{-1}
 \item The attacker intercepts several hundred TLS handshakes using RSA key exchange.
 \item The attacker uses the fractional trimmers as described in Section~\ref{sec:trimmers} to convert the TLS ciphertexts into an \sslconform ciphertext $c_0$.
 \item Once the attacker has obtained a valid \ssltwo ciphertext $c_1$, he repeatedly uses the shifting technique described in Section~\ref{sec:rotations} to rotate the message by 25 bytes each iteration, learning 27 bytes with each shift.  After several iterations, he has learned the entire plaintext.
 \item The attacker then transforms the decrypted \ssltwo plaintext into the decrypted TLS plaintext.
 \end{enumerate}

\paragraph{Attack costs}
Using 40 fractional trimmers, this more efficient oracle attack allows
the attacker to recover one in 260 TLS session keys using only about
17,000 connections to the server.  The computation cost is so low that
we can complete the full attack on a single workstation in under one
minute. \S\ref{sec:special-performance} gives more details.

Mounting the attack using the optimized version of Special DROWN
described in \S\ref{sec:special-performance} allows the
attacker to target one of 100 connections, at the expense of
increasing the number of queries to 27,000.


\documentclass[a4paper]{article}

\usepackage[utf8]{inputenc}
\usepackage{a4wide}
\usepackage{relsize}
\usepackage{indentfirst}
\usepackage{minted}
\usepackage{graphicx}
\usepackage{float}
\usepackage{amsmath, amssymb, amsfonts, amsthm}
\usepackage{hyperref}
\usepackage[capitalise]{cleveref}
\usepackage[toc, page]{appendix}

\AtBeginEnvironment{minted}{\let\itshape\relax}

\begin{document}

\section*{Exercise 1}

See \cref{fig:dhparam}.

\section*{Exercise 2}

See \Cref{fig:dsa-prime,fig:dsa-generator}.

\section*{Exercise 3}

Without the \texttt{dsaparam} option, the generation of DH parameters takes much longer because it
generates a \textit{strong prime}, \textit{i.e.} a prime number $p$ such that $(p-1)/2$ is also prime.
This ensures that the multiplicative group $\mathbb{Z}_p^\ast$ does not contain small subgroups. The
existence of small subgroups within a larger group in a cryptographic protocol is undesirable in the
sense that it confines shared secrets to a smaller set of possible values than if it were to use the
whole group $\mathbb{Z}_p^\ast$.

On the other hand, using the \texttt{dsaparam} DSA rather than DH parameters are read or generate, which
are then they are converted to DH format, making the key exchange process more efficient.

\vspace{\baselineskip}

\textbf{Note:} See \cref{fig:sage-dh,fig:sage-dsa}.

\section*{Exercise 4}

Running the following function with the previously generated DH parameters, we can see that
$X^y \ (\mathrm{mod} \ p) = Y^x \ (\mathrm{mod} \ p)$ holds.

\vspace{\baselineskip}

\begin{minted}{py}
from sage.all import *

def ex4(p, q):
    x = randrange(q)
    y = randrange(q)

    X = Mod(q ** x, p)
    Y = Mod(q ** y, p)

    return Mod(X ** y, p) == Mod(Y ** x, p)
\end{minted}

\vspace{\baselineskip}

\textbf{Note:} See \cref{fig:ex4-dhparam,fig:ex4-dsa}.

\section*{Exercise 5}

We know that $p = 1373, g = 2, X = 974$ and $y = 871$. We can determine $Y$ by computing the
following:

\[
Y = g^y \ \mathrm{mod} \ p = 2^{871} \ \mathrm{mod} \ 1373 = 805
\]

Knowing the value of $X$ and $y$, we can then determine the shared secret $g^{xy}$ by computing
the following:

\[
g^{xy} = (g^x)^y = X ^ y = 974 ^ {871} \ \mathrm{mod} \ 1373 = 397
\]

\pagebreak

Finally, we can find Alice's secret exponent by solving:

\[
X = g^x \ \mathrm{mod} \ p \Leftrightarrow 974 = 2^x \ \mathrm{mod} \ 1373 \Leftrightarrow x = 587
\]

\section*{Exercise 6}

The Computational Diffie-Hellman Problem (CHD) consists in computing the shared secret $g^{ab}$ given
only the public values $g^a$ and $g^b$, and not any of the secret values $a$ or $b$. The motivation
is to ensure that even if an eavesdropper captures $g^a$ and $g^b$, they will not be able to determine
the shared secret $g^{ab}$.

The Decisional Diffie-Hellman Problem (DDH) is stronger than the CDH. To ensure that an attacker can't
learn anything about the shared secret $g^{ab}$, this value needs only to be indistinguishable from a
random group element. This being said, given $g^a, g^b$ and a value that is either $g^{ab}$ or $g^c$
for a random value $c$, each with probability $1/2$, the DDH problem consists of determining whether
$g^{ab}$ was chosen.

An algorithm that solves the Computational Diffie–Hellman problem can be used to solve the Decisional
Diffie–Hellman problem. Given $g^a, g^b$ and $g^c$, and assuming we can solve CDH, we can derive $g^{ab}$
from $g^a$ and $g^b$. Then, to solve DDH, we only have to check if the result equals $g^c$.

\pagebreak

\begin{appendices}

\section*{Exercise 1}

\begin{figure}[H]
    \centering
    \includegraphics[width=.9\textwidth]{img/dhparam.png}
    \caption{Generation of DH parameters without the \texttt{dsaparam} option}
    \label{fig:dhparam}
\end{figure}

\section*{Exercise 2}

\begin{figure}[H]
    \centering
    \includegraphics[width=.8\textwidth]{img/dsa-prime.png}
    \caption{Prime number generated using DH parameter generation with the \texttt{dsaparam} option}
    \label{fig:dsa-prime}
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[width=.8\textwidth]{img/dsa-generator.png}
    \caption{Group generator generated using DH parameter generation with the \texttt{dsaparam} option}
    \label{fig:dsa-generator}
\end{figure}

\section*{Exercise 3}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{img/dh-sage.png}
    \caption{Prime number generated using DH parameter generation without the \texttt{dsaparam} option}
    \label{fig:sage-dh}
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{img/dsa-sage.png}
    \caption{Prime number generated using DH parameter generation with the \texttt{dsaparam} option}
    \label{fig:sage-dsa}
\end{figure}

\section*{Exercise 4}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{img/dh-py.png}
    \caption{Checking that $X^y \ (\mathrm{mod} \ p) = Y^x \ (\mathrm{mod} \ p)$ holds for the DH parameters generated without the \texttt{dsaparam} option}
    \label{fig:ex4-dhparam}
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{img/dsa-py.png}
    \caption{Checking that $X^y \ (\mathrm{mod} \ p) = Y^x \ (\mathrm{mod} \ p)$ holds for the DH parameters generated with the \texttt{dsaparam} option}
    \label{fig:ex4-dsa}
\end{figure}

\end{appendices}

\end{document}


\documentclass[12pt]{article}
    \usepackage{mathtools}
    \usepackage[hidelinks]{hyperref}
    \usepackage{color}
    \usepackage{fancyref}
    \usepackage{lastpage}
    \usepackage{fancyhdr}

    \usepackage{pagecolor}
    % \pagecolor{black}
    % \color{white}

    \pagestyle{fancy}
    \fancyhf{}
    \fancyhead[LO]{TMA4140: Homework set 7}
    \fancyhead[RO]{Henry S. Sjøen \& Toralf Tokheim}
    \fancyfoot[CO]{\thepage\ of \pageref{LastPage}}

    \definecolor{darkred}{RGB}{200, 0, 0}

  \author{Henry S. Sjøen \& Toralf Tokheim}
  \title{
  \textbf{TMA4140 - Homework Exercise Set 7}}
\begin{document}
    \maketitle
    \thispagestyle{empty}
    \pagebreak
    \tableofcontents
    \pagebreak

    \section{Section 8.1}
    % Obligatoriske: 11, 20;
    % Anbefalte: 19, 21.
    \subsection{Exercise 11}
    \textbf{a) Find a recurrence relation for the number of ways to climb n stairs if the person climbing the stairs can take one stair or two stairs at a time.}\\
    $ a_n=a_{n-1}+a_{n-2} $ when $n > 2$\\
    \textbf{b) What are the initial conditions?}\\
    % Only one way to reach "The zero-step", is not to move, witch also is a move. $a_0 = 1$\\
    To reach the first step there is only one solution, taking one single step $a_1=1$\\
    To reach the second step, we can take either two single steps or one double step. $a_2=2$\\
    \textbf{c) In how many ways can this person climb a flight of eight stairs?}\\
    A lot...
    \begin{equation}
        \begin{split}
            % 8 enere
            % 0 toer
            1+1+1+1+1+1+1+1=8\\
            % 6 enere
            % 1 toer
            % Gir oss 7 bokser
            % og også 7 permutasjoner
            2+1+1+1+1+1+1=8\\
            1+2+1+1+1+1+1=8\\
            1+1+2+1+1+1+1=8\\
            1+1+1+2+1+1+1=8\\
            1+1+1+1+2+1+1=8\\
            1+1+1+1+1+2+1=8\\
            1+1+1+1+1+1+2=8\\
            % 4 enere
            % 2 toere
            % Gir oss 6 bokser
            2+2+1+1+1+1=8\\
            2+1+2+1+1+1=8\\
            2+1+1+2+1+1=8\\
            2+1+1+1+2+1=8\\
            2+1+1+1+1+2=8\\
            %
            1+2+2+1+1+1=8\\
            1+2+1+2+1+1=8\\
            1+2+1+1+2+1=8\\
            1+2+1+1+1+2=8\\
            %
            %
            1+1+2+2+1+1=8\\
            1+1+2+1+2+1=8\\
            1+1+2+1+1+2=8\\
            %
            1+1+1+2+2+1=8\\
            1+1+1+2+1+2=8\\
            %
            1+1+1+1+2+2=8\\
            %
            % 2 enere
            % 3 toere
            2+2+2+1+1=8\\
            2+2+1+2+1=8\\
            2+2+1+1+2=8\\
            2+1+2+1+2=8\\
            2+1+1+2+2=8\\
            1+2+2+2+1=8\\
            1+2+2+1+2=8\\
            1+2+1+2+2=8\\
            1+1+2+2+2=8\\
            % 0 enere
            % 4 toere
            2+2+2+2=8\\
        \end{split}
    \end{equation}

    I missed one above, but easier to calculate it...
    $ a_n=a_{n-1}+a_{n-2} $ when $n > 2$\\

    \begin{equation}
        \begin{split}
            a_n&=a_{n-1}+a_{n-2} | n>2\\
            % a_0&=1,
            a_1&=1,a_2=2\\
            % a_2&=a_{2-1}+a_{2-2}=a_{1}+a_{0}=1+1=2\\
            a_3&=a_{3-1}+a_{3-2}=a_{2}+a_{1}=2+1=3\\
            a_4&=a_{4-1}+a_{4-2}=a_{3}+a_{2}=3+2=5\\
            a_5&=a_{5-1}+a_{5-2}=a_{4}+a_{3}=5+3=8\\
            a_6&=a_{6-1}+a_{6-2}=a_{5}+a_{4}=8+5=13\\
            a_7&=a_{7-1}+a_{7-2}=a_{6}+a_{5}=13+8=21\\
            a_8&=a_{8-1}+a_{8-2}=a_{7}+a_{6}=21+13=34
        \end{split}
    \end{equation}

    There are 34 possible ways to climb the flight of $8$ stairs using only $1$ or $2$ steps at a time.

    \subsection{Exercise 20}
    % https://www.chegg.com/homework-help/bus-driver-pays-tolls-using-nickels-dimes-throwing-one-coin-chapter-8.1-problem-20e-solution-9780073383095-exc
    A bus driver pays all tolls, using only nickels and dimes, by throwing one coin at a time into the mechanical toll collector.\\
    \textbf{a)} Find a recurrence relation for the number of different ways the bus driver can pay a toll of n cents (where the order in which the coins are used matters).\\
    1 Nickel =  5 cents, 1 Dime = 10 cents.\\
    Let $a_n$ be the number of ways the busdriver can pay a toll of $n$ cents.\\
    If last coin Nickel $a_{n-5}$, If last coin Dime $a_{n-10}$\\
    Then the recurrence relation can be given as...
    $a_n=a_{n-5}+a_{n-10} | n \geq 10$
    And since nickels and dimes are of multiple of 5 we can further write it as ...
    $a_{5n}=a_{5(n-1)} + a_{5(n-2)} | n \geq 2$.
    Where the initial conditions are $a_0=1; a_5=1;$
    \textbf{b)} In how many different ways can the driver pay a toll of 45 cents?
    We must calculate $a_{45}$
    \begin{equation}
        \begin{split}
            a_{5n}&=a_{5(n-1)} + a_{5(n-2)}\\
            a_0&=1; a_5=1\\
            a_{10}&=2\\
            a_{15}&=3\\
            a_{20}&=5\\
            a_{25}&=8\\
            a_{30}&=13\\
            a_{35}&=21\\
            a_{40}&=34\\
            a_{45}&=55
        \end{split}
    \end{equation}

    \section{Section 8.2}
    % Obligatoriske: 3c,d,e,g, 6, 11, 42;
    % Anbefalte: 40.

    \subsection{Exercise 3} %todo
    % https://www.chegg.com/homework-help/bus-driver-pays-tolls-using-nickels-dimes-throwing-one-coin-chapter-8.2-problem-3E-solution-9780073383095-exc
    Solve these recurrence relations together with the initial conditions given.\\
    \textbf{c)} $ a_n=5a_{n-1} - 6a_{n-2} $ for $ n \geq 2, a_0=1, a_1=0 $\\
    Comparing the given recurrence with the general relation we get $C_1=5,c_2=-6$ and rest of the coefficients are $0$.
    Our characteristic equation will be $r^2=5r^1-6r^0$, so...
    \begin{equation}
        \begin{split}
            r^2-5r^1+6r^0=0\\
            r^2-5r+6r=0\\
            r^2-3r-2r+6=0\\
            (r-3)(r-2)=0
        \end{split}
    \end{equation}
    That is, $r=3,2$.
    Solution will be of the form $a_n=a_1r_2^n$. Setting in the value of $r$ and use the initial condition.
    $n=0,1$...\\
    \begin{equation}
        \begin{split}
            a_n=a_1r_1^n\\
            a_n=a_1(2)^n+a_2(3)^n
        \end{split}
    \end{equation}
    When $n=0$, then...
    \begin{equation}
        \begin{split}
            a_0=a_1(2)^0+a_2(3)^0\\
            1=a_1+a_2\\
            a_1=1-a_2
        \end{split}
    \end{equation}
    When $n=1$, then...
    \begin{equation}
        \begin{split}
            a_1=a_1(2)^1+a_2(3)^1\\
            0=2a_1+3a_2
        \end{split}
    \end{equation}
    Put the value of $a_1=1-a_2$ in the equation $2a_1+3a_2=0$...
    \begin{equation}
        \begin{split}
            2(1-a_2)+3a_2=0\\
            2-2a_2+3a_2=0\\
            a_2=-2
        \end{split}
    \end{equation}
    Put the value of $a_2=-2$ in the equation $a_1=1-a_2$ and we get $a_1=3$.
    Put the value of $a_1,a_2$ in the equation $a_n=a_1r_1^n+a_2r_2^n$ to get the final solution $a_n=3 \times (2)^n-2 \times (3)^n$.

    \textbf{d)} $ a_n=4a_{n-1} - 4a_{n-2} $ for $ n \geq 2, a_0=6, a_1=8 $\\
    Comparing the given recurrence relation with the general relation we get $c_1=4$ $c_2=-4$ and the rest of the coefficients are 0.
    The corresponding characteristic equation will be $r^2=4r^1-4r^0$. So,
    \begin{equation}
        \begin{split}
            r^2-4r+4=0\\
            r^2-2r-2r+4=0\\
            (r-2)(r-2)=0
        \end{split}
    \end{equation}
    Since we have a repeated root, the solution will be of the form $a_n=a_1r_1^n+a_2nr_2^n$.
    Putting the value of $r$ and useing the initial conditions $n=0,1$ and get...
    \begin{equation}
        \begin{split}
            a_n=a_1r_1^n+a_2nr_2^n\\
            a_n=a_1(2)^n+a_2n(2)^n
        \end{split}
    \end{equation}
    when $n=0$...
    \begin{equation}
        \begin{split}
            a_0=a_1(2)^0+a_20(2)^0\\
            6=a_1
        \end{split}
    \end{equation}
When $n=1$...

\begin{equation}
    \begin{split}
        a_1=a_1(2)^1+a_21(2)^1\\
        8=2a_1+2a_2
    \end{split}
\end{equation}
Put the value of $a_1=6$ in the equation $8=2a_1+2a_2$ to get...
\begin{equation}
    \begin{split}
        2a_1+2a_2=8\\
        12+2a_2=8\\
        2a_2=8-12\\
        a_2=-2
    \end{split}
\end{equation}
Put the value of $a_1,a_2$ in the equation $a_n=a_1r_1^n+a_2nr_2^n$ to get the desired solution.
Thus, the solution is $a_n=6\cdot (2)^n-2\cdot n \cdot (2)^n$.
    \textbf{e)} $ a_n=-4a_{n-1} - 4a_{n-2} $ for $ n \geq 2, a_0=0, a_1=1 $\\
    \textbf{g)} $ a_n=\frac{a_{n-2}}{4} $ for $ n \geq 2, a_0=1, a_1=0 $
    \subsection{Exercise 6} %todo
    % https://www.chegg.com/homework-help/bus-driver-pays-tolls-using-nickels-dimes-throwing-one-coin-chapter-8.2-problem-6E-solution-9780073383095-exc
    How many different messages can be transmitted in $n$ microseconds using three different signals if one signal requires $1$ microsecond for transmittal, the other two signals require $2$ microseconds each for transmittal, and a signal in a message is followed immediately by the next signal?

    \subsection{Exercise 11}
    % https://www.chegg.com/homework-help/bus-driver-pays-tolls-using-nickels-dimes-throwing-one-coin-chapter-8.2-problem-11E-solution-9780073383095-exc
    The Lucas numbers satisfy the recurrence relation
    \begin{equation}
        L_n=L_{n-1}+L_{n-2}
    \end{equation}
    and the initial conditions $L_0 = 2$ and $L_1=1$.

    \textbf{a)} Show that $L_n = f_{N-1}+f_{n+1}$ for $n=2,3,...,$ where $f_n$ is the $n$th Fibonacci number.\\
    \textbf{b)} Find an explicit formula for the Lucas numbers.
Answer:
     $Ln = ((1+sqr(5))/2)+((1-sqr(5))/(2))$

    \subsection{Exercise 42} %todo
    % https://www.chegg.com/homework-help/bus-driver-pays-tolls-using-nickels-dimes-throwing-one-coin-chapter-8.2-problem-42E-solution-9780073383095-exc
    Show that if $a_n =a_{n-1} + a_{n-2}, a_0=s$ and $a1 =t$, where $s$ and $t$ are constants, then $a_n = sf_{n-1} + tf_n$ for all positive integers $n$.

    \section{Section 5.1}
    % Obligatoriske: 4, 6, 14, (4, 6, 14);
    % Anbefalte: 9, 10.

    \subsection{Exercise 4}
    % https://www.chegg.com/homework-help/bus-driver-pays-tolls-using-nickels-dimes-throwing-one-coin-chapter-5.1-problem-4E-solution-9780073383095-exc
    Let $P(n)$ be the statement that $1^3 + 2^3 + \cdots + n^3 = (n(n + 1)/2)^2$ for the positive integer $n$.\\
%     \textbf{a) What is the statement $P(1)$?}\\
%     \textbf{b) Show that $P(1)$ is $true$, completing the basis step of the proof.}\\
%     \textbf{c) What is the inductive hypothesis?}\\
%     \textbf{d) What do you need to prove in the inductive step?}\\
%    \textbf{ e) Complete the inductive step, identifying where you use the inductive hypothesis.}\\
%     \textbf{f) Explain why these steps show that this formula is $true$ whenever $n$ is a positive integer.}
    %
    % \begin{enumerate}[label=\alph*)]

        $P(1): 1^3 = (\frac{1(1+1)}{2})^2$
        Basis step:
            $P(1): 1^3 = (\frac{1(1+1)}{2})^2 = 1$ \\
            $P(1)$ is true, which completes the basis step of a proof by
                induction for $P(k)$

        The inductive hypothesis consists of two parts: \\
            - $P(b)$ holds true \\
            - $P(k) \rightarrow P(k+1)$ holds true\\
            Then $P(k), \quad \forall k > b$

            In other words, if $P$ is true for the first step, and $P$ holds true for an arbitrary step implies $P$ holds true for the next step, then $P$ holds true for all steps. \\

        You need to prove the first step $P(b)$, and then you need to prove
            $P(k) \rightarrow P(k+1)$

                    \begin{align}
                        \intertext{Basis step:}
                        1^3 = (1(1+1)/2)^2 = 1 \\
                        \intertext{LHS = RHS}
                        \intertext{Inductive step:}
                        \sum_{n=1}^k{n^3} = \Big(\frac{k(k+1)}{2}\Big)^2 \\ %\label{eq:4e_k}
                        \intertext{We assume $P(k)$ is true for an arbitrary integer $k$. We can replace $k$ with $k+1$. Out goal is to show that if $P(k)$ holds then $P(k+1)$ must hold}
                        \sum_{n=1}^{k+1}{n^3} = \Big(\frac{(k+1)((k+1)+1)}{2}\Big)^2 \\
                        \sum_{n=1}^{k}{n^3} + (k+1)^3 = \Big(\frac{(k+1)(k+2)}{2}\Big)^2 \\
                        = \Big(\frac{(k^2+3k+2)}{2}\Big)^2 \\
                        = \Big(\frac{(k(k+1))}{2} + (k+1)\Big)^2 \\
                        \sum_{n=1}^{k}{n^3} + (k+1)^3 = \Big(\frac{(k(k+1))}{2}\Big)^2 + k(k+1)(k+1) + (k+1)^2 \\
                        \intertext{We subtract  $\sum_{n=1}^k{n^3} = \Big(\frac{k(k+1)}{2}\Big)^2$ from the equation and have}
                        (k+1)^3 = k(k+1)^2 + (k+1)^2 \\
                        (k+1)^3 = (k+1)(k+1)^2 = (k+1)^3 \\
                        \intertext{LHS = RHS. This means that if $P(k)$ holds true,
                        then $P(k+1)$ must also be true, which by the inductive
                        hypothesis means that $\forall k \geq 1, \quad P(k)$ holds true}
                    \end{align}

    \subsection{Exercise 6}
    % https://www.chegg.com/homework-help/bus-driver-pays-tolls-using-nickels-dimes-throwing-one-coin-chapter-5.1-problem-6E-solution-9780073383095-exc
    Prove that $s1 \cdot 1!+2 \cdot 2! + \cdots + n \cdot n! = (n+1)!-1$ whenever $n$ is a positive integer.
    \begin{align}
        \intertext{Basis step:}
        1\cdot 1\! = (1 + 1)! - 1 \\
        1 = (2)! - 1 = 2 - 1 = 1\\
        \intertext{LHS = RHS, so $P(1)$ holds true}
        \intertext{Inductive step:}
        \sum_{n = 1}^k{n\cdot n!} = (k + 1)! - 1 \\ %\label{eq:1-6-k}
        \intertext{We assume that $P(k)$ holds for all $k > 1$, and replace
            $k$ with $k+1$}
        \sum_{n = 1}^{k+1}{n\cdot n!} = ((k+1) + 1)! - 1 \\
        (k+1)\cdot(k+1)! + \sum_{n = 1}^{k}{n\cdot n!}  = (k+2)! - 1 \\
        (k+1)\cdot(k+1)! + \sum_{n = 1}^{k}{n\cdot n!}  = (k+2)(k+1)! - 1 \\
        \intertext{We subtract $\sum_{n = 1}^k{n\cdot n!} = (k + 1)! - 1$ from the equation and get}
        (k+1)\cdot(k+1)! +  = (k+2)(k+1)! - 1 - \big( (k+1)! - 1\big)\\
        (k+1)\cdot(k+1)! +  = (k+2)(k+1)!  -  (k+1)! \\
        (k+1)\cdot(k+1)! +  = k(k+1)! + 2(k+1)! - (k+1)! \\
        (k+1)\cdot(k+1)! +  = k(k+1)! + (k+1)!  \\
        (k+1)\cdot(k+1)! +  = (k+1)\cdot(k+1)!  \\
        \intertext{LHS = RHS, so $P(k)$ must hold for all $k > 1$}
    \end{align}

    \subsection{Exercise 14}
    % https://www.chegg.com/homework-help/bus-driver-pays-tolls-using-nickels-dimes-throwing-one-coin-chapter-5.1-problem-14E-solution-9780073383095-exc
    Prove that for every positive integer $n$, $\sum_{k=1}^{n}k2^k=(n-1)2^{n+1}+2$.
            \begin{align}
                \intertext{$P(k)$:}
                \sum_{n=1}^k{ n2^n} = (k-1)2^{k+1} + 2 %\label{eq:1-14-k}\
                \intertext{Basis step:}
                1\cdot 2^1 = (1-1)2^2{1+1} + 2 = 2\
                \intertext{LHS = RHS, so $P(1)$ holds true}
                \intertext{Induction step:}
                \intertext{Replacing $n$ with $k+1$ in $\sum_{n=1}^k{ n2^n} = (k-1)2^{k+1} + 2$ gives us}
                \sum_{n=1}^{k+1}{ n2^n} = ((k+1)-1)2^{(k+1)+1} + 2 \\
                (k+1)2^{k+1} + \sum_{n=1}^{k}{ n2^n} = k2^{k+2} + 2 \\
                (k+1)2^{k+1} + \sum_{n=1}^{k}{ n2^n} = 2k2^{k+1} + 2 \\
                \intertext{We subtract $\sum_{n=1}^k{ n2^n} = (k-1)2^{k+1} + 2$ from the equation and get}
                (k+1)2^{k+1} = 2k2^{k+1} + 2 - \big((k-1)2^{k+1} + 2\big) \\
                (k+1)2^{k+1} = 2k2^{k+1} - (k-1)2^{k+1} \\
                (k+1)2^{k+1} = (2k - (k-1)) 2^{k+1}\\
                (k+1)2^{k+1} = (k+1)2^{k+1}\\
                \intertext{LHS = RHS, so $P(k)$ must hold for all $k > 1$}
            \end{align}
\end{document}


% !TeX encoding = UTF-8
% !TeX program = pdflatex

\documentclass[11pt]{article}
\usepackage[english]{babel}
\usepackage{graphicx}
\usepackage{listings}
\usepackage[dvipsnames]{xcolor}
\usepackage{lineno}
\lstloadlanguages{Python}
\lstset{%
	basicstyle=\fontsize{10}{11}\ttfamily\color{black},
	commentstyle=\ttfamily\color{red},
	keywordstyle=\ttfamily\color{blue},
	stringstyle=\color{orange},
	tabsize=2,
	numbers=left,
	numberstyle=\tiny,
	firstnumber=1,
	numberfirstline=false,
	frame=single,
	showstringspaces=false,
	inputencoding=utf8,
	breaklines=true
}
\usepackage{amsmath}
\usepackage{algorithm}
\usepackage[noend]{algpseudocode}
\makeatletter
\def\BState{\State\hskip-\ALG@thistlm}
\makeatother

\title{Homework 1 - Algorithm Design \\ \bigskip \large Sapienza University of Rome}
\date{\today}
\author{Marco Costa 1691388}

\begin{document}
\maketitle

\section{Exercise 1}
The goal of this exercise is to cluster the points of \textbf{X} using the first \textit{\textbf{k}} points \textbf{C = $\mathbf{\{ \pi(X)_{1},...,\pi(X)_{k} \}}$} as center (where $\mathit{\pi(X)}$ is a permutation). For any \textit{\textbf{k}}, the output should be optimal with respect of the objective function $\mathbf{\displaystyle\max_{x \in X} \min_{c \in C} d(x,c)}$. \\
This problem is known as k-center clustering problem, and it's NP-HARD. \\
Fortunately, it is possible to find an approximation using a greedy algorithm. \\
The greedy clustering algorithm simply chooses the point farthest away from the current set of centers in each iteration as the new center. It can be described in this way:
\begin{itemize}
	\item Pick an arbitrary point $\mathbf{c_{i}}$ into $\mathbf{C}$;
	\item For every point $\mathbf{x \in X}$ compute $\mathbf{d_{i}(x,c_{i})}$;
	\item Pick the point $\mathbf{c_{i}}$ with the highest distance from $\mathbf{c_{i-1}}$ ($\displaystyle\max_{x \in X} d(x, c_{i})$).
	In this case, distance must be 3. If there are no points that have the distance 3, then the distance must be 1;
	\item Add it to the set of centers $\mathbf{C}$;
	\item Continue till \textit{\textbf{k}} centers are found (\textit{i=1,...,k}).
\end{itemize}

\subsection{Analysis of the algorithm}
\begin{itemize}
	\item The $i^{th}$ iteration of choosing the $i^{th}$ center takes $\mathbf{O(|X|)}$ time.
	\item There are  \textit{\textbf{k }} iterations, so overall the algorithm takes $\mathbf{O(k|X|)}$ time.
\end{itemize}

\subsection{Proof}
The correctness of this algorithm can be proved by contradiction.\\
Assume that $\mathbf{O^{*} = \{c^{∗}_{1},...,c^{∗}_{k}\}}$ is an optimal solution for the problem and $\mathbf{S = \{c_{1},...,c_{k}\}}$ the solution given by the greedy algorithm. If $\mathbf{S \ne O^{*}}$ so there is at
least an optimum center $\mathbf{\bar{c}}$
which belongs to $\mathbf{O^{*}}$ but not to $\mathbf{S}$. This means that, if this point was selected at the $\mathbf{(i-1)^{th}}$ iteration of the optimum
algorithm because it has distance 3 from the center, so the greedy algorithm has
not selected it as the optimum value to satisfy the objective function. But
this is not possible, it first insert into the partition all
the points of the space which have distance equals to 3 from the center
and after the points with distance equals to 1 if needed.

\subsection{2-approximation}
The solution obtained using the greedy algorithm (the set $\mathbf{C}$) is a 2-approximation to the optimal solution ($\mathbf{r^{C}_{\infty}(X) \le 2r^{opt}_{\infty}}(X,k)$).

\subsubsection{Proof}
Case 1: Every cluster of $\mathbf{C_{opt}}$ contains exactly one point of $\mathbf{C}$.
\begin{itemize}
	\item Consider a point $\mathbf{x \in X}$;
	\item Let $\mathbf{\bar{c}}$ be the center it belongs to in $\mathbf{C_{opt}}$;
	\item Let $\mathbf{\bar{k}}$ be the center of $\mathbf{C}$ that is in $\mathbf{\Pi(C_{opt},\bar{c})}$;
	\item $\mathbf{d(x,\bar{c}) = d(x,C_{opt}) \le r^{opt}_{\infty}(X,k)}$;
	\item Similarly, $\mathbf{d(\bar{k},\bar{c}) = d(\bar{k},C_{opt}) \le r^{opt}_{\infty}(X,k)}$;
	\item By the triangle inequality: $\mathbf{d(x,\bar{k}) \le d(x,\bar{c}) + d(\bar{c},\bar{k}) \le 2r^{opt}_{\infty}(X,k)}$	.
\end{itemize}

Case 2: There are two centers $\mathbf{\bar{k}}$ and $\mathbf{\bar{u}}$ of $\mathbf{C}$ that is in $\mathbf{\Pi(C_{opt},\bar{c})}$, for some $\mathbf{\bar{c} \in C_{opt}}$
\begin{itemize}
	\item Assume, without loss of generality, that $\mathbf{\bar{u}}$ was added later to the center set $\mathbf{C}$ by the greedy algorithm in the $\mathbf{i^{th}}$ iteration;
	\item But since the greedy algorithm always chooses the point furthest away from the current set of centers, we have that $\mathbf{\bar{c} \in C_{i-1}}$, and \\ $\mathbf{r^{C}_{\infty}(X) \le r^{C_{i-1}}_{\infty}(X) = d(\bar{u},C_{i-1}) \le d(\bar{u},\bar{k}) \le d(\bar{u},\bar{c}) + d(\bar{c},\bar{k}) \le 2r^{opt}_{\infty}(X,k)}$.
\end{itemize}

\section{Exercise 2}
It's possible to solve this problem using a bipartite graph \textbf{G(S,A,C)}, where \textbf{S} is the set containing streets, \textbf{A} is the set containing avenues and \textbf{C} is the set containing checkpoints (so a checkpoint is an edge that connects a street in \textbf{S} with an avenue in \textbf{A}).
The goal is to find a vertex set $\mathbf{T \subseteq S \bigcup A}$ of minimum size.
So the problem is a minimum vertex cover problem
(because we want to find a set of vertices such that every edge of the graph
has at least one endpoint in the set). It's possible to find \textbf{T} with:
\begin{itemize}
	\item Ford-Fulkerson’s algorithm to find the maximum matching;
	\item Konig's theorem, which states that in any bipartite graph, the number of edges in a maximum matching is equal to the number of vertices in a minimum vertex cover.
\end{itemize}
Let’s consider \textbf{M}, a maximum cardinality matching in \textbf{G}. A matching is an edge set $\mathbf{M \subseteq E}$ such that no two edges of M have the same endpoint. Konig's theorem proves that the size of a
maximum matching M is equal to the size of the minimum cardinality vertex cover \textbf{T}. \\
It's possible to find the maximum matching set \textbf{M} applying Ford-Fulkerson algorithm in the bipartite flow network \textbf{G(S,A,C)}.
\\
Now it's possible to derive from the maximum alternating forest, built starting
from non matched vertices in \textbf{S}, the minimum vertex cover set \textbf{T} using Konig's theorem.\\
This set represent the minimum
set of streets and avenues in which it's necessary to place cameras for covering checkpoints.

\subsection{Analysis}
The cost of this algorithm is dominated by the finding of the maximum
flow (the cost of Ford-Fulkerson’s algorithm). So the total cost is \textbf{O(\textit{f}$|C|$)}, where \textit{\textbf{f}} is the maximum flow found and \textbf{C} is the set of the checkpoints.

\section{Exercise 3}
It's possible to solve this problem considering a complete graph \textbf{G(P,R)} where $\mathbf{P = M \bigcup F}$ is the set of friends (\textbf{M} is the set of male friends and \textbf{F} is the set of female friends) and R is the set of their relations (the edges of the graph). \\
Each  edge
has weight equals to 0 if the two friends don’t like each other, 1 otherwise. \\
The goal is to find a subset I of people who
like each other and who are equally bipartite (so equal number of male and
female friends). \\
The first point of the problem can be solved considering it as a k-clique problem. The algorithm proceed in this way:
\begin{enumerate}
	\item For each $\mathbf{\textbf{\textit{k}} \in \{2,|V|\}}$ it computes the k-clique and stores the cliques with \textbf{\textit{k}} edges in a set \textbf{S};
	\item For each clique in \textbf{S} it computes the maximum bipartite matching in order to satisfy the second point (equal number of male and female);
	\item For each value of \textbf{\textit{k}} it finds the maximum bipartite matching $\mathbf{M_{k}}$ which maximizes the value \\
	$\mathbf{maxVal_{k} = \frac{1}{|M_{k}}} \sum_{(u,v) \in M_{k}}weight(u,v)$;
	\item At the end it finds the maximum value between the various $\mathbf{maxVal_{k}}$ with $\mathbf{\textbf{\textit{k}} \in \{2,|V|\}}$ and returns the maximum matching corresponding to this value.
\end{enumerate}

\subsubsection{Implementation (pseudocode)}
\begin{algorithm}
	\label{euclid}
	\begin{algorithmic}[1]
		\State $V \gets vertices$
		\State $E \gets edges$
		\State $G \gets graph$
		\State $maxD[|V|] \gets [0]$
		\State $maxI[k] \gets$ [ ]
		\For{$k \gets$ 0 to $|V|$}
		\State $S \gets$ find-all k-cliques()
		\For{s in S}
		\State I $\gets$ bipartite matching of s
		\If{sum weight(s)/$|I| >$ maxD[k]}
		\State $maxD[k] \gets weight(s)/|I|$
		\State $maxI[k] \gets I$
		\EndIf
		\EndFor
		\EndFor
		\State $index \gets max(maxD)$
		\State\Return maxI[index]
	\end{algorithmic}
\end{algorithm}

\subsection{Proof}
This algorithm uses the k-clique problem, which is NP-COMPLETE, in fact, the reduction of this problem is polynomial. For this reason the implemented algorithm is NP-COMPLETE too.

\section{Exercise 4}
The input of the algorithm is the following:
\begin{itemize}
	\item costs: a list of tuple where the first element is the timestamp of the task, the second one is the cost of outsourcing;
	\item C: hiring cost;
	\item S: severance cost;
	\item s: daily salary.
\end{itemize}
For first, the algorithm computes how many tasks with different timestamp there are. In this way it's possible to overlap tasks.
After that, it joins the overlapping tasks generating a matrix in which there are the timestamps in the first row and the outsourcing costs in the second one.
Then it generates a total cost matrix and initializes the first column, calculates the lowest cost for each instant using the costs of the previous instant and at the end returns the minimum total cost.

\subsection{Analysis}
The cost of the algorithm depends on iterative loops so it is linear: O(T), where T is the last
time unit to consider. \\

\subsection{Proof}
To prove the correctness of the algorithm we could use proof by induction.
We consider as basis step the first two couple of values inside total cost matrix , at this moment i = 0
and the min of them is the min cost so far. As induction hypothesis we suppose that for i = k the
cost computed as min of the couple of values inside total cost matrix is optimal. The inductive step is to
prove that also for i = k + 1 the cost is optimal.

\subsection{Implementation}
The algorithm is implemented in \textit{\textbf{Python}}.
\begin{lstlisting}[language=Python, caption=minCost]
def minCost(costs, C, S, s):

	# Determine the number of different tasks
	T = 0
	time = 0
	for cost in costs:
		if cost[0] != time:
			time = cost[0]
			T += 1

	# Initialize outsourcing cost matrix
	outsource = [[0 for x in range(0, T+1)] for x in range(0, 2)]

	# Initialize total cost matrix
	tc = [[0 for x in range(0, T+1)] for x in range(0, 2)]

	# Construck outsourcing cost matrix joining different tasks with the same timestamp
	time = 0
	i = 0
	j = 0
	for cost in costs:
		if j == 0:
			outsource[0][i] = cost[0]
			outsource[1][i] = cost[1]
			time = cost[0]
			j += 1
		else:
			if time == cost[0]:
				outsource[1][i] += cost[1]
			else:
				i += 1
				outsource[0][i] = cost[0]
				outsource[1][i] = cost[1]
				time = cost[0]

	# Initialize first column of total cost matrix
	tc[0][0] = s + C
	tc[1][0] = outsource[1][0]

	# Construct rest of the total cost matrix
	for i in range(1, T):
		tc[0][i] = min(tc[0][i-1] + (outsource[0][i] - outsource[0][i-1]) * s, tc[1][i-1] + C + s)
		tc[1][i] = min(tc[0][i-1] + S + outsource[1][i], tc[1][i-1] + outsource[1][i])

	# Return the minimum cost
	return min(tc[0][T-1], tc[1][T-1])
\end{lstlisting}

\section{Exercise 5}
\subsection{First part}
The first part of the exercise can be solved using the Minimum Spanning Tree cycle property: \\
\textit{For any cycle \textbf{C} in the graph,
if the weight of an edge \textbf{e} of \textbf{C} is larger than the weights of all other
edges of \textbf{C}, then e cannot belong to the MST}.
\\
The algorithm is based on a \textbf{DFS}. It determine if the
two endpoints of \textit{\textbf{e}} are connected with a cycle respecting the property
mentioned before. \\
How it works:
\begin{enumerate}
	\item Run the DFS from one of the end-points of the
input edge \textit{\textbf{e}} (\textbf{u} or \textbf{v} for the edge \textit{\textbf{(u, v, weight)}}) considering only outgoing edges which
have weight less than the one of \textit{\textbf{e}};
	\item \begin{itemize}
		\item Case 1: If at the end of the \textbf{DFS}, the two vertices \textbf{u} and \textbf{v}
get connected, then the edge \textit{\textbf{e}} can't be part of any \textbf{MST}
because of the cycle property, in fact in this case there exists
a cycle in the graph \textbf{G} containing the edge \textit{\textbf{e}} where \textit{\textbf{e}} is the
edge with maximum weight.
		\item Case 2: If at the end of the \textbf{DFS} \textbf{u} and
\textbf{v} stay disconnected, then there is not a cycle between \textbf{u} and
\textbf{v} where \textit{\textbf{e}} is the edge with maximum weight, so the edge
considered can be part of the \textbf{MST}.
	\end{itemize}
\end{enumerate}
The cost of this algorithm is dominated from the one of the \textbf{DFS}, which
is \textbf{O($|V| + |E|$)}.

\subsection{Second part}
For this second part, the algorithm described before was used to check if
the given edge \textit{\textbf{e}} could be part of a Minimum Spanning Tree. After
checking the previous condition, Kruskal’s algorithm was used to compute
the \textbf{MST} containing \textit{\textbf{e}}: the cost of this algorithm is
\textbf{O($|E|log|E|$)}.

\subsubsection{Implementation}
The algorithm is implemented in \textit{\textbf{Python}}.
\begin{lstlisting}[language=Python, caption=MST given an edge]
def give_a_mst(self, edge):
	if self.edge_in_mst(edge):
		print("The edge",str(edge),"belongs to MST!")
		print("Minimum Spanning Tree:", end="")
		return self.kruskal()
	else:
		print("The edge",str(edge),"doesn't belong to MST!")
		return None
\end{lstlisting}
This algorithm is very symple: \\
It checks if the edge is contained in the MST using the algorithm described in the first part.\\
If the result is positive, then the Kruskal's algorithm is executed and the MST is output, otherwise nothing is returned.

\begin{lstlisting}[language=Python, caption=Kruskal's algorithm]
def kruskal(self):
	parent = dict()
	rank = dict()

	for vertex in self.vertex:
		parent[vertex] = vertex
		rank[vertex] = 0

	def find(vertex):
		if parent[vertex] != vertex:
			parent[vertex] = find(parent[vertex])
		return parent[vertex]

	def union(vertex1, vertex2):
		root1 = find(vertex1)
		root2 = find(vertex2)
		if rank[root1] < rank[root2]:
			parent[root1] = root2
		elif rank[root1] > rank[root2]:
			parent[root2] = root1
		else:
			parent[root2] = root1
			rank[root2] += 1

	mst = Graph("directed")
	minimum_spanning_tree = set()
	edges = self.sorted_by_weight()
	for edge in edges:
		v1, v2, w = edge
		if find(v1) != find(v2):
			union(v1, v2)
			minimum_spanning_tree.add(edge)
	for node in self.get_nodes():
		mst.add_node(node)
	for edge in minimum_spanning_tree:
		mst.add_edge(edge)
	return mst
\end{lstlisting}

\appendix
\section{Code}
The whole implementation of the exercises are attached in the archive \\ \textit{HW1-AD\_Marco\_Costa\_1691388.tar.gz}

\end{document}


\documentclass[12pt]{article}
\author{David Alves}

\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{dirtytalk}
\usepackage[a4paper]{geometry}
\usepackage{forest}
\usepackage{listings}
\usepackage{mathtools}
\usepackage{multicol}
\usepackage{nth}
\usepackage{relsize}
\usepackage{skak}
\usepackage{tikz}
\usepackage{tikz-qtree}
\usepackage{titling}
\usepackage{wrapfig}
\usepackage{xcolor}

\usetikzlibrary{decorations.pathreplacing}
\usetikzlibrary{patterns}

\DeclarePairedDelimiter\ceil{\lceil}{\rceil}
\DeclarePairedDelimiter\floor{\lfloor}{\rfloor}

\def\multichoose#1#2{\ensuremath{\left(\kern-.3em\left(\genfrac{}{}{0pt}{}{#1}{#2}\right)\kern-.3em\right)}}

\newcommand{\ts}[1]{\textsuperscript{#1}}

\newcommand{\ProblemStatement}[1]{
\subsection*{Problem Statement}
#1
\subsection*{Solution}
}

% If uncommented, next line hides problem statements
%\renewcommand{\ProblemStatement}[1]{}

\title{Math 142 Problem Set 11}
\author{David Alves}
\date{2016-11-10}

\begin{document}
\pagenumbering{gobble}

\begin{center}
\large \thetitle \\
\theauthor \\
\thedate
\end{center}

\subsection*{Sources}

    \begin{itemize}
    \item http://tex.stackexchange.com and https://www.sharelatex.com for help with \LaTeX
    \item Wikipedia for definition of Harmonic numbers
    \item Wolfram$\vert$Alpha for help with factoring polynomials
    \end{itemize}

\section{Strange Dice (Optional)}
\ProblemStatement{
How many ways are there to make two fair 6-sided dice with natural number values (fair in the sense that each face lands up with same probability, but the sides can have freedom of numbers, including duplication) such that rolling them gives the same probability distribution on the sum of the two dice as two normal fair 6-sided dice? (for example, if we have a die with sides (0, 1, 2, 3, 4, 5) and another with sides (2,3,4,5,6,7), we get the same distribution as two normal dice (why?) but this is not legal as I wanted natural numbers)
}

There are two ways: $(1,2,3,4,5,6)\times2$ and $(1,2,2,3,3,4), (1,3,4,5,6,8)$.
\begin{proof}
You can represent normal die using the generating function $x + x^2 + x^3 + x^4 + x^5 + x^6$, since there is one way to get each possible value in $[1,6]$. Rolling two normal dice is represented by squaring the generating function as follows:

\begin{multline*}
(x + x^2 + x^3 + x^4 + x^5 + x^6)^2 = \\x^2+2 x^3+3 x^4+4 x^5+5 x^6+6 x^7+5 x^8+4 x^9+3 x^{10}+2x^{11}+x^{12}
\end{multline*}

This question asks how many ways to factor this polynomial into two polynomials $A$ and $B$ such that:
\begin{enumerate}
    \item Every side has a natural number of dots, meaning that every term in both $A$ and $B$ is at least $x$.
    \item Each die has six sides, meaning that $A$ and $B$ both contain six terms.
\end{enumerate}

We first factor the polynomial:
\begin{align*}
    (x + x^2 + x^3 + x^4 + x^5 + x^6)^2 &= \left(x(1 + x + x^2 + x^3 + x^4 + x^5)\right)^2\\
    &= \left(x\frac{(x^6-1)}{x-1}\right)^2\\
    &= \left(x\frac{(x + 1)(x^2 - x + 1)(x-1)(x^2+x+1)}{x-1}\right)^2\\
    &= \left(x(x + 1)(x^2 - x + 1)(x^2+x+1)\right)^2\\
    &= x^2(x + 1)^2(x^2 - x + 1)^2(x^2+x+1)^2\\
\end{align*}

Since we know that every term in $A$ and $B$ is at least $x$, each of them needs to get one of the two $x$ factors. Trying each of the possible ways to distribute the remaining factors gives two valid arrangements:
\begin{multline*}
    A = B = x(x+1)(x^2 - x + 1)(x^2+x+1) = (1,2,3,4,5,6)\times 2 \text{ (Normal dice)}\\
    A = x(x+1)(x^2 - x + 1)^2(x^2+x+1), B = x(x+1)(x^2+x+1)\\
    = (1,2,2,3,3,4), (1,3,4,5,6,8) \text{ (Strange dice)}\\
\end{multline*}
\end{proof}

\section{Paths Through Trees}
\ProblemStatement{
Recall that a tree has many definitions. Suppose you're only allowed to use the definition that there are $n - 1$ edges on $n$ vertices with no cycles. Prove that you can go from any vertex to any other vertex via a path (i.e. the tree is not separated into 2 parts inaccessible from each other).
}

Let a \emph{connected component} be a set of vertices and edges such that you can get from any vertex in the component to any other vertex via some path which is a subset of that component's edges. Consider the edges of the graph in some arbitrary order. We will add the edges one at a time to the graph. Before any edges have been added, the graph consists of $n$ connected components each with one vertex. Now add the first edge to the graph, which connects two separate connected components (each with 1 vertex), leaving $n-1$ connected components. Each additional edge added to the graph connects two vertices in separate connected components, turning them into a single connected component. It must connect two vertices in different connected components because if the edge connected some $u,v$ inside the same connected component then there would be two paths from $u$ to $v$: One using the new edge and one through the other edges in that connected component, which contradicts our statement that there are no cycles. Thus each edge added to the graph reduces the number of connected components by one, so after adding all $n-1$ edges there is only one connected component, and thus you can get from any vertex to any other vertex via a path.

\section{Triangle Game (Optional)}
\ProblemStatement{
Describe (and prove) an optimal strategy of playing the triangle game we played in class for n = 5 vertices.
}

Below is a diagram showing a strategy for Player 1 that can force a win regardless of how Player 2 plays. Solid edges are Player 1 moves, while dashed edges are Player 2 moves. Only one graph per isomorphic group is shown.
\colorlet{lightred}{red!45}

\begin{center}
\begin{tikzpicture}
    \node[shape=circle,draw=black] (A1) at (4+0,24+1) {};
    \node[shape=circle,draw=black] (B1) at (4+.951,24+.309) {};
    \node[shape=circle,draw=black] (C1) at (4+.588,24-.809) {};
    \node[shape=circle,draw=black] (D1) at (4-.588,24-.809) {};
    \node[shape=circle,draw=black] (E1) at (4-.951,24+.309) {} ;
    \path [-,line width=1pt,color=blue] (A1) edge node[left] {} (B1) node[yshift=-6.5em,color=black] {Initial Move};

    \draw [->,line width=2.5,black!25] (2.5,22.5) -- (1.5,21.5);
    \node[shape=circle,draw=black] (A2) at (0+0,20+1) {};
    \node[shape=circle,draw=black] (B2) at (0+.951,20+.309) {};
    \node[shape=circle,draw=black] (C2) at (0+.588,20-.809) {};
    \node[shape=circle,draw=black] (D2) at (0-.588,20-.809) {};
    \node[shape=circle,draw=black] (E2) at (0-.951,20+.309) {} ;
    \path [-,line width=1pt] (A2) edge node[left] {} (B2) node[yshift=-6.5em] {\parbox{4.5cm}{Player 2's move shares a vertex with initial move}};
    \path [-,line width=1pt,style=dashed,color=blue] (A2) edge node[left] {} (C2);

    \draw [->,line width=2.5,black!25] (5.5,22.5) -- (6.5,21.5);
    \node[shape=circle,draw=black] (A3) at (8+0,20+1) {};
    \node[shape=circle,draw=black] (B3) at (8+.951,20+.309) {};
    \node[shape=circle,draw=black] (C3) at (8+.588,20-.809) {};
    \node[shape=circle,draw=black] (D3) at (8-.588,20-.809) {};
    \node[shape=circle,draw=black] (E3) at (8-.951,20+.309) {} ;
    \path [-,line width=1pt] (A3) edge node[left] {} (B3) node[yshift=-6.5em] {\parbox{5.5cm}{Player 2's move does not share a vertex with initial move}};
    \path [-,line width=1pt,style=dashed,color=blue] (D3) edge node[left] {} (C3);

    \draw [->,line width=2.5,black!25] (0,17.5) -- (0,16.5);
    \node[shape=circle,draw=black] (A4) at (0+0,15+1) {};
    \node[shape=circle,draw=black] (B4) at (0+.951,15+.309) {};
    \node[shape=circle,draw=black] (C4) at (0+.588,15-.809) {};
    \node[shape=circle,draw=black] (D4) at (0-.588,15-.809) {};
    \node[shape=circle,draw=black] (E4) at (0-.951,15+.309) {} ;
    \path [-,line width=1pt] (A4) edge node[left] {} (B4) node[yshift=-6.5em] {};
    \path [-,line width=1pt,style=dashed] (A4) edge node[left] {} (C4);
    \path [-,line width=1pt,color=blue] (A4) edge node[left] {} (E4);

    \draw [->,line width=2.5,black!25] (8,17.5) -- (8,16.5);
    \node[shape=circle,draw=black] (A5) at (8+0,15+1) {};
    \node[shape=circle,draw=black] (B5) at (8+.951,15+.309) {};
    \node[shape=circle,draw=black] (C5) at (8+.588,15-.809) {};
    \node[shape=circle,draw=black] (D5) at (8-.588,15-.809) {};
    \node[shape=circle,draw=black] (E5) at (8-.951,15+.309) {} ;
    \path [-,line width=1pt] (A5) edge node[left] {} (B5) node[yshift=-6.5em] {};
    \path [-,line width=1pt,style=dashed] (D5) edge node[left] {} (C5);
    \path [-,line width=1pt,color=blue] (A5) edge node[left] {} (E5);

    \draw [->,line width=2.5,black!25] (0,13) -- (0,12);
    \node[shape=circle,draw=black] (A6) at (0+0,10+1) {};
    \node[shape=circle,draw=black] (B6) at (0+.951,10+.309) {};
    \node[shape=circle,draw=black] (C6) at (0+.588,10-.809) {};
    \node[shape=circle,draw=black] (D6) at (0-.588,10-.809) {};
    \node[shape=circle,draw=black] (E6) at (0-.951,10+.309) {} ;
    \path [-,line width=1pt] (A6) edge node[left] {} (B6) node[yshift=-6.3em] {Forced move for player 2};
    \path [-,line width=1pt,style=dashed] (A6) edge node[left] {} (C6);
    \path [-,line width=1pt] (A6) edge node[left] {} (E6);
    \path [-,line width=1pt,style=dashed,color=blue] (E6) edge node[left] {} (B6);

    \draw [->,line width=2.5,black!25] (8,13) -- (8,12);
    \node[shape=circle,draw=black] (A7) at (8+0,10+1) {};
    \node[shape=circle,draw=black] (B7) at (8+.951,10+.309*1.3) {};
    \node[shape=circle,draw=black] (C7) at (8+.588,10-.809*1.3) {};
    \node[shape=circle,draw=black] (D7) at (8-.588,10-.809*1.3) {};
    \node[shape=circle,draw=black] (E7) at (8-.951,10+.309*1.3) {} ;
    \path [-,line width=1pt] (A7) edge node[left] {} (B7) node[yshift=-6.3em] {Forced move for player 2};
    \path [-,line width=1pt,style=dashed] (D7) edge node[left] {} (C7);
    \path [-,line width=1pt] (A7) edge node[left] {} (E7);
    \path [-,line width=1pt,style=dashed,color=blue] (E7) edge node[left] {} (B7);

    \draw [->,line width=2.5,black!25] (0,7.75) -- (0,6.75);
    \node[shape=circle,draw=black] (A8) at (0+0,5+1) {};
    \node[shape=circle,draw=black] (B8) at (0+.951,5+.309) {};
    \node[shape=circle,draw=black] (C8) at (0+.588,5-.809) {};
    \node[shape=circle,draw=black] (D8) at (0-.588,5-.809) {};
    \node[shape=circle,draw=black] (E8) at (0-.951,5+.309) {} ;
    \path [-,line width=1pt] (A8) edge node[left] {} (B8) node[yshift=-7em] {\parbox{7.1cm}{Player 1 has forced a win because Player 2 can only play one of the moves in red; Player 1 will use the other to win}};
    \path [-,line width=1pt,style=dashed] (A8) edge node[left] {} (C8);
    \path [-,line width=1pt] (A8) edge node[left] {} (E8);
    \path [-,line width=1pt,style=dashed] (E8) edge node[left] {} (B8);
    \path [-,line width=1pt,color=blue] (A8) edge node[left] {} (D8);
    \path [-,line width=1pt,style=dashed,color=lightred] (E8) edge node[left] {} (D8);
    \path [-,line width=1pt,style=dashed,color=lightred] (B8) edge node[left] {} (D8);

    \draw [->,line width=2.5,black!25] (8,7.75) -- (8,6.75);
    \node[shape=circle,draw=black] (A9) at (8+0,5+1) {};
    \node[shape=circle,draw=black] (B9) at (8+.951,5+.309) {};
    \node[shape=circle,draw=black] (C9) at (8+.588,5-.809) {};
    \node[shape=circle,draw=black] (D9) at (8-.588,5-.809) {};
    \node[shape=circle,draw=black] (E9) at (8-.951,5+.309) {} ;
    \path [-,line width=1pt] (A9) edge node[left] {} (B9) node[yshift=-7em] {\parbox{7.1cm}{Player 1 has forced a win because Player 2 can only play one of the moves in red; Player 1 will use the other to win}};
    \path [-,line width=1pt,style=dashed] (D9) edge node[left] {} (C9);
    \path [-,line width=1pt] (A9) edge node[left] {} (E9);
    \path [-,line width=1pt,style=dashed] (E9) edge node[left] {} (B9);
    \path [-,line width=1pt,color=blue] (A9) edge node[left] {} (D9);
    \path [-,line width=1pt,style=dashed,color=lightred] (E9) edge node[left] {} (D9);
    \path [-,line width=1pt,style=dashed,color=lightred] (B9) edge node[left] {} (D9);
\end{tikzpicture}
\end{center}

\section{Sum of Choices}
\ProblemStatement{
    Calculate $\sum_{k=1}^n \binom{k}{m} \frac{1}{k}$.
}

\[
    \sum_{k=1}^n \binom{k}{m} \frac{1}{k} =
    \begin{cases}
        H_n &: m = 0\\
        \frac{1}{m} \binom{n}{m} &: m \ge 1\\
    \end{cases}
\]

\begin{proof}
For $m=0$, we have $\sum_{k=1}^n \frac{1}{k} = H_n$, the nth harmonic number by definition.
For $m \ge 1$,
\begin{align*}
    \sum_{k=1}^n \binom{k}{m} \frac{1}{k} &=\sum_{k=1}^n \frac{k!}{m!(k-m)!} \frac{1}{k}\\
    &=\sum_{k=1}^n \frac{(k-1)!}{m!(k-m)!}\\
    &=\frac{1}{m} \sum_{k=1}^n \frac{(k-1)!}{(m-1)!(k-m)!}\\
    &=\frac{1}{m} \sum_{k=1}^n \binom{k-1}{m-1}
\end{align*}

From previous work, we know that the sum of diagonal elements in Pascal's triangle is $\sum_{a=0}^b \binom{a}{c} = \binom{b+1}{c+1}$, thus by setting $a=k-1$, $c=m-1$, $b=n-1$ we obtain $\sum_{k=1}^n \binom{k-1}{m-1}=\binom{n}{m}$ and thus $
    \sum_{k=1}^n \binom{k}{m} \frac{1}{k} = \frac{1}{m} \binom{n}{m}: m \ge 1$.
\end{proof}

\section{Counting Functions}
\ProblemStatement{
How many functions $f : [n] \rightarrow [n]$ are there such that for all $i < j$, we have $f(i) \leq f(j)$?
}

There are $\binom{2n-1}{n}$ functions on $[n] \rightarrow [n]$ such that for all $i < j$, $f(i) \leq f(j)$.

\begin{proof}
There is a bijection between these functions and paths from $(0, 0)$ to ${(n, n-1)}$ which consist of single steps $(+1, 0)$ or $(0, +1)$. To construct such a path from a function, do the following:
\begin{enumerate}
    \item For each value $a$ in the domain of $f$, draw a step from at $(a-1, f(a)-1)$ to $(a, f(a)-1)$.
    \item Draw vertical lines to connect the horizontal steps, as needed.
    \item Draw vertical lines to connect $(0,0)$ to $(0, f(1)-1)$ and $(n, f(n)-1)$ to $(n,n-1)$, as needed.
\end{enumerate}

Additionally, any path from $(0,0)$ to $(n,n-1)$ consisting of only steps $(+1, 0)$ or $(0, +1)$ has a unique corresponding function $f(i)$ defined by $f(i) = 1 + y-$coordinate of the step from $x=i$ to $x=i+1$. Thus there's a bijection between these paths and these functions. We know from previous work that the number of such paths is equal to $\binom{2n-1}{n}$ since it is equivalent to having a list made up of $2n-1$ instructions, $n$ of which are \say{go right} while the rest are \say{go up}.
\end{proof}
\newcommand{\bijectpath}[3]{\scalebox{.8}{
\begin{tikzpicture}
\draw[step=1.0,black,thin,color=black,opacity=.35] (0,0) grid (3,2);
\draw [-,line width=1pt] (0,0) -- (0,#1-1) --
    (1, #1-1) node[shape=circle,fill=black,minimum width=.2cm,inner sep=0pt] {} --
    (1, #2-1) --
    (2, #2-1) node[shape=circle,fill=black,minimum width=.2cm,inner sep=0pt] {} --
    (2, #3-1) --
    (3, #3-1)node[shape=circle,fill=black,minimum width=.2cm,inner sep=0pt] {} --
    (3, 2);
\node [above] at (5.85,-.5) {\parbox{3em}{
    \begin{align*}
    \iff f(x) =
    \begin{cases}
    #1&: x=1\\
    #2&: x=2\\
    #3&: x=3\\
\end{cases}
    \end{align*}
}};
\end{tikzpicture}
}
}

\noindent\ignorespacesafterend
\bijectpath{1}{1}{1} \hspace{2em} \bijectpath{1}{1}{2}\\
\bijectpath{1}{1}{3} \hspace{2em} \bijectpath{1}{2}{2}\\
\bijectpath{1}{2}{3} \hspace{2em} \bijectpath{1}{3}{3}\\
\bijectpath{2}{2}{2} \hspace{2em} \bijectpath{2}{2}{3}\\
\bijectpath{2}{3}{3} \hspace{2em} \bijectpath{3}{3}{3}

\section{Time Spent \& Thoughts}
This was really time consuming. I either need to stop trying to do the optional problems or not attempt to take this class while working full time. :) I probably spent about 15 hours on this assignment. I enjoyed problems 1, 3, and 5 quite a bit. Problem 4 took a long time to figure out and wasn't very interesting, although it's probably good practice.
\end{document}