The last chapter introduced some fundamental topics related to image processing, namely filtering, feature detection, and feature description. While these techniques are quite useful for a large number of computer vision applications, they may not be sufficient to extract higher-level information from images. For example, the features that were discussed are \textit{local features} that describe important keypoints of the image, but these may be too localized to discuss higher-level features or semantic content. In some cases it may be possible to correlate local features to extract higher-level information (e.g. image matching), but in other cases higher-level algorithms may be useful (e.g. identifying a particular object in a scene, such as a person). In particular, object recognition is a very important task in robotics and therefore some common methods for object recognition will be discussed in this chapter.

This chapter will additionally focus on geometric feature extraction\cite{SiegwartNourbakhshEtAl2011}, which is used to extract structure from data in the form of geometric primitives (e.g. lines, circles, planes). This is very useful in robotics for localization and mapping, and these algorithms can generally be applied to different types of data, such as data extracted from images or even data collected via laser rangefinders or radar.

\notessection{Information Extraction}
This chapter will focus on common methods for extracting higher-level environmental information from sensor data that is useful for robotics. In particular, common algorithms for \textit{geometric feature extraction} will be presented, as well as methods for \textit{object recognition} in images. Such information is crucial for robots operating in real environments to enable intelligent decision making and task planning, as well as to execute plans safely in unknown environments with obstacles.

\subsection{Geometric Feature Extraction}
It is very common in robotic localization and mapping to represent the environment using simple geometric primitives (e.g. lines, circles, corners, planes) that can be efficiently extracted from sensor data. In particular, in this section techniques for line extraction from range data\footnote{Range data can generally come from a variety of sources, including laser rangefinders, radar, or even computer vision.} will be presented. Lines in particular are one of the most fundamental geometric primitives to be extracted, and generally the techniques for extracting other primitives are conceptually similar.

There are two main challenges with extracting lines from range data. The first is called \textit{segmentation}, which is the task of identifying which data points belong to which line (and inherently also identifying how many lines there are). The second is \textit{fitting}, which is the task of estimating the parameters that define a line given a set of points. For simplicity this chapter will consider line extraction problems based on two-dimensional range data.

\subsubsection{Line Segmentation}
The line segmentation problem is to determine how many lines exist in a given set of data and also which data points correspond to each line. Three popular algorithms for line segmentation will be discussed, the \textit{split-and-merge} algorithm, the \textit{random sample consensus (RANSAC)} algorithm, and the \textit{Hough-transform} algorithm.

\paragraph{Split-and-Merge:}
The split-and-merge algorithm is perhaps the most popular line extraction algorithm and is arguably the fastest (but not as robust to outliers). The concept of this algorithm is quite simple: repeatedly fit lines to sets of points and then split the set of points into two sets if any point lies more than distance $d$ from the line. By repeating this process until no more splits occur, it is guaranteed that all points will lie less than distance $d$ to a line. After this ``split'' process is completed, a second step merges any of the newly formed lines that are colinear. This algorithm is presented in more detail in Algorithm \ref{alg:splitmerge}.
\begin{algorithm}[ht]\caption{Split-and-Merge} \label{alg:splitmerge}
	\KwData{Set $S$ of $N$ points, distance threshold $d > 0$}
	\KwResult{A list $L$ of sets of points each resembling a line}
	$L \xleftarrow{} [S]$ \\
	$i \xleftarrow{} 1$ \\
	\While{$i \leq \text{len}(L)$}{
	    fit a line $(\alpha,r)$ to the set $L[i]$ \\
	    detect the point $P \in L[i]$ with maximum distance $D$ to the line $(\alpha, r)$ \\
	    \eIf{$D < d$}{
	        $i \xleftarrow{} i + 1$
	    }
	    {
	    split $L[i]$ at $P$ into new sets $S_1$ and $S_2$ \\
	    $L[i] \xleftarrow{} S_1$ \\
	    $L[i+1] \xleftarrow{} S_2$ \\
	    }
	}
	Merge colinear sets in $L$
\end{algorithm}
A popular variant of the split-and-merge algorithm is known as the iterative-end-point-fit algorithm. This algorithm is simply the split-and-merge algorithm given in Algorithm \ref{alg:splitmerge} where the line is simply constructed by connecting the first and the last points of the set. This approach is shown graphically in Figure \ref{fig:splitmerge}.
\begin{figure*}[ht]
\centering
	\includegraphics[width=0.85\textwidth]{tex/figs/ch11_figs/iterative_end_point_fit.png}
	\caption{Iterative-end-point-fit variation of the split-and-merge algorithm for extracting lines from data.}
	\label{fig:splitmerge}
\end{figure*}

\paragraph{Random Sample Consensus (RANSAC):}
Random Sample Consensus (RANSAC) is an algorithm to estimate the parameters of a model from a set of data that may contain outliers (i.e. \textit{robust} model parameter estimation). Outliers are data points that do not fit the model and may be the result of high noise in the data, incorrect measurements, or simply points which come from objects that are unrelated to the current model. For example, a typical laser scan of an indoor environment may contain distinct lines from the surrounding walls but also points from other static and dynamic objects (e.g. chairs or humans). In this case, if the goal was to extract lines to represent the walls then any data point corresponding to other objects would be an outlier. In general, RANSAC can be applied to many parameter estimation problems, and typical applications in robotics include line extraction from 2D range data, plane extraction from 3D point clouds, and structure-from-motion (where the goal is to identify image correspondences which satisfy a rigid body transformation). However for simplicity this section focuses on using RANSAC for line extraction from 2D data.

RANSAC is an iterative method and is non-deterministic (i.e. stochastic or random).
Given a dataset $S$ of $N$ points, the algorithm starts by randomly selecting a sample of two points from $S$. Then a line is constructed from these two points and the distance of all other points to this line is computed. A set of \textit{inliers} comprised of all the points whose distance to the line is within a predefined threshold $d$ is then defined. By repeating this process $k$ times, $k$ inlier sets (and their associated lines) are generated and the inlier set with the most points is returned. This procedure is detailed in Algorithm \ref{alg:ransac} and is also illustrated in Figure \ref{fig:ransac-working}.
\begin{algorithm}[ht]\caption{Random Sample Consensus (RANSAC) for Line Extraction} \label{alg:ransac}
	\KwData{Set $S$ of $N$ points, distance threshold $d$}
	\KwResult{Set with maximum number of inliers and corresponding line}
	\While{$i \leq k$}{
	    randomly select 2 points from $S$ \\
	    fit line $l_i$ through the 2 points \\
	    compute distance of all other points to $l_i$ \\
	    construct set of points $\tilde{S}_i$ with distance less than $d$ to $l_i$ \\
	    store line $l_i$ and set of points $\tilde{S}_i$
	    $i \xleftarrow{} i + 1$
	}
	Choose set $\tilde{S}_i$ with maximum number of points
\end{algorithm}

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.85\textwidth]{tex/figs/ch11_figs/ransac.png}
\caption{Example of the RANSAC algorithm, showing four iterations of the algorithm. If the algorithm was terminated after these four iterations, line $l_3$ would be returned since it contains the maximum number of points.}
\label{fig:ransac-working}
\end{figure}

Due to the probabilistic nature of the algorithm, as the number of iterations $k$ increases the probability of finding a good solution increases. This approach is used over a brute force search of all possible combinations of two points since the total number of combinations is $N(N-1)/2$, which can be extremely large. In fact, a simple statistical analysis of RANSAC can be performed.

Let $p$ be the \textit{desired} probability of finding a set of points free of outliers and let $w$ be the probability of selecting an inlier from the dataset $S$ of $N$ points, which can be expressed as:
\begin{equation*}
    w = \frac{\text{\# inliers}}{N}.
\end{equation*}
Assuming point samples are drawn independently from $S$, the probability of drawing two inliers is $w^2$ (and $1-w^2$ is the probability that at least one is an outlier). Therefore, with $k$ iterations the probability that RANSAC \textit{never} selects two points that are both inliers is $(1-w^2)^k$. Therefore the minimum number of iterations $\bar{k}$ needed to find an outlier-free set with probability $p$ can be found by solving:
\begin{equation*}
    1-p = (1-w^2)^k,
\end{equation*}
for $k$. In other words, $\bar{k}$ can be computed as:
\begin{equation*}
    \bar{k} = \frac{\log (1-p)}{\log (1-w^2)}.
    \label{eq:magic-k}
\end{equation*}
While the value of $w$ may not be known exactly\footnote{There also exist advanced versions of RANSAC that can estimate $w$ in an adaptive online fashion.}, this expression can still be used to get a good estimate of the number of iterations $k$ that are needed for good results. It is important to note that this probabilistic approach often leads to a much smaller number of iterations than for brute force searching through all combinations. This can be attributed to the fact that $\bar{k}$ is only a function of $w$ and not the total number of samples $N$ in the dataset.

Overall, the main advantage of RANSAC is that it is a generic extraction method and can be used with many types of features given a feature \textit{model}. It is also simple to implement and is robust with respect to outliers in the data. The main disadvantages are that the algorithm needs to be run multiple times if multiple features are to be extracted, and there are not guarantees that the solutions will be optimal.

\paragraph{Hough Transform:}
In the Hough transform algorithm, each point $(x_i,y_i)$ of the set $S$ ``votes'' for a \textit{set} of possible line parameters $(m,b)$ (i.e. slope and intercept). For any given point $(x_i,y_i)$ the candidate set of line parameters $(m,b)$ that could pass through this point must satisfy $y_i=mx_i+b$, which can also be written as:
\begin{equation*}
\quad b=-mx_i + y_i.
\end{equation*}
Therefore it can be noted that each point in the original space space $(x,y)$ maps to a \textit{line} in the Hough space $(m,b)$ (see Figure \ref{fig:hough1}). The Hough transform algorithm exploits this fact by noting that two points on the same line in the original space will yield two \textit{intersecting} lines in Hough space. In particular, the point where they intersect in the Hough space corresponds to the parameters $m^*$ and $b^*$ that defines the line passing between the points in the original space (see Figure \ref{fig:hough2}).
\begin{figure}[ht]
  \centering
  \includegraphics[width=0.75\textwidth]{tex/figs/ch11_figs/hough.png}
\caption{Each point $(x_i, y_i)$ in the original space maps to a \textit{line} in the Hough space which describes all possible parameters $m$ and $b$ that would generate a line passing through the point $(x_i, y_i)$.}
\label{fig:hough1}
\end{figure}
\begin{figure}[ht]
  \centering
  \includegraphics[width=0.75\textwidth]{tex/figs/ch11_figs/hough2.png}
\caption{All points on a line in the original space yield lines in the Hough space that intersect at a common point.}
\label{fig:hough2}
\end{figure}

This concept can be applied to line segmentation by searching in the Hough space for intersections among the lines that correspond to each point $(x,y)$ in the set $S$. In practice, this can be done by discretizing the Hough space with a grid and simply counting for each grid cell the number of lines corresponding to $(x_i,y_i)$ points from $S$ that pass through it. Local maxima among the cells then can be chosen as lines that ``fit'' the data set $S$.

However, performing a discretization of the Hough space requires a trade-off between range and resolution (in particular because $m$ can range from $-\infty$ to $\infty$. Alternatively, it is possible to use a polar coordinate representation of the Hough space which defines a line as:
\begin{equation*}
x \cos \alpha + y \sin \alpha = r,
\end{equation*}
where $(\alpha,r)$ are the new line parameters. With this representation, a point $(x_i,y_i)$ from the original space gets mapped to the polar Hough space $(\alpha,r)$ as a sinusoidal curve (see Figure \ref{fig:hough3}). An example of the Hough transform using the polar representation is given in Figure \ref{fig:hough4}.
\begin{figure}
\centering
	\includegraphics[width=0.75\textwidth]{tex/figs/ch11_figs/hough3.png}
	\caption{Representation of a point $(x_i, y_i)$ in the Hough space when using a polar coordinate representation of a line with parameters $\alpha$ and $r$.}
	\label{fig:hough3}
\end{figure}
\begin{figure}
\centering
	\includegraphics[width=0.85\textwidth]{tex/figs/ch11_figs/hough4.png}
	\caption{Example of the Hough transformation using a polar coordinate representation of lines.}
	\label{fig:hough4}
\end{figure}

\subsubsection{Line Fitting}
Line segmentation is the process of identifying which data points belong to a line, and line fitting is the process of estimating parameters of a line for those corresponding data points. For the line segmentation algorithms previously discussed (i.e. split-and-merge, RANSAC, and Hough-transform), a line associated with the segmented data points was also implicitly defined. However, the lines implicitly defined from the segmentation algorithms may not always be ideal and so other techniques have been developed to specifically address the line fitting task.

Line fitting algorithms search for lines that best fit a set of data points. In almost all cases the problem is over-determined (i.e. there are more data points than parameters to choose) and noise in the data means that there is not perfect solution. Therefore one of the most common approaches to line fitting is based on \textit{least-squares estimation}, which tries to find a line that minimizes the overall error in the fit. For this approach it is useful to work in polar coordinates defined by:
\begin{equation*}
    x = \rho \cos \theta, \quad y = \rho \sin \theta,
\end{equation*}
where $(x,y)$ is the 2D Cartesian coordinate of a data point and $(\rho,\theta)$ is the 2D polar coordinate. In polar coordinates the equation of a line is given by
\begin{equation} \label{eq:polarline}
\rho \cos(\theta - \alpha) = r, \quad \text{or} \quad  x \cos \alpha + y \sin \alpha = r,
\end{equation}
where $\alpha$ and $r$ are the parameters that define the line. For a visual representation of these definitions see Figure \ref{fig:polarline}.
\begin{marginfigure}
  \centering
  \includegraphics[width=0.85\textwidth]{tex/figs/ch11_figs/polarlinefit.png}
  \caption{Representation of a line in polar coordinates, defined by the parameters $r$ and $\alpha$ which are the distance and angle to the closest point on the line to the origin.}
  \label{fig:polarline}
\end{marginfigure}
For a collection $S$ of $N$ points $(\rho_i, \theta_i)$, the error $d_i$ corresponding to the perpendicular distance from a point to a line defined by parameters $\alpha$ and $r$ can be computed by:
\begin{equation} \label{eq:lineerror}
d_i = \rho_i \cos(\theta_i - \alpha) - r.
\end{equation}
The line fitting task can then be formulated as an optimization problem over the parameters $\alpha$ and $r$ to minimize the combined errors $d_i$ for $i = 1,\dots,N$. In particular, the combined errors are aggregated using a sum of the squared errors:
\begin{equation}
    S(r,\alpha) = \sum_{i=1}^N d_i^2 = \sum_{i=1}^N(\rho_i \cos(\theta_i - \alpha) - r)^2.
    \label{eq:MSL}
\end{equation}
This is a classic least squares optimization problem that can be efficiently solved. However, this cost function generally assumes that each of the data points is equally affected by noise (i.e. the uncertainty of each measurement is the same). In some cases it might be beneficial to account for differences in data quality for each point $i$, which could give preference to well known points.

Accounting for unique uncertainties in each data point leads to a \textit{weighted least squares estimation} problem. In particular, it is assumed that the variance of each range measurement $\rho_i$ is given by $\sigma_i$. The cost function \eqref{eq:MSL} is then modified to be:
\begin{equation} \label{eq:WMSL}
    S_w(r,\alpha) = \sum_{i=1}^N w_i d_i^2 = \sum_{i=1}^N w_i (\rho_i \cos (\theta_i -\alpha) -r)^2,
\end{equation}
where the weights $w_i$ are given by:
\begin{equation*}
    w_i = \frac{1}{\sigma^2_i}.
\end{equation*}
It can be shown that the solution to the optimization problem defined by the weighted cost function \eqref{eq:WMSL} is given by:
\begin{equation}
\begin{split}
r &= \frac{\sum_{i=1}^N w_i\rho_i\cos(\theta_i - \alpha)}{\sum_{i=1}^N w_i}, \\
\alpha &= \frac{1}{2} \text{atan2} \left(\frac{\sum_{i=1}^N w_i\rho_i^2\sin(2\theta_i) - \frac{2}{\sum_{i=1}^N w_i}\sum_{i=1}^N\sum_{j=1}^N w_i w_j \rho_i \rho_j \cos\theta_i \sin \theta_j}{\sum_{i=1}^N w_i \rho_i^2\cos(2\theta_i) - \frac{1}{\sum_{i=1}^N w_i}\sum_{i=1}^N\sum_{j=1}^N w_i w_j \rho_i \rho_j \cos(\theta_i + \theta_j)}\right) + \frac{\pi}{2}.
\end{split}
\end{equation}

\subsection{Object Recognition}
Another high-level information extraction task that is common in robotics is \textit{object recognition}. Object recognition is the task of classifying or naming discrete objects in the world (usually based on images or video). This can be a particularly challenging task because real world scenes are commonly made up of many varying types of objects which can appear at different poses and can occlude each other. Additionally, objects within a specific class can have a large amount of variability (e.g. breeds of dogs or car models). In this section three common methods for object recognition will be introduced, namely template matching, bag of visual words, and neural network methods.

\subsubsection{Template Matching}
Template matching\cite{PerveenKumarEtAl2013} is a machine vision technique for identifying parts of an image that match a given image pattern\footnote{Advanced template matching algorithms allow finding pattern occurrences regardless of their orientation and local brightness.}. This approach has seen success in a variety of applications, including manufacturing quality control, mobile robotics, and more. The two primary components needed for template matching are the source image $I$ and a template image $T$

Given a source and template image, one approach to template matching is to leverage the linear spatial correlation filters discussed in the previous chapter. In particular, a naive approach would be to use the normalized template image as a filter mask in a correlation filter. By applying this filter mask to every pixel in the source image the resulting output would quantify the similarity of that region of the source image to the template. This type of approach is sometimes referred to as a \textit{cross-correlation}.
Another approach based on linear spatial filters from the previous chapter would be to leverage the similarity filters that compute the sum of absolute differences (SAD) metric for each pixel in the source image. Regions of the source image similar to the template would correspond to low SAD scores.
The disadvantages of these approaches is that do not handle rotations or scale changes, which are quite common in real world applications.

One solution to the scaling issue in correlation filter based template matching is to simply re-scale the source image multiple times and perform template matching on each. This concept, referred to as using \textit{image pyramids}\cite{Szeliski2010}, can also be used to accelerate object search by using a coarser resolution image first to localize the object and then using finer resolution images for actual detection. Building image pyramids can be accomplished in several ways. One naive approach is to simply eliminate some rows and columns of the image. Another approach is to first use a Gaussian smoothing filter to remove high frequency content form the image and \textit{then} subsample the image. The sequence of images resulting from this approach is referred to as a \textit{Gaussian pyramid}.

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.6\textwidth]{tex/figs/ch11_figs/image_pyramid.png}
\caption{A traditional image pyramid: each level has half the resolution (width and height), and hence a quarter of the pixels, of its parent level. Figure from Szeliski (2010) \nocite{Szeliski2010}.}
\label{image-pyramid}
\end{figure}

\subsubsection{Bag of Visual Words}
The key idea behind the bag of visual words\footnote{The model originated in natural language processing, where we consider texts such as documents, paragraphs, and sentences as collections of words - effectively ``bags" of words.
} approach is that object representations can be simplified by considering them as a collection of their subparts (e.g. a bike is an object with wheels, a frame, and handlebars), and the subparts are referred to as \textit{visual words}. In this approach a source image is searched for \textit{visual words}, and a distribution of visual words that are found in the image is created (in the form of a histogram). Object detection can then be performed by comparing this distribution to a set of training images. For example, suppose the source image contains a human face and the recognized features included eyes and a nose. Then by comparing the distribution to training images, it would likely be determined that training images that also have eyes and a nose are also images of faces.

\subsubsection{Convolutional Neural Networks}
Convolutional Neural Networks (CNNs) represent a relatively new and very powerful paradigm in object recognition. These approaches were first introduced in the field of computer vision for image recognition in 1989, and since then have significantly boosted performance in image recognition and classification tasks. Research in this field is also still very active.

\subsection{Exercises}
All exercises for this chapter can be found in the online repository:

\vspace{\baselineskip}

\url{https://github.com/PrinciplesofRobotAutonomy/AA274A_HW3}.

\subsubsection{Line Extraction}
Complete \textit{Problem 2: Line Extraction}, where you will implement a line extraction algorithm (Split-and-Merge) to fit lines to simulated Lidar range data.

\subsubsection{Template Matching}
Complete \textit{Problem 4: Template Matching}, where you will explore the use of the classic template matching algorithm, implemented in the open-source OpenCV library.

\subsubsection{Image Pyramids}
Complete \textit{Extra Problem: Image Pyramids}, where you will learn about how template matching algorithms can be enhanced through the use of image pyramids (and image filtering).


\subsection{Simple, recursive summarization}

We thought that a simple way to do extractive summarization on a large body of
text would be to employ a basic summarizer, but to apply it
\textit{recursively}. Our recursive summarizer ``meta-script'' takes a
summarizing function, which can basically be anything, and applies it
recursively. First it divides the text into chunks of \(n\) sentences and runs
the summarizer on those chunks. The resulting summaries are then combined, and
fed as the new input to the recursive summarizer. At the start of every
recursion we check the length of the text, stopping if it's short enough.

With a little glue code, our recursive python script can be made to use any
summarizing function. In our experiments, we used a very simple summarizing
function; It counted the word frequencies, and using those frequencies it
assigned a score to each sentence, selecting those that had the highest scores.

In previous stages of the project we looked into the different nouns appearing
in texts. We compiled some noun frequency data, and found out that
often the most used noun referred to the main character of a book (see
figures in~\ref{appendix:countfigs}), which isn't
admittedly that surprising. In the first version of our recursive algorithm the
summarizer only looks at the words in the current block; We thought it would be
interesting to try to add the frequencies of the most used nouns.

In the final version the code first compiles a dictionary of nouns and their
frequencies. That noun frequency data can be added to the word frequency data
obtained from each block. The importance of the noun frequencies can also be
adjusted.


\documentclass{article}
\usepackage{amsfonts}
\usepackage[operators]{cryptocode}
\usepackage[utf8]{inputenc}

\title{Introduction of the committed encryption}
\author{Weikeng Chen\\
\vspace{0.12em}
for CS294-144 Blockchain course project}
\date{May 2018}

\begin{document}

\maketitle

\section{Motivation}

We want to encrypt one message to several individual parties. With public-key encryption under a group $\mathbb{G}$ and the group generator $g\in\mathbb{G}$, we assume there are three parties, and their public/private key pairs are $(g^{x_1},x_1)$,  $(g^{x_2},x_2)$, and $(g^{x_3},x_3)$, respectively, we can encrypt a message $m$ as follows:
\[
\begin{array}{l}
r_1\sample[0,...,|\mathbb{G}|-1],~r_2\sample[0,...,|\mathbb{G}|-1],~r_3\sample[0,...,|\mathbb{G}|-1],\\
k\sample\{0,1\}^\lambda,\\
c_1=g^{r_1}\parallel\mathsf{KDF}(g^{x_1r_1})\oplus k,\\
c_2=g^{r_2}\parallel\mathsf{KDF}(g^{x_2r_2})\oplus k,\\
c_3=g^{r_3}\parallel\mathsf{KDF}(g^{x_3r_3})\oplus k,\\
c_4=\mathsf{AuthEnc}.\mathsf{Enc}_k(\mathsf{msg}).
\end{array}
\]
where $\mathsf{KDF}(\cdot)$ is a key derivation function and $\mathsf{AuthEnc}$ refers to an authenticated encryption scheme.

For honest encryption, all three parties can decrypt the message using their secret key. However, if the encryption is not honest, different parties may see different messages, we formalize as follows:
\[
\begin{array}{l}
r_1\sample[0,...,|\mathbb{G}|-1],~r_2\sample[0,...,|\mathbb{G}|-1],~r_3\sample[0,...,|\mathbb{G}|-1],\\
k\sample\{0,1\}^\lambda,\\
c_1=g^{r_1}\parallel\mathsf{KDF}(g^{x_1r_1})\oplus k,\\
c_2=g^{r_2}\parallel\mathsf{KDF}(g^{x_2r_2})\oplus k~\boxed{\oplus\Delta_2},\\
c_3=g^{r_3}\parallel\mathsf{KDF}(g^{x_3r_3})\oplus k~\boxed{\oplus\Delta_3},\\
c_4=\mathsf{AuthEnc}.\mathsf{Enc}_k(\mathsf{msg}).
\end{array}
\]
in which the party 2 obtains $\mathsf{k}\oplus\Delta_2$ and the party 3 obtains $\mathsf{k}\oplus\Delta_3$ instead of $\mathsf{k}$. Note that the party 2's key $\mathsf{k}\oplus\Delta_2$ may decrypt $c_4$ into non-$\perp$ result, as the authenticity of authenticated encryption does not necessarily protect against malicious encryption.

We want to provide a multi-recipient encryption that satisfies the following properties: (1) semantic security; (2) party 1 can detect whether $c_2$ and $c_3$ is generated honestly. Therefore, a malicious encryption cam be detected by the party 1 without knowing the secret keys of the party 2 and the party 3.

\section{Our construction, informally}
We encrypt the message as follows with a pseudorandom function:
\[
\begin{array}{l}
s\sample\{0,1\}^\lambda,~k=\mathsf{PRF}_s(1),\\
r_1\sample[0,...,|\mathbb{G}|-1],\\
\qquad c_1=g^{r_1}\parallel\mathsf{KDF}(g^{x_1r_1})\oplus s,\\
r_2=\mathsf{PRF}_s(2)\mod{|\mathbb{G}|},\\
\qquad c_2=g^{r_2}\parallel\mathsf{KDF}(g^{x_2r_2})\oplus k,\\
r_3=\mathsf{PRF}_s(3)\mod{|\mathbb{G}|},\\
\qquad c_3=g^{r_3}\parallel\mathsf{KDF}(g^{x_3r_3})\oplus k,\\
c_4=\mathsf{AuthEnc}.\mathsf{Enc}_k(\mathsf{msg}).
\end{array}
\]
assumed the Hash Diffie-Hellman (HDH) assumption holds on the group $\mathbb{G}$.

\smallskip
\noindent\textbf{Semantic security.} We can prove the semantic security of this construction with the random oracle heuristic.

By hybrid arguments, we have the following proof. We assume the distinguisher has all the public keys, but does not have any private keys of the three parties.

\smallskip
\noindent $\mathbf{H}_0$: the honest generation.

\smallskip
\noindent $\mathbf{H}_1$: in this hybrid, we change the $s$ in $c_1$ into $0^\lambda$:
\[
\begin{array}{l}
s\sample\{0,1\}^\lambda,~k=\mathsf{PRF}_s(1),\\
r_1\sample[0,...,|\mathbb{G}|-1],\\
\qquad \boxed{c_1=g^{r_1}\parallel\mathsf{KDF}(g^{x_1r_1}),}\\
r_2=\mathsf{PRF}_s(2)\mod{|\mathbb{G}|},\\
\qquad c_2=g^{r_2}\parallel\mathsf{KDF}(g^{x_2r_2})\oplus k,\\
r_3=\mathsf{PRF}_s(3)\mod{|\mathbb{G}|},\\
\qquad c_3=g^{r_3}\parallel\mathsf{KDF}(g^{x_3r_3})\oplus k,\\
c_4=\mathsf{AuthEnc}.\mathsf{Enc}_k(\mathsf{msg}).
\end{array}
\]

This hybrid is computationally indistinguishable with $\mathbf{H}_0$ because $\mathsf{KDF}(g^{x_1r_1})$ is computationally indistinguishable to a random $\lambda$-bit string assuming HDH.

\smallskip
\noindent $\mathbf{H}_2$: in this hybrid, we change $k$, $r_2$, and $r_3$ to randomly sampled values and discard $s$:

\[
\begin{array}{l}
\boxed{k\sample\{0,1\}^\lambda,}\\
r_1\sample[0,...,|\mathbb{G}|-1],\\
\qquad c_1=g^{r_1}\parallel\mathsf{KDF}(g^{x_1r_1}),\\
\boxed{r_2\sample[0,...,|\mathbb{G}|-1],}\\
\qquad c_2=g^{r_2}\parallel\mathsf{KDF}(g^{x_2r_2})\oplus k,\\
\boxed{r_3\sample[0,...,|\mathbb{G}|-1],}\\
\qquad c_3=g^{r_3}\parallel\mathsf{KDF}(g^{x_3r_3})\oplus k,\\
c_4=\mathsf{AuthEnc}.\mathsf{Enc}_k(\mathsf{msg}).
\end{array}
\]

This hybrid is computationally indistinguishable with $\mathbf{H}_1$ because of $\mathsf{PRF}$ with hidden seed $s$.

\smallskip
\noindent $\mathbf{H}_3$: in this hybrid, we hide the key $k$:

\[
\begin{array}{l}
r_1\sample[0,...,|\mathbb{G}|-1],\\
\qquad c_1=g^{r_1}\parallel\mathsf{KDF}(g^{x_1r_1}),\\
r_2\sample[0,...,|\mathbb{G}|-1],\\
\qquad \boxed{c_2=g^{r_2}\parallel\mathsf{KDF}(g^{x_2r_2}),}\\
r_3\sample[0,...,|\mathbb{G}|-1],\\
\qquad \boxed{c_3=g^{r_3}\parallel\mathsf{KDF}(g^{x_3r_3}),}\\
k\sample\{0,1\}^\lambda,\\
c_4=\mathsf{AuthEnc}.\mathsf{Enc}_k(\mathsf{msg}).
\end{array}
\]

This hybrid is computationally indistinguishable with $\mathbf{H}_2$ due to the same reason for $\mathbf{H}_1\stackrel{\mathsf{c}}{\approx}\mathbf{H}_0$.

\smallskip
\noindent $\mathbf{H}_4$: in this hybrid, we change the message into $1^l$, where $l$ is the message size:

\[
\begin{array}{l}
r_1\sample[0,...,|\mathbb{G}|-1],\\
\qquad c_1=g^{r_1}\parallel\mathsf{KDF}(g^{x_1r_1}),\\
r_2\sample[0,...,|\mathbb{G}|-1],\\
\qquad c_2=g^{r_2}\parallel\mathsf{KDF}(g^{x_2r_2}),\\
r_3\sample[0,...,|\mathbb{G}|-1],\\
\qquad c_3=g^{r_3}\parallel\mathsf{KDF}(g^{x_3r_3}),\\
k\sample\{0,1\}^\lambda,\\
c_4=\mathsf{AuthEnc}.\mathsf{Enc}_k(\boxed{1^l}).
\end{array}
\]

This hybrid is computationally indistinguishable with $\mathbf{H}_3$ due to the message indistinguishability of the authenticated encryption.

According to the hybrid arguments, $\mathbf{H}_0\stackrel{\mathsf{c}}{\approx}\mathbf{H}_4$, but $\mathbf{H}_4$ can be simulated by a PPT simulator without the message. This shows that our construction achieves semantic security under HDH assumption.

\smallskip
\noindent\textbf{Accountability (in our definition).} As mentioned in the motivation section, we want the party 1 to be able to detect malicious encryption, in which different parties will see different results.

We first introduce the problem. In a nutshell, we want to see if $\Delta_1$, $\Delta_2$, ..., and $\Delta_4$ equal to zero pads and whether $k=k'$ in the following ciphertext:

\[
\begin{array}{l}
c_1=g^{r_1}\parallel\mathsf{KDF}(g^{x_1r_1})\oplus s,\\
c_2=g^{r_2}~\boxed{\oplus\Delta_1}\parallel\mathsf{KDF}(g^{x_2r_2})\oplus k ~\boxed{\oplus\Delta_2,}\\
c_3=g^{r_3}~\boxed{\oplus\Delta_3}\parallel\mathsf{KDF}(g^{x_3r_3})\oplus k ~\boxed{\oplus\Delta_4,}\\
c_4=\mathsf{AuthEnc}.\mathsf{Enc}_{k'}(\mathsf{msg}).
\end{array}
\]
where $k=\mathsf{PRF}_s(1)$, $r_2=\mathsf{PRF}_s(2)$, and $r_3=\mathsf{PRF}_s(3)$.

We now introduce the algorithm to detect the malicious encryption.
\[
\begin{array}{l}
\mathsf{Detect}(c_1,c_2,c_3,c_4,\mathbb{G},g,g^{x_1},g^{x_2},g^{x_3},x_1)
\end{array}
\]
which runs as follows:
\begin{enumerate}
    \item Computes $g^{x_1r_1}$ from $g^{r_1}$ and $x_1$.

    \item Recovers $s$ by XOR the second part of $c_1$ with $\mathsf{KDF}(g^{x_1r_1})$.

    \item Computes the following pseudorandom values:
    \begin{itemize}
        \item $k=\mathsf{PRF}_s(1)$.
        \item $r_2=\mathsf{PRF}_s(2)$.
        \item $r_3=\mathsf{PRF}_s(3)$.
    \end{itemize}

    \item Checks whether the first part of $c_2$ equals to $g^{r_2}$ and the first part of $c_3$ equals to $g^{r_3}$. If not, outputs $\mathsf{CHEAT}$ and terminates. Otherwise, we know $\Delta_1$ and $\Delta_3$ are both zero pads, and the ciphertext has the following format:
    \[
    \begin{array}{l}
    c_1=g^{r_1}\parallel\mathsf{KDF}(g^{x_1r_1})\oplus s,\\
    c_2=\boxed{g^{r_2}}\parallel\mathsf{KDF}(g^{x_2r_2})\oplus k \oplus\Delta_2,\\
    c_3=\boxed{g^{r_3}}\parallel\mathsf{KDF}(g^{x_3r_3})\oplus k \oplus\Delta_4,\\
    c_4=\mathsf{AuthEnc}.\mathsf{Enc}_{k'}(\mathsf{msg}).
    \end{array}
    \]

    \item Computes $g^{x_2r_2}$ from $r_2$ and $g^{x_2}$ and $g^{x_3r_3}$ from $r_3$ and $g^{x_3}$. Note that we only use the public keys of the party 2 and the party 3.

    \item Recovers $\Delta_2$ by XOR the second part of $c_2$ with $\mathsf{KDF}(g^{x_2r_2})\oplus k$ and $\Delta_4$ by XOR the second part of $c_4$ with $\mathsf{KDF}(g^{x_3r_3})\oplus k$.

    \item Checks if $\Delta_2$ and $\Delta_4$ are both zero pads. If not, outputs $\mathsf{CHEAT}$ and terminates. Otherwise, we know the ciphertext has the following format:
    \[
    \begin{array}{l}
    c_1=g^{r_1}\parallel\mathsf{KDF}(g^{x_1r_1})\oplus s,\\
    c_2=g^{r_2}\parallel\boxed{\mathsf{KDF}(g^{x_2r_2})\oplus k,}\\
    c_3=g^{r_3}\parallel\boxed{\mathsf{KDF}(g^{x_3r_3})\oplus k,}\\
    c_4=\mathsf{AuthEnc}.\mathsf{Enc}_{k'}(\mathsf{msg}).
    \end{array}
    \]

    \item Uses $k$ to decrypt $c_4$. If the decrypted result is $\perp$, the ciphertext is not valid, the algorithm outputs $\mathsf{CHEAT}$ and terminates. Otherwise, outputs $\mathsf{ACCEPT}$, as the ciphertext now has the following format the same as the one from honest encryption:

    \[
    \begin{array}{l}
    c_1=g^{r_1}\parallel\mathsf{KDF}(g^{x_1r_1})\oplus s,\\
    c_2=g^{r_2}\parallel\mathsf{KDF}(g^{x_2r_2})\oplus k,\\
    c_3=g^{r_3}\parallel\mathsf{KDF}(g^{x_3r_3})\oplus k,\\
    c_4=\mathsf{AuthEnc}.\boxed{\mathsf{Enc}_{k}}(\mathsf{msg}).
    \end{array}
    \]
\end{enumerate}
According to the discussion above, if the detection algorithm outputs $\mathsf{ACCEPT}$, three parties will see the same decrypted result.

\end{document}


\subsection{Ordering of rationals}


\documentclass[12pt, answers]{exam}
%\documentclass[12pt]{exam}
\usepackage[top=1in, bottom=1in, left=1in, right=1in]{geometry}
\usepackage{setspace}
\PassOptionsToPackage{hyphens}{url}
\usepackage{tabu}
\usepackage{lscape}
\onehalfspacing
\setlength{\parindent}{0mm} \setlength{\parskip}{1em}

% packages
\RequirePackage{amssymb, amsfonts, amsmath, latexsym, verbatim, xspace, setspace}
\RequirePackage{tikz}
% The float package HAS to load before hyperref
\usepackage{float} % for psuedocode formatting
\usepackage{amsthm}
\usepackage{epsfig}
\usepackage{times}
\renewcommand{\ttdefault}{cmtt}
\usepackage{amsmath}
\usepackage{graphicx} % for graphics files

% for creating indented blocks
\usepackage{scrextend}
\usepackage{paralist, tabularx}

% from Denovo Methods Manual
\usepackage{mathrsfs}
\usepackage[mathcal]{euscript}
\usepackage{color}
\usepackage{array}

\usepackage[pdftex]{hyperref}
\usepackage[parfill]{parskip}
\usepackage{cancel}

\newcommand{\nth}{n\ensuremath{^{\text{th}}} }
\newcommand{\ve}[1]{\ensuremath{\mathbf{#1}}}
\newcommand{\Macro}{\ensuremath{\Sigma}}
\newcommand{\vOmega}{\ensuremath{\hat{\Omega}}}
\newcommand{\cc}[1]{\ensuremath{\overline{#1}}}
\newcommand{\ccm}[1]{\ensuremath{\overline{\mathbf{#1}}}}

%--------------------------------------------------------------------
%--------------------------------------------------------------------
\begin{document}
\begin{center}
{\bf NE 155, Topic 15, S21 \\
2-D Finite Difference/Volume methods\\
April 8, 2021}
\end{center}

\setlength{\unitlength}{1in}
\begin{picture}(6,.1)
\put(0,0) {\line(1,0){6.25}}
\end{picture}

Much of this can be found in Duderstadt and Hamilton, Chp.\ 5 Section II.B.

%-------------------------------------------------------------
\section*{PDEs}

We'll start by considering PDEs in general, and then move on to the Diffusion Equation specifically.

Recall: a partial differential equation is an equation containing an unknown function of two or more variables and its derivatives with respect to those variables.

If the PDE is linear in $u$ and all derivatives of $u$, then we say that the PDE is linear.
%
\begin{equation}
A\frac{\partial^2 u}{\partial x^2} + B\frac{\partial^2 u}{\partial x \partial  y} + C\frac{\partial^2 u}{\partial y^2} + D\frac{\partial u}{\partial x} + E\frac{\partial u}{\partial y} + Fu = G \nonumber
\end{equation}
%
This equation is a \textbf{2nd order} PDE in two variables. It is \textbf{linear} if $A$ through $G$ do not depend on $u$ (they may depend on $x$ and $y$).

%-------------------------------------------------------------
\vspace*{1em}

Recall that we can classify second order PDEs based on the geometric behavior of their solutions.
%
\begin{itemize}
\item Elliptic if $B^2 - 4 AC < 0$.

\item Hyperbolic if $B^2 - 4 AC > 0$

\item Parabolic if $B^2 - 4 AC = 0$
\end{itemize}

%-------------------------------------------------------------
\subsection*{Elliptic Equations}

Recall that the 2-D Laplacian for the function $u(x,y)$ is
%
\ifprintanswers
\begin{equation}
\nabla^2 u(x,y) = \frac{\partial^2 u}{\partial x^2} + \frac{\partial^2 u}{\partial y^2} = u_{xx} + u_{yy} \:.\nonumber
\end{equation}
\else
\\ \vspace*{3em} \\
\fi

%
The Laplacian is used in many of the equations that are commonly found in physics:
%
\begin{itemize}
\item Laplace's equation
\[\nabla^2 u(x,y) = 0\]

\item Poisson's equation
\[\nabla^2 u(x,y) = g(x,y)\]

\item Helmholtz's equation (recall that we can make the DE look like this)
\[\nabla^2 u(x,y) +f(x,y)u(x,y) = g(x,y)\]
\end{itemize}

% -----------------------------------------------------
\section*{2-D Finite Difference}

We will use 5 points to define our central-differenced Laplacian:

\begin{minipage}{0.5\textwidth}
\begin{tikzpicture}
\draw (0,0)--(0,7);
\draw (1,0)--(1,7);
\draw (2,0)--(2,7);
\draw (3,0)--(3,7);
\draw (4,0)--(4,7);
\draw (5,0)--(5,7);
\draw (6,0)--(6,7);
\draw (7,0)--(7,7);
\node[below] at (0,-.25) {$x_0$};
\node[below] at (1,-.25) {$x_1$};
\node[below] at (3,-.25) {$x_{i-1}$};
\node[below] at (4,-.25) {$x_i$};
\node[below] at (5,-.25) {$x_{i+1}$};
\node[below] at (7,-.25) {$x_n$};
\node at (3.5, 2.5) {$\Delta x_i$};
% begin y
\draw (0,0)--(7,0);
\draw (0,1)--(7,1);
\draw (0,2)--(7,2);
\draw (0,3)--(7,3);
\draw (0,4)--(7,4);
\draw (0,5)--(7,5);
\draw (0,6)--(7,6);
\draw (0,7)--(7,7);
\node[left] at (-.25, 0) {$y_0$};
\node[left] at (-.25,1) {$y_1$};
\node[left] at (-.25,3) {$y_{j-1}$};
\node[left] at (-.25,4) {$y_j$};
\node[left] at (-.25,5) {$y_{j+1}$};
\node[left] at (-.25,7) {$y_m$};
  \node at (2.5,3.5) {$\Delta y_j$};
% labels
\node at (4,4) [circle,fill=black,scale=0.3] {};
\node[below left] at (4,4) {$i,j$};
\node at (5,4) [circle,fill=black,scale=0.3] {};
\node[below right] at (5,4) {$i+1,j$};
\node at (3,4) [circle,fill=black,scale=0.3] {};
\node[above left] at (3,4) {$i-1,j$};
\node at (4,5) [circle,fill=black,scale=0.3] {};
\node[above right] at (4,5) {$i,j+1$};
\node at (4,3) [circle,fill=black,scale=0.3] {};
\node[below right] at (4,3) {$i,j-1$};
\end{tikzpicture}
\end{minipage} \hfill
%
\begin{minipage}{0.5\textwidth}
  \[u(x_i,y_j) = u_{i,j} \qquad u(x_{i+1},y_j) = u_{i+1,j}\]
  \begin{align}
  \frac{\partial^2 u_{i,j}}{\partial x^2} = \frac{\partial}{\partial x}\bigl(\frac{\partial u_{i,j}}{\partial x}\bigr) =
\frac{u_{i-1,j} - 2u_{i,j} + u_{i+1,j}}{\Delta x_i^2} \nonumber \\
%
  \frac{\partial^2 u_{i,j}}{\partial y^2} = \frac{\partial}{\partial y}\bigl(\frac{\partial u_{i,j}}{\partial y}\bigr) =
\frac{u_{i,j-1} - 2u_{i,j} + u_{i,j+1}}{\Delta y_j^2} \nonumber
\end{align}
\end{minipage}

All together this gives
\ifprintanswers
\[\nabla^2 u_{i,j} =
\frac{\partial^2 u_{i,j}}{\partial x^2} + \frac{\partial^2 u_{i,j}}{\partial y^2} =
\frac{u_{i-1,j} - 2u_{i,j} + u_{i+1,j}}{\Delta x_i^2} + \frac{u_{i,j-1} - 2u_{i,j} + u_{i,j+1}}{\Delta y_j^2}\]
%
If $\Delta x_i = $ constant $= \Delta y_j = h$, then
%
\[ \nabla^2 u_{i,j} = \frac{u_{i+1,j} + u_{i-1,j} + u_{i,j+1} + u_{i,j-1} - 4u_{i,j}}{h^2}\]
\else
\vspace*{9em}
\fi

If we apply this to Laplace's equation and have fixed boundary conditions:
%
\begin{align}
u_{i+1,j} &+ u_{i-1,j} + u_{i,j+1} + u_{i,j-1} - 4u_{i,j} = 0, \quad i = 1, 2, \dots, n-1, \quad j = 1, 2, \dots, m-1 \nonumber \\
%
u_{0,j} &= BC_L , \quad j = 1, 2, \dots, m-1 \nonumber \\
u_{i,0} &= BC_B , \quad i = 1, 2, \dots, n-1 \nonumber \\
u_{n,j} &= BC_R , \quad j = 1, 2, \dots, m-1 \nonumber \\
u_{i.m} &= BC_T , \quad i = 1, 2, \dots, n-1 \nonumber
\end{align}
%
Make sure to check how the corners need to be defined.

We could apply this directly the the 2-D diffusion equation.

%--------------------------------------------------------------------
\section*{2-D Finite Volume Method, Diffusion Equation}

Remember that in 1-D we were solving this
\[-\frac{d}{dx}D(x)\frac{d \phi(x)}{dx} + \Sigma_a(x) \phi(x) = S(x)\]
%
with an equilibrium (reflecting) condition at the centerline ($x_0 = 0$) and vacuum on the right ($x_n = a$):
\begin{align}
\frac{d}{dx}\phi(x) \big|_{x=0} &= 0 \qquad \text{zero net current,} \nonumber\\
\phi(\tilde{a}) &= 0 \qquad \tilde{a} = a + 2D \:.\nonumber
\end{align}

We imposed a spatial mesh and said material discontinuities will coincide with the cell edges, $x_i$. Thus, we can assume that the cross sections and the diffusion coefficient are constant in each cell.
%
%\begin{align}
%D(x) &= D_i \qquad \text{for } x_{i-1} \leq x_i \nonumber \\
%\Sigma_{a}(x) &= \Sigma_{a,i} \qquad \text{for } x_{i-1} \leq x_i \nonumber \\
%h_i &\equiv x_i - x_{i-1} \nonumber
%\end{align}
%
The unknown fluxes and known sources are defined at the mesh or cell edges.
%\begin{align}
%\phi(x_i) &= \phi_i \nonumber \\
%S(x_i) &= S_i \nonumber
%\end{align}
%
We then went on to define things for the finite volume method.
%\begin{center}
%\begin{figure}[h!]
%\includegraphics[height=2.5in]{../figs/FVM-DE}
%\end{figure}
%\end{center}

Now, we're going to extend all of that to two dimensions. We'll start with the multi-D equation:
\[-\nabla \cdot \bigl(D(\vec{r})\nabla \phi(\vec{r})\bigr) + \Sigma_a(\vec{r}) \phi(\vec{r}) = S(\vec{r})\:.\]
%
This equation is elliptic:
\begin{itemize}
\item in general, this is the Helmholtz equation.
\item if $\Sigma_a(\vec{r})=0$ and $D(\vec{r})=$ constant, then this takes the form of the Poisson equation.
\item if $\Sigma_a(\vec{r})=0$ and $S(\vec{r})=0$, then this becomes Laplace's equation.
\end{itemize}

In 2-D:
\ifprintanswers
\[-\frac{\partial}{\partial x}D(x,y)\frac{\partial}{\partial x} \phi(x,y) -\frac{\partial}{\partial y}D(x,y)\frac{\partial}{\partial y} \phi(x,y) + \Sigma_a(x,y) \phi(x,y) = S(x,y)\:.\]
\else
\\ \vspace*{2em} \\
\fi
%
For now we're going to say we have fixed boundary conditions:
\[\phi(-a,y) = \Phi_L\:, \quad \phi(a,y) = \Phi_R\:, \quad \phi(x,-b) = \Phi_B\:, \quad \phi(x,b) = \Phi_T\:,\]
with $x \in [-a,a]$ and $y \in [-b,b]$.

% ---------------------------------------------------------
% ---------------------------------------------------------
\subsection*{Finite Volume}
%
%\begin{figure}[h!]
%\begin{center}
%\includegraphics[height=5in]{../figs/2DfvmGrid}
%\end{center}
%\end{figure}

\begin{center}
\begin{tikzpicture}
% grid
\draw[step=2cm] (2,2) grid (6,6);
% volume labels
\node at (5,3) {$V_{i+1,j}$};
\node at (5,5) {$V_{i+1,j+1}$};
\node at (3,3) {$V_{i,j}$};
\node at (3,5) {$V_{i,j+1}$};
\node at (1,7) {$D_{i,j+1}$};
% surface labels
\node[left] at (1.75,3) {$s_1$};
\node[left] at (1.75,5) {$s_8$};
\node[above] at (3,6.25) {$s_7$};
\node[above] at (5,6.25) {$s_6$};
\node[right] at (6.25,5) {$s_5$};
\node[right] at (6.25,3) {$s_4$};
\node[below] at (3,1.75) {$s_2$};
\node[below] at (5,1.75) {$s_3$};
% surrounding grid and markers
\draw plot[mark=*, mark options={fill=black}] (4,4) node[below] {$(i,j)$};
\draw (4,4) -- plot[mark=*, mark options={fill=black}] (8,4) node[right] {$(i+1,j)$};
\draw (4,4)-- plot[mark=*, mark options={fill=black}] (4,0) node[below] {$(i,j-1)$};
\draw (4,4)-- plot[mark=*, mark options={fill=black}] (4,8) node[above] {$(i,j+1)$};
\draw (4,4)-- plot[mark=*, mark options={fill=black}] (0,4) node[above] {$(i-1,j)$};
% arrows
\draw[<->](7.5,4) -- (7.5,8);
\node[right] at (7.5,6) {$\epsilon_{j+1}$};
\draw[<->](7.5,0) -- (7.5,4);
\node[right] at (7.5,2) {$\epsilon_{j}$};
\draw[<->](1,2) -- (1,4);
\node[left] at (1,3) {$\epsilon_{j}/2$};
\draw[<->](4,0.5) -- (8,0.5);
\node[below] at (6, 0.5) {$\delta_{i+1}$};
\draw[<->](0,0.5) -- (4,0.5);
\node[below] at (2, 0.5) {$\delta_{i}$};
\draw[<->](4,7) -- (6,7);
\node[above] at (5, 7) {$\delta_{i+1}/2$};
\end{tikzpicture}
\end{center}

We assume the diffusion coefficient, absorption cross section, and source are constant in each cell (cell-centered), e.g., for $V_{i,j}$:
\begin{align}
D(x,y) &= D_{i,j}\;, \qquad x_{i-1} \leq x \leq x_i \:\text{ and }\: y_{j-1} \leq y \leq y_j \nonumber \\
%
\Sigma_a(x,y) &= \Sigma_{a,i,j}\;, \qquad x_{i-1} \leq x \leq x_i \:\text{ and }\: y_{j-1} \leq y \leq y_j \nonumber \\
%
S(x,y) &= S_{i,j}\;, \qquad x_{i-1} \leq x \leq x_i \:\text{ and }\: y_{j-1} \leq y \leq y_j \nonumber \\
%
\Delta x_i &\equiv \delta_i = x_{i} - x_{i-1} \nonumber \\
\Delta y_j &\equiv \epsilon_j = y_{j} - y_{j-1} \nonumber
\end{align}
And analogously for the other 3 cells.

We further assume that the fluxes are constant over the interval centered around $(x_i, y_j)$ (edge-centered):
%
\ifprintanswers
\[\phi(x,y) = \phi_{i,j} \qquad \text{for } \bigl(x_i - \frac{\delta_i}{2}\bigr) \leq x \leq \bigl(x_i + \frac{\delta_{i+1}}{2}\bigr) \:\text{ and }\:\bigl(y_j - \frac{\epsilon_j}{2}\bigr) \leq x \leq \bigl(y_j + \frac{\epsilon_{j+1}}{2}\bigr) \]
\else
\\ \vspace*{3em}
\fi

Now, we integrate the 2-D equation over 4 rectangular partial-volumes (well, areas, technically): $V = V_{i,j} + V_{i+1,j} + V_{i,j+1} + V_{i+1,j+1}$.
%
\[\int_V d\vec{r}\:\bigl[-\nabla \cdot \bigl(D(\vec{r})\nabla \phi(\vec{r})\bigr) +\Sigma_a(\vec{r}) \phi(\vec{r}) = S(\vec{r}) \bigr]\]

% ---------------------------------------------------------
\subsubsection*{Streaming Term}
Use Gauss Theorem to replace the first volume integral with a surface integral:
%
\ifprintanswers
\begin{align}
-\int_V d\vec{r}\:\bigl[\nabla \cdot \bigl(D(\vec{r})\nabla \phi(\vec{r})\bigr)\bigr] &= -\int_S d\vec{S} \cdot\bigl(D(\vec{r})\nabla \phi(\vec{r})\bigr) \nonumber \\
%
= -\int_S d\vec{S}\: D(\vec{r})\hat{n} \cdot \nabla \phi(\vec{r}) &= -\int_S d\vec{S} \:D(\vec{r})\frac{\partial}{\partial \hat{n}}\phi(\vec{r})\:. \nonumber
\end{align}
\else
\\ \vspace*{4em} \\
\fi
%
Next, we define the partial derivative w.r.t.\ direction on each surface, using $O(h)$ forward and backward difference schemes since they're simple:

\begin{align}
\frac{\partial}{\partial \hat{n}}\phi(\vec{r}) &= \frac{\phi_{i,j-1} - \phi_{i,j}}{\epsilon_j} \qquad \text{on } S_2 \:, S_3 \quad \text{(bottom)} \nonumber \\
%
&= \frac{\phi_{i,j+1} - \phi_{i,j}}{\epsilon_{j+1}} \qquad \text{on } S_7 \:, S_6 \quad \text{(top)} \nonumber \\
%
&= \frac{\phi_{i-1,j} - \phi_{i,j}}{\delta_{i}} \qquad \text{on } S_1 \:, S_8 \quad \text{(left)} \nonumber \\
%
&= \frac{\phi_{i+1,j} - \phi_{i,j}}{\delta_{i+1}} \qquad \text{on } S_4 \:, S_5 \quad \text{(right)} \nonumber
\end{align}
%
We then use the midpoint rule for the integration and integrate along each surface. Recall that the physics values are cell-centered while the flux is edge-centered. E.g., surfaces $S_2$ and $S_3$:
%
\begin{align*}
-\int_{S_2+S_3} d\vec{S} \:D(\vec{r})\frac{\partial}{\partial \hat{n}}\phi(\vec{r}) &= \boxed{\frac{\phi_{i,j} - \phi_{i,j-1}}{\epsilon_{j}} \biggl(\frac{D_{i,j} \delta_{i} + D_{i+1,j} \delta_{i+1}}{2}\biggr)}\:,\\
%
S_6+S_7 &= \boxed{\frac{\phi_{i,j} - \phi_{i,j+1}}{\epsilon_{j+1}} \biggl(\frac{D_{i,j+1} \delta_{i} + D_{i+1,j+1} \delta_{i+1}}{2}\biggr)}\:,\\
%
S_1+S_8 &= \boxed{\frac{\phi_{i,j} - \phi_{i-1,j}}{\delta_{i}} \biggl(\frac{D_{i,j} \epsilon_{j} + D_{i,j+1} \epsilon_{j+1}}{2}\biggr)}\:,\\
%
S_4+S_5 &= \boxed{\frac{\phi_{i,j} - \phi_{i+1,j}}{\delta_{i+1}} \biggl(\frac{D_{i+1,j} \epsilon_{j} + D_{i+1,j+1} \epsilon_{j+1}}{2}\biggr)}\:,
\end{align*}
%
and we do this for each set of surfaces.

% ---------------------------------------------------------
\subsubsection*{Absorption Term}
To integrate absorption, we do four integrals - one over each sub-volume:
%
\ifprintanswers
\begin{align}
\int_{x_i-\frac{\delta_{i}}{2}}^{x_i+\frac{\delta_{i+1}}{2}} dx \int_{y_j-\frac{\epsilon_{j}}{2}}^{y_j+\frac{\epsilon_{j+1}}{2}}dy\:\Sigma_a(x,y) \phi(x,y) &= \Sigma_{a,i,j}\int\int_{V_{i,j}} dx dy \: \phi(x,y) + \nonumber \\
%
\Sigma_{a,i+1,j}\int\int_{V_{i+1,j}} dx dy \: \phi(x,y) &+ \Sigma_{a,i+1,j+1}\int\int_{V_{i+1,j+1}} dx dy \: \phi(x,y) + \nonumber \\
 \Sigma_{a,i,j+1}\int\int_{V_{i,j+1}} dx dy \: \phi(x,y) \:.\nonumber
\end{align}
\else
\\ \vspace*{8em}\\
\fi
%
Again applying the midpoint scheme and using our edge-centered flux definition:
%
\begin{align}
\int \int dx dy\:\Sigma_a(x,y) \phi(x,y) &= \boxed{\phi_{i,j}\bigl(\Sigma_{a,i,j} V_{i,j} + \Sigma_{a,i+1,j} V_{i+1,j} + \Sigma_{a,i+1,j+1} V_{i+1,j+1} + \Sigma_{a,i,j+1} V_{i,j+1} \bigr) } \nonumber \\
&\equiv \phi_{i,j}\Sigma_{a,ij}\:, \nonumber
\end{align}
%
where
\[V_{i,j} = \frac{1}{4}\delta_i \epsilon_j \:, \quad V_{i+1,j} = \frac{1}{4}\delta_{i+1} \epsilon_{j} \:, \quad V_{i+1,j+1} = \frac{1}{4}\delta_{i+1} \epsilon_{j+1} \:, \quad V_{i,j+1} = \frac{1}{4}\delta_{i} \epsilon_{j+1} \:.\]

% ---------------------------------------------------------
\subsubsection*{Source Term}
Using the same procedure for the source, we get:
\begin{align}
\int \int dx dy \: \:S(x,y) &= \boxed{ S_{i,j} V_{i,j} + S_{i+1,j} V_{i+1,j} + S_{i+1,j+1} V_{i+1,j+1} + S_{i,j+1} V_{i,j+1} }\nonumber \\
&\equiv S_{ij}\:. \nonumber
\end{align}

% ---------------------------------------------------------
\subsubsection*{Discretized Equations}
\ifprintanswers
Collecting all of the terms and separating them, we get a 5-point difference equation for $i=1,\dots,n-1$; $j=1,\dots,m-1$:
%
\[a_{i-1,j}^{ij}\phi_{i-1,j} + a_{i+1,j}^{ij}\phi_{i+1,j} + a_{i,j-1}^{ij}\phi_{i,j-1} + a_{i,j+1}^{ij}\phi_{i,j+1} +  a_{i,j}^{ij}\phi_{i,j} = S_{ij} \:.\]
%
\else
\vspace*{3em}
\fi
The lower index is the cell to which you are coupling, and the upper index is which cell you are in--this becomes important when we're ordering our matrix.
%
\begin{align}
a_{i-1,j}^{ij} &= -\frac{D_{i,j} \epsilon_{j} + D_{i,j+1} \epsilon_{j+1}}{2 \delta_{i}}  \nonumber \\
(a_L^{ij} & \quad \text{capturing influence of \textbf{left} flux on center cell}) \nonumber \\
%
a_{i+1,j}^{ij} &= -\frac{D_{i+1,j} \epsilon_{j} + D_{i+1,j+1} \epsilon_{j+1}}{2 \delta_{i+1}}  \nonumber \\
(a_R^{ij} & \quad \text{capturing influence of \textbf{right} flux on center cell}) \nonumber \\
%
a_{i,j-1}^{ij} &= -\frac{D_{i,j} \delta_{i} + D_{i+1,j} \delta_{i+1}}{2 \epsilon_{j}}  \nonumber \\
(a_B^{ij} & \quad \text{capturing influence of \textbf{lower} flux on center cell}) \nonumber \\
%
a_{i,j+1}^{ij} &= -\frac{D_{i,j+1} \delta_{i} + D_{i+1,j+1} \delta_{i+1}}{2 \epsilon_{j+1}}  \nonumber \\
(a_T^{ij} & \quad \text{capturing influence of \textbf{upper} flux on center cell}) \nonumber \\
%
\nonumber \\
a_{i,j}^{ij} &= \Sigma_{a,ij} - \bigl(a_{i-1,j}^{ij} + a_{i+1,j}^{ij} + a_{i,j-1}^{ij} + a_{i,j+1}^{ij} \bigr)
 \:.\nonumber \\
 (a_C^{ij}) & \nonumber
\end{align}

In this formulation we now have $(n+1) \times (m+1)$ unknowns.

%-------------------------------------------------------
\subsection*{Matrix Form}
To write the system in a way that looks like $\ve{A}\vec{x} = \vec{b}$, we need to choose an ordering strategy for how we want to store the points. Let's look at a 3$\times$3 example:

\begin{minipage}{0.5\textwidth}
%-------------- indices ------------------
\begin{tikzpicture}
\draw[step=2cm] (0,0) grid (4,4);
\foreach \y in {1, 2}
    \foreach \x in  {0, 1, 2}
        \node[below left] at (2*\x cm,2*\y cm) {$(\x,\y)$};
\node[below] at (0,-.25) {$(0,0)$};
\node[below] at (2,-.25) {$(1,0)$};
\node[below] at (4,-.25) {$(2,0)$};
\end{tikzpicture}
\end{minipage} \hfill
%-------------- numbered nodes ------------------
\begin{minipage}{0.5\textwidth}
\begin{tikzpicture}
\draw[step=2cm] (0,0) grid (4,4);
\node[below] at (0,-.25) {$0$};
\node[below] at (2,-.25) {$1$};
\node[below] at (4,-.25) {$2$};
\node[below left] at (-.25,2) {$3$};
\node[below left] at (2,2) {$4$};
\node[below left] at (4,2) {$5$};
\node[below left] at (-.25,4) {$6$};
\node[below left] at (2,4) {$7$};
\node[below left] at (4,4) {$8$};
\end{tikzpicture}
\end{minipage}

Our solution vector length in this example is 9 (3 $\times$ 3). Our matrix size is therefore going to be 9 $\times$ 9. We can think of this as a 3 $\times$ 3 matrix of 3 $\times$ 3 matrices:
\begin{equation}
\underbrace{\begin{pmatrix}
\begin{pmatrix}
a_{0,0}^{00} & a_{1,0}^{00} & 0 \\
a_{0,0}^{10} & a_{1,0}^{10} & a_{2,0}^{10} \\
0            & a_{1,0}^{20} & a_{2,0}^{20}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,1}^{00} & 0 & 0 \\
0 & a_{1,1}^{10} & 0 \\
0 & 0 & a_{2,1}^{20}
\end{pmatrix}
&
\begin{pmatrix}
 & & \\
 & 0 & \\
 & &
\end{pmatrix} \\
%--------------------
\begin{pmatrix}
a_{0,0}^{01} & 0 & 0 \\
0 & a_{1,0}^{11} & 0 \\
0 & 0 & a_{2,0}^{21}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,1}^{01} & a_{1,1}^{01} & 0 \\
a_{0,1}^{11} & a_{1,1}^{11} & a_{2,1}^{11} \\
0            & a_{1,1}^{21} & a_{2,1}^{21}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,2}^{01} & 0 & 0 \\
0 & a_{1,2}^{11} & 0 \\
0 & 0 & a_{2,2}^{21}
\end{pmatrix} \\
%--------------------
\begin{pmatrix}
 & & \\
 & 0 & \\
 & &
\end{pmatrix} &
\begin{pmatrix}
a_{0,1}^{02} & 0 & 0 \\
0 & a_{1,1}^{12} & 0 \\
0 & 0 & a_{2,1}^{22}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,2}^{02} & a_{1,2}^{02} & 0 \\
a_{0,2}^{12} & a_{1,2}^{12} & a_{2,2}^{12} \\
0            & a_{1,2}^{22} & a_{2,2}^{22}
\end{pmatrix} \\
\end{pmatrix}}_{\ve{A}}
%--------------------
%
\underbrace{\begin{pmatrix} \phi_{0,0} \\ \phi_{1,0} \\ \phi_{2,0} \\ \\ \phi_{0,1} \\ \phi_{1,1} \\ \phi_{2,1} \\ \\ \phi_{0,2}\\ \phi_{1,2} \\  \phi_{2,2} \end{pmatrix}}_{\vec{\phi}} =
%
\underbrace{\begin{pmatrix} S_{00} \\ S_{10} \\ S_{20} \\ \\ S_{01} \\ S_{11} \\ S_{21} \\ \\ S_{02} \\ S_{12} \\  S_{22} \end{pmatrix}}_{\vec{S}} \nonumber
\end{equation}

%
%
Another way to think of $\ve{A}$ is
\begin{equation}
\begin{pmatrix}
\begin{pmatrix}
a_{C}^{00} & a_{R}^{00} & 0 \\
a_{L}^{10} & a_{C}^{10} & a_{R}^{10} \\
0            & a_{L}^{20} & a_{C}^{20}
\end{pmatrix}
&
\begin{pmatrix}
a_{T}^{00} & 0 & 0 \\
0 & a_{T}^{10} & 0 \\
0 & 0 & a_{T}^{20}
\end{pmatrix}
&
\begin{pmatrix}
 & & \\
 & 0 & \\
 & &
\end{pmatrix} \\
%--------------------
\begin{pmatrix}
a_{B}^{01} & 0 & 0 \\
0 & a_{B}^{11} & 0 \\
0 & 0 & a_{B}^{21}
\end{pmatrix}
&
\begin{pmatrix}
a_{C}^{01} & a_{R}^{01} & 0 \\
a_{L}^{11} & a_{C}^{11} & a_{R}^{11} \\
0            & a_{L}^{21} & a_{C}^{21}
\end{pmatrix}
&
\begin{pmatrix}
a_{T}^{01} & 0 & 0 \\
0 & a_{T}^{11} & 0 \\
0 & 0 & a_{T}^{21}
\end{pmatrix} \\
%--------------------
\begin{pmatrix}
 & & \\
 & 0 & \\
 & &
\end{pmatrix} &
\begin{pmatrix}
a_{B}^{02} & 0 & 0 \\
0 & a_{B}^{12} & 0 \\
0 & 0 & a_{B}^{22}
\end{pmatrix}
&
\begin{pmatrix}
a_{C}^{02} & a_{R}^{02} & 0 \\
a_{L}^{12} & a_{C}^{12} & a_{R}^{12} \\
0            & a_{L}^{22} & a_{C}^{22}
\end{pmatrix} \\
\end{pmatrix} \nonumber
\end{equation}

This gives us a 5-banded matrix, which is not so difficult to represent with built in functions in Python or MATLAB. We could still solve this directly with Gaussian elimination or LU decomposition, but the size of the system increases pretty rapidly with mesh size.

Therefore using an iterative method like Jacobi, GS, or SOR is probably a better plan.

%-------------------------------------------------------
\subsection*{Boundary Conditions}

This all works fine for central points, but what about the boundaries? When we're at the edges, we get four entries rather than five because the edge values are known. For corners we have only three entries.

Let's see how this impacts our equations. Recall:
%
\[\phi(-a,y) = \Phi_L\:, \quad \phi(a,y) = \Phi_R\:, \quad \phi(x,-b) = \Phi_B\:, \quad \phi(x,b) = \Phi_T\:.\]
%
Let's choose what to do at the corners, and then define the rest of the boundaries:
%
\ifprintanswers
\begin{align}
\phi_{0,0} = \Phi_B\:, \quad \phi_{n,0} &= \Phi_R\:, \quad \phi_{0,m} = \Phi_L\:, \quad \phi_{n,m} = \Phi_T\:. \nonumber \\
\phi_{0,j} &= \Phi_L \qquad j=1,\dots,m-1 \:, \nonumber \\
\phi_{n,j} &= \Phi_R \qquad j=1,\dots,m-1 \:, \nonumber \\
\phi_{i,0} &= \Phi_B \qquad i=1,\dots,n-1 \:, \nonumber \\
\phi_{i,m} &= \Phi_T \qquad i=1,\dots,n-1 \:. \nonumber
\end{align}
\else
\\ \vspace*{8em}
\fi

Let's look at how this would impact the left ($i=0$) boundary. The $i=0$ equations are simply the boundary condition:
\begin{itemize}
\item Change the $a_{C}^{0,j}$ entries to $1$s,
\item the $a_{x}^{0,j}$, where $x$ is $R, T$, and $B$ to $0$, and
\item the $S_{0,j}$ entries to $\Phi_L$.
\end{itemize}

For the $i=1$ equations:
\[\underbrace{a_{2,j}^{1j}}_{R}\phi_{2,j} + \underbrace{a_{1,j-1}^{1j}}_{B}\phi_{1,j-1} + \underbrace{a_{1,j+1}^{1j}}_{T}\phi_{1,j+1} + \underbrace{a_{1,j}^{1j}}_{C}\phi_{1,j} = S_{1j} - \underbrace{a_{0,j}^{1j}}_{L}\phi_L \]

And analogously for the rest of the edges.

\subsection*{Larger Matrix}
What about a bigger matrix?

\begin{minipage}{0.5\textwidth}
%-------------- indices ------------------
\begin{tikzpicture}
\draw[step=2cm] (0,0) grid (6,6);
\foreach \y in {1, 2, 3}
    \foreach \x in  {0, 1, 2, 3}
        \node[below left] at (2*\x cm,2*\y cm) {$(\x,\y)$};
\node[below] at (0,-.25) {$(0,0)$};
\node[below] at (2,-.25) {$(1,0)$};
\node[below] at (4,-.25) {$(2,0)$};
\node[below] at (6,-.25) {$(3,0)$};
\end{tikzpicture}
\end{minipage} \hfill
%-------------- numbered nodes ------------------
\begin{minipage}{0.5\textwidth}
\begin{tikzpicture}
\draw[step=2cm] (0,0) grid (6,6);
\node[below] at (0,-.25) {$0$};
\node[below] at (2,-.25) {$1$};
\node[below] at (4,-.25) {$2$};
\node[below] at (6,-.25) {$3$};
\node[below left] at (-.25,2) {$4$};
\node[below left] at (2,2) {$5$};
\node[below left] at (4,2) {$6$};
\node[below left] at (6,2) {$7$};
\node[below left] at (-.25,4) {$8$};
\node[below left] at (2,4) {$9$};
\node[below left] at (4,4) {$10$};
\node[below left] at (6,4) {$11$};
\node[below left] at (-.25,6) {$12$};
\node[below left] at (2,6) {$13$};
\node[below left] at (4,6) {$14$};
\node[below left] at (6,6) {$15$};
\end{tikzpicture}
\end{minipage}

Our solution vector length in this example is 16 (4 $\times$ 4). Our matrix size is therefore going to be 16 $\times$ 16. We can think of this as a 4 $\times$ 4 matrix of 4 $\times$ 4 matrices:
\begin{equation}
\underbrace{\begin{pmatrix}
\begin{pmatrix}
a_{0,0}^{00} & a_{1,0}^{00} & 0            & 0\\
a_{0,0}^{10} & a_{1,0}^{10} & a_{2,0}^{10} & 0 \\
0            & a_{1,0}^{20} & a_{2,0}^{20} & a_{3,0}^{20} \\
0            & 0            & a_{2,0}^{30} & a_{3,0}^{30}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,1}^{00} & 0 & 0 & 0 \\
0 & a_{1,1}^{10} & 0 & 0 \\
0 & 0 & a_{2,1}^{20} & 0 \\
0 & 0 & 0 & a_{3,1}^{30}  \\
\end{pmatrix}
&
\begin{pmatrix}
 & & & \\
 & 0 & &\\
 & & & \\
 & & &
\end{pmatrix}
&
\begin{pmatrix}
 & & & \\
 & 0 & &\\
 & & & \\
 & & &
\end{pmatrix}\\
%--------------------
\begin{pmatrix}
a_{0,0}^{01} & 0 & 0 & 0\\
0 & a_{1,0}^{11} & 0 & 0\\
0 & 0 & a_{2,0}^{21} & 0\\
0 & 0 & 0 & a_{3,0}^{31}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,1}^{01} & a_{1,1}^{01} & 0            & 0\\
a_{0,1}^{11} & a_{1,1}^{11} & a_{2,1}^{11} & 0\\
0            & a_{1,1}^{21} & a_{2,1}^{21} & a_{3,1}^{21} \\
0            & )            & a_{2,1}^{31} & a_{3,1}^{31}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,2}^{01} & 0 & 0 & 0\\
0 & a_{1,2}^{11} & 0 & 0\\
0 & 0 & a_{2,2}^{21} & 0\\
0 & 0  & 0 & a_{3,2}^{31}
\end{pmatrix}
&
\begin{pmatrix}
 & & & \\
 & 0 & &\\
 & & & \\
 & & &
\end{pmatrix}\\
%--------------------
\begin{pmatrix}
 & & & \\
 & 0 & &\\
 & & & \\
 & & &
\end{pmatrix}
&
\begin{pmatrix}
a_{0,1}^{02} & 0 & 0 & 0\\
0 & a_{1,1}^{12} & 0 & 0\\
0 & 0 & a_{2,1}^{22} & 0\\
0 & 0  & 0 & a_{3,1}^{32}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,2}^{02} & a_{1,2}^{02} & 0            & 0\\
a_{0,2}^{12} & a_{1,2}^{12} & a_{2,2}^{12} & 0\\
0            & a_{1,2}^{22} & a_{2,2}^{22} & a_{3,2}^{22}\\
0            & 0            & a_{2,2}^{32} & a_{3,2}^{32}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,3}^{02} & 0 & 0 & 0\\
0 & a_{1,3}^{12} & 0 & 0\\
0 & 0 & a_{2,3}^{22} & 0\\
0 & 0  & 0 & a_{3,3}^{32}
\end{pmatrix} \\
%--------------------
\begin{pmatrix}
 & & & \\
 & 0 & &\\
 & & & \\
 & & &
\end{pmatrix}
&
\begin{pmatrix}
 & & & \\
 & 0 & &\\
 & & & \\
 & & &
\end{pmatrix}
&
\begin{pmatrix}
a_{0,2}^{03} & 0 & 0 & 0\\
0 & a_{1,2}^{13} & 0 & 0\\
0 & 0 & a_{2,2}^{23} & 0\\
0 & 0  & 0 & a_{3,2}^{33}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,3}^{03} & a_{1,3}^{03} & 0            & 0\\
a_{0,3}^{13} & a_{1,3}^{13} & a_{2,3}^{13} & 0\\
0            & a_{1,3}^{23} & a_{2,3}^{23} & a_{3,3}^{23}\\
0            & 0            & a_{2,3}^{33} & a_{3,3}^{33}
\end{pmatrix}
\end{pmatrix}}_{\ve{A}} \nonumber
\end{equation}
%--------------------
%
\begin{equation}
\underbrace{\begin{pmatrix}
\phi_{0,0} \\ \phi_{1,0} \\ \phi_{2,0} \\ \phi_{3,0} \\ \\
\phi_{0,1} \\ \phi_{1,1} \\ \phi_{2,1} \\ \phi_{3,1} \\ \\
\phi_{0,2} \\ \phi_{1,2} \\ \phi_{2,2} \\ \phi_{3,2} \\ \\
\phi_{0,3} \\ \phi_{1,3} \\ \phi_{2,3} \\ \phi_{3,3} \end{pmatrix}}_{\vec{\phi}} =
%
\underbrace{\begin{pmatrix}
S_{00} \\ S_{10} \\ S_{20} \\ S_{30} \\ \\
S_{01} \\ S_{11} \\ S_{21} \\ S_{31} \\ \\
S_{02} \\ S_{12} \\ S_{22} \\ S_{32} \\ \\
S_{03} \\ S_{13} \\ S_{23} \\ S_{33}
\end{pmatrix}}_{\vec{S}} \nonumber
\end{equation}

\begin{minipage}{0.5\textwidth}
%-------------- indices ------------------
\begin{tikzpicture}
\draw[step=2cm] (0,0) grid (6,6);
\foreach \y in {1}
    \foreach \x in  {0, 1}
        \node[below left] at (2*\x cm,2*\y cm) {$(\x,\y)$};
\node[below] at (0,-.25) {$(0,0)$};
\node[below] at (2,-.25) {$(1,0)$};
\node[below] at (4,-.25) {$...$};
\node[below] at (6,-.25) {$(n,0)$};
\node[below left] at (0,4) {$\vdots$};
\node[below left] at (0,6) {$(0,m)$};
\node[below left] at (2,6) {$(1,m)$};
\node[below left] at (4,6) {$(n-1,m)$};
\node[below left] at (6,6) {$(n,m)$};
\node[below left] at (6,2) {$(n,1)$};
\node[below left] at (6,4) {$(n,m-1)$};
\node[below left] at (6,6) {$(n,m)$};
\end{tikzpicture}
\end{minipage} \hfill
%-------------- numbered nodes ------------------
\begin{minipage}{0.5\textwidth}
\begin{tikzpicture}
\draw[step=2cm] (0,0) grid (6,6);
\node[below] at (0,-.25) {$0$};
\node[below] at (2,-.25) {$1$};
\node[below] at (4,-.25) {$2$};
\node[below] at (6,-.25) {$3$};
\node[below left] at (-.25,2) {$4$};
\node[below left] at (2,2) {$5$};
\node[below left] at (4,2) {$6$};
\node[below left] at (6,2) {$7$};
\node[below left] at (-.25,4) {$8$};
\node[below left] at (2,4) {$9$};
\node[below left] at (4,4) {$10$};
\node[below left] at (6,4) {$11$};
\node[below left] at (-.25,6) {$12$};
\node[below left] at (2,6) {$13$};
\node[below left] at (4,6) {$14$};
\node[below left] at (6,6) {$15$};
\end{tikzpicture}
\end{minipage}

Our solution vector length in this example is $1 \times nm$. Our matrix size is therefore going to be $n m \times n m$. We can think of this as a $n \times m$ matrix of $n \times m$ matrices: ($n$ rows; $m$ columns)

\begin{landscape}
\begin{equation}
\underbrace{\begin{pmatrix}
\begin{pmatrix}
a_{0,0}^{00} & a_{1,0}^{00} & 0            & 0\\
a_{0,0}^{10} & a_{1,0}^{10} & a_{2,0}^{10} & 0 \\
0            & a_{1,0}^{20} & a_{2,0}^{20} & a_{n,0}^{20} \\
0            & 0            & a_{2,0}^{n0} & a_{n,0}^{n0}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,1}^{00} & 0 & 0 & 0 \\
0 & a_{1,1}^{10} & 0 & 0 \\
0 & 0 & a_{2,1}^{20} & 0 \\
0 & 0 & 0 & a_{n,1}^{n0}  \\
\end{pmatrix}
&
\begin{pmatrix}
 & & & \\
 & 0 & &\\
 & & & \\
 & & &
\end{pmatrix}
&
\begin{pmatrix}
 & & & \\
 & 0 & &\\
 & & & \\
 & & &
\end{pmatrix}\\
%--------------------
\begin{pmatrix}
a_{0,0}^{01} & 0 & 0 & 0\\
0 & a_{1,0}^{11} & 0 & 0\\
0 & 0 & a_{2,0}^{21} & 0\\
0 & 0 & 0 & a_{n,0}^{n1}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,1}^{01} & a_{1,1}^{01} & 0            & 0\\
a_{0,1}^{11} & a_{1,1}^{11} & a_{2,1}^{11} & 0\\
0            & a_{1,1}^{21} & a_{2,1}^{21} & a_{n,1}^{21} \\
0            & 0            & a_{2,1}^{n1} & a_{n,1}^{n1}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,2}^{01} & 0 & 0 & 0\\
0 & a_{1,2}^{11} & 0 & 0\\
0 & 0 & a_{2,2}^{21} & 0\\
0 & 0 & 0 & a_{n,2}^{n1}
\end{pmatrix}
&
\begin{pmatrix}
 & & & \\
 & 0 & &\\
 & & & \\
 & & &
\end{pmatrix}\\
%--------------------
\begin{pmatrix}
 & & & \\
 & 0 & &\\
 & & & \\
 & & &
\end{pmatrix}
&
\begin{pmatrix}
a_{0,1}^{02} & 0 & 0 & 0\\
0 & a_{1,1}^{12} & 0 & 0\\
0 & 0 & a_{2,1}^{22} & 0\\
0 & 0 & 0 & a_{n,1}^{n2}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,2}^{02} & a_{1,2}^{02} & 0            & 0\\
a_{0,2}^{12} & a_{1,2}^{12} & a_{2,2}^{12} & 0\\
0            & a_{1,2}^{22} & a_{2,2}^{22} & a_{n,2}^{22}\\
0            & 0            & a_{2,2}^{n2} & a_{n,2}^{n2}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,m}^{02} & 0 & 0 & 0\\
0 & a_{1,m}^{12} & 0 & 0\\
0 & 0 & a_{2,m}^{22} & 0\\
0 & 0 & 0 & a_{n,m}^{n2}
\end{pmatrix} \\
%--------------------
\begin{pmatrix}
 & & & \\
 & 0 & &\\
 & & & \\
 & & &
\end{pmatrix}
&
\begin{pmatrix}
 & & & \\
 & 0 & &\\
 & & & \\
 & & &
\end{pmatrix}
&
\begin{pmatrix}
a_{0,2}^{0m} & 0 & 0 & 0\\
0 & a_{1,2}^{1m} & 0 & 0\\
0 & 0 & a_{2,2}^{2m} & 0\\
0 & 0 & 0 & a_{n,2}^{nm}
\end{pmatrix}
&
\begin{pmatrix}
a_{0,m}^{0m} & a_{1,m}^{0m} & 0            & 0\\
a_{0,m}^{1m} & a_{1,m}^{1m} & a_{2,m}^{1m} & 0\\
0            & a_{1,m}^{2m} & a_{2,m}^{2m} & a_{n,m}^{2m}\\
0            & 0            & a_{2,m}^{nm} & a_{n,m}^{nm}
\end{pmatrix}
\end{pmatrix}}_{\ve{A}} \nonumber
\end{equation}
\end{landscape}
%--------------------
%
\begin{equation}
\underbrace{\begin{pmatrix}
\phi_{0,0} \\ \phi_{1,0} \\ \phi_{n-1,0} \\ \phi_{n,0} \\ \\
\phi_{0,1} \\ \phi_{1,1} \\ \phi_{n-1,1} \\ \phi_{n,1} \\ \\
\phi_{0,m-1} \\ \phi_{1,m-1} \\ \phi_{n-1,m-1} \\ \phi_{n,m-1} \\ \\
\phi_{0,m} \\ \phi_{1,m} \\ \phi_{n-1,m} \\ \phi_{n,m} \end{pmatrix}}_{\vec{\phi}} =
%
\underbrace{\begin{pmatrix}
S_{00} \\ S_{10} \\ S_{(n-1)0} \\ S_{n0} \\ \\
S_{01} \\ S_{11} \\ S_{(n-1)1} \\ S_{n1} \\ \\
S_{0(m-1)} \\ S_{1(m-1)} \\ S_{(n-1)(m-1)} \\ S_{n(m-1)} \\ \\
S_{0m} \\ S_{13} \\ S_{(n-1)m} \\ S_{nm}
\end{pmatrix}}_{\vec{S}} \nonumber
\end{equation}

%--------------------------------------------------------------------
%--------------------------------------------------------------------
%\bibliographystyle{plain}
%\bibliography{LinearSolns}

\end{document}


\section*{Conclusion}

In this work we described three approaches to parallelize the FW algorithm with three different architectures:
distributed with MPI, shared-memory multiprocessing with OpenMP and GPGPU with CUDA.

The fastest "pure" implementation is \emph{CUDA FW} thanks to the high computation capability and high memory bandwidth,
but it requires more expensive hardware.

\emph{MPI FW} (with one thread per node)  is still faster than the \emph{serial}, but the implementation
is more complex, the cost of the cluster (hosting and maintenance) makes the solution non convenient,
the efficiency depends on the network bandwidth and the speedup is not that high.

\emph{OpenMP FW} is almost 95 times faster than \emph{serial FW} but 13 times slower than \emph{CUDA FW}. The absence of
overhead due to communication, fast memory access, easy development process and relatively low costs make this solution the
most affordable, maintenable and cost-efficient one.

Talking about hybrid solutions, \emph{MPI + CUDA} is obviously faster than \emph{MPI + OpenMP} as long as the matrix is not small,
but the benefits may not justify the total cost of the infrastructure: the monthly cost for hosting a server with a GPU is at least
four times the cost of one without a GPU. This solution is suggested to those systems that really need the lowest response time possible
(\emph{e.g.} real-time systems or systems that cannot rely on a caching mechanism in front of them).


\documentclass[11pt]{article}
\usepackage{geometry}
\geometry{a4paper,top=2cm,bottom=2cm,left=2cm,right=2cm}
\usepackage[english]{babel}
\usepackage{graphicx}
\usepackage{fancyhdr}
\usepackage{lineno}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{algorithm}
\usepackage[noend]{algpseudocode}
\usepackage {tikz}
\usetikzlibrary {positioning}
\usepackage {xcolor}
\usepackage{titlesec}
\usepackage{varwidth}

\makeatletter
\def\BState{\State\hskip-\ALG@thistlm}
\makeatother

\title{\textbf{Algorithm Design - Homework 1} \\ \bigskip \large \textbf{Sapienza University of Rome}}
\date{\textbf{\today}}
\author{\textbf{Marco Costa, 1691388}}
\pagestyle{fancy}
\fancyhead[L]{Marco Costa, 1691388}
\fancyhead[R]{Algorithm Design - Homework 1}

%section, subsection: space - left, before, after
\titlespacing{\section}{0em}{0em}{0em}
\titlespacing{\subsection}{0em}{0.5em}{0em}
\titlespacing{\subsubsection}{0em}{0.5em}{0em}
\setlength\parindent{0pt}

\begin{document}
\maketitle
\newpage

\section*{Exercise 1}
\subsection*{Problem}
The problem consists in finding the expected optimal reward playing the game ``\textit{Open the Boxes and keep the Best!}''. The input is composed by the number of boxes $k$, the number of different rewards $n$ an
the cost $c_i$ of every box. Two algorithms are required. The first one's complexity must be \textbf{\textit{O}}($n^2 \cdot k$), while the second one's complexity must be \textbf{\textit{O}}($n \cdot k$)
\subsubsection*{Solution}
In order to solve the problem, we can define a matrix \textit{M} composed of $k$ rows and $n$ columns, where the element $M[i, j]$ represents the expected value by opening the box i and having as current maximum reward j. \\
To populate the matrix we can use \textbf{backward induction}, proceeding first considering the last box and choosing what to do for all the possible rewards. Using this information, we can then determine what to do at the penultimate box. This process continues backwards until we have determined the best action for every possible reward for each box. So the element $\mathbf{M[0, 0]}$ is the expected optimal reward.\\
Let's explain the mathematical formula: \\
- $j \cdot \frac{j+1}{n+1}$ (or $M[i+1, j] \cdot \frac{j+1}{n+1}$) is the probability of obtaining a reward $r \le j$; \\
- $\frac{1}{n+1}\sum\limits_{h = j + 1}^n{h}$ (or $\frac{1}{n+1}\sum\limits_{h = j + 1}^n{M[i+1, j]}$) is the probability of obtaining a reward $r > j$; \\
- $c[i]$ is the cost to pay opening the $i^{th}$ box. \\
- the sum of the previous values is the expected value opening the box $i$ having as current maximum reward $j$ \\
To improve the complexity to \textbf{\textit{O}}($n \cdot k$) we can eliminate the summations, since in the worst case we have a sum over $n$ (\textbf{\textit{O}}($n$)) in nested cycles (\textbf{\textit{O}}($n \cdot k$)) having a total cost of\textbf{\textit{O}}($n^2 \cdot k$). To do this we must distinguish the two cases:
\begin{itemize}
	\item \textbf{\textit{Last box}}: we can remove this summation using a mathematical trick: \\
	$\sum\limits_{i = j + 1}^n{i} = \sum\limits_{i = 0}^n{i} - \sum\limits_{i = 0}^j{i} = \frac{n \cdot (n+1)}{2} - \frac{j \cdot (j+1)}{2} = \frac{n \cdot (n+1) - j \cdot (j+1)}{2}$;
	\item \textbf{\textit{Other boxes}}: we can use an auxiliary vector in which we keep track of the sum of the elements of the matrix related to the box next to the current one.
\end{itemize}

Here there are the two \textbf{\textit{Algorithms}}:

\begin{minipage}[t]{0.49\textwidth}
\begin{algorithm}[H]
	\caption{Get optimal expected value (\textbf{\textit{O}}($n^2 \cdot k$))}\label{euclid}
	\begin{algorithmic}[1]
		\State $M[k, n+1] \gets 0$
		\For{$i$ in $(k-1, ..., 0)$}
		\For{$j$ in $(n, ..., 0)$}
		\State $r \gets 0$
		\If{$i = k-1$}
		\State $r \gets j \cdot \frac{j+1}{n+1} + \frac{1}{n+1}\sum\limits_{h = j + 1}^n{h} - c[i]$
		\Else
		\State \begin{varwidth}[t]{\linewidth} $r \gets M[i+1, j] \cdot \frac{j+1}{n+1} + \frac{1}{n+1} \cdot$ \par
			\hskip\algorithmicindent $\sum\limits_{h = j + 1}^n{M[i+1, h]} - c[i]$
		\end{varwidth}
		\EndIf
		\State $M[i, j] \gets \max(j, r)$
		\EndFor
		\EndFor
		\State \Return $M[0, 0]$
	\end{algorithmic}
\end{algorithm}
\end{minipage}
\hfill
\begin{minipage}[t]{0.49\textwidth}
\begin{algorithm}[H]
	\caption{Get optimal expected value (\textbf{\textit{O}}($n \cdot k$))}\label{euclid}
	\begin{algorithmic}[1]
		\State $M[k, n+1] \gets 0$
		\State $sums[n+1] \gets 0$
		\For{$i$ in $(k-1, ..., 0)$}
		\For{$j$ in $(n, ..., 0)$}
		\State $r \gets 0$
		\If{$i = k-1$}
			\State $r \gets j \cdot \frac{j+1}{n+1} + \frac{1}{n+1} \cdot \frac{n \cdot (n+1)-j \cdot (j+1)}{2} - c[i]$
		\Else
			\State $r \gets M[i+1, j] \cdot \frac{j+1}{n+1} + \frac{sums[j]}{n+1} - c[i]$
		\EndIf
		\State $M[i, j] \gets \max(j, r)$
		\If{$j = n$}
			\State $sums[j] \gets 0$
		\Else
			\State $sums[j] \gets sums[j+1] + M[i, j+1]$
		\EndIf
		\EndFor
		\EndFor
		\State \Return $M[0, 0]$
	\end{algorithmic}
\end{algorithm}
\end{minipage}
\newpage

\section*{Exercise 2}
\subsection*{First problem}
The problem is to find the complete graph \textbf{\textit{G}} of minimum weight given a weighted tree \textbf{\textit{T}}, such that  \textbf{\textit{T}} is the unique minimum spanning tree of \textbf{\textit{G}}. Algorithm's run time must be polynomial in $n$.
\subsubsection*{Solution}
Insert edges that are not in the tree so as to obtain the complete graph. These edges must have a greater weight than those of the tree, so that \textbf{\textit{T}} is the only \textbf{\textit{MST}} of \textbf{\textit{G}}. Since \textit{Kruskal}'s algorithm uses the \textit{Union-Find} structures for representing the cuts, we use this structures to solve the problem. \\
Let's define with \textbf{\textit{V}} the set of nodes of \textbf{\textit{T}} and with \textbf{\textit{E}} the set of edges of \textbf{\textit{T}}.
\begin{algorithm}
	\caption{Find complete graph}\label{euclid}
	\begin{algorithmic}[1]
		\For{$v \in V$} \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad (\textbf{\textit{O}}($n$))
			\State $v.initializeUnionFindSingleton()$
		\EndFor
		\State $T.sortEdgesByAscendingWeights()$ \qquad \qquad \qquad \qquad \qquad \qquad \qquad (\textbf{\textit{O}}($n\log{n}$))
		\State $G \gets \emptyset$
		\For {$e \gets (u, v) \in E$} \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad (\textbf{\textit{O}}($n^3$))
			\State  $G.addEdge(e)$
			\State  $set_1 \gets u.findComponents()$
			\State  $set_2 \gets v.findComponents()$
			\For {$u \in set_1$}
				\For {$v \in set_2$}
					\State $\hat{e} \gets (u, v)$
					\If {$\hat{e} \notin G$}
						\State $\hat{e}.setWeight(e.getWeight() + 1)$
						\State $G.addEdge(\hat{e})$
					\EndIf
				\EndFor
			\EndFor
			\State $union(set_1, set_2)$
		\EndFor
		\State \Return $G$
	\end{algorithmic}
\end{algorithm} \\
\textbf{Cost}: \textbf{\textit{O}}($|V|\log{|V|} + |E| \cdot |V|^2$) = \textbf{\textit{O}}($n\log{n} + (n-1) \cdot n^2$) = \textbf{\textit{O}}($n\log{n} + n^3 - n^2$) = \textbf{\textit{O}}($n^3$), given by the three nested cycles over $n$.
\subsection*{Second problem}
Find the total weight of the complete graph \textbf{\textit{G}} of minimum weight given a weighted tree \textbf{\textit{T}}, such that  \textbf{\textit{T}} is the unique minimum spanning tree of \textbf{\textit{G}}. Algorithm's complexity must be \textbf{\textit{O}}($n \log{n}$).
\subsubsection*{Solution}
This problem is quite similar to the previous one, but in this case there is no need to create all the edges of the graph.
\begin{algorithm}
	\caption{Find weight of the complete graph}\label{euclid}
	\begin{algorithmic}[1]
		\For{$v \in V$} \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad (\textbf{\textit{O}}($n$))
			\State $v.initializeUnionFindSingleton()$
		\EndFor
		\State $T.sortEdgesByAscendingWeights()$ \qquad \qquad \qquad \qquad \qquad \qquad \qquad (\textbf{\textit{O}}($n\log{n}$))
		\State $w_{total} \gets 0$
		\For {$e \gets (u, v) \in E$} \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad (\textbf{\textit{O}}($n \log{n}$))
			\State $w_{total}$ \textit{+=} $e.getWeight()$
			\State  $set_1 \gets u.findComponents()$
			\State  $set_2 \gets v.findComponents()$
			\State $w_{total}$ \textit{+=} $(set_1.size() \times set_2.size() - 1) \times (e.getWeight() + 1)$
			\State $union(set_1, set_2)$
		\EndFor
		\State \Return $w_{total}$
	\end{algorithmic}
\end{algorithm} \\
\textbf{Cost}: \textbf{\textit{O}}($|V|\log{|V|} + |E| \log{|V|}$) = \textbf{\textit{O}}($n\log{n} + n\log{n}$) = \textbf{\textit{O}}($n\log{n}$) (\textit{union} costs \textbf{\textit{O}}($\log{n}$)).
\newpage

\section*{Exercise 3}
\subsection*{First Problem}
The problem consists in modeling Federico's business as a flow problem in a graph \textbf{\textit{G}}, only consisting of regular vertices, one source, one sink, and capacitated edges to find out how many chocolates he should make every week. An example with $|F| = 4$ is required.
\subsubsection*{Solution}
The model consists in a source node \textit{F} that represents Federico, a sink node \textit{S} that represents the costumers and different nodes $f_i$ representing Federico's friends. Moreover, for each friend there is a node $w_i$ that represents the friend's ``warehouse''. These nodes help us better understand that the amount of total chocolates received by a friend is exactly $n_i$. Each friend can receive the chocolate from Federico (represented with an edge of path from Federico to the friend and capacity $n_i$) or from another friend (represented with an edge of path from the new friend to the first one and capacity $n_i - s_i$). Notice that when a friend meets another, they can exchange chocolates for each other (represented with two edges: the first one has the path from the first friend to the warehouse of the second one and capacity $n_{i1} - s_{i1}$, while the second one has the path from the second friend to the first one and capacity $n_{i2} - s_{i2}$). The chocolates sold by each friend is represented with an edge having the path from the friend to the costumers and capacity $s_i$. Notice that for each friend $s_i \le n_i$ and the total amount of chocolates sold is $\sum\limits_{i \in |F|}s_i$. It's possible to find the \textit{Maximum Flow} in order to find out how many chocolates Federico should
actually make every week. Let's show the example with $|F| = 4$:

\begin {tikzpicture}
\begin{scope}[auto=left,every node/.style={circle,draw, minimum width = 1 cm}]
\node(f) at (0,0) {F};
\node(wa) at (3,1) {w$_a$};
\node(a) at (6.5,1) {f$_a$};
\node(wb) at (3,-1) {w$_b$};
\node(b) at (6.5,-1) {f$_b$};
\node(wc) at (9.5,1) {w$_c$};
\node(c) at (13,1) {f$_c$};
\node(wd) at (9.5,-1) {w$_d$};
\node(d) at (13,-1) {f$_d$};
\node(s) at (16,0) {s};
\end{scope}
\begin{scope}[every edge/.style={draw=black,very thick}, every node/.style={scale=0.8}]
\path [->] (f) edge node[sloped,above]{$n_1$} (wa);
\path [->] (f) edge node[sloped,above]{$n_2$} (wb);

\path [->] (a) edge node[sloped,below]{$n_a - s_a$} (wc);
\path [->] (b) edge node[sloped,above]{$n_b - s_b$} (wd);
\path [->] (c) edge[bend right=20] node[sloped,above]{$n_c - s_c$} (wa);
\path [->] (d) edge[bend left=20] node[sloped,below]{$n_d - s_d$} (wb);

\path [->] (a) edge[sloped,above,pos=0.75] node{$n_a - s_a$} (wb);
\path [->] (d) edge[sloped,above,pos=0.2] node{$n_d - s_d$} (wc);
\path [->] (b) edge[sloped,above,pos=0.2] node{$n_b - s_b$} (wa);
\path [->] (c) edge[sloped,above,pos=0.75] node{$n_c - s_c$} (wd);

\path [->] (wa) edge node[sloped,above]{$n_a$}(a);
\path [->] (wb) edge node[sloped,above]{$n_b$}(b);
\path [->] (wc) edge node[sloped,above]{$n_c$}(c);
\path [->] (wd) edge node[sloped,above]{$n_d$}(d);

\path [->] (a) edge[bend left=30] node[sloped,above]{$s_a$} (s);
\path [->] (b) edge[bend right=30] node[sloped,below]{$s_b$} (s);
\path [->] (c) edge node[sloped,above]{$s_c$} (s);
\path [->] (d) edge node[sloped,above]{$s_d$} (s);
\end{scope}
\end{tikzpicture}
\subsection*{Second Problem}
Adjust the network knowing that each friend will have an associated building $b_f \in B$ and find out if and how this
impacts Federico’s business.
\subsubsection*{Solution}
In this case we must modify a bit the previous model adding some new nodes $b_i$ that represent the buildings. Now each friend can sell chocolates only in his associated building, so we have the edges with capacity $s_i$ with path from the friend to his associated building, and another edge with capacity $c_i$ with path from the building to the costumers. The main difference is that now the total amount of chocolate sold is $\sum\limits_{i \in B}c_i$, and we have that $\sum\limits_{i \in |B|}c_i \le \sum\limits_{i \in |F|}s_i$. It means that Federico could make less chocolates than before because of the possible drop of sales.
Let's show the adjusted example:

\begin {tikzpicture}
\begin{scope}[auto=left,every node/.style={circle,draw, minimum width = 1 cm}]
	\node(f) at (0,0) {F};
	\node(ma) at (2,1) {m$_a$};
	\node(a) at (5.8,1) {f$_a$};
	\node(mb) at (2,-1) {m$_b$};
	\node(b) at (5.8,-1) {f$_b$};
	\node(mc) at (8.2,1) {m$_c$};
	\node(c) at (12,1) {f$_c$};
	\node(md) at (8.2,-1) {m$_d$};
	\node(d) at (12,-1) {f$_d$};
	\node(b1) at (14,2) {$b_1$};
	\node(b2) at (14,0) {$b_2$};
	\node(b3) at (14,-2) {$b_3$};
	\node(s) at (16,0) {s};
\end{scope}
\begin{scope}[every edge/.style={draw=black,very thick}, every node/.style={scale=0.8}]
	\path [->] (f) edge node[sloped,above]{$n_1$} (ma);
	\path [->] (f) edge node[sloped,above]{$n_2$} (mb);

	\path [->] (a) edge node[sloped,below]{$n_a - s_a$} (mc);
	\path [->] (b) edge node[sloped,above]{$n_b - s_b$} (md);
	\path [->] (c) edge[bend right=20] node[sloped,above]{$n_c - s_c$} (ma);
	\path [->] (d) edge[bend left=20] node[sloped,below]{$n_d - s_d$} (mb);

	\path [->] (a) edge[sloped,above,pos=0.75] node{$n_a - s_a$} (mb);
	\path [->] (d) edge[sloped,above,pos=0.2] node{$n_d - s_d$} (mc);
	\path [->] (b) edge[sloped,above,pos=0.2] node{$n_b - s_b$} (ma);
	\path [->] (c) edge[sloped,above,pos=0.75] node{$n_c - s_c$} (md);

	\path [->] (ma) edge node[sloped,above]{$n_a$}(a);
	\path [->] (mb) edge node[sloped,above]{$n_b$}(b);
	\path [->] (mc) edge node[sloped,above]{$n_c$}(c);
	\path [->] (md) edge node[sloped,above]{$n_d$}(d);

	\path [->] (a) edge[bend left=10] node[sloped,above]{$s_a$} (b1);
	\path [->] (b) edge[bend right=10] node[sloped,below]{$s_b$} (b3);
	\path [->] (c) edge node[sloped,above]{$s_c$} (b1);
	\path [->] (d) edge node[sloped,above]{$s_d$} (b2);

	\path [->] (b1) edge node[sloped,above]{$c_1$} (s);
	\path [->] (b2) edge node[sloped,above]{$c_2$} (s);
	\path [->] (b3) edge node[sloped,above]{$c_3$} (s);
\end{scope}
\end{tikzpicture}
\newpage

\section*{Exercise 4}
\subsection*{Problem}
The problem consists in showing that it is not possible to solve the problem of scheduling $n$ tasks with quick algorithm. Each job  $j$ of the $n$ jobs has an earliest time $s_j$ to start task, a deadline $d_j$ when it has to be finished, and the length $l_j \in \mathbb{N}$ that specifies the time needed to complete it. There is also an upper bound time $k$ and $\sum\limits_{j \in J}l_j \le k$.
\subsubsection*{Solution}
In order to prove that this problem is not solvable with quick algorithm we need to prove that it is \textbf{\textit{NP-Hard}}. We can do it reducing another \textit{NP-Hard} problem to the current problem. Let's define our problem as $\Theta$, we need to find a well known \textit{NP-Hard} problem $\Omega$ such that:
\begin{itemize}
	\item $\omega \in \Omega \implies  \theta \in \Theta$
	\item $\theta \in \Theta \implies  \omega \in \Omega$ (or equivalently $\neg\omega \in \Omega \implies  \neg\theta \in \Theta$)
\end{itemize}
\textbf{\underline{Observation}}: The earliest start time  $s_j$ indicate only at what time the job $j$ is available (it does not need to be done at that moment, but it does have to be done before its deadline $d_j$). \\ \\
\textbf{\underline{Observation}}: Our scheduling problem is solvable if and only if each job $j$ meets its deadline $d_j$ and $\sum\limits_{j \in J}l_j \le k$ (we know that this last is satisfied). \\ \\
We can use \textbf{\textit{Subset Sum}} for our purpose ($\Omega$ = \textit{Subset Sum}). \\
\textbf{Subset sum}: Given a set of positive integers and an integer $k$, is there any non-empty subset whose sum to $k$.
\begin{enumerate}
	\item Prove that $\omega \in \Omega \implies  \theta \in \Theta$: \\
	Let's suppose we start with a solvable instance of \textit{Subset Sum} Problem. Given $X = \{x_1, ..., x_n\}$ a set of positive integers to which there is a subset $X'$ such that its elements add up to $k$ and the sum of the whole set is $\lambda$ ($\sum\limits_{x_i \in X'}x_i = k$ and $\sum\limits_{x_i \in X}x_i = \lambda$). Let's define jobs $\{j_1, ..., j_n\}$ such that $s_i = 0$, $l_i = x_i$ and $d_i = \lambda + 1$ for every $j_i$ (earliest start time, duration of the job and deadline respectively). This instance solves the scheduling problem because it's possible to arrange the jobs in any order and they will always meet their deadline.

	\item Prove that $\theta \in \Theta \implies  \omega \in \Omega$: \\
	Let's suppose we start with a solvable instance of our \textit{Scheduling} problem. Given the set of jobs $\{j_1, ..., j_n, j_{n+1}\}$ where the first $n$ jobs are defined the same as before and job $j_{n+1}$ having $s_{n+1} = k$, $l_{n+1} = 1$ and $d_{n+1} = k+1$. The extra job must be done in $[k, k+1]$, but there would still be $\lambda$ units of time available in the time interval $[0, \lambda+1]$. Since this instance is solvable, we have $|X'|$ jobs that can be executed before the job $j_{n+1}$ and the remaining jobs after the job $j_{n+1}$. The set $X'$ solves the \textit{Subset Sum} problem.
\end{enumerate}
We proved that our scheduling problem is \textit{NP-Hard}, so we can say that it is not solvable with a quick algorithm.

\end{document}


\subsection{Variables and Strings}
Texts such as "Hello World!" can be represented by a variable
\footnote{there is no declaration of types, such as number or string, in ImageJ macro.}.
Let's understand this by examining a short macro below.
\lstinputlisting{code/code02.ijm}

\ilcom{text} is a "String Variable" or simply a "String".
ImageJ prepares a memory space for this variable, and you can change the content by re-defining the content. Two (or maybe more) variables could be used to construct another variable.

\lstinputlisting{code/code03.ijm}

The above operation concatenates content of \ilcom{text2} to the content of \ilcom{text1} and produces a third variable \ilcom{text3} that holds the result of concatenation. It should be noted here, that macro has two ways of usage for \ilcom{+}. What we tested in above is ``concatenation''. Another usage is ``addition'' in the next section.

\begin{indentexercise}{1}
\item Add more string variables and make a longer sentence.\\
\end{indentexercise}

It is also possible to store a number in a variable. For example, \\
\begin{lstlisting}[numbers=none]
text = 256;
\end{lstlisting}
With this assignment, the variable is now a "numerical variable" or simply "variable".
In other programming languages such as C or Java, difference between numbers and characters matters a lot.
In ImageJ macro you do not have to care whether the variable is number or string (we call them ``types'') ad they are defined automatically by the type of value provided for a variable, and this makes the macro coding to be light and easy. However, since types are implicitly defined without declaration it could cause simple mistakes such as type mismatching.
So be sure keep the difference in types DOES matter but they are not shown in the code. We will see an example of such confusion,
and also a way to avoid the confusion.

Test the following macro to see how the numerical variable works.
\lstinputlisting{code/code04.ijm}
Did you get some results printed out? It should, but you should read the code carefully as there is a small trick in this code.  This trick is something special in IJ macro language compared to other general scripting languages.

You might have noticed a strange expression at line 8, in the way it assigns the variable \ilcom{txt}.
It starts with double quotation marks. \\
%\lstinputlisting[language=Java, linerange={8-8}, numbers=none]{code/code04.ijm}
\begin{lstlisting}[numbers=none]
txt= "" + a + "+" + b + "=" + c;
\end{lstlisting}
Seemingly this looks like meaningless.
If you define ilcom{txt} without the first "useless" quotation marks, then it will be like\\
\begin{lstlisting}[numbers=none]
txt= a + "+"+ b + "=" + c;
\end{lstlisting}
Theoretically this should work,
since the double quote does not have any content so its presence should be meaningless. But if you try to run this what it seems to be straight-forward assignment,
ImageJ returns an error message.

\begin{figure}[htbp]
\begin{center}
\includegraphics[scale=0.6]{fig/ErrorStringNumericFunction.png}
\caption{Error with Variable Assignment} \label{fig_ErrorVariable}
\end{center}
\end{figure}

This is because when ImageJ scans through the macro from top to bottom, line by line,
it reaches the line for the assignment of the variable \ilcom{txt} and first sees the variable \ilcom{a} and interprets that \ilcom{txt} should be a numerical variable
(or function), since \ilcom{a} is known to be a number as it was defined so in one of the lines above. Then ImageJ goes on interpreting rightward thinking that this is math. Then finding a "+" which surprisingly is a character
ImageJ cannot interpret string variable within a numerical function, so it returns an error message. The macro aborts.

To overcome this problem, the programmer can tell ImageJ that
\textit{txt} is a string function at the beginning of the assignment
by putting a set of double quote. This tells the interpreter that this assignment is a string concatenation assignment and not a numerical assignment.
ImageJ does handle numerical values within string function,
so the line is interpreted without problem and prints out the result successfully. Note that such confusion of string and numerical types are rarely seen in general scripting languages and specific to ImageJ macro language.

\begin{indentexercise}{2}
Modify the code 4, so that the calculation involves subtraction (-), multiplication (*) and division (/).
\end{indentexercise}


\usetikzlibrary{backgrounds,calc}

\begin{document}
	\chapter{Probability}
	\section{Introduction}
	Let an event be denoted by $A$. The probability that event $A$ occurs is denoted by P$(A)$ and is given by \[\Pr(A) = \frac{\text{no. of ways in which } A \text{ can occur}}{\text{total no. of outcomes}}\]

	Also note that the probability of event $A$ not happening is defined by $1-\Pr(A)$ and is denoted by $\Pr(A^c)$.

	\begin{example}
		A letter is chosen at random from the world `CALCULUS'. Find the probability that it is:

		\quad \textbf{a) } a `C'.

		\quad \textbf{b) } a vowel.
	\end{example}

\begin{example}
	A card is chosen at random from a pack of 52 playing cards. Find the probability that the card is:

	\quad \textbf{a) } an ace.

	\quad \textbf{b) } black.

	\quad \textbf{c) } a heart.

	\quad \textbf{d) } a royal card.
\end{example}
	\section{Using Permutations and Combinations}

	In the following problem, the number of successful outcomes and the total number of outcomes are calculated using the counting techniques in the previous chapter.

	\begin{example}
		A team of 6 children is chosen at random from a class of 10 girls and 9 boys. Find the probability thattge selected team contains:

		\quad \textbf{a) } girls only.

		\quad \textbf{b) } boys only.

		\quad \textbf{c) } more girls than boys.

		\quad \textbf{d) } the oldest 5 children in the class.

	\end{example}
	\section{Bernoulli Trials}
	A Binomial Experiment is an experiment which satisfies the following 4 conditions:
	\begin{itemize}
		\item {A fixed number of trials.}
		\item {Each trial is independent of the others.}
		\item {There are only \textbf{two} outcomes.}
		\item {The probability of each outcome remains constant from trial to trial.}
	\end{itemize}
	\section{Probability Space Diagrams and Tree Diagrams.}
	Tree diagrams and possibility spaces are simple graphical tools which help us calculate probabilities. Possibility space diagrams are particularly useful when the problem involves independent events.
	\section{Venn Diagrams}
	Venn diagrams ilustrate sets of objects or items and set operations. Consider for instance the sets $M_2$, $M_3$ and $M_5$ which stand for multiples of 2, 3 and 5 respectively. The \textbf{universal} set $U$ is the set of all integers between 1 and 30. For convenience, let us consider the set $P$ to be the set of all primes between 1 and 30.

	\begin{tikzpicture}[venncircle/.style={draw, circle, minimum size=15em, align=center}, node distance=12.5em, framed]
		\node[venncircle] (circle1) {$M_5$};
		\node[venncircle, right of=circle1] (circle2) {$M_3$};
		\node (MN) at ($(circle1)!0.5!(circle2)$){9};
		\node[venncircle, below of=MN, yshift=3em] (circle3) {$M_2$}; %yshift value by pythagoras
		\node (ML)  at ($(circle1)!0.4!(circle3)$){6};
		\node (ML)  at ($(circle1)!0.5!(circle3)$){12};
		\node (MLa)  at ($(circle1)!0.6!(circle3)$){18};
		\node (NL)  at ($(circle2)!0.5!(circle3)$){4};
		\node (MNL)  at ($(MN)!0.3!(circle3)$){2};
		\node (Lleft) [below of=ML, yshift=5em] {16};
		\node (Lright) [below of=NL, yshift=5em] {21};
		\node (outbotrightnum) [right of=circle3, yshift=1.5em, xshift=-2em] {11};
		\node (outtoprightnum) [above right of=circle2, yshift=-2.5em, xshift=-1em] {13};
		\node (outbotleftnum) [below left of=circle3, yshift=4em, xshift=-4em]{12};
		\node (outtopleftnum) [left of=circle3, yshift=1.5em]{23};
		\node (U)[above left of=circle1, xshift=8em, yshift=-3em]{5};
		\node (M)[above of=circle1, yshift=-4em]{\textbf{\textit{M}}};
		\node (N)[above of=circle2, yshift=-4em]{\textbf{\textit{N}}};
		\node (L)[below right of=circle3, yshift=2.5em, xshift=-2.5em]{\textbf{\textit{L}}};
	\end{tikzpicture}
\begin{itemize}
	\item{The intersection between sets $M_2$ and $M_3$ is denoted by $M_2 \cap M_3$ and hence $M_2 \cap M_3 = \set{6,12,18,25,30}$ and $M_2 \cap M_3 \cap M_5 = \set{30}$.}

	\item{The union between the sets $M_3$ and $M_5$ is denoted by $M_3 \cup M_5$ and hence $M_3 \cup M_5 = \set{3,5,6,7,10,12,15,18,20,21,24,25,27,30}$.}

	\item{We already know that the set that is formed by the members of the universal set which are not in the set $M_2$ is called the complement of the set $M_2$ denoted by $M_2^c$. Hence, $M_2^c = \set{1,3,5,7,9,11,13,15,17,19,21,23,25,27,29}$.}
\end{itemize}

We already know that the set that is formed by the members of the universal set which are not in the set $M_2$ is called the complement of the set $M_2$ denoted by $M_2^C$

\end{document}