\lab{Markov Chains}{Markov Chains}
\label{lab:Markov}
\objective{
A Markov chain is a collection of states with specified probabilities for transitioning from one state to another.
They are characterized by the fact that the future behavior of the system depends only on its current state.
In this lab, we learn to construct, analyze, and interact with Markov chains, then apply a Markov chain to a natural language processing problem.}

\section*{State Space Models} % ===============================================

Many systems can be described by a finite number of states.
For example, a board game where players move around the board based on die rolls can be modeled by a Markov chain.
Each space represents a state, and a player is said to be in a state if their piece is currently on the corresponding space.
In this case, the probability of moving from one space to another only depends on the player's current location; where the player was on a previous turn does not affect their current turn.

Finite Markov chains have an associated \emph{transition matrix} that stores the information about the transitions between the states in the chain.
The $(i,j)$th entry of the matrix gives the probability of moving \textbf{from state $j$ to state $i$}.
Thus each of the columns of the transition matrix sum to $1$.

\begin{info} % Generate random row stochastic transitino matrices.
A transition matrix where the columns sum to 1 is called \emph{column stochastic} (or \emph{left stochastic}).
The rows of a \emph{row stochastic} (or \emph{right stochastic}) transition matrix each sum to 1 and the $(i,j)$th entry of the matrix is the probability of moving from state $i$ to state $j$.
Both representations are common, but in this lab we exclusively use column stochastic transition matrices for consistency.
\end{info}

Consider a very simple weather model where the probability of being hot or cold depends on the weather of the previous day.
If the probability that tomorrow is hot given that today is hot is 0.7, and the probability that tomorrow is cold given that today is cold is 0.4, then by assigning hot to the $0$th row and column, and cold to the $1^{st}$ row and column, this Markov chain has the following transition matrix.

\begin{align*}
\begin{blockarray}{ccc}
& \text{\textcolor{red}{hot today}} & \text{\textcolor{blue}{cold today}} \\
\begin{block}{c[cc]}
\text{\textcolor{red}{hot tomorrow}}   & 0.7 & 0.6 \\
\text{\textcolor{blue}{cold tomorrow}} & 0.3 & 0.4 \\
\end{block}\end{blockarray}
\end{align*}

The $0$th column of the matrix says that if it is hot today, there is a $70\%$ chance that tomorrow will be hot ($0$th row) and a $30\%$ chance that tomorrow will be cold ($1$st row).
The $1$st column says if it is cold today, then there is a $60\%$ chance of heat and a $40\%$ chance of cold tomorrow.

Markov chains can be represented by a \emph{state diagram}, a type of directed graph.
The nodes in the graph are the states, and the edges indicate the state transition probabilities.
The Markov chain described above has the following state diagram.

\begin{figure}[H] % 2-state chain.
\centering
\begin{tikzpicture}[normalcircle/.style={draw, circle, minimum size=1.5cm, thick, node distance=2.5cm}]
    % Place the circles
    \node[normalcircle] (Hot) {};
    \node[red] at (Hot) {Hot};

    \node[normalcircle, right=of Hot] (Cold) {};
    \node[blue] at (Cold) {Cold};

    % Draw loop, place number, draw line
    \draw[thick,->,>=stealth',red!50!black] (Hot)+(-.52,.52) arc (325:40:.35 and -.85);
    \node[left,red!50!black] at (Hot.west) [shift={+(-.5,0)}] {0.7};
    \draw[thick,->,>=stealth',red!50!black] (Hot.north east) -- node[above] {0.3} (Cold.north west);

    \draw[thick,->,>=stealth',blue!50!black] (Cold)+(.52,.52) arc (325:40:-.35 and -.85);
    \node[right,blue!50!black] at (Cold.east) [shift={+(.5,0)}] {0.4};
    \draw[thick,->,>=stealth',blue!50!black] (Cold.south west) -- node[below] {0.6} (Hot.south east);

\end{tikzpicture}
\end{figure}

%
\begin{problem} % Make a column stochastic matrix.
Transition matrices for Markov chains are efficiently stored as NumPy arrays.
Write a function that accepts an integer $n$ and returns the transition matrix for a random Markov chain with $n$ states.
\\ (Hint: use array broadcasting to avoid looping.)
\label{prob:random-markov-chain}
\end{problem}

\subsection*{Simulating State Transitions} % ----------------------------------

A single draw from a \emph{binomial distribution} with parameters $n$ and $p$ indicates the number of successes out of $n$ independent experiments, each with probability $p$ of success.
The classic example is a series of coin flips, where $p$ is the probability that the coin lands heads side up.
NumPy's \li{random} module has an efficient tool, \li{binomial()}, for drawing from a binomial distribution.

\begin{lstlisting}
>>> import numpy as np

# Draw from the binomial distribution with n = 1 and p = .5 (flip 1 coin).
>>> np.random.binomial(1, .5)
0                             # The coin flip resulted in tails.
\end{lstlisting}

Consider again the simple weather model and suppose that today is hot.
The column that corresponds to ``hot''in the transition matrix is $[0.7, 0.3]$.
To determine whether tomorrow is hot or cold, draw from the binomial distribution with $n = 1$ and $p = 0.3$.
If the draw is 1, which has $30\%$ likelihood, then tomorrow is cold.
If the draw is 0, which has $70\%$ likelihood, then tomorrow is hot.
The following function implements this idea.

\begin{lstlisting}
def forecast():
    """Forecast tomorrow's weather given that today is hot."""
    transition = np.array([[0.7, 0.6], [0.3, 0.4]])

    # Sample from a binomial distribution to choose a new state.
    return np.binomial(1, transition[1, 0])
\end{lstlisting}

\begin{problem} % 2-state chain with binomial transitioning.
Modify \li{forecast()} so that it accepts an integer parameter \li{days} and runs a simulation of the weather for the number of days given.
Return a list containing the day-by-day weather predictions (0 for hot, 1 for cold).
Assume the first day is hot, but do not include the data from the first day in the list of predictions.
The resulting list should therefore have \li{days} entries.
\label{prob:small-markov-transitions}
\end{problem}

\subsection*{Larger Chains} % -------------------------------------------------

The \li{forecast()} function makes one random draw from a binomial distribution to simulate a state change.
Larger Markov chains require draws from a \emph{multinomial distribution}, a multivariate generalization of the binomial distribution.
A draw from a multinomial distribution parameters $n$ and $(p_1,\ p_2,\ \ldots,\ p_k)$ indicates which of $k$ outcomes occurs in $n$ different experiments.
In this case the classic example is a series of dice rolls, with $6$ possible outcomes of equal probability.

\begin{lstlisting}
>>> die_probabilities = np.array([1./6, 1./6, 1./6, 1./6, 1./6, 1./6])

# Draw from the multinomial distribution with n = 1 (roll a single die).
>>> np.random.multinomial(1, die_probabilities)
array([0, 0, 0, 1, 0, 0])     # The roll resulted in a 4.
\end{lstlisting}

\begin{problem} % 4-state chain with multinomial transitioning.
Let the following matrix be the transition matrix for a Markov chain modeling weather with four states: hot, mild, cold, and freezing.

\begin{align*}
\begin{blockarray}{ccccc}
& \text{\textcolor{red}{hot}} & \text{\textcolor[rgb]{0,.6,0}{mild}} & \text{\textcolor{blue}{cold}} & \text{\textcolor{cyan}{freezing}} \\
\begin{block}{c[cccc]}
\text{\textcolor{red}{hot}}              & 0.5 & 0.3 & 0.1 & 0   \\
\text{\textcolor[rgb]{0,.6,0}{mild}}     & 0.3 & 0.3 & 0.3 & 0.3 \\
\text{\textcolor{blue}{cold}}            & 0.2 & 0.3 & 0.4 & 0.5 \\
\text{\textcolor{cyan}{freezing}}        &   0 & 0.1 & 0.2 & 0.2 \\
\end{block}\end{blockarray}
\end{align*}

Write a new function that accepts an integer parameter and runs the same kind of simulation as \li{forecast()}, but that uses this new four-state transition matrix.
This time, assume that the first day is mild.
Return a list containing the day-to-day results (0 for hot, 1 for mild, 2 for cold, and 3 for freezing).
\label{prob:markov-larger-chain}
\end{problem}

% TODO: Turn this into a Note down below?
% \begin{info}
% A single experiment with probability $p$ of success is called a \emph{Bernoulli trial}.
% In general, the binomial distribution is the sum of $n$ repeated Bernoulli trials.
% Like the Bernoulli is a special case of the binomial, the \emph{categorical distribution} is a special case of the \emph{multinomial distribution}, when n = 1.
% NumPy only has direct implementations for the binomial and multinomial distributions, since the Bernoulli and categorical distributions are just special cases of these.
% \end{info}

\section*{General State Distributions} % ======================================

\begin{comment} % The 1-Norm. Might be able to get away without this.
For an $n\times 1$ vector $\x$ with entries $x_i$ and an $n\times n$ matrix $A$ with entries $a_{ij}$, the \emph{1-norm} is defined as follows.
\begin{align*}
\|\x\|_1 = \sum_{i=1}^n|x_i| && \|A\|_1 = \sup_j \sum_{i=1}^n |a_{ij}|
\end{align*}
In other words, the $1$-norm for both vectors and matrices is the maximum absolute column sum.
Then if $A$ is a transition matrix, $\|A\|_1 = 1$, since each of the entries of the matrix are positive and each of the columns sum to $1$ by definition.
The power method with the 1-norm can be used to find the unique stable steady state distribution of $A$.
\end{comment}

For a Markov chain with $n$ states, the probability of being in each of the states can be encoded by a single $n \times 1$ vector $\x$, called a \emph{state distribution vector}.
The entries of $\x$ must be nonnegative and sum to 1.
Then the $i$th entry $x_i$ of $\x$ is the probability of being in state $i$.
For example, the state distribution vector $\x = [0.8,\ 0.2]\trp$ corresponding to the 2-state weather model of Problem \ref{prob:small-markov-transitions} indicates that there is a $80\%$ chance that today is hot and a $20\%$ chance that today is cold.
On the other hand, the vector $\x = [0, 1]\trp$ implies that today is, with $100\%$ certainty, cold.

If $A$ is an $n\times n$ transition matrix for a Markov chain and $\x$ is a state distribution vector, then $A\x$ is also a state distribution vector.
% To verify this fact, let $a_{ij}$ be the entries of $A$ and $x_i$ the entries of $\x$.
% The columns of $A$ sum to $1$, so $\sum_{j=1}^n a_{ij} = 1$ for $i = 1,\ 2,\ \ldots,\ n$.
% In addition, $\sum_{j=1}^n x_j = 1$ since the entries of $\x$ also sum to $1$.
% From matrix multiplication, the $i$th entry of $A\x$ is given by $\sum_{j=1}^n a_{ij}x_j$, so the sum of the entries of $A\x$ is
% \[\sum_{i=1}^n\sum_{j=1}^n a_{ij}x_j
% = \sum_{j=1}^n\left(x_j\left(\sum_{j=1}^n a_{ij}\right)\right)
% = \sum_{j=1}^n x_j = 1.\]
In fact, if $\x_k$ is the state distribution vector corresponding to a certain time $k$, then $\x_{k+1} = A\x_k$ contains the probabilities of being in each state after allowing the system to transition again.
For the weather model, this means that if there is an $80\%$ chance that it will be hot 5 days from now, written $\x_{5} = [0.8,\ 0.2]\trp$, then since
\[
\x_{6} = A\x_{5} =
\left[\begin{array}{cc}
0.7 & 0.6 \\
0.3 & 0.4 \\
\end{array}\right]
\left[\begin{array}{c}0.8 \\ 0.2\end{array}\right]
=
\left[\begin{array}{c}0.68 \\ 0.32\end{array}\right],
\]
there is a $68\%$ chance that 6 days from now will be a hot day.

\subsection*{Convergent Transition Matrices} % --------------------------------

Given an initial state distribution vector $\x_{0}$, defining $\x_{k+1} = A\x_k$ yields the following significant relation.
\[\x_k = A\x_{k-1} = A(A\x_{k-2}) = A(A(A\x_{x-3})) = \cdots = A^k\x_{0}\]

This indicates that the $(i,j)$th entry of $A^k$ is the probability of transition from state $j$ to state $i$ in $k$ steps.
For the transition matrix of the 2-state weather model, something curious happens to $A^k$ for even small values of $k$.
\[
A = \left[\begin{array}{cc}
0.7 & 0.6 \\
0.3 & 0.4 \\
\end{array}\right]
\quad
A^2 = \left[\begin{array}{cc}
0.67 & 0.66 \\
0.33 & 0.34 \\
\end{array}\right]
\quad
A^3 = \left[\begin{array}{cc}
0.667 & 0.666 \\
0.333 & 0.334 \\
\end{array}\right]
\]
As $k\rightarrow\infty$, the entries of $A^k$ converge, written as follows.
\begin{equation}
\lim_{k \rightarrow \infty} A^k = \left[\begin{array}{ccc}
2/3 & 2/3 \\
1/3 & 1/3 \\
\end{array}\right].
\label{eq:markov-steady-transition}
\end{equation}
In addition, for any initial state distribution vector $\x_{0} = [a,\ b]\trp$, $a + b = 1$,
\[
\lim_{k \rightarrow \infty} \x_k = \lim_{k \rightarrow \infty}A^k\x_{0}
=
\left[\begin{array}{ccc}
2/3 & 2/3 \\
1/3 & 1/3 \\
\end{array}\right]
\left[\begin{array}{c}a\\b\end{array}\right]
=
\left[\begin{array}{c}2(a+b)/3\\(a+b)/3\end{array}\right]
=
\left[\begin{array}{c}2/3\\1/3\end{array}\right].
\]

Thus as $k\rightarrow\infty$, $\x_k \rightarrow \x = [2/3,\ 1/3]\trp$, regardless of the initial state distribution $\x_{0}$.
So according to this model, no matter the weather today, the probability that it is hot a week from now is approximately $66.67\%$.
In fact, approximately 2 out of 3 days in the year should be hot.

\subsection*{Steady State Distributions} % ------------------------------------

The state distribution $\x = [2/3,\ 1/3]\trp$ has another important property.
\[
A\x =
\left[\begin{array}{cc}
7/10 & 3/5 \\
3/10 & 2/5 \\
\end{array}\right]
\left[\begin{array}{c}2/3 \\ 1/3\end{array}\right]
=
\left[\begin{array}{c}14/30 + 3/15 \\ 6/30 + 2/15\end{array}\right]
=
\left[\begin{array}{c}2/3 \\ 1/3\end{array}\right]
= \x.
\]
Any $\x$ satisfying $A\x = \x$ is called a \emph{steady state distribution} or a \emph{stable fixed point} of $A$.
In other words, a steady state distribution is an eigenvector of $A$ with corresponding eigenvalue $\lambda = 1$.

% TODO: Verify this paragraph.
Every Markov chain has at least one steady state distribution.
If some power $A^k$ of $A$ has all positive (nonzero) entries, then the steady state distribution is unique.%
\footnote{This is a consequence of the \emph{Perron-Frobenius theorem}, which is presented in conjunction with spectral calculus in Volume I.}
In this case, $\lim_{k\rightarrow\infty}A^k$ is the matrix whose columns are all equal to the unique steady state distribution, as in (\ref{eq:markov-steady-transition}).
Under these circumstances, the steady state distribution $\x$ can be found by iteratively calculating $\x_{k+1} = A\x_k$, as long as the initial vector $\x_{0}$ is a state distribution vector.

\begin{warn}
Though every Markov chain has at least one steady state distribution, the procedure described above fails if $A^k$ fails to converge.
Consider the following example.
\[
A = \left[\begin{array}{ccc}
0 & 0 & 1 \\
0 & 1 & 0 \\
1 & 0 & 0
\end{array}\right]
,\quad A^k = \begin{cases}
A\quad\text{if } k\ \text{is odd}
\\
I\quad\text{if } k\ \text{is even}
\end{cases}
\]
In this case as $k\rightarrow\infty$, $A^k$ oscillates between two different matrices.

Furthermore, the steady state distribution is not always unique; the transition matrix defined above, for example, has infinitely many.
\end{warn}

\begin{problem} % Use the power method (simple) to get the steady state.
Write a function that accepts an $n\times n$ transition matrix $A$, a convergence tolerance $\epsilon$, and a maximum number of iterations $N$.
Generate a random state distribution vector $\x_{0}$ and calculate $\x_{k+1} = A\x_k$ until $\|\x_{k-1} - \x_k\| < \epsilon$.
If $k$ exceeds $N$, raise a \li{ValueError} to indicate that $A^k$ does not converge.
Return the approximate steady state distribution $\x$ of $A$.

To test your function, use Problem \ref{prob:random-markov-chain} to generate a random transition matrix $A$.
Verify that $A\x = \x$ and that the columns of $A^k$ approach $\x$ as $k\rightarrow\infty$.
To compute $A^k$, use NumPy's (very efficient) algorithm for computing matrix powers. % (which is not part of \li{scipy.linalg}).

\begin{lstlisting}
>>> A = np.array([[.7, .6],[.3, .4]])
>>> np.linalg.matrix_power(A, 10)       # Compute A^10.
array([[ 0.66666667,  0.66666667],
       [ 0.33333333,  0.33333333]])
\end{lstlisting}

Finally, use your function to validate the results of Problems \ref{prob:small-markov-transitions} and \ref{prob:markov-larger-chain}:
\begin{enumerate}
    \item Calculate the steady state distributions corresponding to the transition matrices for each simulation.
    \item Run each simulation for a large number of days and verify that the results match the steady state distribution (for example, check that approximately 2/3 of the days are hot for the smaller weather model).
\end{enumerate}

\label{prob:markov-power-method}
\end{problem}

\begin{info}
Problem \ref{prob:markov-power-method} is a special case of the \emph{power method}, an algorithm for calculating an eigenvector of a matrix corresponding to the eigenvalue of largest magnitude.
The general version of the power method, together with a discussion of its convergence conditions, is discussed in another lab.
\end{info}

\section*{Using Markov Chains to Simulate English} % ==========================
% TODO: is it okay to make this reference?
One of the original applications of Markov chains was to study natural languages.
% \footnote{The term \emph{natural language} refers to a spoken language, like English or Russian. See \url{http://langvillea.people.cofc.edu/MCapps7.pdf} for some details on the early applications of Markov chains, including the study of natural languages.}
In the early $20$th century, Markov used his chains to model how Russian switched from vowels to consonants.
By mid-century, they had been used as an attempt to model English.
It turns out that Markov chains are, by themselves, insufficient to model very good English.
However, they can approach a fairly good model of bad English, with sometimes amusing results.

By nature, a Markov chain is only concerned with its current state.
Thus a Markov chain simulating transitions between English words is completely unaware of context or even of previous words in a sentence.
For example, a Markov chain's current state may be the word ``continuous.''
Then the chain would say that the next word in the sentence is more likely to be ``function'' rather than ``raccoon.''
However, without the context of the rest of the sentence, even two likely words stringed together may result in gibberish.

We restrict ourselves to a subproblem of modeling the English of a specific file.
The transition probabilities of the resulting Markov chain reflect the sort of English that the source authors speak.
Thus the Markov chain built from \emph{The Complete Works of William Shakespeare} differs greatly from, say, the Markov chain built from a collection of academic journals.
We call the source collection of works in the next problems the \emph{training set}.

\subsection*{Making the Chain} % ----------------------------------------------

With the weather models of the previous sections, we chose a fixed number of days to simulate.
However, English sentences are of varying length, so we do not know beforehand how many words to choose (how many state transitions to make) before ending the sentence.
To capture this feature, we include two extra states in our Markov model: a \emph{start state} (\textcolor[rgb]{0,.6,0}{\$tart}) marking the beginning of a sentence, and a \emph{stop state} (\textcolor{red}{\$top}) marking the end.
Thus a training set with $N$ unique words has an $(N+2)\times (N+2)$ transition matrix.

The start state only transitions to words that appear at the beginning of a sentence in the training set, and only words that appear at the end a sentence in the training set transition to the stop state.
The stop state is called an \emph{absorbing state} because once we reach it, we cannot transition back to another state.
% Because every state has a possible path to the stop state, this model is called an \emph{absorbing Markov chain}.

After determining the states in the Markov chain, we need to determine the transition probabilities between the states and build the corresponding transition matrix.
Consider the following small training set from Dr. Seuss as an example.

\begin{lstlisting}
<<I am Sam Sam I am.
Do you like green eggs and ham?
I do not like them, Sam I am.
I do not like green eggs and ham.>>
\end{lstlisting}

If we include punctuation (so ``ham?'' and ``ham.'' are counted as distinct words) and do not alter the capitalization (so ``Do'' and ``do'' are also different), there are 15 unique words in this training set:
%
\begin{align*}
\text{I\quad am\quad Sam\quad am.\quad Do\quad you\quad like\quad green}
\\
\text{eggs\quad and\quad ham?\quad do\quad not\quad them,\quad ham.}
\end{align*}

With start and stop states, the transition matrix should be $17 \times 17$.
Each state must be assigned a row and column index in the transition matrix.
As easy way to do this is to assign the states an index based on the order that they appear in the training set.
Thus our states and the corresponding indices will be as follows:
%
\begin{align*}
\begin{array}{ccccccc}
\text{\textcolor[rgb]{0,.6,0}{\$tart}} & \text{I} & \text{am} & \text{Sam} & \ldots & \text{ham.} & \text{\textcolor{red}{\$top}}
\\
0 & 1 & 2 & 3 & \ldots & 15 & 16
\end{array}
\end{align*}

The start state should transition to the words ``I'' and ``Do'', and the words ``am.'', ``ham?'', and ``ham.'' should each transition to the stop state.
We first count the number of times that each state transitions to another state:

\begin{align*}
\begin{blockarray}{cccccccc}
& \text{\textcolor[rgb]{.3,.6,.1}{\$tart}} & \text{I} & \text{am} & \text{Sam} & & \text{ham.} & \text{\textcolor[rgb]{1,0,0}{\$top}} \\
\begin{block}{c[ccccccc]}
\text{\textcolor[rgb]{.3,.6,.1}{\$tart}}    & 0 & 0 & 0 & 0 & \ldots & 0 & 0\\
\text{I}        & 3 & 0 & 0 & 2 & \ldots & 0 & 0\\
\text{am}       & 0 & 1 & 0 & 0 & \ldots & 0 & 0\\
\text{Sam}      & 0 & 0 & 1 & 1 & \ldots & 0 & 0\\
& \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots\\
\text{ham.}     & 0 & 0 & 0 & 0 & \ldots & 0 & 0\\
\text{\textcolor[rgb]{1,0,0}{\$top}}        & 0 & 0 & 0 & 0 & \ldots & 1 & 1\\
\end{block}\end{blockarray}
\end{align*}

Now divide each column by its sum so that each column sums to 1.

\begin{align*}
\begin{blockarray}{cccccccc}
& \text{\textcolor[rgb]{.3,.6,.1}{\$tart}} & \text{I} & \text{am} & \text{Sam} & & \text{ham.} & \text{\textcolor[rgb]{1,0,0}{\$top}} \\
\begin{block}{c[ccccccc]}
\text{\textcolor[rgb]{.3,.6,.1}{\$tart}} & 0 & 0 & 0 & 0 & \ldots & 0 & 0\\
\text{I}        & 3/4 & 0 & 0 & 2/3 & \ldots & 0 & 0\\
\text{am}       & 0 & 1/5 & 0 & 0 & \ldots & 0 & 0\\
\text{Sam}      & 0 & 0 & 1 & 1/3 & \ldots & 0 & 0\\
& \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots\\
\text{ham.}     & 0 & 0 & 0 & 0 & \ldots & 0 & 0\\
\text{\textcolor[rgb]{1,0,0}{\$top}}    & 0 & 0 & 0 & 0 & \ldots & 1 & 1\\
\end{block}\end{blockarray}
\end{align*}

The $3/4$ indicates that 3 out of 4 times, the sentences in the training set start with the word ``I''.
Similarly, the $2/3$ and $1/3$ tell us that ``Sam'' is followed by ``I'' twice and by ``Sam'' once in the training set.
Note that ``am'' (without a period) always transitions to ``Sam'' and that ``ham.'' (with a period) always transitions the stop state.
Finally, to avoid a column of zeros, we place a 1 in the bottom right hand corner of the matrix (so the stop state always transitions to itself).

The entire procedure of creating the transition matrix for the Markov chain with words from a file as states is summarized below.

\begin{algorithm} % Read a file and convert it into a Markov chain.
\begin{algorithmic}[1]
\Procedure{MakeTransitionMatrix}{}
\State Count the number of unique words in the training set.
\State Initialize a square array of zeros of the appropriate size to be the transition \par\quad matrix (remember to account for the start and stop states).
\State Initialize a list of states, beginning with \li{"\$tart"}.
\For {each sentence in the training set}
    \State Split the sentence into a list of words.
    \State Add each \textbf{new} word in the sentence to the list of states.
    \State Convert the list of words into a list of indices indicating which row and \par\qquad\enspace column of the transition matrix each word corresponds to.
    \State Add 1 to the entry of the transition matrix corresponding to
    \par\qquad\enspace transitioning from the start state to the first word of the sentence.
    \For {each consecutive pair $(x, y)$ of words in the list of words}
        \State Add 1 to the entry of the transition matrix corresponding to \par\qquad\qquad transitioning from state $x$ to state $y$.
    \EndFor
    \State Add 1 to the entry of the transition matrix corresponding to
    \par\qquad\enspace transitioning from the last word of the sentence to the stop state.
\EndFor
\State Make sure the stop state transitions to itself.
\State Normalize each column by dividing by the column sums.
\EndProcedure
\end{algorithmic}
\caption{Convert a training set of sentences into a Markov chain.}
\label{alg:MarkovSentencesTransitionMatrix}
\end{algorithm}

\begin{problem} % Class that makes a Markov chain from a file.
Write a class called \li{SentenceGenerator}.
The constructor should accept a filename (the training set).
Read the file and build a transition matrix from its contents as described in Algorithm \ref{alg:MarkovSentencesTransitionMatrix}.

You may assume that the file has one complete sentence written on each line, and your implementation may be either column- or row-stochastic.
\label{problem:markov-random-sentences-init}
\end{problem}

\begin{problem} % Create random sentences.
Add a method to the \li{SentenceGenerator} class called \li{babble()}.
Begin at the start state and use the strategy from Problem \ref{prob:markov-larger-chain} to repeatedly transition through the object's Markov chain.
Keep track of the path through the chain and the corresponding sequence of words.
When the stop state is reached, stop transitioning to terminate the simulation.
Return the resulting sentence as a single string.

For example, your \li{SentenceGenerator} class should be able to create random sentences that sound somewhat like Yoda speaking.

\begin{lstlisting}
>>> yoda = SentenceGenerator("yoda.txt")
>>> for _ in range(3):
... 	print(yoda.babble())
...
<<Impossible to my size, do not!
For eight hundred years old to enter the dark side of Congress there is.
But beware of the Wookiees, I have.>>
\end{lstlisting}

\label{prob:markov-random-sentences-babble}
\end{problem}

\newpage

\section*{Additional Material} % ==============================================

\subsection*{Large Training Sets} % -------------------------------------------

The approach in Problems \ref{problem:markov-random-sentences-init} and \ref{prob:markov-random-sentences-babble} begins to fail as the training set grows larger.
For example, a single Shakespearean play may not be large enough to cause memory problems, but \emph{The Complete Works of William Shakespeare} certainly will.

To accommodate larger data sets, consider use a sparse matrix from \li{scipy.sparse} for the transition matrix instead of a regular NumPy array.
Specifically, construct the transition matrix as a \li{lil_matrix} (which is easy to build incrementally), then convert it to the \li{csc_matrix} format (which supports fast column operations).
Ensure that the process still works on small training sets, then proceed to larger training sets.
How are the resulting sentences different if a very large training set is used instead of a small training set?

\subsection*{Variations on the English Model} % -------------------------------

Choosing a different state space for the English Markov model produces different results.
Consider modifying your \li{SentenceGenerator} class so that it can determine the state space in a few different ways.
The following ideas are just a few possibilities.

\begin{itemize}
\item Let each punctuation mark have its own state.
In the example training set, instead of having two states for the words ``ham?'' and ``ham.'', there would be three states: ``ham'', ``?'', and ``.'', with ``ham'' transitioning to both punctuation states.

\item Model paragraphs instead of sentences.
Add a \textcolor[rgb]{0,.6,0}{\$tartParagraph} state that always transitions to \textcolor[rgb]{0,.6,0}{\$tartSentence} and a \textcolor{red}{\$topParagraph} state that is sometimes transitioned to from \textcolor{red}{\$topSentence}.

\item Let the states be individual letters instead of individual words.
Be sure to include a state for the spaces between words.
We will explore this particular state space choice more in Volume III together with hidden Markov models.

\item Construct the state space so that the next state depends on both the current and previous states.
This kind of Markov chain is called a \emph{Markov chain of order 2}.
This way, every set of three consecutive words in a randomly generated sentence should be part of the training set, as opposed to only every consecutive pair of words coming from the set.

\item Instead of generating random sentences from a single source, simulate a random conversation between $n$ people.
Construct a Markov chain $M_i,$ for each person, $i=1,\ \ldots,\ n$, then create a Markov chain $C$ describing the conversation transitions from person to person; in other words, the states of $C$ are the $M_i$.
To create the conversation, generate a random sentence from the first person using $M_1$.
Then use $C$ to determine the next speaker, generate a random sentence using their Markov chain, and so on.
\end{itemize}

\subsection*{Natural Language Processing Tools} % -----------------------------

The Markov model of Problems \ref{problem:markov-random-sentences-init} and \ref{prob:markov-random-sentences-babble} is a \emph{natural language processing} application.
Python's \li{nltk} module (natural language toolkit) has many tools for parsing and analyzing text for these kinds of problems.
For example, \li{nltk.sent_tokenize()} reads a single string and splits it up into sentences.

\begin{lstlisting}
>>> from nltk import sent_tokenize
>>> with open("yoda.txt", 'r') as yoda:
...     sentences = sent_tokenize(yoda.read())
...
>>> print(sentences)
<<['Away with your weapon!',
 'I mean you no harm.',
 'I am wondering - why are you here?',
 ...>>
\end{lstlisting}

The \li{nltk} module is \textbf{not} part of the Python standard library.
For instructions on downloading, installing, and using \li{nltk}, visit \url{http://www.nltk.org/}.


\subsection{Bilinear maps}

A bilinear map (or function) is a map from two inputs to an output which preserves addition and scalar multiplication. This is in contrast to a linear map, which only has one input.

In addition, the function is linear in both arguments.

That is if function \(f\) is bilinear then:

\(X=aM+bN\)

\(Y=cO+dP\)

\(f(X,Y)=f(aM+bN,cO+dP)\)

\(f(X,Y)=f(aM,cO+dP)+f(bN,cO+dP)\)

\(f(X,Y)=f(aM,cO)+f(aM,dP)+f(bN,cO)+f(bN,dP)\)

\(f(X,Y)=acf(M,O)+adf(M,P)+bcf(N,O)+bdf(N,P)\)

Note that:

\(f(X,Y)=f(X+0,Y)\)

\(f(X,Y)=f(X,Y)+f(0,Y)\)

\((0,Y)=0\)

That is, if any input is \(0\) in an additative sense, the value of the map must be zero.


\section{Topological Spaces}
\begin{definition}
    Consider a set $X$.
    A topology $\tau$ is a collection of subsets of $X$ such that the following axioms hold:\\
    1. $\varnothing, X\in\tau$.\\
    2. $\forall i\in I,U_i\in\tau\implies \bigcup_{i\in I}U_i\in\tau$.\\
    3. $U,V\in\tau\implies U\cap V\in\tau$.\\
    A topological space is a pair $(X,\tau)$ where $X$ is a set and $\tau$ is a topology on $X$.
\end{definition}
Note that the third axiom can be extended to any finite set of elements of $\tau$.\\
Members of $\tau$ are called open sets of $X$.
\begin{example}
    For any metric space, we can induce the metric topology by Proposition \ref{metric_topology}.
    For example, the Euclidean distance on $\mathbb R^n$ induce the usual topology on $\mathbb R^n$.
\end{example}
\begin{definition}
    A topological space $X$ (or the topology of $X$) is called metrizable if it can be induced by some metric on $X$.
\end{definition}
In the case where the topology is metrizable, any other metric that is equivalent to the previous metric gives the same topology.
\begin{example}
    The indiscrete topology on a set $X$ is $\{\varnothing, X\}$.
\end{example}
\begin{definition}
    Give topologies $\tau_1,\tau_2$ on $X$, we say $\tau_1$ is coarser than $\tau_2$ or $\tau_2$ is finer than $\tau_1$ if $\tau_1\subset\tau_2$.
\end{definition}
We know that the indiscrete topology is coarser than any topology on $X$.
It is then immediate that if $|X|\ge 2$, then the indiscrete topology is not metrizable.
Indeed, suppose $x,y\in X$, the open ball $D_{d(x,y)}(x)$, then it contains $x$ but not $y$ and is open under the metric topology under $d$, so $d$ cannot induce the indiscrete topology on $X$.
\begin{example}
    The discrete topology on a set $X$ is $\tau=2^X$.
    This is metrizable.
    Indeed, it can be induced by the discrete metric.
    It is also the finest topology on $X$.
\end{example}
\begin{example}
    The cofinite topology on a set $X$ consists of all subsets of $X$ whose complement is finite and the empty set.
    When $X$ is finite, this topology is just the discrete topology.
    If it is infinite, it is not metrizable.
    Fix $x\neq y\in X$, whenever there is open sets $U,V$ such that $x\in U, y\in V$, we know that $X\setminus(U\cap V)$ is finite, thus $U\cap V$ is not empty, but it would mean that the topological space that is not Hausdorff (which we will define below), but any metric space is (also below), so it is not metrizable.
\end{example}
\begin{definition}
    A topological space $X$ is called Hausdorff if any two distinct elements $x,y$ in $X$, there are open sets $U,V$ such that $x\in U,y\in V$ and $U\cap V=\varnothing$.
\end{definition}
\begin{proposition}
    Any metric space is Hausdorff.
\end{proposition}
\begin{proof}
    Consider $U=D_{d(x,y)/2}(x),V=D_{d(x,y)/2}(y)$, they obviously contain $x,y$ respectively and have empty intersection due to triangle inequality.
\end{proof}
\begin{definition}
    A subset of $X$ is called closed if its complement is open.
\end{definition}
This coincides with the definition of closed sets in a metric space by Lemma \ref{metric_complement}.
\begin{proposition}
    1. $\varnothing,X$ are closed\\
    2. If $A_i$ is closed for all $i\in I$, then $\bigcap_{i\in I}A_i$ is closed.\\
    3. If $A,B$ are closed, then $A\cup B$ is closed.
\end{proposition}
\begin{proof}
    Trivial.
\end{proof}
Again the last one can be generalized to any finite indices.
\begin{example}
    In cofinite topology, a subset is closed if and only if it is finite.
\end{example}
\begin{definition}
    For a topological space $X$, $x\in X$ and $U\subset X$.
    We say $U$ is a neighbourhood of $x$ if $\exists V\subset X$ open such that $x\in V\subset U$.
\end{definition}
Note again that in a metric space, this reduced to our previous definition.
The proof of this is trivial.
\begin{proposition}
    Let $U\subset X$, then $U$ is open if and only if every $x\in U$ has a neighbourhood contained in $U$.
\end{proposition}
\begin{proof}
    Completely trivial.
\end{proof}
\begin{definition}
    A sequence $(x_n)\in X$ converges to some $x\in X$, or $x_n\to x$, if for any neighbourhood $V$ of $x$, $\exists N\in\mathbb N,\forall n>N, x_n\in V$.
\end{definition}
This again and again coincides with previous definition in metric spaces by Proposition \ref{metric_nbhdconv}.
\begin{example}
    In a indiscrete space, any sequence converge to any element.
\end{example}
\begin{theorem}
    In a Hausdorff space, limits are unique.
\end{theorem}
\begin{proof}
    If $x_n\to x$ and $x_n\to y$ but $x\neq y$, then there are disjoint open sets $U,V$ containing $x,y$ respectively.
    But then there is some $N_1\in\mathbb N,\forall n>N, x_n\in U$, and there is also some $N_2\in\mathbb N,\forall n>N, x_n\in V$, but then for any $n>\max\{N_1,N_2\}$, $x_n\in U\cap V=\varnothing$, contradicion.
\end{proof}
\begin{remark}
    In a metric space, $A$ is closed if and only if whenever $x_n$ converges in the metric space, then its limit is in $A$.
    The $\implies$ part is true in all topological space, but not necessarily $\impliedby$.
\end{remark}
\begin{definition}
    Let $X$ be a topological space and $A\subset X,x\in X$.
    $x$ is called an accumulation point (aka limit point/cluster point) of $A$ if for any neighbourhood $U$ of $x$, $(A\setminus\{x\})\cap U\neq\varnothing$.\\
    The derived set $A'$ of $A$ is the set of all accumulation points of $A$.
\end{definition}
\begin{example}
    In $\mathbb R$, suppose $A=[0,1)\cup \{2\}$, then $A'=[0,1]$.
    Also $\mathbb Q'=\mathbb R$, and $\mathbb Z'=\varnothing$.
\end{example}
\begin{proposition}
    Let $X$ be a topological space and $A\subset X$, then $A$ is closed if and only if $A'\subset A$.
\end{proposition}
\begin{proof}
    If $A$ is closed, then $U=X\setminus A$ is open, so for any $x\in X\setminus A$, $U$ is a neighborhood of $x$ but $U\cap A=\varnothing$, so every accumulation points of $A$ are inside of $A$.\\
    Conversely, given $x\in X\setminus A$, then $x\notin A'$, so there is a neighbourhood $U$ of $x$ with $U\cap A=\varnothing$, so $x\in U\subset X\setminus A$, so $X\setminus A$ is open, hence $A$ is closed.
\end{proof}
\begin{definition}
    Let $A$ be a subset of a topological space $X$, then the interior of $A$, $\operatorname{int}A$ or $A^\circ$ is defined by
    $$\operatorname{int}A=\bigcup_{U\subset A\text{, $U$ open}}U$$
    The closure, $\operatorname{cl}A$ or $\bar A$, is defined by
    $$\operatorname{cl}A=\bigcap_{A\subset F\text{, $F$ closed}}F$$
\end{definition}
Note that $A^\circ\subset A\subset\bar{A}$, and $A^\circ =A$ if and only if $A$ is open, $\bar A=A$ if and only if $A$ is closed.
\begin{proposition}
    \begin{align*}
        A^\circ&=\{x\in X:\text{$A$ is a neighbourhood of $x$}\}\\
        \bar{A}&=\{x\in X:\text{$\forall U\subset X$ such that $U$ is a neighbourhood of $x$, $U\cap A=\varnothing$}\}\\
        &=A\cup A'
    \end{align*}
\end{proposition}
\begin{proof}
    Trivial.
\end{proof}
\begin{example}
    In $\mathbb R$, $\overline{[0,1)\cup\{2\}}=[0,1]\cup\{2\}$, $([0,1)\cup\{2\})^\circ=(0,1)$, $\bar{\mathbb{Q}}=\mathbb R, \mathbb Q^\circ=\varnothing=\mathbb Z^\circ, \bar{\mathbb Z}=\mathbb Z$.
\end{example}
\begin{remark}
    Convergent sequences determine the metric topology, since $x\in\bar{A}\iff \exists (x_n)\in A, x_n\to x$.
    Again we have the $\impliedby$ direction for all topological spaces but not necessarily for the $\implies$ direction.
\end{remark}
\begin{definition}
    Let $X$ be a topological space and $A\subset X$.
    We say $A$ is dense if $\bar{A}=X$.\\
    We say $X$ is seperable if there is a countable dense set in $X$.
\end{definition}
\begin{definition}
    1. $\mathbb R^n$ is seperable since $\bar{\mathbb Q^n}$.
    2. (non-example) An uncountable set in the discrete topology is not seperable.
\end{definition}
As usual we can try to construct new spaces from old.
\begin{definition}
    Let $(X,\tau)$ be a topological space and $Y\subset X$.
    The subspace (or relative) topology on $Y$ is the collection $\{U\cap Y:U\in\tau\}$.
    This is also called the topology on $Y$ induced by $\tau$.
\end{definition}
One can check that this is indeed a topology.
\begin{example}
    Let $X=\mathbb R$, $Y=[0,2]$, then $U=(1,2]$ is open in $Y$ since $U=Y\cap (1,3)$.
    Note that $U$ is not open in $X$.
\end{example}
\begin{remark}
    1. If $Z\subset Y\subset X$, then the topology on $Z$ induced by the topology on $X$ is the topology on $Z$ induced by the topology on $Y$ which is induced by the topology on $X$.
    So the subspace of a subspace is a subspace.\\
    2. If $N\subset M$ where $N,M$ are metric spaces, then the metric on $N$ induced by the metric on $M$ induces the metric topology on $N$ induced by the metric topology on $M$.
\end{remark}
\begin{proposition}
    Let $Y$ be the subspace of a topological space $X$.\\
    1. $A\subset Y$ is closed in $Y$ if and only if there is closed set $B\subset X$ such that $B\cap Y=A$.\\
    2. $\forall A\subset Y, \bar{A}^Y=Y\cap\bar{A}^X$.
\end{proposition}
\begin{remark}
    The analogy of $2$ on interiors does not always work.
    Take $X=\mathbb R,Y=\{0\}$.
\end{remark}
\begin{proof}
    Trivial.
\end{proof}
\begin{definition}
    A base for a topological space $(X,\tau)$ is a family $\mathscr B\subset\tau$ such that $\forall U\in\tau,\exists \mathscr C\subset\mathscr B$ such that
    $$U=\bigcup_{B\in\mathscr C}B$$
\end{definition}
In other words, the topology $\tau$ consists of the arbitrary unions of some family of open sets which is a subset of $\mathscr B$.
So a base determines topology.
\begin{example}
    1. The set of all open intervals is a base of the usual topology on $\mathbb R$.
    In general, the collection of all open balls in a metric space is a base for the metric topology on it.
\end{example}
However, what we want to do is not to construct $\mathscr B$ from $\tau$, but the other way around.
\begin{lemma}
    let $X$ be a set and $\mathscr B\subset 2^X$.
    Assume that\\
    1. $X=\bigcup_{B\in\mathscr B}B$.\\
    2. $\forall B_1,B_2\in\mathscr B,\forall x\in B_1\cap B_2, \exists B\in\mathscr B,x\in B\subset B_1\cap B_2$.\\
    Then there is an unique topology on $X$ that is generated by the base $\mathscr B$.
\end{lemma}
\begin{proof}
    We must have the topology
    $$\tau=\left\{\bigcup_{B\in\mathscr C}B:\mathscr C\subset\mathscr B\right\}$$
    It is immediate that $\tau$ is a topology on $X$.
    Indeed, $\varnothing,X\in\tau$ and it is closed under arbitrary union.
    For intersection, consider
    $$U_1=\bigcup_{B\in\mathscr C_1}B,U_2=\bigcup_{B\in\mathscr C_2}B$$
    Given $x\in U_1\cap U_2$, so $\exists B_1\in\mathscr C_1, B_2\in\mathscr C_2$, so there is some $B_x\in\mathscr B$ such that $x\in B_x\subset B_1\cap B_2\subset U_1\cap U_2$, thus
    $$U_1\cap U_2=\bigcup_{x\in U_1\cap U_2}B_x$$
    By definition $\mathscr B$ is a base for $\tau$.
\end{proof}
\begin{definition}
    A topological space is called second-countable if it has a countable base.
\end{definition}
\begin{example}
    The set of all open balls of rational radii and centres is a countable base for $\mathbb R^n$.
    So $\mathbb R^n$ is seond-countable.
\end{example}
\begin{definition}
    A map $f:(X,\tau)\to (Y,\rho)$ is continuous if $V\in\rho\implies f^{-1}(V)\in\tau$.
\end{definition}
This extends our previous defintion of continuity in metric space by Proposition \ref{metric_preimage}.
\begin{proposition}
    Let $f:X\to Y$ be a map between topological spaces, then\\
    1. $f$ is continuous if and only if the preimage of any closed set is closed.\\
    2. If $\mathscr B$ is a base for $Y$, then $f$ is continuous if and only if for all $B\in\mathscr B$, $f^{-1}(B)$ is open in $X$.\\
    3. Composition of continous functions is continuous.
\end{proposition}
\begin{proof}
    Trivial.
\end{proof}
\begin{example}
    Constant, identity and inclusion are always continuous.
    Hence the restriction of a continuous map is continuous.
\end{example}
\begin{definition}
    Let $f:X\to Y$ be a map between topological spaces, then we say $f$ is a homeomorphism if $f$ is a bijection and both $f,f^{-1}$ are continuous.
    We say $X,Y$ are homeomorphic, or $X\cong Y$, if there is a homeomorphism between them.
\end{definition}
\begin{definition}
    $f$ is a open map if for every $U$ open in $X$, $f(U)$ is open in $Y$.
    So $f$ is a homeomorphism if and only if $f$ is a continuous open bijection.
\end{definition}
\begin{definition}
    A property $P$ of topological spaces is called a topological property if it is preserved under homeomorphisms.
\end{definition}
\begin{definition}
    Let $(X,\tau),(Y,\rho)$ be topological spaces and let
    $$\mathscr B=\{U\times V:U\in\tau, V\in\rho\}$$
    Then $X\times Y\in\mathscr B$ and $U_1\times V_1\cap U_2\times V_2=(U_1\cap U_2)\times (V_1\times V_2)\in\mathscr B$.
    Thus there is an unique topology on $X\times Y$ with base $\mathscr B$.
    This is called the product topology.
\end{definition}
So a set $W$ in the product topological space is open if and only if $\forall (x,y)\in W,\exists U\in\tau, V\in\rho, x\in U\times V\subset W$.
\begin{example}
    $\mathbb R^2$ in the usual topology is homeomorphic to $\mathbb R\times\mathbb R$ in the product topology.
    In general, the topology induced by the ($p$-)product metric is the product topology of metric topologies.
    So products of metrizable topologies are metrizable.
\end{example}
\begin{proposition}
    Consider $\pi_X:X\times Y\to X,\pi_Y:X\times Y\to Y$ be the projections.
    Then $\pi_X,\pi_Y$ are continuous and if $Z$ is a topological space, and $f:Z\to X\times Y$ is continuous if and only if $\pi_X\circ f,\pi_Y\circ f$ are both continuous.
\end{proposition}
Note that $f(z)=(\pi_X\circ f(z),\pi_Y\circ f(z))$.
\begin{proof}
    Given an open set $U\subset X$, $\pi_X^{-1}(U)=U\times Y$, which is open in $X\times Y$, so $\pi_X$ is continuous.
    Similarly, $\pi_Y$ is continuous.\\
    Given such an $f$, if $f$ is continuous, then both of $\pi_X\circ f,\pi_Y\circ f$ are continuous since composition of continuous functions is continuous.
    Conversely, if both of $\pi_X\circ f,\pi_Y\circ f$ are continuous, then it is enough to check that any member of the base $U\times V\subset X\times Y$ has an open preimage.
    Indeed,
    \begin{align*}
        f^{-1}(U\times V)&=f^{-1}(U\times Y)\cap f^{-1}(X\times V)\\
        &=f^{-1}(\pi_X^{-1}(U))\cap f^{-1}(\pi_Y^{-1}(V))\\
        &=(\pi_X\circ f)^{-1}(U)\cap (\pi_Y\circ f)^{-1}(V)
    \end{align*}
    which is open by assumption.
\end{proof}
It is trivial to extend all the above to finite products.
It is interesting to know that $(X\times Y)\times Z\cong X\times (Y\times Z)$, and $X\times Y\cong Y\times X$.
%But how about arbitrary products?
Now we turns to quotient topology.
\begin{definition}
    Start with a topological space $(X,\tau)$ and let $R$ be an equivalence relation on $X$.
    We let $X/R$ be the set of equivalence classes (the ``quotient set'').
    Let $q:X\to X/R$ be the quotient map sending $x\mapsto [x]$ where $[x]=\{y\in X:yRx\}$ is the equivalence class containing $x$.
    The quotient topology on $X/R$ is the family
    $$\tau_R=\{V\subset X/R:q^{-1}(V)\in\tau\}$$
\end{definition}
\begin{proposition}
    the quotient topology is indeed a topology.
\end{proposition}
\begin{proof}
    $q^{-1}(X/R)=X,q^{-1}(\varnothing)=\varnothing$, so $\varnothing,X\in\tau_R$.
    $$\forall (V_i)_{i\in I}\in\tau_R,q^{-1}\left(\bigcup_{i\in I}V_i\right)=\bigcup_{i\in I}q^{-1}(V_i)$$
    which is open.
    $$\forall U,V\in\tau_R,q^{-1}(U\cap V)=q^{-1}(U)\cap q^{-1}(V)$$
    which is also open.
\end{proof}
\begin{remark}
    1. Note that $q$ is surjective and continuous under $\tau_R$.\\
    2. For $x\in X,t\in X/R, x\in t\iff q(x)=t$, hence
    $$\forall V\subset X/R,q^{-1}(V)=\{x\in X:q(x)\in V\}=\{x\in X:\exists t\in V, q(x)=t\}=\bigcup_{t\in V}t$$
\end{remark}
\begin{example}
    $\mathbb Q\le\mathbb R$ as (additive) groups, so $\mathbb R/\mathbb Q$ gives a equivalence relation.
    So we can induce a quotient topology on $\mathbb R/\mathbb Q$, which immediately we can find to be the indiscrete topology which is not metrizable, which is why we do not do quotients in metric spaces.
\end{example}
Consider $q:X\to X/R$ the quotient map and any map $f:X\to Y$ such that $xRy\implies f(x)=f(y)$, then there is a map $\tilde{f}:X/R\to Y$ such that $\tilde{f}\circ q=f$.
That is, the following diagram commutes.
$$
\begin{tikzcd}
    X\arrow{r}{f}\arrow[swap]{d}{q}&Y\\
    X/R\arrow[swap,dashed]{ur}{\tilde{f}}&
\end{tikzcd}
$$
If $f$ is surjective, so is $\tilde{f}$.
Also, if $f(x)=f(y)\iff xRy$, then $\tilde{f}$ is injective.
\begin{proposition}
    Let $X,Y$ be topological spaces, $R$ an equivalence relation on $X$, $q:X\to X/R$ the quotient map, $f:X\to Y$ some map with $xRy\implies f(x)=f(y)$, then let $\tilde{f}$ be as above, then\\
    1. If $f$ is continuous so is $\tilde{f}$.\\
    2. If $f$ is an open map so is $\tilde{f}$.
\end{proposition}
\begin{proof}
    1. Let $V$ be open in $Y$, then $f^{-1}(V)$ is open in $X$, so $q^{-1}(\tilde{f}^{-1}(V))=f^{-1}(V)$ is open, so $\tilde{f}^{-1}(V)$ is open, hence $\tilde{f}$ is continuous.\\
    2. Given open $V\in X$, $U=q^{-1}(V)$ is open in $X$, and $V=q(U)$, so $\tilde{f}(V)=f(U)$ which is open.
\end{proof}
\begin{corollary}
    If $f(x)=f(y)\iff xRy$, $f$ is surjective, continuous and open, then $\tilde{f}$ is a homeomorphism.
\end{corollary}
\begin{remark}
    Work ``upstairs''!
\end{remark}
\begin{example}
    Take $\mathbb R/\mathbb Z$ where the equivalence relation is as if they are additive groups.
    $\mathbb R/\mathbb Z\cong S^1=\{z\in\mathbb C:|z|=1\}$.
    Indeed, consider the map $f(t)=e^{2\pi it}$, then the induced $\tilde{f}$ is a homeomorphism by the preceding corollary.
    If $f$ is not open, then there is some open $U\in\mathbb R$ such that $f(U)$ is not open.
    $\exists (z_n)\in S^1\setminus f(U)$ such that $z_n\to z$, then due to surjectivity we know that there is some $x\in U$ such that $f(x)=z$ and $(x_n)\in [x-1/2,x+1/2]$ such that $f(x_n)=z_n$ and we know that $x_n\notin U$, but $x_n$ has a convergent subsequence $(x_{k_n})\to y\in \mathbb R\setminus U$ which is closed, so due to continuity we must have $f(x)=f(y)\implies x-y\in\mathbb Z\implies x=y\notin U$, which is a contradiction.
\end{example}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsfonts, amsmath, enumitem, float, graphicx, hyperref}

\title{Numerical Computing - Notes}
\author{Matteo Alberici}
\date{January 2022}

\begin{document}

\maketitle
\newpage
\tableofcontents
\newpage

% --------------------------------- %
% Assignment 1 - PageRank Algorithm
% --------------------------------- %
\section{PageRank Algorithm}
\subsection{Eigenvectors}
An \textbf{eigenvector} $v \in \mathbb{R}$ of a matrix $A$ is a nonzero vector such that the relation
\begin{center}
    $Av = \lambda v$
\end{center}
holds, where $\lambda$ is the \textbf{eigenvalue} associated with vector $v$. \\
An \textbf{eigenbasis} is a basis in which every vector is an eigenvector. \\
Given a vector $v$, the \textbf{Rayleigh quotient} computes the eigenvalue of $v$ whether $v$ is an eigenvector, or the eigenvalue associated with closest eigenvector of $v$ otherwise:
\begin{center}
    $\mu(v) = \displaystyle\frac{v^TAv}{v^Tv}$
\end{center}
\subsection{PageRank Algorithm}
The \textbf{PageRank} is determined by the structure of the World Wide Web. For any query, Google lists the matching Web pages in their PageRank order: a Web page has a high rank if other pages with a high rank link to it. \\
The algorithm is based on the \textbf{Random Surfer Model}, in which a user goes from one page to another by randomly choosing an outgoing link through \textbf{exploitation}. Since surfing randomly could lead to dead ends or cycles of Web pages, a random Web page is chosen through \textbf{exploration}. The random walk generated by exploitation and exploration is known as the \textbf{Markov chain}. \\
Let's introduce some definitions for the PageRank computation:
\begin{itemize}
    \item $n \ = \ $ number of Web pages
    \item $G \ = \ $ $n-$by$-n$ connectivity matrix of a Web portion with:
        \begin{center}
            $g_{ij} =
            \begin{cases}
                1 & \text{hyperlink from page } j \text{ to page } i  \\
                0 & \text{otherwise}
            \end{cases}$
        \end{center}
    \item $r_i \ = \ $ row sum of $G$ and in-degree of $j-$th page:
        \begin{center}
            $r_i = \displaystyle\sum_j g_{ij}$
        \end{center}
    \item $c_j \ = \ $ column sum of $G$ and out-degree of $j-$th page:
        \begin{center}
            $c_j = \displaystyle\sum_i g_{ij}$
        \end{center}
    \newpage
    \item $p \ = \ $ probability that the random walk follows a link
        \begin{itemize}
            \item the typical value is: $\ p = 0.85$
            \item the probability that some arbitrary page is chosen is: $\ 1 - p$
            \item the probability that a particular random page is chosen is:
                \begin{center}
                    $\delta = \displaystyle\frac{1 - p}{n}$
                \end{center}
        \end{itemize}
    \item $A \ = \ $ $n-$by$-n$ transition probability matrix of the Markov chain with:
        \begin{center}
            $a_{ij} =
            \begin{cases}
                p \cdot \frac{g_{ij}}{c_j} + \delta & c_j \neq 0 \\
                \frac{1}{n} & c_j = 0
            \end{cases}$
        \end{center}
\end{itemize}
All the elements in matrix $A$ are strictly between $0$ and $1$ and all the column sums are equal to $1$. \\
The \textbf{Perron-Frobenius theorem} states that a nonzero solution of the equation
\begin{center}
    $x = Ax$
\end{center}
exists and is unique to within a scaling factor. If this factor is chosen so that:
\begin{center}
    $\displaystyle\sum_i x_i = 1$,
\end{center}
then $x$ is the \textbf{state vector} of the Markov chain and Google's PageRank. Its elements are strictly between $0$ and $1$ and it is the solution to the following linear system:
\begin{center}
    $(I - A)x = 0$
\end{center}
The best way to compute PageRank in MATLAB is defining matrix $A$ as follows:
\begin{center}
    $A = p GD + ez^T$, where:
\end{center}
\begin{itemize}
    \item $D\ = \ $ diagonal matrix formed from the out-degrees reciprocals:
        \begin{center}
            $d_{jj} =
            \begin{cases}
                1/ c_j & c_j \neq 0 \\
                0 & c_j = 0
            \end{cases}$
        \end{center}
    \item $e \ = \ $ $n-$vector of all ones
    \item $z \ = \ $ vector with:
        \begin{center}
            $z_j =
            \begin{cases}
                \delta & c_j \neq 0 \\
                1/n & c_j = 0
            \end{cases}$
        \end{center}
\end{itemize}
The assignment statement can now be written as follows:
\begin{center}
    $(I - pGD)x = \lambda e$,
\end{center}
where $ \ \lambda = z^Tx \ $ has the temporary value of $1$.
\subsection{Power Method}
The \textbf{power method} is used to compute the dominant eigenvector $\lambda_1$ of a matrix $A$:
\begin{enumerate}
    \item Start with an initial guess vector $v_0$:
    \item Compute $w$ in the following way:
    \begin{center}
        $w = Av_k$
    \end{center}
    \item Set $v_{k+1} = ||w||$
    \item Find the eigenvalue of $v_{k+1}$ by using the Rayleigh quotient
    \item Repeat from step $2$ incrementing $k$ by $1$ until the difference between the previous and the current eigenvectors is lower than a certain threshold
\end{enumerate}
In order to guarantee convergence, we assume that matrix $A$ has an eigenvalue $\lambda$ whose magnitude is strictly greater than the others and that vector $x$ has a nonzero component in the direction of the eigenvector associated with $\lambda$.
\subsection{Shift and Invert Method}
Since the power method converges linearly and its error constant is $|\lambda_2 / \lambda_1|$, the convergence could be too slow if $||\lambda_2||$ and $||\lambda_1||$ are close: the \textbf{shift and invert method} is used to improve the power method's converging rate. \\
Given a matrix $A$, we define matrix $B$ as:
\begin{center}
    $B = (A - \alpha I)^{-1}$
\end{center}
The eigenvalues of $B$ are defined as:
\begin{center}
    $u_j = \displaystyle\frac{1}{\lambda_j - \alpha}$.
\end{center}
By applying the power method to matrix $B$ and assuming $\lambda_2$ is the closest-to-$\lambda_1$ eigenvalue of $A$, we obtain the following improved converging rate:
\begin{center}
    $|\displaystyle\frac{u_2}{u_1}| = |\displaystyle\frac{\lambda_1 - \alpha}{\lambda_2 - \alpha}|$
\end{center}
The cost of a power method iteration is given by a vector-matrix product, the cost of an iteration in the shift and invert method is given by a linear system.

\newpage

% ------------------------------ %
% Assignment 2 - Social Networks
% ------------------------------ %
\section{Social Networks}
\subsection{Cholesky Factorization}
The \textbf{Cholesky factorization} is the decomposition of a symmetric, positive-definite matrix $A$ into the product of its lower triangular form $L$ and its conjugate transpose $L^T$:
\begin{center}
    $A = LL^T$
\end{center}
It is twice as efficient as the LU decomposition.
\subsection{Reverse Cuthill McKee Ordering}
The \textbf{reverse Cuthill McKee ordering} is the permutation of a symmetric sparse matrix into a banded matrix with a smaller bandwidth. The resulting matrix has the non-zero elements closer to the diagonal. \\
In MATLAB, we can use the following implementation:
\begin{verbatim}
    1) r = symrcm(A(2:end, 2:end));
    2) prcm = [1 r+1];
\end{verbatim}
This method is useful to reduce the number of fill-ins, making less expensive the Cholesky factorization.
\subsection{Spectral Graph Partitioning}
In \textbf{spectral graph partitioning}, we plant a random partition assigning values to the probabilities of intra-sets and inter-sets edges, then use the \textbf{Fiedler's vector}, which is the eigenvector $v_2$ associated with the second smallest eigenvalue $\lambda_2$, to find the partition. \\
In the Fiedler's vector, all indices of entries $ > 0$ belong to one set and those of entries $< 0$ belong to the other. The partition minimizes the number of edges between the sets.
\subsection{Degree Centrality}
The \textbf{degree centrality} consists of ranking the number of incident links upon a node. For a given graph $G = (V,E)$ it is defined as the number of edges of a vertex $v$. \\
The \textbf{eigenvector centrality} measures the influence of a node in a network. The relationships between nodes with a high score contribute more to the node score.

% --------------------------------- %
% Assignment 3 - Graph Partitioning
% --------------------------------- %
\section{Graph Partitioning}
\subsection{Graphs Matrices}
\subsubsection{Degree Matrix}
A \textbf{degree matrix} $D$ is a diagonal matrix containing the degree of each vertex. If the graph is weighted, the entries are the sum of all the weights of the connected edges.
\subsubsection{Adjacency Matrix}
An \textbf{adjacency matrix} is a square matrix representing the connections of an unweighted graph and its entries are defined as follows:
\begin{center}
    $a_{ij} =
    \begin{cases}
        1 & v_i \text{ connected to } v_j \\
        0 & \text{otherwise}
    \end{cases}$
\end{center}
\subsubsection{Weight Matrix}
A \textbf{weight matrix} $W$ is a square matrix representing the connections of a weighted graph and its entries are defined as follows:
\begin{center}
    $w_{ij} =
    \begin{cases}
        w & v_i \text{ connected to } v_j \\
        0 & \text{otherwise}
    \end{cases}$
\end{center}
\subsubsection{Laplacian Matrix}
The \textbf{Laplacian matrix} $L$ is the matrix representation of a graph and is computed in the following ways:
\begin{itemize}
    \item For an undirected and unweighted graph
        \begin{center}
            $L = D - A $
        \end{center}
    \item For an undirected and weighted graph
        \begin{center}
            $L = D - W $
        \end{center}
\end{itemize}
Matrix $L$ only has real and non-negative eigenvalues and its eigenvectors are real and orthogonal.
\subsection{Graph Partitioning}
The \textbf{graph partitioning} problem is defined on a graph $G = (V, E) $ such that it is possible to partition $G$ into smaller components with specific properties, generally with small cuts and equal-size partitions.
\subsection{Spectral Bisection}
\textbf{Spectral bisection} enables the decomposition of a symmetric matrix into eigenvalues within an orthonormal base of eigenvectors. The following procedure performs a spectral bisection:
\begin{enumerate}
    \item Compute the Laplacian matrix $L$
    \item Compute the Fiedler's vector $w_2$
    \item Set a threshold to $0$ or to the median of $w_2$
    \item Choose $V_1 = \{v_i \in V | w_i < \text{threshold}\}$ and $V_2 = \{v_i \in V | w_i \geq \text{threshold}\}$
    \item Return $V_1$ and $V_2$
\end{enumerate}
Thresholding the values of $w_2$ around $0$ results in two roughly equal-sized partitions with minimum edgecut, while thresholding around the median value of $w_2$ produces two strictly balanced partitions.
\subsection{Inertial Bisection}
\textbf{Inertial bisection} relies on the vertices' geometric coordinates: it finds the hyperplane running through the center of mass of the points. \\
In $2D$, the line is chosen such that the sum of squares of the distances of the nodes to the line is minimized. \\
The following procedure performs an inertial bisection:
\begin{enumerate}
    \item Compute the center of mass of the points as follows:
        \begin{center}
            $x = \displaystyle\frac{1}{n} \displaystyle\sum_{i = 1}^n x_i \ \ \ \ \ y = \displaystyle\frac{1}{n}\displaystyle\sum_{i = 1}^n y_i$
        \end{center}
    \item Compute matrix $M$ as follows:
        \begin{center}
            $S_{xx} = \displaystyle\sum_{i = 1}^n(x_i - x)^2 \ \ \ \ S_{yy} = \displaystyle\sum_{i = 1}^n(y_i - y)^2 \ \ \ \ S_{xy} = \displaystyle\sum_{i = 1}^n((x_i - x)(y_i - y))$ \\
            \vspace{0.5cm}
            $ M =
            \begin{bmatrix}
                S_{xx} & S_{xy} \\
                S_{xy} & S_{yy}
            \end{bmatrix}$
        \end{center}
    \item Compute the smallest eigenvalue of matrix $M$ and the associated eigenvector $u$ \item Minimize the distance of the nodes to the line:
        \begin{center}
            $u^TMu$
        \end{center}
    \item Project each point to the line and compute the median to partition the nodes
    \item Return $V_1$ and $V_2$
\end{enumerate}
\subsection{Recursive Bisection}
\textbf{Recursive bisection} depends on the decisions made during the earlier stages of the process and lacks of global information, meaning that it may result in suboptimal partitions.
\subsection{$K-$Way Partitioning}
\textbf{$K-$way partition} starts with partitioning a small set of vertices, then projects it back towards the original set to refine it. The main difference with recursive bisection is that $k-$way partitioning stores global information about the graph.
\subsection{Partitioning Metrics}
The number of cut edges between partitions determines the result's quality. The size of an \textbf{edgecut} partitioning the graph into two complementary vertices subsets is defined as follows:
\begin{center}
    $cut(V_1,V_2) = \displaystyle\sum_{i \in V_1, j \in V_2} w_{ij}$
\end{center}
The \textbf{cardinality}, i.e. the number of nodes, of each subset is given by:
\begin{center}
    $x^Tx = ||x||_2^2 = |V_1|$
\end{center}
Let $\Tilde{V}$ be the average partition weight, then for a $k-$way partition the \textbf{load imbalance} $b^k$ is defined as follows:
\begin{center}
    $b^k = max_{|V_i^k|} \ \ \displaystyle\frac{|V_i^k|}{|\Tilde{V}^k|}$ \\
    \vspace{0.3cm}
    $b^k = 1 + b^k_r \geq 1$ \\
    \vspace{0.35cm}
    $b^k_r \geq 0 $
\end{center}
It characterizes the deviation from obtaining a balanced partitioning. \\
The optimal value for $b^k_r$ is $0$ and implies that the $k$ partitions contain the same number of nodes.

\newpage

% ---------------------------------------- %
% Assignment 4 - Spectral Graph Clustering
% ---------------------------------------- %
\section{Spectral Graph Clustering}
\subsection{Graph Clustering}
Given a set of data points $x_1,...,x_n$ and some notion of similarity $s_{ij} \geq 0$, clustering extracts information by grouping similar data points.
\subsection{Trees}
A \textbf{tree} is an undirected graph in which any two vertices are connected by exactly one path. A \textbf{minimum spanning tree} is a subgraph including all the vertices of the graph, the minimum possible number of edges and the minimum possible total weight.
\subsection{Similarity Graphs}
In \textbf{similarity graphs} two vertices $v_i$ and $v_j$ are connected based on the similarity $s_{ij}$ between the corresponding data points $x_i$ and $x_j$. The edge between them is weighted by $s_{ij}$. \\
The goal is to model the neighborhood relationships between the data points.
\subsubsection{$\epsilon-$Neighborhood Graph}
In $\epsilon-$neighborhood graphs, we connect two vertices if their pairwise distances are smaller than $\epsilon$. Weighting the edges would not incorporate more information about the data to the graph, thus they are unweighted.
\subsubsection{$k-$Nearest Neighbor Graphs}
In \textbf{$k-$nearest neighbor graphs}, we connect vertices $v_i$ and $v_j$ if the latter is among the $k-$nearest neighbors of the first. The graph is directed and we have two ways to make it undirected: either we ignore the directions obtaining a simple $k-$nearest neighbors graph, or we connect the vertices if each of them is among the $k-$nearest neighbors of the other, obtaining a \textbf{mutual $k-$nearest neighbors graph}.
\subsubsection{Fully Connected Graph}
In \textbf{fully connected graphs}, we connect two vertices if the similarity $s_{ij} > 0$. This construction is useful only if the similarity function models local neighborhoods.
\newpage
\subsection{Spectral Clustering}
In \textbf{spectral clustering}, we create clusters with the same number of nodes through the following procedure:
\begin{enumerate}
    \item Compute the minimum spanning tree to determine $\epsilon$
    \item Create the $\epsilon-$neighbourhood graph
    \item Compute the adjacency matrix $W$ as follows:
    \begin{center}
        $W = S \odot G$
    \end{center}
    \item Compute the Laplacian matrix
    \item Compute the $k-$smallest eigenvectors based on $L$
    \item Use k-means algorithm to cluster the nodes
    \item Use the eigenvector matrix in order to obtain the clusters
\end{enumerate}
\subsection{K-Means Clustering}
The \textbf{k-means clustering} divides the clusters based on the distance between a node and a \textbf{centroid}, which represents the center of a cluster. The algorithm is the following:
\begin{enumerate}
    \item Start with some random centroids
    \item Assign each node to the nearest cluster based on its centroid
    \item Recompute the centroids as the mean of the points in the cluster
    \item Repeat from step $2$ until no further improvement can be made
\end{enumerate}

\newpage

% ------------------------------------------------------ %
% Assignment 5 - Image Deblurring and Conjugate Gradient
% ------------------------------------------------------ %
\section{Image Deblurring and Conjugate Gradient}
\subsection{Blurred Image Problem Definition}
Let matrix $B \in \mathbb{R}^{n^2 \times n^2}$ be a blurred image we want to deblur given the transformation and let $X \in \mathbb{R}^{n \times n}$ be the original square and greyscale image matrix where each entry corresponds to a pixel value. We can perform \textbf{vectorization} by converting matrix $X$ into a column vector $x \in \mathbb{R}^{n^2}$ and matrix $B$ into vector $b \in \mathbb{R^{n^2}}$. \\
We can write the following equation:
\begin{center}
    $Ax = b$,
\end{center}
where $A \in \mathbb{R}^{n^2 \times n^2}$ indicates the transformation matrix coming from the repeated application of the image kernel. A blurred pixel is the weighted average of the surrounding ones, with the weights defined by the kernel matrix $K \in \mathbb{R}^{d\times d}$. The non-zero elements of each row of matrix $A$ correspond to the values of matrix $K$.
Matrix $A$ ignores the elements outside the matrix borders:
\begin{center}
    $(max(i, j) > n)$
\end{center}
Matrix $A$ is a $d^2-$banded matrix, where:
\begin{center}
    $d << n$,
\end{center}
meaning that matrix $A$ is sparse.
\subsection{Direct and Iterative Methods}
Since the complexity of Gaussian elimination for large linear systems is too high, we need another approach: we use \textbf{direct methods} to compute the exact solution in $n$ steps, while we use \textbf{iterative methods} with an arbitrary starting point to compute an approximate solution. Iterative methods converge after a few iterations, but they are less robust and not as general as direct ones.
\subsection{Error and Residual}
Let $x$ be an exact solution and $x^m$ be the computed point, then the \textbf{error} in step $m$ is the deviation of the computed point from the exact solution:
\begin{center}
    $e^{m} = x - x^m$,
\end{center}
The error is not known during the iterations, otherwise we would know the solution. \\
The \textbf{residual} provides us with a measure of the real error:
\begin{center}
    $r^m = b - Ax^m$
\end{center}
\newpage
\subsection{Steepest Descent}
The \textbf{steepest descent} algorithm is a precursor to the Conjugate Gradient algorithm that performs the following procedure:
\begin{enumerate}
    \item Start with a random initial guess
    \item Take the \textbf{gradient}, i.e. the direction of the deepest descent
    \item Compute the minimum of the gradient
    \item Take the new gradient such that it is orthogonal to the previous one
    \item Repeat from step $2$ until a certain convergence criterion is met
\end{enumerate}
This algorithm has a low convergence rate, but it does not take optimal steps to find the next approximate solution.
\subsection{Conjugate Gradient}
The \textbf{conjugate gradient method} is used to solve equations of the type:
\begin{center}
    $Ax = b$,
\end{center}
where $A$ is symmetric and positive-definite. It can be used as an iterative method since it provides monotonically improving approximations $x_k$ to the exact solution after each iteration, with a number of iterations not larger than the size of the matrix, in the absence of round-off error. The improvement is typically linear and its speed is determined by the \textbf{condition number} $\kappa(A)$ of the matrix $A$:
\begin{center}
    $\kappa(A) = \displaystyle\frac{\sigma_{max}}{\sigma_{min}}$,
\end{center}
with $\sigma$ indicating the singular values of $A$. For a real symmetric matrix, the values are equal to the eigenvalues absolute value:
\begin{center}
    $\sigma = |\lambda|$ \\
\end{center}
The larger the condition number is, the slower the improvement.
\newpage
The algorithm starts from an initial guess $x_0$ and applies a series of operations until the desired tolerance is reached:
\begin{enumerate}
    \item $r = b - Ax_0$
    \item $d = r$
    \item $\rho_{old} = <r,r>$
    \item $for \ i = 0,1,... \ do:$
    \begin{enumerate}[label=\arabic*.]
        \item $ s = A \cdot d_i$
        \item $\alpha = \rho_{old} / <d,s>$
        \item $x = x + \alpha \cdot d$
        \item $r = r - \alpha \cdot s$
        \item $\rho_{new} = <r,r>$
        \item $\beta = \rho_{new} / \rho_{old}$
        \item $d = r + \beta \cdot d$
        \item $\rho_{old} = \rho_{new}$
    \end{enumerate}
    \item $end \ for$
\end{enumerate}
\vspace{0.2cm}
Since matrix $A$ is not positive-definite we must solve the following equations:
\begin{center}
    $A^TAx = A^Tb \ \rightarrow \ \Tilde{A}x = \Tilde{b}$
\end{center}
The pre-multiplication with $A^T$ results in the positive-definite augmented transformation matrix $\Tilde{A}$. \\
The condition number is the relation of the sensitivity of the solution $x$ to changes in $b$: if small changes result in large changes in $x$, then the system is \textbf{ill-conditioned} and its condition number is large. The convergence rate of the algorithm is hindered when the system has a high condition number.
\subsection{Preconditioned Conjugate Gradient}
In the \textbf{preconditioned conjugate gradient method} (PCG), a symmetric and positive-definite \textbf{preconditioner} $P$ is selected such that:
\begin{center}
    $P^{-1} \Tilde{A} \approx I$
\end{center}
Furthermore, we can decompose the preconditioner such that:
\begin{center}
    $P = LL^T$,
\end{center}
where $L$ is the Cholesky factor.
\newpage
Now we have to solve the preconditioned augmented system:
\begin{center}
    $P^{-1}\Tilde{A}x = P^{-1}\Tilde{b}$ \\
    \vspace{0.25cm}
    $(L^{-1}\Tilde{A}L^{-1})(Lx) = L^{-1}\Tilde{b}$
\end{center}
This is done to decrease the condition number and the range of the eigenvalues. The preconditioner should be computationally inexpensive to find.
We can use the \textbf{incomplete Cholesky factorization} to compute the Cholesky factorization of the non-zero elements of $\Tilde{A}$, returning the following preconditioner:
\begin{center}
    $P = F^TF$,
\end{center}
where $F$ is the sparse incomplete Cholesky factor. Due to the fact that the routine fail since the existence of $F$ is not guaranteed, a heuristic approach is used to apply a diagonal shift of $P$, enforcing positive-definiteness and making $F$ computable. \\
PCG would be worth the added computational cost whether the conditional number $\kappa(A)$ is large, i.e. matrix $A$ is ill-conditioned since with CG it will require a lot of iterations. While deblurring lots of images with PCG, the computational cost would decrease since it is possible to use the same precondition in order to save time.

\newpage

% -------------------------------------------------------- %
% Assignment 6 - Linear Programming and the Simplex Method
% -------------------------------------------------------- %
\section{Linear Programming and the Simplex Method}
\subsection{Linear Programming}
\textbf{Linear Programming} is an optimization technique used to either maximize or minimize a linear objective function subject to linear equality and inequality constraints. An example of a linear program is the following:
\begin{align*}
	\max~~ & \sum_{i = 1}^n c_ix_i \\
	\text{s.t.}~~ & \sum_{j = 1}^n a_{1,j}x_j \leq h_1 \\
	~~&~~ \vdots \\
	~~& \sum_{j = 1}^n a_{m,j}x_j \leq h_m
\end{align*}
Vector $x_i$ must satisfy some constraints and the non-negativity condition. \\
Linear programming problems can be written in the following \textbf{standard form}:
\begin{equation*}
  \begin{split}
    \max~~ & z = c^Tx \\
	\text{s.t.}~~ & Ax \leq h \\
	~~& x \geq 0
  \end{split}
\quad\quad\quad\quad
  \begin{split}
    \min~~ & z = c^Tx \\
	\text{s.t.}~~ & Ax \geq h \\
	~~& x \geq 0
  \end{split}
\end{equation*} \\
where $z $ is the value of the objective function, $c \in \mathbb{R}^n$ is the coefficients vector, $x \in \mathbb{R}^n$ is the unknowns vector, $A \in \mathbb{R}^{m \times n}$ is the coefficients matrix, and $h \in \mathbb{R}^m$ is the vector of the constraints coefficients. \\
Given that the \textbf{feasible region} is generated by the vertices that satisfy the constraints, the optimal value can be found with the \textbf{Fundamental Theorem of Linear Programming}: \\ \\
\textbf{Theorem 1} \textit{If a linear program admits a solution, it will lie on a vertex of the polytope defined by the feasible region. If two vertices are both maximizers or minimizers of the function, then all the points lying on the segment between them will represent the optimal solutions to the problem.}
\subsection{The Simplex Method}
Since computing the value of $z$ at all the vertices of the feasible region could be expensive, we use the \textbf{simplex method}, which has an exponential worst-case complexity.
\newpage
\subsubsection{Slack and Surplus Variables}
The method introduces \textbf{slack variables} for maximization and \textbf{surplus variables} for minimization to the standard form of the problem, denoting them with $s_m$ as in the following example:
\begin{equation*}
  \begin{split}
  	\max~~ & z = 3x+2y \\
	\text{s.t.}~~ & x+2y+s_1 = 4 \\
	~~& x-y+s_2 = 1 \\
	~~& x,y \geq 0; \ ~ s_1, s_2 \geq 0
  \end{split}
\quad\quad\quad\quad
  \begin{split}
    \min~~ & z = 3x+2y \\
	\text{s.t.}~~ & x+2y-s_1 = 4 \\
	~~& x-y-s_2 = 1 \\
	~~& x,y \geq 0;~ \ s_1, s_2 \geq 0
  \end{split}
\end{equation*}
\subsubsection{Basic and Nonbasic Variables}
We can swap the rows of matrix $A$ as long as we swap the elements of vector $h$. Furthermore, we can swap the columns of matrix $A$ as long as we swap the elements of vector $x$. Matrix $A$ can be split in two submatrices:
\begin{center}
    $A = [B \ D]$,
\end{center}
where matrix $B$ contains the linearly independent columns of $A$, while matrix $D$ contains the remaining ones. We can split vectors $x$ and $c$ in the same way:
\begin{center}
    $x =
    \begin{bmatrix}
    x_B \\
    x_D
    \end{bmatrix}
    ~~~~ c =
    \begin{bmatrix}
    c_B \\
    c_D
    \end{bmatrix}$
\end{center}
Finally, we obtain the following relation:
\begin{center}
    $x_B = B^{-1}h - B^{-1}Dx_D$
\end{center}
Vector $x_B$ contains the \textbf{basic variables}, while vector $x_D$ contains the \textbf{nonbasic variables}. By setting $ \ x_D = 0 \ $, we obtain the following equation:
\begin{center}
    $x_B = B^{-1}h$,
\end{center}
and if the non-negativity condition for $x_B$ is satisfied, then this is a \textbf{basic solution} and corresponds to the feasible region. If $x_B$ has some zero values, then the solution is \textbf{degenerate}. \\
The existence of a feasible solution implies the existence of a feasible basic solution, and the existence of an optimal solution implies the existence of an optimal basic solution. \\
The number of possible basic solutions grows exponentially with unknowns and constraints. The maximum possible number of iteration $N$ is:
\begin{center}
    $N = \displaystyle\frac{(m +n)!}{m! n!}$
\end{center}
\subsubsection{Optimality Condition}
We must check if the solution satisfies the \textbf{optimality condition}, based on the basis of the \textbf{reduced cost coefficients}, at every iteration:
\begin{center}
    $r_D = c_D^T - c_B^T B^{-1}D$,
\end{center}
where $c_B$ is the basic coefficient vector, $c_D$ is the nonbasic coefficient vector, $B$ is the basic matrix, and $D$ is the nonbasic matrix. \\
The optimality condition for maximization is $r_D \leq 0$, while it is $r_D \geq 0$ for minimization. Both are satisfied when all the components of the reduced cost coefficients vectors are less or bigger than $0$, respectively.
\subsubsection{Iterative Rule}
If the optimality condition is not met, we take the variable with the highest reduced cost coefficient, in the case of maximization, or the variable with the lowest cost coefficient, in the case of minimization, into the basis. \\
If two variables have the same value, we could end up swapping them over and over again creating a \textbf{cycle}. The \textbf{iterative rule} identifies the variables that must be taken out of the basis:
\begin{center}
    $\displaystyle\frac{B^{-1}h}{B^{-1}D}$
\end{center}
We select the ratio with the smallest positive value.
\subsubsection{Simplex Method Procedure}
The method procedure can be summarized as follows:
\begin{enumerate}
    \item Write the problem in standard form
    \item Add slack or surplus variables
    \item Apply the iterative rule by exchanging basic with nonbasic variables
    \item Repeat from step $3$ until the optimality condition is met
\end{enumerate}
\subsection{The Auxiliary Problem}
We could find a feasible initial basic solution by solving an \textbf{auxiliary problem}, which can be defined by introducing the \textbf{artificial variables} $u_m$:
\begin{align*}
	\max~~ & z_{\text{aux}} = \sum_{i = 1}^n u_i \\
	\text{s.t.}~~ & \sum_{j = 1}^n a_{1,j}x_j+s_1+u_1 = h_1 \\
	~~&~~ \vdots \\
	~~& \sum_{j = 1}^n a_{m,j}x_j+s_m+u_m = h_m \\
	~~& x_i, ..., x_n \geq 0;~ s_1, ..., s_m \geq 0;~ u_1, ..., u_m \geq 0
\end{align*}
The auxiliary problem aims at minimizing the sum of artificial variables and its optimal solution would be achieved when all the artificial variables are $ = 0$. If $z_{aux} = 0$ is not achieved, then the auxiliary problem and the original problem do not admit a feasible solution. \\
In order to obtain a starting basic solution for the auxiliary problem, we can set the original and the slack variables to $0$ and the artificial variables equal to the right-hand side.

\end{document}


\section{Bottleneck structure in MgNet by using subspace correction}
Recall the standard MgNet iteration
\begin{equation}\label{eq:mgnetiteration}
u^{\ell,i} = u^{\ell,i-1} + \sigma \circ B^{\ell,i} \ast \sigma ({f^\ell -  A^{\ell} \ast u^{\ell,i-1}}),
\end{equation}
which corresponds to the classical residual correction scheme in multigrid as
\begin{equation}\label{key}
u^{\ell,i} = u^{\ell,i-1} + B^{\ell,i} ({f^\ell -  A^{\ell} \ast u^{\ell,i-1}}).
\end{equation}

Now let us recall the subspace correction scheme on a fixed level (for example $\ell$-th level),
we have the following iterative scheme
\begin{equation}\label{eq:bottleneckmgnet}
u^{\ell,i} = u^{\ell,i-1} + P^{\ell,i} B^{\ell,i} R^{\ell,i}({f^\ell -  A^{\ell} \ast u^{\ell,i-1}}).
\end{equation}

Here let us recall the dimension of $f^\ell$ and $u^{\ell,i}$ as
\begin{equation}\label{key}
f^\ell, u^{\ell,i} \in \mathbb{R}^{c_\ell \times m_\ell \times n_\ell },
\end{equation}
which leads to the dimension of $B^{\ell,i}$ in standard MgNet in \eqref{eq:mgnetiteration} to be
\begin{equation}\label{key}
B^{\ell,i} \in \mathbb{R}^{c_\ell \times c_\ell \times 3 \times 3}.
\end{equation}

However, for a subspace correction scheme, we can take $R^{\ell,i}$ as the restriction operator as
\begin{equation}\label{key}
R^{\ell,i}: \mathbb{R}^{c_\ell \times m_\ell \times n_\ell } \mapsto \mathbb{R}^{ \alpha c_\ell \times m_\ell \times n_\ell },
\end{equation}
where $\alpha \in (0,1]$ so example $\alpha = \frac{1}{4}$.
A rational choice for $R^{\ell,i}$ and $P^{\ell,i}$ should be
\begin{equation}\label{key}
R^{\ell,i} \in \mathbb{R}^{\alpha c_\ell \times c_\ell \times 1 \times 1},
\end{equation}
and
\begin{equation}\label{key}
P^{\ell,i} \in \mathbb{R}^{ c_\ell \times \alpha c_\ell \times 1 \times 1}.
\end{equation}
Of course, we can just take the $R^{\ell,i} = [P^{\ell.i}]^T$ based on the
theory of subspace corrections.

Then, the size of $B^{\ell,i}$ in \eqref{eq:bottleneckmgnet} can be reduced to
\begin{equation}\label{key}
B^{\ell,i} \in \mathbb{R}^{\alpha c_\ell \times\alpha c_\ell \times 3 \times 3}.
\end{equation}
Thus the dimension of all operations $R^{\ell,i}$, $P^{\ell,i}$ and $B^{\ell,i}$ will be
\begin{equation}\label{key}
\begin{aligned}
&\alpha c_\ell \times c_\ell \times 1 \times 1 +  c_\ell \times \alpha c_\ell \times 1 \times 1 + \alpha c_\ell \times\alpha c_\ell \times 3  \times 3 \\
&= ((3\alpha)^2 +  2\alpha) c_\ell^2\\
&= \frac{17}{16} c_\ell^2 \quad ( \alpha = \frac{1}{4}),
\end{aligned}
\end{equation}
which is much less than the size of $B^{\ell,i}$ in original MgNet in~\eqref{eq:mgnetiteration} which
is $9c_\ell^2$.

To follow the linear constrained model assumption, we may take the nonlinearity as
 \begin{equation}\label{eq:bottleneckmgnet-1}
 u^{\ell,i} = u^{\ell,i-1} +   \sigma \circ P^{\ell,i} \ast  \sigma \circ B^{\ell,i} \ast  \sigma \circ R^{\ell,i} \ast \sigma ({f^\ell -  A^{\ell} \ast u^{\ell,i-1}}).
 \end{equation}

 Following the similar derivation from MgNet to ResNet, we can also derive
 the next "enhanced" bottleneck ResNet from \eqref{eq:bottleneckmgnet-1} as
 \begin{equation}\label{key}
  r^{\ell,i} = r^{\ell,i-1} -  A^\ell \ast \sigma \circ   P^{\ell,i} \ast  \sigma \circ B^{\ell,i} \ast  \sigma \circ R^{\ell,i} \ast \sigma (r^{\ell,i-1}).
 \end{equation}


\subsection{Test 2 Answers}
\begin{enumerate}[label=\arabic*.]
	\item
		For both (a) and (b), we'll need to compute the general solution to make sure any terms that we'd guess as part of the particular solution aren't already part of the general solution.
		The auxiliary equation and its roots are
		\begin{equation*}
			r^2 + 10r + 25 \implies r = -5 \text{ (double root)}.
		\end{equation*}
		So, the general solution is
		\begin{equation*}
			C_1e^{-5t} + C_2te^{5t}.
		\end{equation*}
		\begin{enumerate}[label=(\alph*)]
			\item
				We have a trig term and a linear term times a trig term, so the particular solution has the form
				\begin{equation*}
					y_p = A\cos{(-5t)} + B\sin{(-5t)} + Ct\cos{(-5t)} + Dt\sin{(-5t)}.
				\end{equation*}
				So, without solving for $A$, $B$, $C$, and $D$, the general form of a particular solution is
				\begin{equation*}
					C_1e^{-5t} + C_2te^{5t} + A\cos{(-5t)} + B\sin{(-5t)} + Ct\cos{(-5t)} + Dt\sin{(-5t)}.
				\end{equation*}
			\item
				We have a linear term time $e^{-5t}$.
				However, up to linear terms are already represented in the general solution, so we need to include another factor of $t$.
				So, the particular solution has the form
				\begin{equation*}
					y_p = At^2e^{-5t}.
				\end{equation*}
				So, without solving for $A$, the general form of a particular solution is
				\begin{equation*}
					C_1e^{-5t} + C_2te^{5t} + At^2e^{-5t}.
				\end{equation*}
		\end{enumerate}
	\item
		Finding $p(\lambda)$,
		\begin{equation*}
			p(\lambda) = \det\begin{bmatrix}
				1 & 0 & 6 \\
				3 & 1 & 3 \\
				-3 & 3 & -8
			\end{bmatrix} = -\lambda^3 - 6\lambda^2 + 6\lambda + 55.
		\end{equation*}
		So, $p(\lambda) = 0$ when
		\begin{equation*}
			\lambda = -5, \frac{-1 \pm 3\sqrt{5}}{2}.
		\end{equation*}
		We'll find the eigenvector for $\lambda = -5$.
		\begin{equation*}
			\left[
				\begin{array}{ccc|c}
					6 & 0 & 6 & 0\\
					3 & 6 & 3 & 0\\
					-3 & 3 & 3 & 0
				\end{array}
			\right] \to \left[
				\begin{array}{ccc|c}
					1 & 0 & 1 & 0 \\
					0 & 1 & 0 & 0 \\
					0 & 0 & 0 & 0
				\end{array}
			\right] \implies \vec{v} = t\begin{bmatrix}
				-1 \\
				0 \\
				1
			\end{bmatrix}
		\end{equation*}
	\item
		The instructions say only do one, but we'll do both for the answers.
		\begin{enumerate}[label=(\alph*)]
			\item
				The auxiliary equation and its roots are
				\begin{equation*}
					r^2 - 5r + 6 = 0 \implies r = 2, 3.
				\end{equation*}
				So, the homogeneous solution is
				\begin{equation*}
					y_h = C_1e^{2t} + C_2e^{3t}.
				\end{equation*}
				We use the method of undetermined coefficients to find $y_p$.
				We'll guess that $y_p$ has the form
				\begin{equation*}
					y_p = Ae^{4t} + Be^{t}.
				\end{equation*}
				Solving for $A$ and $B$\footnote{The algebra of solving for $A$ and $B$ have been omitted for brevity.},
				\begin{equation*}
					y_p'' - 5y_p' + 6y_p = 6e^{4t} - 10e^t \implies A = 3, B = -5.
				\end{equation*}
				So, we can write the particular solution
				\begin{equation*}
					y_p = 3e^{4t} - 5e^{t}.
				\end{equation*}
				So, the general solution is
				\begin{equation*}
					y = C_1e^{2t} + C_2e^{3t} + 3e^{4t} - 5e^{t}.
				\end{equation*}
			\item
				Although this is a second-order equation, we'll do the version of variation of parameters that also works for higher orders too.
				First, we need to find our fundamental solutions that are part of the homogeneous solution by finding the roots of the auxiliary equation.
				\begin{equation*}
					r^2 + 9 = 0 \implies r = \pm 3i.
				\end{equation*}
				So, the homogeneous solution is
				\begin{equation*}
					y_h = C_1\cos{(3t)} + C_2\sin{(3t)},
				\end{equation*}
				and the fundamental solutions are
				\begin{align*}
					y_1 &= \cos{(3t)} \\
					y_2 &= \sin{(3t)}.
				\end{align*}
				So, the Wronskian matrix and its determinant are
				\begin{equation*}
					\left[W\right] = \begin{bmatrix}
						\cos{(3t)} & \sin{(3t)} \\
						-3\sin{(3t)} & 3\cos{(3t)}
					\end{bmatrix} \text{, } W = 3\cos^2{(3t)} + 3\sin^2{(3t)} = 3.
				\end{equation*}
				The sub-matrices are
				\begin{align*}
					W_1 &= \det\begin{bmatrix}
						0 & \sin{(3t)} \\
						1 & 3\cos{(3t)}
					\end{bmatrix} = -\sin{(3t)} \\
					 W_2 &= \det\begin{bmatrix}
						\cos{(3t)} & 0 \\
						-3\sin{(3t)} & 1
					\end{bmatrix} = \cos{(3t)}.
				\end{align*}
				Solving for $v_1$ and $v_2$,
				\begin{align*}
					v_1 &= \int{\frac{-\csc{(3t)}\sin{(3t)}}{3} \mathrm{d}t} = \frac{-1}{3}t + C_1 \\
					v_2 &= \int{\frac{\csc{(3t)}\cos{(3t)}}{3} \mathrm{d}t} = \frac{1}{9}\ln{\abs{\sin{(3t)}}} + C_2.
				\end{align*}
				Solving for $y$,
				\begin{align*}
					y &= \cos{(3t)}\left(\frac{-1}{3}t + C_1\right) + \sin{(3t)}\left(\frac{1}{9}\ln{\abs{\sin{(3t)}}} + C_2\right) \\
					&= C_1\cos{(3t)} + C_2\sin{(3t)} - \frac{1}{3}t\cos{(3t)} + \frac{1}{9}\sin{(3t)}\ln{\abs{\sin{(3t)}}}.
				\end{align*}
		\end{enumerate}
		\item
			Below is a diagram that depicts the situation.

			\begin{center}
				\begin{tikzpicture}
					\tikzstyle{spring}=[thick,decorate,decoration={zigzag, pre length=0.5cm, post
						length=0.5cm, segment length=0.33cm, amplitude=0.25cm}]

					\tikzstyle{ground}=[fill, pattern=north east lines, draw=none, minimum
					width=1cm, minimum height=0.5cm]

					\fill[pattern = north east lines] (0, 0) rectangle (0.5cm, 2.5cm);
					\draw[thick] (0, 0) -- (8cm, 0);
					\node (m) at (5cm, 1cm) [draw, thick, minimum width=2cm, minimum height=2cm] {$m = 1$};
					\draw[spring] (0.5cm, 1cm) -- (m);
					\node (k) at (1.5cm, 1.75cm) {$k = 8$};
					\node (b) at (5cm, 2.5cm) {$b = 6$};
					\node (ini) at (10cm, 1cm) {$\begin{cases} y'(0) = 1 \\ F_\text{ext}(t) = 8\sin{(2t)} \end{cases}$};
					\draw[dashed] (5cm, -0.25cm) -- (5cm, 0.25cm);
					\node (y) at (5cm, -0.5cm) {$y = 0$};
				\end{tikzpicture}
			\end{center}

			\begin{enumerate}[label=(\alph*)]
				\item
					Our IVP to model this is
					\begin{equation*}
						\begin{cases}
							y'' + 6y' + 8y = 8\sin{(2t)} \\
							y'(0) = 1 \\
							y(0) = 0
						\end{cases}.
					\end{equation*}
					Solving the auxiliary equation,
					\begin{equation*}
						r^2 + 6r + 8 = 0 \implies r = -4, -2.
					\end{equation*}
					So, our homogeneous solution is
					\begin{equation*}
						y_h = C_1e^{-4t} + C_2e^{-2t}.
					\end{equation*}
					We'll use the method of undetermined coefficients to find $y_p$.
					We'll guess that $y_p$ has the form
					\begin{equation*}
						y_p = A\cos{(2t)} + B\sin{(2t)}.
					\end{equation*}
					Solving for $A$ and $B$\footnote{The algebra of solving for $A$ and $B$ has been omitted for brevity.},
					\begin{equation*}
						y_p'' + 6y_p' + 8y_p = 8\sin{(2t)} \implies A = \frac{-3}{5}, B = \frac{1}{5}.
					\end{equation*}
					So, our particular solution is
					\begin{equation*}
						y_p = \frac{-3}{5}\cos{(2t)} + \frac{1}{5}\sin{(2t)},
					\end{equation*}
					and our general solution is
					\begin{equation*}
						y = C_1e^{-4t} + C_2e^{-2t} - \frac{3}{5}\cos{(2t)} + \frac{1}{5}\sin{(2t)}.
					\end{equation*}
					Plugging in our initial condition to solve for $C_1$ and $C_2$\footnote{The algebra of solving for $C_1$ and $C_2$ has been omitted for brevity.},
					\begin{equation*}
						y'(0) = 1, y(0) = 0 \implies C_1 = \frac{-9}{10}, C_2 = \frac{3}{2}.
					\end{equation*}
					So, our solution to the IVP is
					\begin{equation*}
						y(t) = \frac{-9}{10}e^{-4t} + \frac{3}{2}e^{-2t} - \frac{3}{5}\cos{(2t)} + \frac{1}{5}\sin{(2t)} \text{ m}.
					\end{equation*}
				\item
					As $t$ grows large, the exponential terms will decrease to 0 and have minimal effect.
					So, the steady-state solution $y_{ss}$ is just the terms with $\cos$ and $\sin$.
					\begin{equation*}
						y_{ss}(t) = \frac{-3}{5}\cos{(2t)} + \frac{1}{5}\sin{(2t)} \text{ m}
					\end{equation*}
				\item
					\begin{equation*}
						y_{ss}(t) = \sqrt{\frac{2}{5}}\cos{\left(2x + \arctan{\left(\frac{1}{3}\right)} + \pi\right)} \text{ m}.
					\end{equation*}
					The amplitude, frequency, period, and phase shift are
					\begin{align*}
						A &= \sqrt{\frac{2}{5}} \text{ m} \\
						f &= \frac{1}{\pi} \text{ Hz} \\
						T &= \pi \text{ secs} \\
						\phi &= \arctan{\left(\frac{1}{3}\right)} + \pi.
					\end{align*}
			\end{enumerate}
		\item
			The instructions say only do one, but we'll do both for the answers.
			\begin{enumerate}[label=(\alph*)]
				\item
					\begin{enumerate}[label=(\roman*)]
						\item
							The system in matrix form is
							\begin{equation*}
								\vec{x}' = \begin{bmatrix}
									2 & -3 \\
									1 & -2
								\end{bmatrix}\vec{x} \text{, } \vec{x}(0) = \begin{bmatrix}
									2 \\
									3
								\end{bmatrix}.
							\end{equation*}
						\item
							We'll check the first vector.
							\begin{equation*}
								\begin{bmatrix}
									3e^t \\
									e^t
								\end{bmatrix} = \begin{bmatrix}
									2 & -3 \\
									1 & -2
								\end{bmatrix} \begin{bmatrix}
									3e^t \\
									e^t
								\end{bmatrix} = \begin{bmatrix}
									3e^t \\
									e^t
								\end{bmatrix}.
							\end{equation*}
							Next we'll check the second vector.
							\begin{equation*}
								\begin{bmatrix}
									-e^{-t} \\
									-e^{-t}
								\end{bmatrix} = \begin{bmatrix}
									2 & -3 \\
									1 & -2
								\end{bmatrix} \begin{bmatrix}
									e^{-t} \\
									e^{-t}
								\end{bmatrix} = \begin{bmatrix}
									-e^{-t} \\
									-e^{-t}
								\end{bmatrix}.
							\end{equation*}
							So, both vectors are solutions to the homogeneous equation.
						\item
							Since we know two linearly independent solutions from (ii), we can write the general solution as
							\begin{equation*}
								\vec{x} = C_1 \begin{bmatrix}
									3e^t \\
									e^t
								\end{bmatrix} + C_2 \begin{bmatrix}
									-e^{-t} \\
									-e^{-t}
								\end{bmatrix}.
							\end{equation*}
							Applying the initial conditions\footnote{The algebra of solving for $C_1$ and $C_2$ has been omitted for brevity},
							\begin{equation*}
								\vec{x}(0) = \begin{bmatrix}
									2 \\
									3
								\end{bmatrix} \implies C_1 = \frac{-1}{2}, C_2 = \frac{-7}{2}.
							\end{equation*}
							So, the solution to the IVP is
							\begin{equation*}
								\vec{x} = \frac{-1}{2} \begin{bmatrix}
									3e^t \\
									e^t
								\end{bmatrix} - \frac{7}{2} \begin{bmatrix}
									-e^{-t} \\
									-e^{-t}
								\end{bmatrix}.
							\end{equation*}
					\end{enumerate}
				\item
					Since the eigenvalues are a complex conjugate pair, we only need to consider one eigenvalue to find both corresponding eigenvectors.
					We'll use $\lambda = -2 + i$.
					\begin{equation*}
						\left[
							\begin{array}{cc|c}
								1-i & 2 & 0 \\
								-1 & -1-i & 0
							\end{array}
						\right] \to \left[
							\begin{array}{cc|c}
								1 & 1-i & 0 \\
								0 & 0 & 0
							\end{array}
						\right] \to t\left(\begin{bmatrix}
							-1 \\
							1
						\end{bmatrix} + \begin{bmatrix}
							1 \\
							0
						\end{bmatrix}\right).
					\end{equation*}
					So, the two eigenvectors are
					\begin{equation*}
						\begin{bmatrix}
						-1 \\
						1
						\end{bmatrix}, \begin{bmatrix}
						1 \\
						0
						\end{bmatrix}.
					\end{equation*}
					Remembering our solution form for complex eigenvalues, we get the solution
					\begin{equation*}
						\vec{x} = C_1e^{-2t}\left(\cos{(t)}\begin{bmatrix}
							-1 \\
							1
						\end{bmatrix} - \sin{(t)}\begin{bmatrix}
							1 \\
							0
						\end{bmatrix}\right) + C_2e^{-2t}\left(\cos{(t)}\begin{bmatrix}
							1 \\
							0
						\end{bmatrix} + \sin{(t)}\begin{bmatrix}
							-1 \\
							1
						\end{bmatrix}
						\right).
					\end{equation*}
			\end{enumerate}
		\item
			We'll use the method of undetermined coefficients for systems to find $\vec{x_p}$.
			We'll guess that $\vec{x_p}$ has the form
			\begin{equation*}
				\vec{x_p} = \vec{a}t + \vec{b}.
			\end{equation*}
			Solving for $\vec{a}$ and $\vec{b}$\footnote{The algebra of solving for $\vec{a}$ and $vec{b}$ has been omitted for brevity.},
			\begin{equation*}
				\vec{x_p}' = \begin{bmatrix}
					2 & -1 \\
					3 & -2
				\end{bmatrix}\vec{x_p} \implies \vec{a} = \begin{bmatrix}
					-1 \\
					0
				\end{bmatrix}, \vec{b} = \begin{bmatrix}
					1 \\
					3
				\end{bmatrix}.
			\end{equation*}
			So, we have our solution for $\vec{x_p}$,
			\begin{equation*}
				\vec{x_p} = t\begin{bmatrix}
					-1 \\
					0
				\end{bmatrix} + \begin{bmatrix}
					1 \\
					3
				\end{bmatrix}.
			\end{equation*}
			The general solution is thus
			\begin{equation*}
				\vec{x} = C_1e^{t}\begin{bmatrix}
					1 \\
					1
				\end{bmatrix} + C_2e^{-t}\begin{bmatrix}
					1 \\
					3
				\end{bmatrix} + t\begin{bmatrix}
					-1 \\
					0
				\end{bmatrix} + \begin{bmatrix}
					1 \\
					3
				\end{bmatrix}.
			\end{equation*}
\end{enumerate}