\author{Bautrelle Fotso}
\graphicspath{ {./src/chapters/developer/media/} }

\section{Networks}
In the previous section, many types of datasets and their characteristics have been presented.
However, the use of datasets is not sufficient to perform the Optical Character Recognition (OCR).
These datasets have to be applied to different networks which learn from them.
The goal is to obtain a model with the best accuracy for each OCR case handled in this project, namely: number, text and text without number.
This process is called deep learning.

\noindent
Accordingly, deep learning can be defined as a subset of machine learning to preprocess data in artificial
intelligence. For this task, it imitates the working of the human nervous system.
Like machine learning, deep learning has the particularity of learning in an unsupervised way from unstructured datasets.
Learning in an unsupervised way means that, the network receives inputs without outputs.
During the training, the network must decide itself which output is corresponding to the input received.(\cite{[1]}, p.6)
Deep learning is a term to describe deep neural networks and their functionalities.
The word "deep" specifies that the network always contains more than one intermediate layer.(\cite{[1]}, p.8) \hfill \break

\noindent
Three types of deep neural network have been used in this project and are described in this section. These are:

\begin{itemize}
    \item Deep Feedforward Neural Network(DFFNN)
    \item Convolutional Neural Network(CNN)
    \item Recurrent Neural Network(RNN).

\end{itemize}

\subsection{Deep Feedforward Neural Network(DFFNN)}
The Deep Feedforward Neural Network, also called Multi-Layer Perceptron(MLP), is a network composed of an input layer,
at least two hidden layers and one output layer.
In this network, each neuron in the previous layer is connected with all neurons of the next layer to form a fully connected layer.
Additionally, the connection between two neurons in the network has a certain amplitude called weight. (\cite{[1]}, p.4)

\noindent
The EMNIST-letters and the MNIST datasets have been applied on this network for the main reason that, they are compatible together. \hfill \break

\noindent
The picture (\ref{Abb:feed_forward}) below illustrates a general model of DFFNN.

\begin{figure}[htb]
	\centering
	\includegraphics[height=0.2\textheight]{simple_FFNN}
	\caption[Deep Feedforward Neural Network]{Deep Feedforward Neural Network (\cite{[3]}, p.1)}\label{Abb:feed_forward}
\end{figure}

%\newpage
\noindent
This is just a general overview of DFFNN, however, a specific network needs to be built accordingly to
the requirements of this project. \hfill \break

\subsubsection{Building the Deep Feedforward Neural Network}
An example of DFFNN built on MNIST dataset is presented in the figure(\ref{Abb:build_feed_forward}).

\begin{figure}[htb]
	\centering
	\includegraphics[width=1.0\textwidth]{ffnn}
	\caption[Build Deep Feedforward Neural Network]{Building the Deep Feedforward Neural Network on MNIST dataset} \label{Abb:build_feed_forward}
\end{figure}

\noindent
A sniped code of a built DFFNN an its abstract representation can be seen respectively on the left
and on the right side of the picture(\ref{Abb:build_feed_forward}).
The architecture used to build this network is the sequential mode of the keras library.
It is always appplied when the network takes only one input and one output layer.
This sequential mode specifies to keras that the output of each layer added should be used as input in the next layer.
The function for adding each layer to the network is \emph{add()}.
The first layer of the network takes a vector as input.
But the dataset images used to train the network have the shape of 28x28 pixels (28 for the height x 28 for the width of the image).
Each image is transformed to a vector of 784 pixels with the function \emph{Flatten()}, which receives as parameter
the shape of the image to be converted. This shape can be a tuple of (28, 28) or a triplet of (1, 28, 28).
The "1" in the triplet just specifies the number of image with the shape 28x28 pixels.
These notations are equivalent because the input layer takes only one image at the time.
The two hidden layers are also added with 512 neurons each.
With the function \emph{Dense()}, a full connection is created between neurons of the previous layer and the next layer.
The activation function \emph{"relu"}(Rectified Linear Unit) is a linear function used to remove negative activations.
It is the most used activation in deep learning(\cite{[1]}, p.12).
An activation is the occupancy rate that an image takes on each of its pixels.
This will be more explained in the training part.
On the last layer the number of neurons is determined by the number of classes available in the used dataset.
The output contains ten neurons ranging from 0 to 9 corresponding to the ten classes of MNIST.
The activation function \emph{"softmax"} maps the prediction vector obtained in the output layer to a probability distribution.
In this case, the prediction vector has ten indices, whose values are represented as probabilities in the range [0,1].
At the end, all added layers are put together using the function \emph{compile()}.
The network uses \emph{"adam"} optimizer responsible to update weights between neurons after a forward pass.
By the forward pass, activations are diffused only forward from the input layer, through the hidden layers to the output
layer and consequently has no loops.
Updating the weights means adjusting it so that the difference between the real values and the values predicted by
the trained model will be minimized.
This difference is called \emph{"loss"} and is calculated using the loss function \emph{sparse\_categorical\_cross\_entropy}.
This type of loss function is especially applied on this network because the labels of the MNIST dataset have not been one-hot-encoded.
One hot encoding transforms a label, which is represented as an integer, to a binary value represented as a vector.
Using this loss function also help to save time as well as computation because it simply takes a single integer rather than a whole vector.
The chosen metric is the accuracy, which indicates how good the prediction rate on training data is, after each forward pass. (\cite{[4]})

The network training is then started by the function \emph{fit()} called on the built network(the variable model).

\noindent
In the following part, the training of the model  and the role of each parameter used in the function \emph{fit()} will be explained.

\subsubsection{Training the Deep Feedforward Neural Network}
The next step, after building a network, is to train data on it to obtain a model suitable for prediction.
The following picture presents the training process of the DFFNN on the MNIST dataset.

\begin{figure}[htb]
	\centering
	\includegraphics[width=1.0\textwidth]{train_cnn}
	\caption[Train Deep Feedforward Neural Network]{Training the Deep Feedforward Neural Network on MNIST dataset \cite{[12]}} \label{Abb:train_cnn}
\end{figure}
The MNIST dataset, namely the train and the test set have to be first loaded from the \emph{Keras.dataset.mnist} library.
After loading the dataset,
the train and the test set images are rescaled to be sure that the required size is kept.
Afterward, each image pixel is downscaled from [0 255] to [0 1], to maintain a general pixel value distribution on the entire dataset
and avoid working with big decimal values.
For a better training of the dataset, the network uses mini-batches of the MNIST training data instead of the entire dataset at once.
The actual training is started using the function model.fit() as mentioned above.
At the end of each epoch, the network will be evaluated on the test data, which have not been trained.
An epoch describes an unique passage of the entire MNIST dataset through the network.
After the dataset images have been normalized and reshaped, the built network then receives a digit image of 28x28 pixels as input.
This image is flattened to a vector of 784 pixels.
Each of these pixels represents a neuron in the input layers. Each neuron holds a specific number
in the interval of 0 to 1 due to the previous normalization (division of each pixel value by 255).
The value contained in these neurons represents the value of the corresponding
pixel, which is called \emph{activation}. When a neuron is brighter, the activation is high and tends towards 1.
Otherwise, the activation is low and tends towards 0.
One important thing to notice on training networks is that, the activation value of the previous layer is always used to determine
the one of the next layer. This is realized by doing a particular calculation.
An example of the computation with some results is shown on the figure(\ref{Abb:train_cnn}). The activation values
are not real, but it has been used for illustration purpose.
For this example, there are two neurons on the third layer with the highest activation 1. They have the form of a "zero" and a
"dash", whose combination gives on the fourth layer the correct predicted answer nine(9).
After each forward pass from a layer to the other, the weights muss be updated before a next pass.
This loss value obtained from the function \emph{sparse\_categorical\_cross\_entropy} is sent back through each layer and the
optimizer(adam) bases its computation on this error rate. This helps to update
the weights accordingly to the error rate of each of this weights, in order to minimize the error by the next forward pass.

\subsubsection{Results and interpretation}
The prediction results of the model obtained by training DFFNN on the MNIST dataset are in the figure(\ref{Abb:mnist_metrics}).

\begin{figure}[htb]
	\centering
	\includegraphics[width=0.4\textwidth]{MNIST_metrics}
	\caption[Results of DFFNN model on MNIST]{Loss and Accuracy from DFFNN model on MNIST train and test set} \label{Abb:mnist_metrics}
\end{figure}

The model prediction results of the MNIST training and test set are obtained during a defined number of epochs.
It is observable that, the training set has a higher accuracy and a lower loss value
than the test set.
This can be due to the fact that the training set is the one applied on the network during the training.
However, the error rate by the test set differs very little from the one by the training set.
It means that, the probability to correctly predict a new image is also high, although the image have not
been trained on the network before.

\subsection{Convolutional Neural Network (CNN)}
Convolutional Neural Network is a type of neural network which uses a particular processing for recognizing images.
CNN is basically used for image analysis. (\cite{[6]}, p.3)
They are three types of CNN:

\begin{itemize}
    \item One dimensional CNN
    \item two dimensional CNN
    \item three dimensional CNN.
  \end{itemize}

One dimensional CNN takes one dimensional vectors as input.
It is also applied for digits recognition like DFFNN. The two other types of CNN use a matrix as input(2D or 3D matrices according to the type of CNN).
CNN is an excellent network for object recognition, behavior recognition, natural language processing and more.
The CNN used in this project is the 2D-CNN. (\cite{[1]}, p.16)

The figure(\ref{Abb:schema_cnn}) presents a schematic diagramm of this network type.
\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.6]{schema_cnn}
	\caption[Convolutional Neural Network]{General view of a Convolutional Neural Network (\cite{[14]})} \label{Abb:schema_cnn}
\end{figure}

As seen in figure(\ref{Abb:schema_cnn}) a CNN generally contains an input layer, a convolutional layer and  a pooling layer. They help not only to
analyze an entire image in pieces but also extract its features for a better recognition; this makes the difference CNNs and DFFNNs.
The second part of this network is made up of a fully connected layer for predictions of images. At the end of the prediction process the
results are then forwarded to the output layer.
Figure(\ref{Abb:schema_cnn}) has been chosen for illustrative reasons.
For better performance, a CNN must have more than one of each layer mentioned above. (\cite{[1]}, p.16)

\subsubsection{Building the Convolutional Neural Net}
This part explains how the 2D-CNN was constructed appropriately to the requirements of the data to be applied on it.
Three datasets have been used on this network, namely the EMNIST-balanced, EMNIST-byMerge and EMNIST-digits datasets.
The following application example is based on the building of the 2D-CNN model to recognize the EMNIST-Balanced dataset.

\begin{figure}[htb]
	\centering
	\includegraphics[width=1.0\textwidth]{build_cnn}
	\caption[Building Convolutional Neural Network]{Building the Convolutional Neural Network on EMNIST-Balanced dataset} \label{Abb:build_cnn}
\end{figure}

\noindent
As shown in the figure above, The 2D-CNN is also built using a sequential architecture.
It is constituated of some components, similar to those in the DFFNN.
As explained above, the 2D-CNN takes a 2D-matrix as input parameter. In this case either an image of the
size (28, 28, 1) or the size (1, 28, 28, 1) can be used.
The last value "1" of each size specifies the channel of the image. The channel 1 describes grayscaled images and the channel 3 color images.
Like DFFNN, each input layer recognizes automatically the value "1" of each size, as first element of the input shape.
The first layer is the \emph{2D-convolutional layer}. On this layer 32 filters are defined with a size of 5x5 each.
Each of these filters are used to filter each area of the image.
Activation functions are similar to those described in the DFFNN.
The next layer is the pooling layer where the max pooling operation is performed. The \emph{Max pooling layer} is used to downstream
images obtained from the convolution step, done in the first layer.
It is used to speed up the processing of the image.
The \emph{BatchNormalization} is the following layer where the downsampled images obtained from the \emph{Max Pooling layer} are normalized.
The build process of this three layers is repeated once more to form the next three layers.
The only difference is the fact that the 32 filters used on the second 2D-convolutional layer has a size of 3x3.
The \emph{Flatten} layer converts the obtained feature images to a vector in order to make it ready to be used as input in the fully connected
layer.
By adding of the \emph{Dropout layer}, 20\% of the neurons in the network will be randomly deactivated during the forward pass on the network.
The weights updating is not applied on dropped out neurons during the backward pass.
By the Backward pass, information about the difference between the real and the predicted value (loss) are sent back to neurons.
During the dropout, the remaining neurons have to step in, take the role of the ignored neurons in order to
handle the representation required to do the prediction.
This technique avoids the neurons to be dependent from each other and from its context.
Furthermore, the network will not just memorize the training data but is able to
realize better generalizations and is more flexible to predict any data.
This problem of memorizing the training data is called overfitting and the general solution is to implement a
dropout layer using the activation function \emph{softmax}.
The Results in the output layer are distributed among 47 neurons because the EMNIST-Letters has 47 classes.

\subsubsection{Training the Convolutional Neural Network}

\begin{figure}[htb]
	\centering
	\includegraphics[width=0.8\textwidth]{conv2}
	\caption[Convolutional Neural Network]{Trainings process of the CNN (\cite{[5]})} \label{Abb:cnn_process}
\end{figure}

The upper part of the figure(\ref{Abb:cnn_process}) shows each step of the training on CNN, regardless on whether the CNN is
a 1D or 2D-CNN because both have almost the same process.
The only difference is the shape of the input image used for the training.
Firstly, the image is loaded into the convolutional layer of the CNN.
In this layer the convolution of the image is performed with each of the 5x5 filter(conv).
These filters are randomly initialized so that, each of them recognizes specific features of the image such as edges and curves.
The bottom part of the figure(\ref{Abb:cnn_process}) illustrates how the convolution of an image works.
It is an element-wise multiplication of image pixels with a filter to detect specific features.
The resulting image after the convolution with one filter is called the feature map.
Accordingly,there are 8 feature maps in the first column(conv) of the third picture in figure(\ref{Abb:cnn_process}).
The feature map can contain negative values.
The relu activation function is then used to set all negative values to zero, in order to have only positive
activations in the feature map.
This feature maps are then forwarded to the pooling layer which uses the \emph{maxpool} method.
The maxpool method itself modifiies the image size by reducing the number of overall parameters in the image.
As illustrated in the third column of the third picture in figure(\ref{Abb:cnn_process}), the size of the eigth images has reduced to 14x14 pixels.
In the second 2D-convolutional layer, 32 filters have been used for the convolution.
The previous steps are repeated again up to the last pooling layer, which has reduced the size of the 32 images (obtained
after the convolution) to 7x7 pixels.
The process performed from the first convolutional layer up to the last maxpooling layer is called \emph{feature learning and extraction process}.
The output of the pooling layers are then flattened and connected to the dense layers.
After the images have been flattened, the prediction happens similarly to the process in the DFFNN and is called \emph{classification process}. \hfill \break

\subsubsection{Results and interpretation}
The figure(\ref{Abb:emnist_balaced_metrics}) presents the results obtained by the training of CNN on the
EMNIST-balanced dataset after a defined number of epoch.
It is remarkable that, the training set has a lower loss value than the test set.
This can also be due to the fact that the training set is the one applied on the network during the training.
However the test data has a model accuracy closed to 90 percent.
This accuracy describes the performance of the model on data which have not be trained on it.
Through this results, it can be prognosticated that the model is able to correctly predict any character image
with an accuracy of more than 80 percent.

\begin{figure}[htb]
	\centering
	\includegraphics[width=0.4\textwidth]{EMNIST-balanced_metrics}
	\caption[Results of DFFNN training on MNIST]{Loss and Accuracy rates on EMNIST\_balanced got from the 2D-CNN model} \label{Abb:emnist_balaced_metrics}
\end{figure}

\subsection{Recurent Neural Network(RNN)}
The Recurrent Neural Network is a Network, which has been designed for the analysis of sequential data such as
videos and speechs.
The particularity of the RNN is that, the information are not only sent forward but also backward
through the network; this means, the RNNs have loops and also self-feedback loops.
Additionally RNNs have a short memory.

The figure(\ref{Abb:schema_rnn}) shows a general model on how the RNN is representated.

\begin{figure}[htb]
	\centering
	\includegraphics[height=0.3\textheight]{rnn}
	\caption[Schema Recurrent Neural Network]{Representation of the Recurrent Neural Network (\cite{[1]})} \label{Abb:schema_rnn}
\end{figure}

\noindent
The capacity to loop over the layers while training, gives the possibility to get the activation of the next layer
by computing the current activation (got at the time \emph{t}) with the activation of the previous
layer (got at the time \emph{t-1}).
This example can be taken as illustration to this concept: when the network is provided with the input "HELLO", the RNN still knows
by reaching the character "E", that "H" is the character trained before.
This helps especially on sequential data like speechs because each words pronounced have to be saved and not
immediately thrown away after working on it.
This is not possible by DFFNN and CNN because they have no memory. It is the reason why RNNs have became the state-of-art in natural
language processing(handwriting, speech, etc.).(\cite{[6]}, p.2)
The main type of the RNN is the Long-short Term Memory (LSTM) because it does not conserves its memory only for a
short moment but for a long period of time.
LSTM memory has a read and write function as the one of a computer, with the only difference that
its memory is analog and not digital. (\cite{[7]})

\noindent
A custom model was not built for this network type. But a tool called \emph{Tesseract-ocr engine}(\cite{[11]}) (which already contains
the built LSTM), was used.
Developped from Hewlett Packard betweeen 1984 and 1994 and later taken over from Google(2006), tesseract-ocr
is an open source for the extraction of texts from images.
It has a considerable quantity of models for recognition, available in about two hundred different languages.
Despite the fact that tesseract excellently performs the recognition of printed texts, it is inconsistent by the
prediction on handwritten texts.
Nevertheless, tesseract can learn to recognize handwriting, when a custom dataset is built and trained on it.
This is the reason why the SWTP-AI dataset previously presented has been created.
In the following, the general trainings process of tesseract will be explained.

\subsubsection{Use tesseract to generate a model for text recognition}

\begin{itemize} \bfseries
	\item Installation of dependecies
\end{itemize}
Firstly tesseract-ocr was installed on linux operating system to execute the training process from the command line.
Then the JTessboxeditor was also installed but this time on windows for the labelling of the data.
The labelling and the training could happen directly from the JTessBoxEditor because it has integrated functionalities
to perform both tasks and therefore generate new fonts (models) for tesseract.
But the goal was first of all to learn and understand step by step, how this training works in the background; reason why it has
been done on Linux kernel.
Since python is used on this project, there is a need to have a module which can be connected with tesseract on local system,
in order to use some functions on python for predictions with tesseract.
This module name is \emph{pytesseract}. (\cite{[9]}, \cite{[8]})
The installation of tesseract and pytesseract with all necessary packages can be followed here(\cite{[9]})

\begin{itemize} \bfseries
	\item Labelling handwritten characters for the training
\end{itemize}
About 15 full and long texts have been written from differents persons to have a diversity and a flexibility when using the new model.
When each character of those texts has to be labelled from hand it would have taken too much time.
Therefore, this step was taken off from JTessBoxEditor.
The labelling consists on correcting wrong characters and box coordinates.

An Example of this can be found on in the figure below.

\begin{figure}[htb]
	\centering
	\includegraphics[width=1.0\textwidth]{jtessEditing}
	\caption[Labelling with JTEssboxEditor]{Example of Labelling with JTessBoxEditor} \label{Abb:jtess_editing}
\end{figure}

The picture left shows the characters after they have been predicted and surrounded by boxes using tesseract.
But some of the predictions are wrong and it is not possible to correctly train the network with such errors.
Hence the need for labelling.
After labelling with JTessBoxEditor as shown in the picture on the right, the characters are all correct and ready for training.

\begin{itemize} \bfseries
	\item Training tesseract-ocr on SWTP-AI dataset
\end{itemize}
The lifecycle from the labelling up to the obtain of the ready model (also called font) will be explained using the figure(\ref{Abb:tess_training}):
For better understanding, the example is done for only one word.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.8]{tesseract_training}
	\caption[From writting to trained data File]{Process to get a ready model for training(.traineddata file)} \label{Abb:tess_training}
\end{figure}

The case treated here is to build a model using tesseract which can recognizse the word "Backer".
tesseract does not necessarily need an image preprocessing.

\noindent
It is firstly important to clarify some concepts like (\cite{[10]}):
\begin{itemize}
    \item \emph{lang}: is the font language
    \item \emph{font}: is the name of the font
    \item \emph{exp0}: used to tag the the box name used.
	\item The "exp" varies from exp0 to expn, depending on how many files is used fot the training.
\end{itemize}
The word is firstly written per hand and converted to a .tif format so that, tesseract can be able to use it.
The conversion is done with the \emph{IMageMagick} tool. The file resulting has the name \emph{lag.font.exp0.tif}:
this is a standard naming convention
Afterwards the .tif format is sent to tesseract and boxes are made as well as each characters predicted.
The .tif file and the .box files obtained from tesseract are saved in the same folder.
It is required to save both in the same folder for JTessBoxEditor to recognize them by their names.
Once in the Editor, the errors are corrected and box coordinates are also correctly adjusted.
The changes are saved and the two files are sent back to tesseract for the training.
And after the training all resulting files are combined and the output is a binary file ready  for the prediction process.
All fonts have the extension \emph{.traineddata}.
This is generally how the training works.

\newpage


\documentclass[11pt]{article}

\usepackage[margin=1.0in]{geometry}
\usepackage{titling}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{amssymb}

\setlength{\droptitle}{0em}

\title{Description of the contact matrix extrapolation}

\begin{document}
\maketitle

\newcommand{\myCountry}{the Philippines}
\newcommand{\myProxyCountry}{China}
\newcommand{\myProxyCountryDate}{2017}
\newcommand{\myProxyCountryReference}{\cite{RN10}}

\subsection{Contact matrices calculation}
For each location $L$ (home, school, work, other locations) the age-specific contact matrix $\mathbf{C^L} = (c_{i,j}^L) \in \mathbb{R}_{+}^{16 \times 16}$ is defined such that $c_{i,j}^L$ is the average number of contacts that a typical individual aged $i$ has with individuals aged $j$. As there is no contact survey avalaible for \myCountry{}, the matrices $\mathbf{C^L}$ were obtained by extrapolating contact matrices from \myProxyCountry{}, where a contact survey was conducted in \myProxyCountryDate{} \myProxyCountryReference{}. The original matrices from \myProxyCountry{} are denoted $\mathbf{Q^L} = (q_{i,j}^L) \in \mathbb{R}_{+}^{16 \times 16}$, where $q_{i,j}^L$ is defined using the same convention as for $c_{i,j}^L$. The matrices $\mathbf{Q^L}$ were extracted using the R package ``socialmixr'' (v 0.1.8) and the next paragraph describes how these contact matrices were then adjusted to account for age distribution differences between \myCountry{} and \myProxyCountry{}.

Let $\pi_j$ denote the proportion of people aged $j$ in \myCountry{}, and $\rho_j$ the proportion of people aged $j$ in \myProxyCountry{}. The contact matrices $\mathbf{C^L}$ were obtained from:
$$
c_{i,j}^L = q_{i,j}^L \times \frac{\pi_j}{\rho_j} .
$$

\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}

\title{MAT257 Notes}
\author{Jad Elkhaleq Ghalayini}
\date{October 12 2018}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{mathtools}
\usepackage{enumitem}
\usepackage{graphicx}
\usepackage{cancel}

\usepackage[margin=1in]{geometry}

\newtheorem{theorem}{Theorem}
\newtheorem{lemma}{Lemma}
\newtheorem{definition}{Definition}
\newtheorem*{corollary}{Corollary}
\newtheorem{exercise}{Exercise}

\newcommand{\reals}[0]{\mathbb{R}}
\newcommand{\nats}[0]{\mathbb{N}}
\newcommand{\ints}[0]{\mathbb{Z}}
\newcommand{\rationals}[0]{\mathbb{Q}}
\newcommand{\brac}[1]{\left(#1\right)}
\newcommand{\sbrac}[1]{\left[#1\right]}
\newcommand{\mc}[1]{\mathcal{#1}}
\newcommand{\eval}[3]{\left.#3\right|_{#1}^{#2}}
\newcommand{\ip}[2]{\left\langle#1,#2\right\rangle}
\newcommand{\prt}[2]{\frac{\partial #1}{\partial #2}}

\begin{document}

\maketitle

\section*{The Implicit Function Theorem}

In this course, we have three important theorems, of which this is one. Each is a generalization of something you've seen in first year calculus, but we'll see that these generalizations are very far-reaching and involve new techniques. Let's begin with a brief ``plan'' for the next month:

\subsection*{Plan}
\begin{enumerate}
  \item Today: the inverse function theorem
  \item The inverse function theorem, which we'll see is equivalent.
  \item Proof of the inverse function theorem (which will take a few lectures)
  \item Implications of the implicit function theorem, in particular applications to extreme value problems, introducing the idea of Lagrange Multipliers. Afterwards, we'll talk about the idea of a differentiable manifold.
\end{enumerate}
So this is what we're aiming to do in the next few weeks.

\subsection*{The Inverse Function Theorem}
Recall: suppose \(f: \reals \to \reals\) is a continuously differentiable function in an open inverval containing \(a\), and \(f'(a) \neq 0\). Then \(f'\) is either greater than or lessthan zero in an open interval containing \(a\). Therefore \(f\) is one-to-one, and so has an inverse defined on an open interval \(W\) containing \(f(a)\). Moreover, \(f^{-1}\) is differentiable, and
\[(f^{-1})'(f(a)) = \frac{1}{f'(a)}\]
We're going to discuss a generalization of this idea to several variables.

\begin{theorem}[Inverse Function Theorem]
  Let \(f: U \to \reals^n\) be a continuously differentiable (\(\mc{C}^1\)) on an open set \(U \subset \reals^n\). Let \(a \in U\) be such that \(\det f'(a) \neq 0\). Then there exist open sets \(a \in V, f(a) \in W\) such that \(f: V \to W\) with a continuous inverse \(f^{-1}: W \to V\). Moreover, \(f^{-1}\) is differentiable on \(W\) and
  \[\forall y \in W, (f^{-1})'(y) = (f'(f^{-1}(y)))^{-1}\]
\end{theorem}
Remark: if we know already that \(f^{-1}\) is differentiable, the formula for it follows from the chain rule:
\[f(f^{-1})(y) = y \implies f'(f^{-1}(y))(f^{-1})'(y) = I \iff (f^{-1})'(y) = (f(f^{-1}(y)))^{-1}\]
\begin{corollary}
  \begin{enumerate}
    \item \(f^{-1}\) is continuously differentiable (\(\mc{C}^1\))
    \item If \(f\) is \(\mc{C}^r\) then \(f^{-1}\) is also \(\mc{C}^r\)
  \end{enumerate}
\end{corollary}
\begin{proof}
  \begin{enumerate}

    \item  We know that since \(f'\) is continuous, and \(f^{-1}\) is continuous, \(f' \circ f^{-1}\) is continuous. Now what about the inversion? What is the inverse of a matrix? It's given by a formula. So this is really a composite of 3 functions. So what about the formula for the inverse of a matrix? It follows from Cramer's rule that this formula is continuous, since the entries of the inverse matrix \(B\) of \(A\) are given as rational functions of the entries of \(A\), that is,
    \[b_{ij} = \frac{(-1)^{i + j}\det A^{ji}}{\det A}\]
    where \(A^{ji}\) is the matrix obtained by deleting the \(j^{th}\) row and \(i^{th}\) column from \(A\). So hence the derivative, being the composition of 3 continuous functions, must be continuous.

    \item We proceed by induction on \(r\). Assume that if \(f\) is \(\mc{C}^{r - 1}\), then \(f^{-1}\) is \(\mc{C}^{r - 1}\).

    If \(f\) is \(\mc{C}^r\), then \(f\) is \(\mc{C}^{r - 1}\) implying that \(f^{-1}\) is \(\mc{C}^{r - 1}\) by the inductive hypotheis. So, using the formula
    \[(f^{-1})' = (f'(f^{-1}))^{-1}\]
    is \(\mc{C}^{r - 1}\) implying that \(f^{-1}\) is \(\mc{C}^r\).

  \end{enumerate}
\end{proof}
This corollary is just to show that we could have stated the above theorem in a stronger way. Examples:
\begin{enumerate}
  \item Continuity of \(f'\) cannot be removed from the hypotheses: consider \(f: \reals \to \reals\),
  \[f(x) = \left\{\begin{array}{cc}x + x^2\sin\frac{1}{x} & x \neq 0 \\ 0 & x = 0\end{array}\right.\]
  We have that
  \[x \neq 0 \implies f'(x) = 1 + 2x\sin\frac{1}{x} - \cos\frac{1}{x}\]
  but the limit does not exist as \(x \to 0\).

  \item Consider \(f: \reals^2 \to \reals^2\),
  \[f(x, y) = (e^x\cos y, e^x\sin y)\]
  We have
  \[f'(x, y) = \begin{pmatrix} e^x\cos y & -e^x\sin y \\ e^x\sin y & e^x\cos y \end{pmatrix}\]
  implying that
  \[\det f'(x, y) = (e^x)^2 = e^{2x} \neq 0 \forall x \in \reals\]
  But this function is not 1-1, since it is periodic in \(y\). The inverse function theorem says that we can make some neighborhood around a point where \(f\) is one to one, but it \textit{doesn't} say that it's \textit{globally} one to one.

\end{enumerate}
Remarks:
\begin{enumerate}
  \item \(f\) may be invertible even though \(f'(a) = 0\)
\end{enumerate}

\end{document}


\section{Genetic drift and Neutral alleles}

\subsection{Comparing polymorphism and divergence}

\subsection{Deviations from the constant population model.}
We've seen previously that changes in our population size can be
captured by an effective population size. However, this will only be a
useful measure if population sizes vary rapidly enough, that the
harmonic mean effective population size over short time periods ($\ll
N_e$ generations) is representative of the effective population size averaged over
longer time periods. If this is not the case there is no one effective
population size, as we can not approximate our rate of drift by a
single constant population. Furthermore, we've ignored the effect of
population structure and selection which will violate our modeling
assumptions. \\

We can hope to detect violations from our constant population size
neutral model, by comparing aspects of our dataset to their expectations
and distributions under our neutral model. \\

For example we have devised two estimates of $\theta$,
$\widehat{\theta_{\pi}}$ and $\widehat{\theta_{W}}$, using
expectations of different aspects of our data (pairwise diversity and
number of segregating sites respectively). Under our constant neutral
model if we have sufficient data those two estimates should be
equal to each other on average. But if there's some violation of our model they might not
be. So one test statistic might be to take
\begin{equation}
D = \widehat{\theta_{\pi}} - \widehat{\theta_{W}}
\end{equation}
which will be zero in expectation if our data was generated by a
neutral constant population model.

\newpage


\chapter{Optimizations on low-level representation}

\section{Register allocation}

\subsection{Estimated distance of use}

We define a metric associated with every control arc in the flow
graph.  The metric is called \emph{Estimated Distance to Use} or EDU
for short.  It assigns a number to each lexical variable, reflecting
how far in the future that lexical variable is needed.  The EDU metric
is a generalization of \emph{liveness} information, where and EDU
value of $\infty$ means that the variable is dead.

For some instruction $I$ with an incoming control arc $A$ and a
variable $V$ that is read or written by $I$, the EDU value of $V$ in
$A$, is defined to be $1$, i.e. $E(V)_A = 1$.

For some instruction $I$ with an incoming control arc $A$ and a single
outgoing control arc $B$ such that $V$ is neither read nor written by
$I$, if $E(V)_B \not = \infty$ the EDU value of $V$ in $A$, $E(V)_A =
1 + E(V)_b$ where $E(V)_b$ is the EDU value of $V$ in $B$.  If
$E(V)_b = \infty$, then $E(V)_A = \infty$ as well.

For some instruction $I$ with an incoming control arc $A$ and two
outgoing control arcs $B_1$ $B_2$ such that $V$ is neither read nor
written by $I$, the EDU value of $V$ in $A$, we compute $E(V)_A$ as
follows.  Let $p_1$ be the estimated probability that the control arc
$B_1$ will be used, and $p_2$ be the estimated probability that the
control arc $B_2$ will be used.  If $E(V)_{B_1} \not = \infty$ and
$E(V)_{B_2} \not = \infty$ then $E(V)_A = 1 + {{E(V_{B_1})E(V_{B_2})}
  \over {p_2 E(V_{B_1}) + p_1 E(V_{B_2})}}$.  If $E(V)_{B_1} = \infty$
and $E(V)_{B_2} \not = \infty$ then $E(V)_A = 1 + {{E(V_{B_2})} /
  p_2}$.  If $E(V)_{B_1} \not = \infty$ and $E(V)_{B_2} = \infty$ then
$E(V)_A = 1 + {{E(V_{B_1})} / p_1}$.  If $E(V)_{B_1} = \infty$ and
$E(V)_{B_2} = \infty$ then $E(V)_A = \infty$.

We use an iterative procedure to compute the EDU values, propagating
from \texttt{return} instructions and following control arcs in
reverse.  This procedure does not automatically halt, because of loops
in the control graph.  But EDU values are only approximate, so we stop
propagating modifications when new values differ little from old
ones.

\subsection{Variable map}

Next, we use the EDU values to compute a \emph{variable map}
associated with each control arc in the flow graph.  The map contains
two items:

\begin{enumerate}
\item A vector of \emph{stack location} that are either currently in
  use, or that that have been used in the past.  The size of the
  vector reflects the required number of locations required in the
  stack frame.  An element of the vector is either a lexical variable,
  or \texttt{nil} if the corresponding stack location contains no
  valid datum.
\item A vector with the same size as the number of registers
  available.  An element of the vector is either a lexical variable,
  or \texttt{nil} if the corresponding register contains no valid
  datum.
\end{enumerate}

For a particular control arc, every live variable is contained either
in the first vector, in the second vector, or in both vectors.  For
the time being, we imagine that a lexical variable can be present in
at most one register and in at most one stack location.

Initially, suppose we have a load/store architecture, so that the flow
graph contains instructions that require all their operands in
registers.  We assume that the flow graph does not contain any
\texttt{load} or \texttt{store} instructions.

We compute variables maps from the start of the flow graph.  The
initial map contains the situation upon function entry, as dictated by
the function-call protocol.  For all other control arcs, initialize
all maps to \emph{unknown}.

Let $I$ be an instruction with a single incoming control arc $A$.  For
each Variable $V$ such that $E(V)_A = 1$, we must make sure that $V$
is in a register before $I$ is executed.  If $V$ is already in a
register, no action needs to be taken.  If $V$ is in a stack location,
and there is at least one register containing no valid datum, then
emit a \texttt{load} instruction and update the map.  If $V$ is in a
stack location, but every register contains some valid datum, then we
must choose a \emph{victim}, i.e. a lexical variable $W$ that is
currently in a register.  The decision depends on whether $W$ is also
present in a stack location, and on $E(W)_a$.  It is preferable to
choose a $W$ with a large value of $E(W)_a$ and it is preferable to
choose one that is also present in some stack location.  If a $W$ is
chosen that is also present in a stack location, then a \texttt{load}
instruction is emitted to load $V$ to the register previously occupied
by $W$ and the map is updated accordingly.  If $W$ is not present in a
stack location, then a \emph{store} instruction must first be emitted
to store the variable into a stack location.  If all stack locations
are occupied, then the vector of stack locations will have its size
increased.

Now, let $I$ be an instruction with several incoming control arcs.%
\footnote{To be continued.}


% !TEX TS-program = pdflatex
% !TEX encoding = UTF-8 Unicode

% This is a simple template for a LaTeX document using the "article" class.
% See "book", "report", "letter" for other types of document.

\documentclass[10pt]{article} % use larger type; default would be 10pt
\usepackage{amsmath , amssymb , amsthm}
\usepackage[table]{xcolor}
\usepackage{circuitikz}
\usepackage{subcaption}
\usepackage{tikz}
\usetikzlibrary{automata, positioning, arrows}
\usepackage{placeins}
\usepackage{centernot}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{qtree}
\usepackage{latexsym}
%\usepackage[utf8]{inputenc} % set input encoding (not needed with XeLaTeX)

\usepackage{geometry} % to change the page dimensions
\geometry{a4paper} % or letterpaper (US) or a5paper or....
 \geometry{margin=3cm} % for example, change the margins to 2 inches all round

\renewcommand{\thesubsection}{\thesection.\alph{subsection}}
\renewcommand{\thesubsubsection}{\thesubsection.\roman{subsubsection}}

\title{Fundamentals of Computing \\
      \Large Coursework 1 \\  MSc Data Science}
\author{Mark Rotchell\\13181875}
\date{}
\begin{document}
\maketitle

\pagebreak
\hspace{0pt}
\vfill
\begin{center}
\section*{Academic Declaration}

I have read and understood the sections of plagiarism in the College Policy on assessment offences and confirm that the work is my own, with the work of others clearly acknowledged. I give my permission to submit my report to the plagiarism testing database that the College is using and test it using plagiarism detection software, search engines or meta-searching software.
\end{center}
\vfill
\hspace{0pt}
\pagebreak
\section{}
\subsection{}
\subsubsection{}
\begin{tabular}{c|c|c||c|c|c}
A & B & C & A $\vee$  B & A $\to$ C & C $\wedge$ $\neg$B\\
\hline
0 & 0 & 0 & 0 & 1 & 0 \\
0 & 0 & 1 & 0 & 1 & 1 \\
0 & 1 & 0 & 1 & 1 & 0 \\
0 & 1 & 1 & 1 & 1 & 0 \\
1 & 0 & 0 & 1 & 0 & 0 \\
\rowcolor[HTML]{CCCCCC}
1 & 0 & 1 & 1 & 1 & 1 \\
1 & 1 & 0 & 1 & 0 & 0 \\
1 & 1 & 1 & 1 & 1 & 0 \\
\end{tabular}

\vspace{20px}
The set is consistent for A=1, B=0 and C=1, as hightlighted in the above truth table
\subsubsection{}
\begin{tabular}{c|c|c||c|c|c}
A & B & C & $\neg$A $\wedge$  $\neg$B &  $\neg$C $\to$ A & $\neg$C $\vee$ B \\
\hline
0 & 0 & 0 & 1 & 0 & 1 \\
0 & 0 & 1 & 1 & 1 & 0 \\
0 & 1 & 0 & 0 & 0 & 1 \\
0 & 1 & 1 & 0 & 1 & 1 \\
1 & 0 & 0 & 0 & 1 & 1 \\
1 & 0 & 1 & 0 & 1 & 0 \\
1 & 1 & 0 & 0 & 1 & 1 \\
1 & 1 & 1 & 0 & 1 & 1 \\
\end{tabular}

\vspace{20px}
The set is not consistent as there exists no set of truth values for ${A, B, C}$ such that all of the formulae in the set are true

\subsection{}

\begin{tabular}{c|c|c||c|c|c|c}
A & B & C & $p_1=\neg(A\wedge B)$ & $p_2 = C \to A$ &  $p_3 = C \wedge B$ & $(p_1 \wedge p_2 \wedge p_3) \to C$\\
\hline
0 & 0 & 0 & 1 & 1 & 0 & 1 \\
0 & 0 & 1 & 1 & 0 & 0 & 1 \\
0 & 1 & 0 & 1 & 1 & 0 & 1 \\
0 & 1 & 1 & 1 & 0 & 1 & 1 \\
1 & 0 & 0 & 1 & 1 & 0 & 1 \\
1 & 0 & 1 & 1 & 1 & 0 & 1 \\
1 & 1 & 0 & 0 & 1 & 0 & 1 \\
1 & 1 & 1 & 0 & 1 & 1 & 1 \\
\end{tabular}

\vspace{20px}
The argument is logically correct. There exist no situations in which all of the premises are true and the conclusion is false.
\section{}
\subsection{}
\begin{tabular}{c|c|c||c|c|c}
A & B & C & $A \to \neg B$ & $C \to A$ & $\neg\left((A \to \neg B)\wedge(C \to A)\right)$\\
\hline
0 & 0 & 0 & 1 & 1 & 0 \\
0 & 0 & 1 & 1 & 0 & 1 \\
0 & 1 & 0 & 1 & 1 & 0 \\
0 & 1 & 1 & 1 & 0 & 1 \\
1 & 0 & 0 & 1 & 1 & 0 \\
1 & 0 & 1 & 1 & 1 & 0 \\
1 & 1 & 0 & 0 & 1 & 1 \\
1 & 1 & 1 & 0 & 1 & 1 \\
\end{tabular}

\vspace{20px}
\subsection{}
\begin{center}
\begin{circuitikz} \draw
(0,4) node (B) {B}
(0,2) node (A) {A}
(0,0) node (C) {C}
(1.5,1.5) node[not port] (notA) {}
(4,3) node[and port] (andAB) {}
(4,1) node[and port] (andAC) {}
(6,2) node[or port] (or) {}
(B) -- (andAB.in 1)
(A) -- (andAB.in 2)
(A) -- (notA.in)
(notA.out) -- (andAC.in 1)
(C) -- (andAC.in 2)
(andAB.out) -- (or.in 1)
(andAC.out) -- (or.in 2);
\end{circuitikz}
\end{center}

\subsection{}
\[
\begin{aligned}
       & \neg ((     A \to  \neg B) \wedge   (     C \to  A)  ) \\
\equiv & \neg ((\neg A \vee \neg B) \wedge   (     C \to  A)  ) \\
\equiv & \neg ((\neg A \vee \neg B) \wedge   (\neg C \vee A)  ) \\
\equiv & \neg ((\neg B \vee \neg A)  \wedge   (\neg C \vee A)  ) \\
\equiv & \neg ((     B \to  \neg A)  \wedge   (\neg C \vee A)  ) \\
\equiv &       \neg(      B \to  \neg A)  \vee \neg(\neg C \vee A) \\
\equiv &       \neg(      B \to  \neg A)  \vee     (C \wedge \neg A) \\
\equiv &           (      B \to  \neg A) \to     (C \wedge \neg A) \\
\end{aligned}
\]

\section{}
\begin{center}
$N = A_2 \cdot 2^2 + A_1 \cdot 2^1 + A_0 \cdot 2^0$\\

\vspace{20px}

\begin{tabular}{c|c|c||c|c|c}
$A_2$ & $A_1$ & $A_0$ & $N<3_{10}$ & $A_1 \wedge A_0$ & $A_2 \downarrow (A_1 \wedge A_0)$\\
\hline
0 & 0 & 0 & 1 & 0 & 1 \\
0 & 0 & 1 & 1 & 0 & 1 \\
0 & 1 & 0 & 1 & 0 & 1 \\
0 & 1 & 1 & 0 & 1 & 0 \\
1 & 0 & 0 & 0 & 0 & 0 \\
1 & 0 & 1 & 0 & 0 & 0 \\
1 & 1 & 0 & 0 & 0 & 0 \\
1 & 1 & 1 & 0 & 1 & 0 \\
\end{tabular}

\vspace{20px}

\begin{circuitikz} \draw
(0,0) node (A2) {$A_2$}
(0,1) node (A1) {$A_1$}
(0,3) node (A0) {$A_0$}
(2,2) node[and port] (and01) {}
(4,1) node[nor port] (nor) {}
(A0) -- (and01.in 1)
(A1) -- (and01.in 2)
(and01.out) -- (nor.in 1)
(A2) -- (nor.in 2)
;
\end{circuitikz}
\end{center}
\section{}
The argument is not correct. With the following propositional variables:
\renewcommand{\labelenumi}{\Alph{enumi}}
\begin{enumerate}
\item : 'Jones did not meet Smith last night'
\item : 'Smith was the murderer'
\item : 'Jones is lying'
\item : 'The murder took place after midnight'
\end{enumerate}
the propositions can be summarized as:
\renewcommand{\labelenumi}{$p_{\arabic{enumi}}$}
\begin{enumerate}
\item $= A \to (B\vee C)$
\item $= \neg B \to (A \wedge D)$
\item $= D \to (B \vee C)$
\end{enumerate}

These propositions can all be true whilst the conclusion is false, as show by the below row from the truth table, therefore the argument is not correct

\vspace{20px}

\begin{tabular}{c|c|c|c||c|c|c}
A & B & C & D & $p_1 = A \to (B\vee C)$ & $p_2= \neg B \to (A \wedge D)$ & $p_3= D \to (B \vee C)$ \\
\hline
\multicolumn{4}{c||}{$\cdots$} & \multicolumn{3}{c}{$\cdots$} \\
1 & 0 & 1 & 1 & 1 & 1 & 1\\
\multicolumn{4}{c||}{$\cdots$} & \multicolumn{3}{c}{$\cdots$} \\
\hline
\end{tabular}
\section{}
\subsection{}

Initial digit is a {\ttfamily 1} therefore number is negative. Finding absolute value via following steps:
\begin {tabbing}
Initial word \hspace{30px} \= {\ttfamily 1100 0001 1011 0000 0000 0000 0000 0000} \\
Invert digits \>   {\ttfamily 0011 1110 0100 1111 1111 1111 1111 1111} \\
Add one \> {\ttfamily \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ 1}\\
Result \> {\ttfamily 0011 1110 0101 0000 0000 0000 0000 0000} \\
\> $=2^{29}+2^{28}+2^{27}+2^{26}+2^{25}+2^{22}+2^{20}=1,045,430,272_{10}$
\end{tabbing}
Therefore the number is $-1,045,430,272_{10}$
\subsection{}
\begin{center}
{\ttfamily 1100 0001 1011 0000 0000 0000 0000 0000}

$=2^{31}+2^{30}+2^{24}+2^{23} + 2^{21} + 2^{20}=3,249,537,024_{10}$
\end{center}
\subsection{}
\begin{center}
{\ttfamily 1 $\Big|$ 100 0001 1 $\Big|$ 011 0000 0000 0000 0000 0000}
\end{center}

Sign bit is {\ttfamily 1}, so $S = 1$.

Exponent bits are {\ttfamily 100 0001 1} $=2^7+2^1+2^0=131$, and the bias is 127, so $E=131-127=4$.

Fraction bits are {\ttfamily 011 000 \ldots}, so $F=1+2^{-2}+2^{-3} = 1\dfrac{3}{8}$

So number is $(-1)^S \cdot F \cdot 2^ E = -1 \cdot 1\dfrac{3}{8} \cdot 2^4 = -22$
\section{}
\subsection{}
First find binary representation of $|-107| = 107$
\[
\begin{aligned}
107 / 2 = 53 \text{ remainder } & \mathtt{1} \\
53 / 2 = 26 \text{ remainder } & \mathtt{1} \\
26 / 2 = 13 \text{ remainder } & \mathtt{0} \\
13 / 2 = 6 \text{ remainder } & \mathtt{1} \\
6 / 2 = 3 \text{ remainder } & \mathtt{0} \\
3 / 2 = 1 \text{ remainder } & \mathtt{1} \\
1 / 2 = 0 \text{ remainder } & \mathtt{1} \\
\end{aligned}
\]
reading updwards $107_{10} = \mathtt{110 1011}_2$, then convert to negative in two's complement via
\begin {tabbing}
Initial word \hspace{30px} \= {\ttfamily 0000 0000 0000 0000 0000 0000 0110 1011} \\
Invert digits \> {\ttfamily 1111 1111 1111 1111 1111 1111 1001 0100} \\
Add one \> {\ttfamily \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ 1}\\
Result \> {\ttfamily 1111 1111 1111 1111 1111 1111 1001 0101} \\
\end{tabbing}
\subsection{}
\[-107_{10} = -\mathtt{110 \ 1011}_2 = -\mathtt{1.1010 \ 11}_2 \times 2^6\]

So, sign bit is {\ttfamily 1}

Exponent is $6_{10} + 127_{10} = 133_{10} = \mathtt{1000 \ 0101}_2$

Fraction is $10 1011$

And full word is

\[\mathtt{1100 \ 0010 \ 1101 \ 0110 \ 0000 \ 0000 \ 0000 \ 0000}\]
\subsection{}
\[-14.375 = -\left(8 + 4 + 2 + \frac{1}{4} + \frac{1}{8}\right) = -\mathtt{1110.011}_2 = -\mathtt{1.1100 \ 11}_2+2^3\]

So, sign bit is {\ttfamily 1}

Exponent is $3_{10} + 127_{10} = 130_{10} = \mathtt{1000 \ 0010}_2$

Fraction is $1100 \ 11$

And full word is

\[\mathtt{1100 \ 0001 \ 0110 \ 0110 \ 0000 \ 0000 \ 0000 \ 0000}\]

\section{}
\subsection{}
%Question 7 a
\[ f(x) = x+1 \]
\subsection{}
%Question 7 b
\[
  f(x) =
  \begin{cases}
   \ \dfrac{x}{2} & \text{if} \ x \ \text{is even} \\
   \\
   \ \dfrac{x-1}{2} & \text{if} \ x \ \text{is odd} \\
  \end{cases}
\]
\subsection{}
%Question 7 c
\[
  f(x) =
  \begin{cases}
   \ x + 2 & \text{if} \ 3 \ \text{divides} \ x \\
   \ x - 1 & \text{if} \ 3 \ \text{does not divide} \ x \\
  \end{cases}
\]
\section{}
%Question 8
\subsection{}
%Question 8 a
Note that $f(2) = (2+1) / (2-2) = 3/0$ is undefined, so $f$ is not a function over the whole real domain as stated in the question. As it is not a function, it cannot be an injective nor surjective function, so \underline{$f$ is not injective (one-to-one)} and \underline{$f$ is not surjective (onto)}.

However, considering instead $f^\prime:\mathbb{R}^\prime \to \mathbb{R}$, where $\mathbb{R}^\prime = \mathbb{R} \setminus \{2\}$, given by $f^\prime(x)=\dfrac{x+1}{x-2}$.

\paragraph{Injectivity}
Suppose:
\[
  \begin{aligned}
    f^\prime(x) &= f^\prime(y), \quad x,y \in \mathbb{R}^\prime \\
    \dfrac{x+1}{x-2} &= \dfrac{y+1}{y-2}\\
    (x+1)(y-2) &= (y+1)(x-2) \\
    xy + y -2x -2 &= xy +x -2y -2 \\
    y-2x &= x - 2y \\
    y &= x \\
  \end{aligned}
\]
so
\[
\forall \ x,y \in \mathbb{R}^\prime : f^\prime(x) = f^\prime(y) \implies x = y
\]
therefore \underline{$f^\prime$ is injective (one-to-one)}.

\paragraph{Surjectivity} Let $y = f^\prime(x)$, then
\[
  \begin{aligned}
    f^\prime(x) = \dfrac{x+1}{x-2} & = y \\
    x+1 &= y(x-2)\\
    x+1 &= xy - 2y \\
    x(y - 1) &= 1+2y \\
    \end{aligned}
\]
\[ x = \dfrac{1+2y}{y-1} \]
\[\therefore x \ \text{is undefined for} \ y = 1 \]
\[\therefore \ \nexists \ x \in \mathbb{R}^\prime , \ f^\prime(x) = 1\]
\[\therefore \ \neg \left(\forall y \in \mathbb{R}, \exists x \in \mathbb{R}^\prime, f^\prime(x) = y \right) \]
therefore \underline{$f^\prime$ is not surjective (onto)}.
\subsection{}
%question 8(b)
\paragraph{Injectivity}
\[f(0,1) = 0 = f(1,0)\]
however
\[(0,1) \neq (1,0)\]
\[\therefore \neg \left(\forall (m,n),(p,q) \in \mathbb{Z} \times \mathbb{Z}: f(m,n) = f(p,q) \implies (m,n) = (p,q) \right)
\]
therefore \underline{$f$ is not injective (one-to-one)}

\paragraph{Surjectivity} Suppose $n=1$ then $f(m,n) = f(m,1) = m + 1 -1 = m$. So,

\[\forall y \in \mathbb{Z}: f(y,1) = y \]
\[\therefore \ \forall y \in \mathbb{Z}, \ \exists (m,n) \in \mathbb{Z} \times \mathbb{Z}, \ f(m,n) =y \]
therefore \underline{$f$ is surjective (onto)}
\subsection{}
%question 8(c)

\paragraph{Injectivity}
\[f(0,-1) = 0 = f(-1,0)\]
however
\[(0,-1) \neq (-1,0)\]
\[\therefore \neg \left(\forall (m,n),(p,q) \in \mathbb{Z} \times \mathbb{Z} :f(m,n) = f(p,q) \implies (m,n) = (p,q) \right)
\]
therefore \underline{$f$ is not injective (one-to-one)}

\paragraph{Surjectivity}
\[
  \min(|m| - |n| -1) = 0 - 0 - 1 = -1
\]
\[
  \therefore \nexists (m,n) \in \mathbb{Z} \times \mathbb{Z}, f(m,n) < -1
\]
\[\therefore \neg \left( \forall y \in \mathbb{Z}, \exists (m,n) \in \mathbb{Z} \times \mathbb{Z}, f(m,n) =y \right) \]
therefore \underline{$f$ is not surjective (onto)}

\section{}
%Question 9
\subsection{}
\begin{center}
{\centering
\begin{minipage}[b]{0.4\textwidth}
\centering
\begin{tabular}{c|c c c c}
    & A & B & C & D\\
    \hline
    A & 1 & 2 & 0 & 1\\
    B & 2 & 0 & 3 & 0\\
    C & 0 & 3 & 1 & 1\\
    D & 1 & 0 & 1 & 0\\
\end{tabular}
\end{minipage}
\hspace{20px}
\centering
\begin{minipage}[c]{0.4\textwidth}
\begin{tikzpicture}[node distance = 2.5cm]
    \begin{scope}[every node/.style={circle,thick,draw}]
        \node (A) {A};
        \node (B) [below of = A] {B};
        \node (C) [right of = B] {C};
        \node (D) [right of = A] {D};
    \end{scope}

    \begin{scope}[every edge/.style={draw=black,thick}, every loop/.style={min distance=10mm}]
        \path (A) edge [loop above] (A);
        \path (A) edge [bend right] (B);
        \path (A) edge [bend left]  (B);
        \path (B) edge (C);
        \path (B) edge [bend left]  (C);
        \path (B) edge [bend right] (C);
        \path (C) edge [loop right] (C);
        \path (C) edge (D);
        \path (A) edge (D);
    \end{scope}
\end{tikzpicture}
\end{minipage}}
\end{center}
The graph is not simple as there are vertices with more than one edge between them.
\subsection{}
Let the left hand graph be $G$ such that
\[
	G = (V_G,E_G) = (\{1,2,3,4,5\},\{1,2\},\{1,3\},\{1,5\},\{2,3\},\{2,4\},\{4,5\})
\]
and the right graph be $H$ such that
\[
	H = (V_H,E_H) = (\{a,c,b,d,e\},\{a,c\},\{a,b\},\{a,e\},\{c,b\},\{c,d\},\{d,e\})
\]
Then let the function $f:V_G\to V_H$ be the vertex bijection given by the following set of pairs of the form $(v,f(v))$:
\[
	\{(1,a),(2,c),(3,b),(4,d),(5,e)\}
\]
Applying the function $f$ to each vertex of each edge $\{v_1,v_2\}$ in $E_G$ gives a set of subsets, ${\{f(v_1),f(v_2)\} \subset V_H}$ which is exactly $E_H$, i.e. all the edges of $H$.

\vspace{20px}

\begin{tabular}{c|c}
	$\{v_1,v_2\}$ & $\{f(v_1),f(v_2)\}$ \\
	\hline
	\{1,2\} & \{a,c\} \\
	\{1,3\} & \{a,b\} \\
	\{1,5\} & \{a,e\} \\
	\{2,3\} & \{c,b\} \\
	\{2,4\} & \{c,d\} \\
	\{4,5\} & \{d,e\} \\
\end{tabular}

\vspace{20px}

All the edges from the two graphs are in above table. No set of vertices which isn't an edge is in the table. If two vertices are adjacent in $G$ their $f$-mapped vertices in $H$ are also adjacent. If two vertices are not adjacent in $G$ their $f$-mapped vertices in $H$ are also not adjacent. Formally

\[
\{v_1,v_2\} \in E_G \iff \{f(v_1),f(v_2)\} \in E_H
\]
therefore $f$ is an isomorphism and $G$ and $H$ are isomorphic.

\subsection{}
The two graphs have a different number of edges, which is an invariant, so the two graphs cannot be isomorphic.
\section{}
\subsection{}

\noindent Computations for $bb$:

\vspace{20px}

\begin{tikzpicture}[every node/.style = {align=center}]]

 \node {$(s,bb)$}
    child { node {$(q,b)$}
      child {node {stuck}}
    }
    child { node  {$(p,bb)$}
      child { node {$(q,b)$}
        child {node {stuck}}
      }
    };
\end{tikzpicture}

\vspace{20px}

\begin{itemize}
  \item $(s,bb), (q,b)$ $\leadsto$ stuck
  \item $(s,bb), (p,bb), (q,b)$  $\leadsto$ stuck
\end{itemize}

\noindent word $bb$ is not accepted

\vspace{20px}

\noindent Computations for $ab$:

\vspace{20px}

\begin{tikzpicture}[every node/.style = {align=center}]]
 \node {$(s,ab)$}
    child { node {$(p,ab)$}
      child {node {stuck}}
    }
    child { node {$(p,b)$}
      child {node {$(q,\epsilon)$}
        child {node {accepted}}
      }
    }
    child { node [right = 1cm] {$(s,b)$}
      child {node {$(q,\epsilon)$}
        child {node {accepted}}
      }
      child { node {$(p,b)$}
        child {node {$(q,\epsilon)$}
          child {node {accepted}}
        }
      }
    }
    ;
\end{tikzpicture}

\begin{itemize}
  \item $(s,ab), (p,ab)$ $\leadsto$ stuck
  \item $(s,ab), (p,b), (q,\epsilon)$ $\leadsto$ accepted
  \item $(s,ab), (s,b), (q,\epsilon)$ $\leadsto$ accepted
  \item $(s,ab), (s,b), (p,b), (q,\epsilon)$ $\leadsto$ accepted
\end{itemize}

\noindent word $ab$ is accepted

\vspace{20px}

\noindent Computations for $aba$:

\vspace{20px}

\begin{tikzpicture}[every node/.style = {align=center}]]
 \node {$(s,aba)$}
    child { node {$(p,aba)$}
      child {node {stuck}}
    }
    child { node [right=2mm] {$(p,ba)$}
      child {node {$(q,a)$}
        child {node {$(q,\epsilon)$}
          child {node {accepted}}
        }
        child {node {$(p,\epsilon)$}
          child {node {not \\ accepted}}
        }
      }
    }
    child { node [right=25mm] {$(s,ba)$}
      child {node {$(q,a)$}
        child {node {$(q,\epsilon)$}
          child {node {accepted}}
        }
        child {node {$(p,\epsilon)$}
          child {node {not \\ accepted}}
        }
      }
      child {node [right=1cm] {$(p,ba)$}
      child {node {$(q,a)$}
        child {node {$(q,\epsilon)$}
          child {node {accepted}}
        }
        child {node {$(p,\epsilon)$}
          child {node {not \\ accepted}}
        }
      }
      }
    }
    ;
\end{tikzpicture}

\begin{itemize}
  \item $(s,aba), (p,aba)$ $\leadsto$ stuck
  \item $(s,aba), (p,ba), (q,a), (q,\epsilon)$ $\leadsto$ accepted
  \item $(s,aba), (p,ba), (q,a), (p,\epsilon)$ $\leadsto$ not accepted
  \item $(s,aba), (s,ba), (q,a), (q,\epsilon)$ $\leadsto$ accepted
  \item $(s,aba), (s,ba), (q,a), (p,\epsilon)$ $\leadsto$ not accepted
  \item $(s,aba), (s,ba), (p,ba), (q,a), (q,\epsilon)$ $\leadsto$ accepted
  \item $(s,aba), (s,ba), (p,ba), (q,a), (p,\epsilon)$ $\leadsto$ not accepted
\end{itemize}
\noindent word $aba$ is accepted
\subsection{}
\begin{center}
\begin{tikzpicture}[->,
					>=triangle 45,
					semithick,
					node distance=2cm,
					initial text=$ $,
					every loop/.style={min distance=10mm}]

\node[state] (s) at (-3,1) {$s$};
\node[state] (p) at (3,1)  {$p$};
\node[state, accepting] (q) at (0,-1) {$q$};
\draw 	(s) edge [loop above, above] node{$a$} (s)
		(s) edge [bend left=10, above] node{$a$} (p)
		(s) edge [bend right=10, below] node{$\varepsilon$} (p)
		(s) edge [below] node{$b$} (q)
		(q) edge [bend right = 10, below] node{$a$} (p)
		(p) edge [bend right = 10, above] node{$b$} (q)
		(q) edge [loop below, below] node{$a$} (q);
\draw[-, semithick] (node cs:name=s,anchor=west) -- ++(-3mm,1mm);
\draw[-, semithick] (node cs:name=s,anchor=west) -- ++(-3mm,-1mm);
\end{tikzpicture}
\end{center}

\vspace{20px}

\begin{tabular}{c|c|c}
	subset of states & reachable by $a$ & reachable by $b$ \\
	& followed by one or more  $\varepsilon$ & followed by one or more  $\varepsilon$ \\
	\hline
	$\{s\}$ & $\{s,p\}$ & $\{q\}$\\
	$\{p\}$ & $\O$ &  $\{q\}$\\
	$\{q\}$ & $\{p,q\}$& $\O$ \\
	$\{s,p\}$ & $\{s,p\}$ & $\{q\}$\\
	$\{s,q\}$ & $\{s,p,q\}$ & $\{q\}$\\
	$\{p,q\}$ & $\{p,q\}$ & $\{q\}$\\
	$\{s,p,q\}$ & $\{s,p,q\}$ & $\{q\}$\\
	$\O$ &$\O$  &$\O$  \\
\end{tabular}

\vspace{20px}

The DFA starting state is the $\varepsilon$-closure of the NFA starting state, which is $\{s,p\}$. All subsets with $q$ are accepting states. Full subset construction:

\vspace{20px}

\begin{center}
\begin{tikzpicture}[->,
					>=triangle 45,
					semithick,
					node distance=2cm,
					initial text=$ $,
					every loop/.style={min distance=10mm}]

	\node[state] (s) at (-3,0) {$\{s\}$};
	\node[state] (p) at (3,0)  {$\{p\}$};
	\node[state, accepting] (q) at (0,0)  {$\{q\}$};
	\node[state] (sp) at (-2,3)  {$\{s,p\}$};
	\node[state, accepting] (sq) at (0,-3)  {$\{s,q\}$};
	\node[state, accepting] (pq) at (-3,-3)  {$\{p,q\}$};
	\node[state, accepting] (spq) at (3,-3)  {$\{s,p,q\}$};
	\node[state] (o) at (2,3)  {$\O$};
	\draw	(s) edge [left] node{$a$} (sp)
			(s) edge [above] node{$b$} (q)
			(p) edge [right] node{$a$} (o)
			(p) edge [above] node{$b$} (q)
			(q) edge [bend right=10, above left] node{$a$} (pq)
			(q) edge [above left] node{$b$} (o)
			(sp) edge [loop right] node{$a$} (sp)
			(sp) edge [above right] node{$b$} (q)
			(sq) edge [above] node{$a$} (spq)
			(sq) edge [right] node{$b$} (q)
			(pq) edge [loop left] node{$a$} (pq)
			(pq) edge [bend right =10, below right] node{$b$} (q)
			(spq) edge [loop right] node{$a$} (sqp)
			(spq) edge [above right] node{$b$} (q)
			(o) edge [loop left] node{$a$} (o)
			(o) edge [loop right] node{$b$} (o)
;

\draw[-, semithick] (node cs:name=sp,anchor=west) -- ++(-3mm,1mm);
\draw[-, semithick] (node cs:name=sp,anchor=west) -- ++(-3mm,-1mm);
\end{tikzpicture}
\end{center}

then removing the unreachable states:

\begin{center}
\begin{tikzpicture}[->,
					>=triangle 45,
					semithick,
					node distance=2cm,
					initial text=$ $,
					every loop/.style={min distance=10mm}]

	\node[state, accepting] (q) at (0,0)  {$\{q\}$};
	\node[state] (sp) at (-3,0)  {$\{s,p\}$};
	\node[state, accepting] (pq) at (3,0)  {$\{p,q\}$};
	\node[state] (o) at (0,2)  {$\O$};
	\draw	(q) edge [bend right=10, below] node{$a$} (pq)
			(q) edge [left] node{$b$} (o)
			(sp) edge [loop above] node{$a$} (sp)
			(sp) edge [above] node{$b$} (q)
			(pq) edge [loop above] node{$a$} (pq)
			(pq) edge [bend right =10, above] node{$b$} (q)
			(o) edge [loop left] node{$a$} (o)
			(o) edge [loop right] node{$b$} (o)
;

\draw[-, semithick] (node cs:name=sp,anchor=west) -- ++(-3mm,1mm);
\draw[-, semithick] (node cs:name=sp,anchor=west) -- ++(-3mm,-1mm);
\end{tikzpicture}
\end{center}

then re-labelling the states

\begin{center}
\begin{tikzpicture}[->,
					>=triangle 45,
					semithick,
					node distance=2cm,
					initial text=$ $,
					every loop/.style={min distance=10mm}]

	\node[state, accepting] (q) at (0,0)  {$x$};
	\node[state] (sp) at (-3,0)  {$w$};
	\node[state, accepting] (pq) at (3,0)  {$z$};
	\node[state] (o) at (0,2)  {$y$};
	\draw	(q) edge [bend right=10, below] node{$a$} (pq)
			(q) edge [left] node{$b$} (o)
			(sp) edge [loop above] node{$a$} (sp)
			(sp) edge [above] node{$b$} (q)
			(pq) edge [loop above] node{$a$} (pq)
			(pq) edge [bend right =10, above] node{$b$} (q)
			(o) edge [loop left] node{$a$} (o)
			(o) edge [loop right] node{$b$} (o)
;

\draw[-, semithick] (node cs:name=sp,anchor=west) -- ++(-3mm,1mm);
\draw[-, semithick] (node cs:name=sp,anchor=west) -- ++(-3mm,-1mm);
\end{tikzpicture}
\end{center}

we have a deterministic finite automaton $A=(Q,\Sigma,\delta,q_0,F)$ with
\begin{itemize}
	\item a set of states $Q=\{w, x, y, z\}$
	\item a set of symbols $\Sigma = \{a,b\}$
	\item a transition function $f:Q \times \Sigma \to Q$ given by the below table

	\begin{tabular}{c|c c}
		&  \multicolumn{2}{c}{symbol} \\
		state & a & b \\
		\hline
		$w$ & $w$ & $x$ \\
		$x$ & $z$ & $y$ \\
		$y$ & $y$ & $y$ \\
		$z$ & $z$ & $x$ \\
	\end{tabular}
	\item an initial state $q_0 = w$
	\item a set of accepting states $F = \{x, z\}$
\end{itemize}

\subsection{}
The automaton accepts words comprised of $a$ and $b$ that contain at least one $b$, but no consecutive $b$s.
\subsection{}
$\mathtt{a^*b(aa^*b)^*a^*}$
\subsection{}
Let $G=(V,\Sigma,R,s)$ be a context free grammar with
\begin{itemize}
	\item a set of variables $V=\{S,X,Y\}$
	\item a set of terminals $\Sigma = \{a,b\}$
	\item a set of production rules $R = \{S \to XbYX, \ X \to \varepsilon, \ X \to aX, \ Y \to \varepsilon, \ Y \to aXbY\}$
	\item a start variable $s=S$
\end{itemize}
\section{}
Let $A=(Q,\Sigma,\delta,q_0,F)$ be a deterministic finite automaton with
\begin{itemize}
	\item a set of states $Q=\{s,p,q,r\}$
	\item a set of symbols $\Sigma = \{1,0\}$
	\item a transition function $f:Q \times \Sigma \to Q$ given by the below table

	\begin{tabular}{c|c c}
		&  \multicolumn{2}{c}{symbol} \\
		state & 0 & 1 \\
		\hline
		$s$ & $p$ & $s$ \\
		$p$ & $q$ & $p$ \\
		$q$ & $q$ & $r$ \\
		$r$ & $q$ & $p$ \\
	\end{tabular}
	\item an initial state $q_0 = s$
	\item a set of accepting states $F = \{q,r\}$
\end{itemize}
We can visualise $A$ with the following state diagram
\begin{center}
\begin{tikzpicture}[->,
					>=triangle 45,
					semithick,
					node distance=2cm,
					initial text=$ $,
					every loop/.style={min distance=10mm}]

	\node[state] (s) at (-3,0)  {$s$};
	\node[state] (p) at (-1,0)  {$p$};
	\node[state, accepting] (q) at (2,0)   {$q$};
	\node[state, accepting] (r) at (0.5,-2)  {$r$};
	\draw 	(s) edge [loop above] node{$1$} (s)
			(s) edge [above] node{$0$} (p)
			(p) edge [loop above] node{$1$} (p)
			(p) edge [above] node{$0$} (q)
			(q) edge [bend right=10, above left] node{$1$} (r)
			(q) edge [loop above] node{$0$} (q)
			(r) edge [below left] node{$1$} (p)
			(r) edge [below right, bend right=10] node{$0$} (q)
	;
	\draw[-, semithick] (node cs:name=s,anchor=west) -- ++(-3mm,1mm);
	\draw[-, semithick] (node cs:name=s,anchor=west) -- ++(-3mm,-1mm);
\end{tikzpicture}
\end{center}
This can be represented by the following regular expression $\mathtt{1^*0(0\cup 1)^*(0\cup 01)}$
\pagebreak
\section{}
\begin{figure}[!htbp]
  \begin{subfigure}[c]{0.3\textwidth}
    \begin{center}
    \begin{tikzpicture}[->, >=triangle 45, semithick, node distance=2cm]
    	\node[state] (1) at (-1,0) {};
    	\node[state, accepting] (2) at (1,0)  {};
    	\draw 	(1) edge [above] node{$a$} (2);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,1mm);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,-1mm);
    \end{tikzpicture}
    \end{center}
    \caption*{Automaton accepting $L[a]$}
  \end{subfigure}
  %
  \begin{subfigure}[c]{0.3\textwidth}
    \begin{center}
    \begin{tikzpicture}[->, >=triangle 45, semithick, node distance=2cm]
    	\node[state] (1) at (-1,0) {};
    	\node[state, accepting] (2) at (1,0)  {};
    	\draw 	(1) edge [above] node{$b$} (2);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,1mm);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,-1mm);
    \end{tikzpicture}
    \end{center}
    \caption*{Automaton accepting $L[b]$}
  \end{subfigure}
  %
  \begin{subfigure}[c]{0.4\textwidth}
    \begin{center}
    \begin{tikzpicture}[->, >=triangle 45, semithick, node distance=2cm]
    	\node[state] (1) at (-1,0) {};
    	\node[state] (2) at (1,0)  {};
    	\node[state, accepting] (3) at (3,0)  {};
    	\draw 	(1) edge [above] node{$a$} (2)
    	 		(2) edge [above] node{$b$} (3);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,1mm);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,-1mm);
    \end{tikzpicture}
    \end{center}
    \caption*{Automaton accepting $L[ab]$}
  \end{subfigure}
\end{figure}
\begin{figure}[!htbp]
  \begin{subfigure}[c]{0.3\textwidth}
    \begin{center}
    \begin{tikzpicture}[->, >=triangle 45, semithick, node distance=2cm]
    	\node[state] (1) at (-1,0) {};
    	\node[state, accepting] (2) at (1,0)  {};
    	\draw 	(1) edge [above] node{$d$} (2);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,1mm);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,-1mm);
    \end{tikzpicture}
    \end{center}
    \caption*{Automaton accepting $L[d]$}
  \end{subfigure}
  %
  \begin{subfigure}[c]{0.3\textwidth}
    \begin{center}
    \begin{tikzpicture}[->, >=triangle 45, semithick, node distance=2cm]
    	\node[state] (1) at (-1,0) {};
    	\node[state, accepting] (2) at (1,0)  {};
    	\draw 	(1) edge [above] node{$e$} (2);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,1mm);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,-1mm);
    \end{tikzpicture}
    \end{center}
    \caption*{Automaton accepting $L[e]$}
  \end{subfigure}
  %
  \begin{subfigure}[c]{0.4\textwidth}
    \begin{center}
    \begin{tikzpicture}[->, >=triangle 45, semithick, node distance=2cm]
    	\node[state] (1) at (-1,0) {};
    	\node[state] (2) at (1,0)  {};
    	\node[state, accepting] (3) at (3,0)  {};
    	\draw 	(1) edge [above] node{$d$} (2)
    	 		(2) edge [above] node{$e$} (3);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,1mm);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,-1mm);
    \end{tikzpicture}
    \end{center}
    \caption*{Automaton accepting $L[de]$}
  \end{subfigure}
\end{figure}
\vspace{20px}
\begin{figure}[!htbp]
  \begin{subfigure}[c]{0.3\textwidth}
    \begin{center}
    \begin{tikzpicture}[->, >=triangle 45, semithick, node distance=2cm]
    	\node[state] (1) at (-1,0) {};
    	\node[state, accepting] (2) at (1,0)  {};
    	\draw 	(1) edge [above] node{$c$} (2);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,1mm);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,-1mm);
    \end{tikzpicture}
    \end{center}
    \caption*{Automaton accepting $L[c]$}
  \end{subfigure}
  %
  \begin{subfigure}[c]{0.6\textwidth}
    \begin{center}
    \begin{tikzpicture}[->, >=triangle 45, semithick, node distance=2cm]
    	\node[state] (1) at (-3,0) {};
    	\node[state] (2) at (-0.25,1)  {};
    	\node[state] (3) at (-1,-1)  {};
    	\node[state, accepting] (4) at (2.25,1)  {};
    	\node[state] (5) at (1,-1)  {};
    	\node[state, accepting] (6) at (3,-1)  {};
    	\draw 	(1) edge [above] node{$\varepsilon$} (2)
				(1) edge [above] node{$\varepsilon$} (3)
				(2) edge [above] node{$c$} (4)
				(3) edge [above] node{$d$} (5)
				(5) edge [above] node{$e$} (6)
	;
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,1mm);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,-1mm);
    \end{tikzpicture}
    \end{center}
    \caption*{Automaton accepting $L[c\cup de]$}
  \end{subfigure}
\end{figure}
\begin{figure}[!htbp]
  \begin{subfigure}[c]{0.15\textwidth}
    \begin{center}
    \begin{tikzpicture}[->, >=triangle 45, semithick, node distance=2cm]
    	\node[state, accepting] (1) at (0,0) {};
    	\draw 	(1) edge [loop above] node{$b$} (1);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,1mm);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,-1mm);
    \end{tikzpicture}
    \end{center}
    \caption*{Automaton accepting $L[b^*]$}
  \end{subfigure}
  %
  \begin{subfigure}[c]{0.85\textwidth}
    \begin{center}
    \begin{tikzpicture}[->, >=triangle 45, semithick, node distance=2cm]
    	\node[state] (1) at (-3,0) {};
    	\node[state] (2) at (-0.25,1)  {};
    	\node[state] (3) at (-1,-1)  {};
    	\node[state] (4) at (2.25,1)  {};
    	\node[state] (5) at (1,-1)  {};
    	\node[state] (6) at (3,-1)  {};
    	\node[state] (7) at (5,0)  {};
    	\node[state, accepting] (8) at (7,0)  {};
    	\draw 	(1) edge [above] node{$\varepsilon$} (2)
				(1) edge [above] node{$\varepsilon$} (3)
				(2) edge [above] node{$c$} (4)
				(3) edge [above] node{$d$} (5)
				(5) edge [above] node{$e$} (6)
				(4) edge [above] node{$\varepsilon$} (7)
				(6) edge [above left] node{$\varepsilon$} (7)
				(7) edge [above] node{$a$} (8)
				(8) edge [loop above] node{b} (8)
	;
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,1mm);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,-1mm);
    \end{tikzpicture}
    \end{center}
    \caption*{Automaton accepting $L[(c\cup de)ab^*]$}
  \end{subfigure}
\end{figure}
\begin{figure}[!htbp]
  \begin{subfigure}[c]{0.5\textwidth}
    \begin{center}
    \begin{tikzpicture}[->, >=triangle 45, semithick, node distance=2cm]
    	\node[state] (1) at (-3,0) {};
    	\node[state] (2) at (-1.5,1)  {};
    	\node[state] (3) at (0,0)  {};
    	\node[state, accepting] (4) at (2,0)  {};
    	\draw 	(1) edge [above left] node{$d$} (2)
				(1) edge [below] node{$c$} (3)
				(2) edge [above right] node{$e$} (3)
				(3) edge [above] node{$a$} (4)
				(4) edge [loop above] node{$b$} (4)
	;
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,1mm);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,-1mm);
    \end{tikzpicture}
    \end{center}
    \caption*{Simplified Automaton accepting $L[(c\cup de)ab^*]$}
  \end{subfigure}
  \begin{subfigure}[c]{0.5\textwidth}
    \begin{center}
    \begin{tikzpicture}[->, >=triangle 45, semithick, node distance=2cm]
    	\node[state] (1) at (-3,0) {};
    	\node[state] (2) at (-1.5,1)  {};
    	\node[state] (3) at (0,0)  {};
    	\node[state, accepting] (4) at (2,0)  {};
    	\draw 	(1) edge [above left] node{$d$} (2)
				(1) edge [below] node{$c$} (3)
				(2) edge [above right] node{$e$} (3)
				(3) edge [above] node{$a$} (4)
				(4) edge [loop above] node{$b$} (4)
				(4) edge [bend left, below] node{$\varepsilon$} (1)
	;
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,1mm);
    	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,-1mm);
    \end{tikzpicture}
    \end{center}
    \caption*{Automaton accepting $L[\big(c\cup de)ab^*\big)^*]$}
  \end{subfigure}
\end{figure}
\begin{figure}[!htbp]
  \begin{subfigure}[c]{1\textwidth}
    \begin{center}
    \begin{tikzpicture}[->, >=triangle 45, semithick, node distance=2cm]
    	\node[state] (r) at (-3,0) {$r$};
    	\node[state] (s) at (-1.5,1)  {$s$};
    	\node[state] (t) at (0,0)  {$t$};
    	\node[state] (u) at (2,0)  {$u$};
    	\node[state] (p) at (-7,0)  {$p$};
    	\node[state] (q) at (-5,0)  {$q$};
    	\node[state,accepting] (v) at (4,0)  {$v$};
		\draw 	(r) edge [above left] node{$d$} (s)
				(r) edge [below] node{$c$} (t)
				(s) edge [above right] node{$e$} (t)
				(t) edge [above] node{$a$} (u)
				(u) edge [loop above] node{$b$} (u)
				(u) edge [bend left, below] node{$\varepsilon$} (r)
				(p) edge [above] node{$a$} (q)
				(q) edge [above] node{$b$} (r)
				(u) edge [above] node{$c$} (v)
	;
    	\draw[-, semithick] (node cs:name=p,anchor=west) -- ++(-3mm,1mm);
    	\draw[-, semithick] (node cs:name=p,anchor=west) -- ++(-3mm,-1mm);
    \end{tikzpicture}
    \end{center}
    \caption*{Automaton accepting $L[ab\big(c\cup de)ab^*\big)^*c]$}
  \end{subfigure}
\end{figure}

\FloatBarrier

Let $A=(Q,\Sigma,\delta,q_0,F)$ be a nondeterministic finite automaton with $\varepsilon$ moves, comprising
\begin{itemize}
	\item a set of states $Q=\{p,q,r,s,t,u,v\}$
	\item a set of symbols $\Sigma = \{a,b,c,d,e\}$
	\item a transition function $f:Q \times (\Sigma \cup \{\varepsilon \}) \to \mathsf{Pow}(Q)$ given by the below table

	\begin{tabular}{c|c c c c c c}
		&  \multicolumn{6}{c}{symbol} \\
		state & $a$ & $b$ & $c$ & $d$ &$e$ & $\varepsilon$ \\
		\hline
		$p$ & $\{q\}$ & $\varnothing$ & $\varnothing$ & $\varnothing$ & $\varnothing$ & $\varnothing$\\
		$q$ & $\varnothing$ & $\{r\}$ & $\varnothing$ & $\varnothing$ & $\varnothing$ & $\varnothing$\\
		$r$ & $\varnothing$ & $\varnothing$ & $\{t\}$ & $\{s\}$ & $\varnothing$ & $\varnothing$\\
		$s$ & $\varnothing$ & $\varnothing$ & $\varnothing$ & $\varnothing$ & $\{t\}$ & $\varnothing$\\
		$t$ & $\{u\}$ & $\varnothing$ & $\varnothing$ & $\varnothing$ & $\varnothing$ & $\varnothing$\\
		$u$ & $\varnothing$ & $\{u\}$ & $\{v\}$ & $\varnothing$ & $\varnothing$ & $\{r\}$\\
		$v$ & $\varnothing$ & $\varnothing$ & $\varnothing$ & $\varnothing$ & $\varnothing$ & $\varnothing$\\
	\end{tabular}
	\item an initial state $q_0 = p$
	\item a set of accepting states $F = \{v\}$
\end{itemize}
Then $A$ accepts the language $L[ab\big(c\cup de)ab^*\big)^*c]$.
\section{}
\subsection{}
$\mathtt{aaa^*ba^*}$
\subsection{}
$L = \{a^n b^m a^n|n\geq0,m\geq3\}$

\vspace{10px}

$L$ is not regular. Consider words of the form $a^nbbba^n$ which are all in $L$ and can be of any length greater than 3 -- no substring can be removed from these words whilst they remain in $L$: if the substring contains any $b$, and that substring is removed, there will be fewer than 3 $b$; if the substring contains only $a$, and that substring is removed, there will different numbers of $a$ on each side. As there is no substring that can be removed, there is no substring that can be pumped. Therefore, for any length greater than 3 there is a word in $L$ which does not satisfy the pumping property, therefore it is not regular.

\section{}
Let $G=(V,\Sigma,R,s)$ be a context free grammar with
\begin{itemize}
	\item a set of variables $V=\{S,X,Y\}$
	\item a set of terminals $\Sigma = \{a,b,c\}$
	\item a set of production rules $R = \{S \to XY, \ X \to \varepsilon, \ X \to aXb, \ Y \to \varepsilon, \ Y \to bYc\}$
	\item a start variable $s=S$
\end{itemize}

\vspace{20px}

\noindent This would be accepted by a push-down automaton, $P=(Q,\Sigma,\Gamma,\delta,s_0,F)$ with
\begin{itemize}
	\item a set of states, $Q=\{p,q,r,s,t,u\}$
	\item an input alphabet, $\Sigma=\{a,b,c\}$
	\item a stack alphabet, $\Gamma=\{\bot,a,b\}$
	\item a transition relation, $\delta$, consisting of the following instructions of the form ${((q_0,\sigma,\gamma_0),(q_1,\gamma_1))}$
	\begin{itemize}
		\item $((p,\varepsilon,\varepsilon),(q,\bot))$
		\item $((q,a,\varepsilon),(q,a))$
		\item $((q,\varepsilon,\varepsilon),(r,\varepsilon))$
		\item $((r,b,a),(r,\varepsilon))$
		\item $((r,\varepsilon,\bot),(s,\bot))$
		\item $((s,b,\varepsilon),(s,b))$
		\item $((s,\varepsilon,\varepsilon),(t,\varepsilon))$
		\item $((t,c,b),(t,\varepsilon))$
		\item $((t,\varepsilon,\bot),(u,\varepsilon))$
	\end{itemize} where
	\begin{itemize}
		\item $q_0 \in Q$ is the starting state of the transition
		\item $\sigma \in (\Sigma \cup \{\varepsilon\})$ is the input symbol read during the transition
		\item $\gamma_0 \in (\Gamma \cup \{\varepsilon\})$ is the symbol popped from the stack during the transition
		\item $q_1 \in Q$ is the ending state of the transition
		\item $\gamma_1 \in (\Gamma \cup \{\varepsilon\})$ is the symbol pushed onto the stack during the transition
	\end{itemize}
	\item an initial state, $s_0=p$
	\item a set of accepting states $F=\{u\}$
\end{itemize}

\vspace{20px}

\noindent $P$ can be represented by the following state diagram:

\begin{center}
\begin{tikzpicture}[->, >=triangle 45, semithick, node distance=2cm]
	\node[state] (1) at (0,0)  {$p$};
	\node[state] (2) at (2.5,0)  {$q$};
	\node[state] (3) at (5,0)  {$r$};
	\node[state] (4) at (7.5,0)  {$s$};
	\node[state] (5) at (10,0)  {$t$};
	\node[state, accepting] (6) at (12.5,0)  {$u$};

		\draw 	(1) edge [above] node{$\varepsilon, \ \varepsilon /\bot$} (2)
				(2) edge [loop above] node{$a, \ \varepsilon / a$} (2)
				(2) edge [above] node{$\varepsilon, \ \varepsilon /\varepsilon$} (3)
				(3) edge [loop above] node{$b, \ a / \varepsilon$} (3)
				(3) edge [above] node{$\varepsilon, \ \bot /\bot$} (4)
				(4) edge [loop above] node{$b, \ \varepsilon / b$} (4)
				(4) edge [above] node{$\varepsilon, \ \varepsilon /\varepsilon$} (5)
				(5) edge [loop above] node{$c, \ b / \varepsilon$} (5)
				(5) edge [above] node{$\varepsilon, \ \bot /\varepsilon$} (6)
		;

		\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,1mm);
	\draw[-, semithick] (node cs:name=1,anchor=west) -- ++(-3mm,-1mm);
\end{tikzpicture}
\end{center}
\end{document}


% Created 2022-01-16 Sun 11:18
% Intended LaTeX compiler: pdflatex
\documentclass[presentation]{beamer}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{grffile}
\usepackage{longtable}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{amsmath}
\usepackage{textcomp}
\usepackage{amssymb}
\usepackage{capt-of}
\usepackage{hyperref}
\usepackage{xypic}
\xyoption{pdf}
\usetheme{default}
\newcommand\cat[1]{\ensuremath{\mathcal{#1}}}
\newcommand\hm[2]{\hom(#1,#2)}
\newcommand\nat[2]{{#1}\overset{\cdot}{\rightarrow}{#2}}
\newcommand\tup[1]{\langle #1\rangle}
\newcommand\id[1]{\mathrm{id}_{#1}}
\author{jens.jensen@stfc.ac.uk \\0000-0003-4714-184X\\CC-BY 4.0}
\date{March 13, 2022}
\title{Functional Programming 2}
\hypersetup{
 pdfauthor={jens.jensen@stfc.ac.uk},
 pdftitle={Functional Programming 2},
 pdfkeywords={functional, monad, programming theory},
 pdfsubject={functional programming},
 pdfcreator={Emacs 27.1 (Org mode 9.3) then by hand},
 pdflang={English}}
\begin{document}

\maketitle
\begin{frame}{Outline of Talk 2}

  \begin{itemize}
    \item Previous talk (talk 1):
    \begin{itemize}
\item Introibo
\item Pure Functional Programming Principles
    \end{itemize}
  \item This talk (and probably the next):
    \begin{itemize}
    \item Mapping
    \item Labels and naming
    \item Lists
    \end{itemize}

  \item Advanced(ish) Topics
\item Impure Functional? Side Effects
\item Category Theory
\item Categories and Functions
\item Categories and Computation
\end{itemize}

Still written in the author's spare time!

\medskip

Very much a personal perspective, and not following any particular textbook.  Using \emph{meditations} and \emph{exercises} -- solutions to all exercises given during the talks.

\end{frame}

\section{Introibo}
\label{sec:org0edb596}

\begin{frame}{Summary of Talk 1}
  \begin{itemize}
  \item Basic Principles
    \begin{itemize}
    \item Variables are immutable
    \item Functions have no side effect
      \begin{itemize}
      \item Side effects permitted on r-values (often preferred for efficiency)
      \item In this talk we look at functions with side effects
      \end{itemize}
    \item ``Divide and conquer'' approach to problems
    \end{itemize}
  \end{itemize}
\end{frame}
\begin{frame}[fragile]{Summary of Talk 1}
\begin{verbatim}
(defun fact (k)
  "Calculate the factorial of a number"
  (unless (and (numberp k) (integerp k) (>= k 0))
    (error "Unable to take factorial of %s" k))
  (fact-1 k))
\end{verbatim}
\texttt{fact} is the entry point; it delegates to a helper function which is guaranteed to be called with a non-negative integer.  This means that (in principle) checks can be turned off for the helper, and it can be optimised to run faster:
\begin{verbatim}
(defun fact-1 (k)
  (if (zerop k) 1 (* k (fact-1 (1- k)))))
\end{verbatim}
\end{frame}
\begin{frame}[fragile]{More Mapping}
  Meditation exercise: why do mapping \emph{et al.} not work with macros and special forms?
\begin{verbatim}
(apply #'and '(nil t t))
\end{verbatim}
raises an error, though these will work in EL, but not in CL:
\begin{verbatim}
(funcall #'if t 'yes 'no)
yes
(funcall #'and t t nil)
nil
\end{verbatim}
\end{frame}

\begin{frame}[fragile]{Example -- Sorting Months}
Where does August come before July, April before January?
\begin{verbatim}
(defvar +dates+ '((10 . "Aug") (2 . "Dec") (17 . "Mar") (30 . "Apr") (4 . "Jan") (2 . "Aug")))
\end{verbatim}
Let's start with a function to calculate the month number:
\begin{verbatim}
(defun month-number (m)
  (1+ (floor
        (search m
          "JanFebMarAprMayJunJulAugSepOctNovDec")
        3)))
month-number
(month-number "Jan")
1
(month-number "Dec")
12
\end{verbatim}
This function is \emph{correct} in the sense of producing the right output given the right input, but it does have some flaws and inefficiencies (exercise) -- we shall return to these later.
\end{frame}

\begin{frame}[fragile]{Example -- Sorting Months}
Now the problem is simple: first we add the month number...
\begin{verbatim}
(mapcar (lambda (d) (cons (month-number (cdr d)) d)) +dates+)
((8 10 . "Aug") (12 2 . "Dec") (3 17 . "Mar") (4 30 . "Apr") (1 4 . "Jan") (8 2 . "Aug"))
\end{verbatim}
then we sort those
\begin{verbatim}
(mapcar #'cdr
 (sort
  (mapcar (lambda (d) (cons (month-number (cdr d)) d)) +dates+)
  (lambda (d1 d2) (or (< (first d1) (first d2))
                      (and (= (first d1) (first d2))
                           (< (second d1) (second d2)))))))
((4 . "Jan") (17 . "Mar") (30 . "Apr") (2 . "Aug") (10 . "Aug") (2 . "Dec"))
\end{verbatim}
This is what Perl calls the ``Schwarzian Transform'' -- add the order
value to the entries, sort on the order value, and then strip it off
again at the end.
\end{frame}

\begin{frame}[fragile]{Example -- Sorting Months}
Of course with this being Lisp, we can do better:
\begin{verbatim}
(sort (copy-seq +dates+)
  (lambda (d1 d2)
    (let ((m1 (month-number (cdr d1)))
          (m2 (month-number (cdr d2))))
      (or (< m1 m2) (and (= m1 m2) (< (car d1) (car d2)))))))
((4 . "Jan") (17 . "Mar") (30 . "Apr") (2 . "Aug") (10 . "Aug") (2 . "Dec"))
\end{verbatim}
Meditation: why do we need \texttt{copy-seq} here when we didn't need it before?
 Is this solution better than the Schwarzian transform?

\medskip
In CL, the \texttt{:key} parameter is used to extract the field to be sorted on, but it can also be used to calculate the order:
\begin{verbatim}
(sort (copy-seq +dates+) #'<
  :key (lambda (d) (+ (car d) (* 40 (month-number (cdr d))))))
\end{verbatim}

\end{frame}

\begin{frame}[fragile]{Example -- Magic Mapping with Apply}
From the author's AoC21, Day 9, in CL rather than EL:
\begin{verbatim}
(defun transpose (rows)
  "Transpose rows and columns, list of lists version"
  (apply #'mapcar #'list rows))
\end{verbatim}
This magic works because \texttt{apply} accepts the extra argument (\texttt{\#'list}) to pass to \texttt{mapcar}:
\begin{verbatim}
(transpose '((1 2 3) (4 5 6)))
((1 4) (2 5) (3 6))
\end{verbatim}
\end{frame}
\begin{frame}[fragile]{Advanced Maps}
  Meditate on the advanced mapping functions: this CL example is also from AoC21 Day 9 (uses \texttt{incf} to get map-indexed):
\begin{verbatim}
(defun row-troughs (row)
  "Return the locations of the \"troughs\" in a row of
   numbers, local minima"
  ;; General case: haven't thought too much about it
  (let* ((ext-row (cons most-positive-fixnum
                     (append row
                        (cons most-positive-fixnum nil))))
         (idx 0))
    (mapcan (lambda (a b c)
              (prog1 (if (and (> a b) (< b c))
                       (list idx) nil) (incf idx)))
          ext-row (cdr ext-row) (cddr ext-row))))
\end{verbatim}
\end{frame}
\begin{frame}[fragile]{Example Koan - fizzbuzz 1}
\begin{verbatim}
(defun buzz (num) (if (zerop (mod num 5)) (list 'buzz) nil))
(defun fizz (num)
  (funcall
   (if (zerop (mod num 3))
       (lambda (x) (cons 'fizz x)) #'identity)
   (buzz num)))
(fizz 2)
nil
(fizz 6)
(fizz)
(fizz 10)
(buzz)
(fizz 30)
(fizz buzz)
\end{verbatim}
Meditation: why is \texttt{fizz} defined like this?  (We'll write a cleaner version later)
\end{frame}
\begin{frame}[fragile]{Example Koan - fizzbuzz 2}
As an aside, would this work (i.e.\ without using \texttt{funcall})?
\begin{verbatim}
...
((if (zerop ...) (lambda (x) ...) #'identity) (buzz num))
\end{verbatim}

\end{frame}
\begin{frame}[fragile]{Example Koan - fizzbuzz 3}
Are these better -- and which of these is the best?
\begin{verbatim}
(defun fizz (num)
  (let ((b (buzz num)))
    (if (zerop (mod num 3)) (cons 'fizz b) b)))
\end{verbatim}
or
\begin{verbatim}
(defun fizz (num)
  (if (zerop (mod num 3)) (cons 'fizz (buzz num))
      (buzz num)))
(fizz 7)
nil
(fizz 3)
(fizz)
(fizz 5)
(buzz)
(fizz 15)
(fizz buzz)
\end{verbatim}

\end{frame}
\begin{frame}[fragile]{Example Koan - fizzbuzz 4}
Digression: We need to generate lists of integers (called iota from APL, A+ et al):
\begin{verbatim}
(defun iota (k)
  "Generate a list of integers from 1 to k"
  (labels ((iota1 (k1)
                  (if (< k1 1) nil
                    (cons k1 (iota1 (1- k1))))))
    (nreverse (iota1 k))))
\end{verbatim}
Meditations:
\begin{itemize}
\item Ponder the use of \texttt{nreverse} -- why it is needed, why it is safe
  \begin{itemize}
  \item We shall see later how to build better \texttt{iota}s
  \end{itemize}
\item Why might the variable/function naming not be ideal?  Could we have used \texttt{k} in \texttt{iota1}?  (We will return to naming shortly)
\item \texttt{(defun fact (k) (reduce \#'* (iota k)))}
\end{itemize}

\end{frame}
\begin{frame}[fragile]{Example Koan - fizzbuzz 5}
This is not true fizzbuzz but we want to show \texttt{mapcan} in EL:
\begin{verbatim}
(defun fizzbuzz (num)
  (mapcan #'fizz (iota num)))
(fizzbuzz 15)
(fizz buzz fizz fizz buzz fizz fizz buzz)
\end{verbatim}
The point is that \texttt{fizz} can return 0, 1 or 2 results and they are merged into the result list (destructively!)
\begin{itemize}
\item \texttt{mapcan} merges lists together (destructively), so the map function can return multiple results
\item For 0 or 1 values, it may be cleaner to use \texttt{delete}, \texttt{delete-if} or \texttt{delete-if-not} (as appropriate)
  \begin{itemize}
  \item Note that logically \texttt{delete-if-not} does ``select-if''
  \item \texttt{remove}, \texttt{remove-if} and \texttt{remove-if-not} are the side-effect-free versions
  \end{itemize}
\end{itemize}
\end{frame}
\begin{frame}[fragile]{Example Koan -- FizzBuzz \$$-1$}
  Since we've started, we might as well solve FizzBuzz (with the definitions from previous slides):
\begin{verbatim}
(defun fizzbuzz-number (k)
  "Return symbol from fizzbuzzing a number, or nil"
  (let ((fb (fizz k)))
    (if (cdr fb) 'fizzbuzz  ; >1 elt
      (car fb))))

(defun fizzbuzz (k)
  "FizzBuzz from 1 to k"
  (mapcar (lambda (x) (or (fizzbuzz-number x) x))
    (iota k)))

(fizzbuzz 21)
(1 2 fizz 4 buzz fizz 7 8 fizz buzz 11 fizz ...)
\end{verbatim}
\end{frame}
\begin{frame}[fragile]{Example Koan -- FizzBuzz \$}
Incidentally, in CL if we are worried about the extra list being generated (and eventually gc'ed) in
\begin{verbatim}
(mapcar (lambda (x) ...) (iota k))
\end{verbatim}
we can reuse the list with (permitted) side effect (as the value produced by \texttt{iota} is an r-value):
\begin{verbatim}
(let ((m (iota k)))
  (map-into m (lambda (x) ...) m))
\end{verbatim}
This tells Lisp that we want to reuse \texttt{m} (on the LHS of the lambda) while mapping the values of \texttt{m} (on the RHS), but it obviously less readable than a \texttt{mapcar} and requires the \texttt{let} binding.

\medskip
Unfortunately, while EL has a function called \texttt{map-into}, it does something quite different.

\medskip
We will return to fizzbuzz in the Advanced Topics sections 10 and 13.
\end{frame}

\begin{frame}[fragile]{Advanced maps: Advanced List Iterations}
\texttt{dolist} iterates over a list.  Functionally we have done the same either with \texttt{first} and \texttt{rest} and \emph{recursion}, or with \texttt{mapcar}.  The latter is like \texttt{dolist} except it produces an output:
\begin{verbatim}
(mapcar
  (lambda (state) (when (goalp state) (throw 'found state)))
  data)
(nil nil nil nil nil ...)
\end{verbatim}
An output list is needlessly produced, as we here are calling only for side effect (more on side effects later).

\medskip
Lisp has functions which will happily do the same and discard the result of $\lambda$:
\begin{verbatim}
(map nil (lambda (state)
           (when (goalp state)
             (throw 'found state)))
  data)
\end{verbatim}
\end{frame}
\begin{frame}[fragile]{Advanced maps}
How does \texttt{map} differ from \texttt{mapcar}?  It works on \emph{sequences}:
\begin{verbatim}
(map 'list (lambda (x) (+ x 2)) [1 2 3 4])
(3 4 5 6)
(map 'vector (lambda (x) (- x 2)) '(3 4 5 6))
[1 2 3 4]
(map 'string (lambda (x) (+ x 32)) "ABCD")
"abcd"
\end{verbatim}
(the latter being EL only) but accepts \texttt{nil} as its type to discard the output.

\medskip
Of course this is cleaner if only the type change is desired:
\begin{verbatim}
(coerce (list 1 2 3 4) 'vector)
[1 2 3 4]
\end{verbatim}
\end{frame}

\begin{frame}[fragile]{Two patterns implementing \texttt{mapcar}}
We could have written EL's (simpler) \texttt{mapcar} like this:
\begin{verbatim}
(defun our-mapcar (func lst)
  (if (endp lst) nil
    (cons (funcall func (first lst))
          (our-mapcar func (rest lst)))))
our-mapcar
(our-mapcar #'sqrt '(16 4 36 5))
(4.0 2.0 6.0 2.23606797749979)
\end{verbatim}
except that it may run out of stack for a longer list (we return to this problem later, in Advanced Topics section 5)

\begin{itemize}
\item Indeed much of Lisp can be (or is) constructed from a smaller set of primitives (McCarthy)
\item As we have seen before, the \texttt{first}/\texttt{rest} (or \texttt{car}/\texttt{cdr}) pattern is very common
\item As is the \texttt{cons}/\texttt{nil} constructing the result
\item Usually this (combined) pattern is implemented with \texttt{mapcar}
\end{itemize}
\end{frame}

\begin{frame}[fragile]{Advanced maps -- sliding window}
But maps can do more advanced stuff.  Let us first return to the sliding window exercise from the first talk.  We want to sum values
\begin{verbatim}
(sum-window 3 '(1 5 4 1 3 6 2))
(10 10 8 10 11)
(sum-window 2 '(1 5 4 1 3 6 2))
(6 9 5 4 9 8)
(sum-window 4 '(1 2 3))
nil
\end{verbatim}
\end{frame}
\begin{frame}[fragile]{Sliding Window}
Let's start by writing a helper function: it is to sum variables from a list:
\begin{verbatim}
(sum-helper 4 '(2 5 1 3 6 9 1))
11
(sum-helper 1 '(2 5 1 3 6 9 1))
2
(sum-helper 0 '(2 5 1 3 6 9 1))
0
(sum-helper 5 '(1 2 3 4))
nil
\end{verbatim}
This example is Interesting because it has two stop conditions: the counter reaching zero, and the list becoming empty.

\medskip
Meditation: what is \texttt{(sum-helper 0 nil)}?  Should it be \texttt{0} or \texttt{nil}?
\end{frame}
\begin{frame}[fragile]{Sliding Window}
\begin{verbatim}
(defun sum-helper (n data)
  (cond
   ((zerop n) 0)
   ((endp data) nil)
   (t (let ((result (sum-helper (1- n) (rest data))))
        (and result (+ (first data) result))))))
\end{verbatim}
\end{frame}

\begin{frame}[fragile]{Sliding Window}
Now we can write a function to call the helper... we can't just do \texttt{mapcar} as the function would see only \texttt{first} (as with \texttt{dolist}) -- so, instead, we could almost do:
\begin{verbatim}
(defun sum-window (k data)
  (if (endp data) nil
    (cons (sum-helper k data)
          (sum-window k (cdr data)))))
(sum-window 3 '(1 5 4 1 3 6 2))
(10 10 8 10 11 nil nil)
\end{verbatim}
This works, but obviously it generates extra \texttt{nil}s at the end as it has to map across the whole list - so we could write another function to truncate them away.

\medskip
Notice the \texttt{cons}/\texttt{rest} pattern again -- \emph{nearly} like the \texttt{mapcar} pattern
\end{frame}

\begin{frame}[fragile]{Sliding Window -- Advanced maps}
The \texttt{cons}-\texttt{cdr} pattern from the previous slide is a very common functional pattern.  Lisp has a mapping function that can do the same in a single line:
\begin{verbatim}
(defun sum-window (k data)
  (maplist (lambda (d) (sum-helper k d)) data))
(10 10 8 10 11 nil nil)
\end{verbatim}
\texttt{maplist} \texttt{cdr}s across the list, \texttt{cons}ing the results:
\begin{verbatim}
(maplist #'identity '(1 2 3 4 5))
((1 2 3 4 5) (2 3 4 5) (3 4 5) (4 5) (5))
\end{verbatim}
These are successive \texttt{cdr}s of the \emph{same} list:
\begin{verbatim}
(let ((m (maplist #'identity '(1 2 3 4 5))))
  (eq (cdar m) (cadr m)))
t
\end{verbatim}
\end{frame}

\begin{frame}[fragile]{Sliding Window -- Advanced maps}
So we either do
\begin{verbatim}
(defun sum-window (k data)
  (delete nil (maplist (lambda (d) (sum-helper k d)) data)))
\end{verbatim}
and we're done; or we do
\begin{verbatim}
(defun sum-window (k data)
  (mapcon (lambda (d)
            (let ((v (sum-helper k d)))
              (and v (list v))))
          data))
(sum-window 3 '(1 5 4 1 3 6 2))
(10 10 8 10 11)
\end{verbatim}
\texttt{mapcon} is to \texttt{maplist} what \texttt{mapcan} is to \texttt{mapcar} -- it \texttt{nconc}s the results; here we use it to delete the \texttt{nil} entries.

\end{frame}
\begin{frame}[fragile]{Mapping for side effect}
This call appears to have the side effect of the lambda, but it doesn't -- why?
\begin{verbatim}
(mapcar (lambda (x) (setq x 3)) (list 1 2 3 4))
(3 3 3 3)
\end{verbatim}
but in fact it has no effect at all on the original list:
\begin{verbatim}
(let ((y (list 1 2 3 4)))
  (mapcar (lambda (x) (setq x 3)) y)
  y)
\end{verbatim}
Meditation: why are we using \texttt{list} to create the list instead of the macro \texttt{'}?
\end{frame}

\begin{frame}[fragile]{Mapping for side effect}
This works, though -- why?
\begin{verbatim}
(let ((y (list 1 2 3 4)))
  (maplist (lambda (x) (rplaca x (1+ (car x)))) y)
  y)
(2 3 4 5)
\end{verbatim}
but of course \texttt{maplist} still creates a temporary list which is also updated:
\begin{verbatim}
(let ((y (list 1 2 3 4)))
  (maplist (lambda (x) (rplaca x (1+ (car x)))) y))
(2 3 4 5)
\end{verbatim}
The lists are not \texttt{eq} (though their elements are):
\begin{verbatim}
(let ((y (list 1 2 3 4)))
  (eq y (maplist (lambda (x) (rplaca x 3)) y)))
nil
\end{verbatim}
\bigskip
This ends the review of the mapping functions!
\end{frame}
\end{document}