\section{Machine Learning - Supervised Learning}

Supervised learning concerns the problems where the objective is to predict something based on previous data. The counterpart Unsupervised Learning tries to find patterns in unlabelled data.

More generally, the dataset for supervised learning problems consists on a feature vector $\mathbf{x}$ and a output vector $\mathbf{y}$ as opposed to unsupervised learning where everything is features / data.

There are two main types of problems, regression and classification. The only difference between them is the expected output: regression aims to predict continuous outcomes while classification regards separating inputs in classes, thus a discrete output.

Some tools can be used to solve both problems, like Neural Networks. We'll have a look which tools are best for which problems.

\subsection{Regression Problems - Least Squares}
These are the two most commons types of problems. Probably every supervised learning problem can be \ii{classified} as one of these.

Regression is when the output should be continuous, classification when the output should be in discrete classes.

About the first, a measure to minimize is the difference between our prediction to the value we want to achieve. The Sum of the Square Errors (SSE) is very standard cost function to minimize.

The function that is required to minimize is loss/cost/risk function. Nomenclature wise is a problem... Therefore, the following letters/terms can be will be used interchangibly: $L$ (Loss Function) or $J$ (more used when the weights or coefficients are $\theta$) or $R$ (Risk Function):
\begin{equation}
    R = \frac{1}{2m} \sum_{i = 1}^m \left(y^{(i)} - \hat{y}^{(i)}\right)^2
\end{equation}

Where $\hat{y}$ is our prediction or estimation of the true value of $y$ and \bb{m} is the number of training samples we have. Therefore $y^(i)$ constitutes the outcome of sample $i$.

One prediction can be made with:
\begin{equation}
    \hat{y} = \beta_0 + \beta_1 x_1 + \dots + \beta_n x_n
\end{equation}
Where \bb{n} is the number of features we are using. Note that features and data points are different things. We can have data regarding only one measure but take the square and the cube of the measure multiplied to different coefficients in order to try to better estimate the function.

$\beta$'s are usually used in regressions while in neural networks letters like $\theta$ or $w$ are more common.

One intuition that is important to have is that \bb{the more we increase the features, the more likely it is that we end up overfitting our training set and loosing generalization capabilities for the actual test data}. This is why it is important to divide all the available data in sets, the model will be trained to guess the training samples, not samples that it never saw before.
\begin{itemize}
    \item One set for training and one set for testing the prediction capabilities;
    \item One training set, one validation set and one test set.
\end{itemize}

The last option is meant to validate the model. The model is trained with the training set, then some parameters are tuned with the validation set, namely the number of polynomial features, regularization term and step size related parameters (like momentum or adaptive step size), and the actual performance is testing in the test set. This way, we avoid optimistic measures of performance by not testing in data used for training.

\quickimage{AIML/AA-cap1-017.png}{0.4}

The result wouldn't be very different if done with the number of iterations or number of features. In particular, it is called doing an ``early stop'' when the iteration that minimises the loss in the test set is early in the minimization process. It is useful when the model starts overfitting the data.

\subsubsection{How to calculate the coefficientes}
\label{sec:calcBetas}
After having the coefficients, given any other set of data points we can already give predictions on the output.

Note that we are trying to minimize the cost/risk/loss function. The actual cost function would have to be some sort of prediction because it's impossible to know exactly how much the actual outcome will be, despite knowing  exactly what the outcome of the model will be to a certain feature vector. As opposed to the empirical risk function where the training outcome and the training predicted result are used, therefore being able to calculate the difference between each estimation and the supposed outcome. To compute the real risk, the expected value of the SSE is necessary. Also, there are continuous results so an integral is required:
\begin{equation}
    R = E[y-\hat{y}] = \int_{-\infty}^{+\infty} L(y, x) \phi(y) dx dy
\end{equation}

Because a potential function to give us a measure on how frequent certain values are is not known, the only way is to approximate the actual error empirically, using the model with some test data. This will degenerate in the actual cost function presented before. L here is meant to denote the loss of one sample which is nothing more than the squared error.

One thing that won't happen in all problems is having an analytical and optimum solution for them. Actually, minimizing the SSE is a kind of problem is called the \bb{Least Squares}. \ul{This kind of problem is very usually used in optimisation and often a closed solution is possible.}

In this case, since we are searching for the function's minimum, the functions partial derivatives need to be zero in order to have a critical point (maximum, minimum or saddle point).

In this case,

\begin{align*}
    \frac{\delta R}{\delta \beta_0} = -2 \sum_{i = 1}^n (y^{(i)} - \beta_0 - \beta_1 x^{(i)}) = 0 \\
    \frac{\delta R}{\delta \beta_1} = -2 \sum_{i = 1}^n (y^{(i)} - \beta_0 - \beta_1 x^{(i)})x^{(i)} = 0 \\
\end{align*}

Is possible to simplify further these equations, putting the $\beta$'s in evidence and separating sums, arriving at:

\quickimage{AIML/AA-cap1-001.png}{0.6}

Is possible to invert these equations and get the expressions for the coefficients. However there's one important factor to have in mind. Are we aiming at a minimum, maximum or something different like a saddle point? The Hessian Matrix will tell us.

\subsubsection{Extrema Conditions and Hessian Matrix} \label{sec:Hessian}

Videos: \href{https://www.youtube.com/watch?v=nRJM4mY-Pq0&list=PLSQl0a2vh4HC5feHa6Rc5c0wbRTx56nF7&index=87}{\ul{87 - Warm up to the second partial derivative test}} to\dots \href{https://www.youtube.com/watch?v=sJo7D74PAak&list=PLSQl0a2vh4HC5feHa6Rc5c0wbRTx56nF7&index=89}{\ul{89 - Second partial derivative test intuition}}

\quickimage{AIML/AA-cap1-002.png}{0.6}

In one dimension, to find an extrema is necessary to equalise the first derivative to zero, and the second derivative must be positive - in case of a minimum - or negative - in case of a maximum. If the second derivative is zero at the critical point, then there's an inflexion point. A similar analysis must be done in ($n$+1)-D where $n$ is the number of features used in the regression.

Guaranteeing that the first derivative is zero, which in N dimensions is correspondent to guaranteeing that the gradient is zero in that point, is the first step. Setting $\triangledown f = 0$ means that, in that point, the function is not increasing or decreasing in any of the N directions. So the first derivative makes sense.

However, when we go to the second derivative, the meanings get a bit more complicated.

In 2D, if the second derivative was 0, it was certainly a saddle point, if it was $> 0$ or $< 0$ it was, respectively, a local minimum or maximum.

The conditions we should impose in 3D is to have a positive (for finding a minimum) or negative (for finding a maximum) definite Hessian Matrix. While it is possible to attribute a meaning to second derivatives in order to just one variable, being nothing more than the concavity in those 2 directions, why do the cross derivatives play a role as well? And, why do they mean really?

Well, first the explanation on why it is needed: there are functions that across multiple dimensions still show that it is an extrema but then there's an inflexion along directions that are not along the axis. So, checking the axis is not enough. Why checking the cross partial derivatives makes it enough?

\ul{The Second Derivative Test}

\begin{equation}
    f_{xx}(x_o, y_o) f_{yy}(x_o, y_o) - f_{xy}(x_o, y_o)^2 \gtrless 0
\end{equation}

If it is greater than 0, we have a maximum or a minimum and have to check the value of $f_{xx}$ or $f_{yy}$ to be sure. If it is less than 0, we have a saddle point. If it equals 0, then we don't know if it is a saddle point, but it is not a min or max therefore, at least for now, we certainly don't care.

Cross or Mixed partial derivatives can be switched? Yes if the function is $C^2$.
(Boring to prove theorem called: Schwarz' Theorem)

Therefore, we just need to compute one of the cross derivatives.

Also this works because the second derivative test is nothing more than the determinant of the Hessian matrix. The determinant is the product of every eigenvalue of that matrix, therefore it can only be positive if they are both positive or both negative, in which cases there is, respectively, a minimum or a maximum.

But why do eigenvalues tell us this? Because they tell us how the eigenvectors are scaled! And the eigenvectors of such matrix will be the greatest and the least curvatures. Therefore, they either have the same signal / are scaled the same amount, or

Some other links that helped with this:
\begin{itemize}
    \item \href{http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/MORSE/diffgeom.pdf}{\ul{Differential Geometry}}
    \item \href{http://math.mit.edu/classes/18.013A/HTML/chapter11/section02.html}{\ul{Criterior for critical points - Maximum, Minimum or Saddle?}}
    \item \href{https://www.adelaide.edu.au/mathslearning/play/seminars/evalue-magic-tricks-handout.pdf}{David Butler - Facts about Eigenvalues}
\end{itemize}

The two main properties of eigenvalues that allow us to quickly calculate them from the Hessian matrix (specially if it is 2x2) are:

\begin{equation}
    \text{tr}(A) = \sum^n_{i = 1} \lambda_i
\end{equation}

\begin{equation}
    \text{det}(A) = \prod^n_{i = 1} \lambda_i
\end{equation}

Because there are only 2 variables, there can't be very big changes across more than 2 main directions, so it is possible to quantify the main directions which will be the eigenvectors directions.

\bb{The eigenvalues of the Hessian Matrix are also called principal curvatures and the eigenvectors the principal directions.}

\subsubsection{Analytical Expression for the Coefficientes}

From the equation presented in the end of \ref{sec:calcBetas}, we can re-write the SSE and the normal equations in the following way.

\quickimage{AIML/AA-cap1-003.png}{0.6}

And arrive at the analytical expression through the simple inversion of the normal equations. Another way of reaching the analytical expression is deriving the cost function.
\quickimage{AIML/AA-cap1-004.png}{0.6}
\quickimage{AIML/AA-cap1-006.png}{0.6}

Note however that:
\quickimage{AIML/AA-cap1-005.png}{0.6}

A final remark on multiple outputs: in case our feature vector serves to estimate more than one output, we can simply use it separately for each output!

\subsubsection{Regularization}

This is the method of taking importance away from the minimization of the errors between the training set supposed outcomes and the actual model outcomes for those samples. If we don't take importance away, the model may become too good at predicting training examples and may forget that it should predict a tendency and generalize well for the test data.

\quickimage{AIML/overfitting.png}{0.3}

Performing a regularization consists on nothing more than adding a new parameter to the cost function, in order to shift away the focus of minimizing the SSE.

There are generally two terms that can be added. One with the \bb{norm of the coefficients squared} and the other is the with the module of the coefficients squared.

For the norm squared, if the regularization is applied to a regression - \bb{which is not a necessity since regularization can even be applied to Neural Networks} - it's called Ridge Regression:
\quickimage{AIML/AA-cap1-007.png}{0.6}

\quickimage{AIML/AA-cap1-008.png}{0.6}

Two key things to note:

\begin{itemize}
    \item Note that $\beta_0$ is usually not included as the data should be normalised already for much better results. Normalised data means data that has zero mean in every feature and outcome.
    \item If $\left(X^TX\right)$ is singular, the least squares estimate is not unique. Regularization will help finding an estimate even then because $\left(X^TX + \lambda I\right)$ is always non-singular.
\end{itemize}

For the simple norm of the coefficients, when applied to a regression problem it is called the Lasso Regression:

\quickimage{AIML/AA-cap1-009.png}{0.6}

\begin{equation}
    ||\beta||_1 = \sum_{j=1}^{n} |\beta_j|
\end{equation}

The key difference between the two is that Ridge aims to minimize the norm of all of them while Lasso aims to minimize the sum of module of each of them. Therefore, Ridge it is much likely to pull closer to zero the biggest ones as those are the ones that matter the most for the Euclidean norm, while Lasso will try to pull each of them to 0, there's a direct dependency between a coefficients and the cost function. This is also why the Lasso Regression is called to do feature selection: because if the SSE doesn't depend on the coefficients, the regularization term with the sum of the norms will put that coefficients to zero very quickly.

\quickimage{AIML/AA-cap1-010.png}{0.4}

Again, recall that the data should be centered - have zero mean - and that after calculating the model we need to de-centre it to obtain the real predictions!
\quickimage{AIML/AA-cap1-011.png}{0.5}

\subsubsection{Optimization problems - Gradient Descent and Newton's Method}

The gradient descent is probably the most know method to approximate a functions minimum. By changing the direction of the step we have the gradient ascent.

\quickimage{AIML/AA-cap1-012.png}{0.6}

The above expression works because the gradient points the direction of the maximum growth of the function. Therefore, taking a step in the opposite direction will lead to the minimum.

\subsubsection*{Momentum and Adaptive StepSize}
Momentum $0 \leq \alpha \leq 1$:
\quickimage{AIML/AA-cap1-013.png}{0.6}

Note that we want to pick the $\mathbf{x}$ that brings the function to a minimum. Therefore, the ``exponent'' will simply refer to the iteration number, not the sample like in the previous section.
If $\alpha$ is closer to 1 the memory of the previous increment is more taken into account, meaning that the increment will change only slightly. The closer the parameter gets to 0, the closer we get to the normal situation. This is called the momentum term because it gives the convergence some inercia, the behaviour of momentum. By changing the increment slowly, it may converge faster and have less abrupt changes.

Adaptive Step size, with typical values: $u = 1.2, d = 0.8$:

\quickimage{AIML/AA-cap1-014.png}{0.6}

Let f be the function where the objective minimum lies, the Divergence criterium is given by:
\begin{equation}
    f(x^{(n+1)}) > f(x^{(n)})
\end{equation}
Where the threshold is found in the equality.

\subsubsection*{Newton's method}

Given by:
\quickimage{AIML/AA-cap1-015.png}{0.4}

Where the gradient and the Hessian Matrix are given by:

\quickimage{AIML/AA-cap1-016.png}{0.4}

The Newton's method converges insanely fast! But requires the inversion of the Hessian matrix which can be a serious problem...

\subsubsection*{Newton's Method - Intuition and Demonstration}
This is a very very good proof!

%do. it. It is in the notebook...

\subsubsection{How to optimise hyperparameters}
There will always be parameters to optimise in order to obtain the best model possible.
It was said before that 3 sets should be selected: training, validation and testing.
And that it was in the validation set that all hyperparameter tunning should be done.
The simplest way is calculate the model with all combinations of the parameters possible and see which one performs better in the validation set.

There can be one other problem: little data. If the data is too little, dividing it into sets can start to give biased results to the accuracy.

One way of calculating the accuracy with more ... accuracy \dots is splitting the data in k folds and rotate:

\quickimage{AIML/AA-cap1-018.png}{0.4}

Finally, if both things need to be done at the same time, then:
1- k folds need to be made.
2- for each fold, all values of the hyperparameters need to be used for training and tested in the test set.
Note: it will be used for training T except ($T_i$ and $T_j$) that are, respectively, the test set and the validation set. So the hyperparameters testing will be done with $T_i$.
When the all combinations are done, the hyperparameters are selected for the bet one and the actual model is trained with T except $T_i$ depending on the fold considered.
3- use the performances of each fold to get the best average of performance.

(Note that this is not very used...)
\quickimage{AIML/AA-cap1-019.png}{0.4}

\subsection{Neural Networks}

\subsubsection{Formalisation}

On the surface, a NN is nothing more than a set of weights connecting a set of neurons. This is represented in Figure \ref{mlp}.

\image{AIML/mlp.png}{Overview of a Neural Network}{mlp}{.7}

This particular architecture is called a Multilayer Perceptron (MLP), the standard NN. In other architectures not all layers are required to be fully connected e.i every neuron from the previous layer is connected to all neurons from the next layer, however, for simplicity, let's restrict this formalisation to MLPs.

Let $w_{ij}^{(l)}$ be the weight that connects the output of the $j$-th neuron of layer $l-1$ to the $i$-th neuron of layer $l$. If we call the output of a neuron $j$-th of layer $l$, $z_j^{(l)}$, and the input of the $i$-th neuron of layer $l$ $s_i^{(l)}$, one may write Equation \eqref{eq:s}.

\begin{equation} \label{eq:s}
    s_{i}^{(l)} = w_{i 0}^{(l)} + \sum_{j=1}^{N^{(l)}} w_{ij}^{(l)} z_j^{(l-1)}
\end{equation}

With $N^{(l)}$ being the number of neurons of layer $l$. Also, the input of the first layer is the input of the network, i.e $z^{(0)} = \mathbf{x}$.

Note further that is possible to relate the input of a neuron to its output through that neuron's activation function $g(x)$. Equation \eqref{eq:act} shows this relation, with L the number of layers in the MLP.

\begin{equation}\label{eq:act}
    z_{i}^{(l)} = g\left(s_{i}^{(l)}\right) \quad , i = 1, ..., N^{(l)} \quad, l = 1, ... , L
\end{equation}

Activation functions of neurons may vary across layers and they simply relate the input with the output.

There are many kinds of activation functions, each with its advantages and disadvantages - refer to \href{towardsdatascience.com/comparison-of-activation-functions-for-deep-neural-networks-706ac4284c8a}{\ul{actFunctions}} for a more in-depth analysis. In this work only two are used, Rectified Linear Unit (ReLu) and Softmax, which are represented in the set of Equations \eqref{eq:actFuncs}.

\begin{equation} \label{eq:actFuncs}
    \begin{cases}
        \text{ReLu}\left(s_{i}^{(l)}\right) = \max\left(0,s_{i}^{(l)}\right) \\
        \text{Softmax}\left(s_{i}^{(l)}\right) = \frac{\exp\left(s_i^{(l)}\right)}{\sum_{j=1}^{N^{(l)}} \exp(s_j)}
    \end{cases}
\end{equation}

So far we've see how to get the input to the output - forward propagation is the technical term - but wasn't explained yet how to adjust the weights such that the networks starts behaving like expected. It is done with backpropagation.

Backpropagation is an algorithm that consists of calculating the effect that each weight has on the output and adjust that weight accordingly to that relation and accordingly to how wrong the output is. Backpropagation can be done with Gradient Descent methods and all their associated optimization techniques. The simple version of backpropagation with the classic gradient descent is presented in Equation \eqref{gradDesc} where $\eta$ denotes the step size.

\begin{equation} \label{gradDesc}
    w_{ij}^{(l)} = w_{ij}^{(l)} - \eta \ \bigtriangledown_{ij}^{(l)} \quad , \text{with} \ \ \bigtriangledown_{ij}^{(l)} = \frac{\delta J}{\delta w_{ij}^{(l)}}
\end{equation}

The complete expression for the partial derivatives of every weight is in Equation \eqref{gradExpr}.

\begin{equation} \label{gradExpr}
    \bigtriangledown_{ij}^{(l)} = \delta_i^{(l)} z_j^{(l-1)} \quad,
\end{equation}

\begin{equation}
    \text{with} \
    \begin{cases}
    \delta_i^{(l)} = g\left(s_i^{(l)}\right)
    \sum_{k=1}^{N^{(l+1)}} \delta_k^{(l+1)} w_{ki}^{(l+1)} \quad, \text{for} \ l = 1, ..., L-1\\

    \delta_i^{(L)} = g\left(s_i^{(L)}\right) \ \frac{\delta J}{\delta z_i^{(L)}}
    \quad, \ \text{otherwise, i.e for} \ l = L)
    \end{cases}
\end{equation}

Note that $z_i^{(L)} = \hat{y_i}$. Additionally, J is the cost/loss function, the function that tells us how far from the correct result the output is. For classification problems, a good cost function usually is Cross Entropy, Equation \eqref{crossEnt}. However, bear in mind that modifying the cost function to one that is more frequently used in Regression problems one can easily use the NN in regression problems.

\begin{equation} \label{crossEnt}
    J(y, \hat{y}) = -\frac{1}{N} \sum_{i=1}^{N} \sum_{k=1}^K \mathbb{1}_{y_i = y_k} \log\left(\hat{y_i}\right)
\end{equation}

Where $N$ is the number of samples and $\mathbb{1}_{y_i = y_k}$ denotes the indicator function that only is 1 when the supposed output is $y_k$. In other words, the inner sum should always have just one term corresponding to the logarithm of exit of the neural network that should be 1 for that class. This is because due to the common use of Softmax activation function in the last layer, the outputs will be percentages of certainty. And the cost function should be the logarithm of that the certainty of that class only.

More specifically, if we want to categorise images of digits, our neural networks will have 10 outputs, one for each class/digit. When a sample that has the number 3 written on it is propagated until the end, the loss of that computation should be the logarithm of the probability in the $3^{rd}$ exit, because a perfect NN would output a 1 in the $3^{rd}$ exit and 0 in the others. In fact, $\log(1)$ is 0 loss and $-\log(0)$ is infinite (positive) loss.

\subsubsection*{Further optimizations}

In order to achieve an efficient implementation, the previous equations can be written in a vectorized way and the forward and backward propagation will be reduced to matrix multiplications. Additionally, for a low level analysis it becomes relevant to keep track of all matrices dimensions, thus they are the following:
\begin{itemize}
    \item $z^{(l)}$ is $\left(1 + N^{(l)}\right) \times 1$ and   $z^{(l)} = $ %that vertical matrix;
    \item $s^{(l)}$ is $N^{(l)} \times 1$ and $s^{(l)} = W^{(l)} z^{(l-1)}$;
    \item $W^{(l)}$ is the weights matrix and is $ N^{(l)} \times \left(1 + N^{(l-1)}\right)$ which should make sense when looking for the above formula and that along its rows are the weights multiplied to the previous layer plus one for the bias unit;
    \item $\bigtriangledown^{(l)} = \delta^{(l)} \dots$
\end{itemize}

Note that feed forward of all samples at once is possible through the correct definition of $X$ matrix and the correct changes.

\subsubsection*{Some history}
\quickimage{AIML/AA-cap1-018.png}{0.4}

\quickimage{AIML/AA-cap1-023.png}{0.4}

However, there's a big problem with only being able to distinguish data that can be separate with an hyperplane: very often the data doesn't behave that way.

\quickimage{AIML/AA-cap1-024.png}{0.4}

One can make a more complex analysis of the situation...

\subsubsection*{Activation Functions and Architecture}

Some activation functions:

\quickimage{AIML/AA-cap1-021.png}{0.4}

Two other are Rectifier Linear Unit and the softmax:
\quickimage{AIML/AA-cap1-022.png}{0.4}

\quickimage{AIML/softmax.png}{0.2}

In the all layers with the exception of the last one, ReLu or the logistic function work very well. The last layer must have a function that returns results between 0 and 1, therefore only SoftMax and sigmoid functions like the logistic function would work properly.

Training methods:
\quickimage{AIML/AA-cap1-025.png}{0.4}

One very interesting fact is that the image features are not selected by anyone. The network itself crafts its features throught backpropagating the changes required to get the images right. This is true for all Neural Networks.

Also, in image recognition for instance, but in deep neural networks in general, the last layers usually are fully connected.

\subsubsection{Neural Networks - BackPropagation}

Some of the most useful websites to check while trying to demonstrate the backpropagation algorithm:
\begin{itemize}
    \item \href{https://medium.com/@pdquant/all-the-backpropagation-derivatives-d5275f727f60}{\ul{all backpropagation derivatives}}
    \item \href{https://stats.stackexchange.com/questions/94387/how-to-derive-errors-in-neural-network-with-the-backpropagation-algorithm}{\ul{Error (deltas) derivation for backpropagation in neural networks}}
    \item \href{https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi}{3Blue1Brown Neural Networks} - Specially the last one, on backpropagation.
\end{itemize}

The fastest way is the following:

Hand written demonstration on Drive, link in Intro - the first page.

\vspace{1cm}

Note that multiple training modes are possible. The normal one is on-line, where the increment to the weight is nothing more than the step multiplied by the respective partial derivative - Equation \eqref{eq1:online}. Then one can do the batch-mode where all the samples are considered for the weights update - Equation \eqref{eq2:batch}. Finally, there's the mini-batch mode that doesn't use all the training samples.

\begin{align}
    w_{ij}^{(l)} &= w_{ij}^{(l)} + \bigtriangleup w_{ij}^{(l)} \\
    \bigtriangleup w_{ij}^{(l)} &= -\eta \frac{\delta C^k}{\delta w_{ij}^{(l)}} \label{eq1:online} \\
    \bigtriangleup w_{ij}^{(l)} &= -\frac{\eta}{m} \sum_{k=1}^m  \frac{\delta C^k}{\delta w_{ij}^{(l)}} \label{eq2:batch}
\end{align}

\subsubsection{Neural Networks - Convolutional}

Convolutional Neural Networks are extremely useful for applications like image recognition. They take a width x height x image depth 3D array and convolve it with a kernel, generating an 2D array.

Because many elements in an image are translation invariant, considering patches of the image is one of the best ways of acquiring features.

Convolutional neural networks allow us to use less inputs to our neural networks. We map a region of a picture to just one pixel/input. We tend to use many different kernels - a set of weights to multiply to each of pixel in the set we chose - to perform this step.

\quickimage{AIML/AA-cnn.png}{0.6}

At the end of the convolution, an activation function (like ReLu) is applied.

Note that this convolutional layer can be applied to a 3D array to reduce it to 2D. Moreover, many different kernels can be convolved with the section of the 3D array creating several layers. If there are 20 kernels, we'll have 20 2D arrays, therefore a new 3D array that is all the pixels in the several vicinities, weighted with different kernels.

\quickimage{AIML/AA-cnn3.png}{0.6}

\subsubsection*{Pooling}
2D to 2D but with smaller dimensions.
Downsize the feature array by choosing what we consider to be the most important values. For instance, for each 4x4 square of pixels, we choose the maximum of them (the highest value in grey scale, for instance). Therefore, the end result will have 16x less pixels/inputs.

\quickimage{AIML/AA-cnn2.png}{0.6}

Overall, the tendency is the kernel to be smaller, but the network to have many many layers.

\subsubsection{Kernels}
Why kernels are important? They facilitate a lot the mapping of features to higher dimensions!

\href{https://medium.com/@zxr.nju/what-is-the-kernel-trick-why-is-it-important-98a98db0961d}{\ul{Why kernels?}}

They will become specially important in support vector machines when the data is not separable in certain dimensions but if we increase the dimensions is possible to separate the data by an hyperplane.
%therefore, I'll write here in a few days/weeks

\subsubsection{Classification Problems}

When the required output is a discrete class. Instead of regressing something, the objective is to separate data into classes. Through the learning of what features each class has, be able to classify new data accordingly.

A Bayes Estimator is a Maximum Likelihood estimator, a perfect estimator. However, it requires information that we usually don't have. Estimators like the Naive Bayes are useful because they enable simplifications if one is willing to accept the assumptions they entail.

We'll start with the maximum likelihood estimator and then particularize to Naive Bayes.

Let's assume K classes and $y_k$ being each one. We want to choose $y_k$ such that $k = \underset{k}{\operatorname{argmax}} p(y_k|\mathbf{x})$.
This is a maximum likelihood estimator because we want to choose the class that is more likely given the data we received.

From the Bayes Theorem, one can rewrite that probability as $p(y_k|\mathbf{x}) = \frac{p(\mathbf{x}|y_k) p(y_k)}{p(\mathbf{x})}$. Given the denominator as a scaling factor and, normally, an equal probability of each class, one can rewrite the maximum likelihood estimator as:

\begin{equation}
    y_k : k = \underset{k}{\operatorname{argmax}} \ p(\mathbf{x}|y_k)
\end{equation}

Note that the rigorous shape of the above probability is $p(x_1, x_2, ..., x_n|y_k)$, where $n$ is the number of features in our feature vector $\mathbf{x}$.

So far, some considerations have been made but no approximations. The Naive Bayes estimator simplifies the estimation at a cost: assuming the features are independent. If the features are independent of each other, the joint probability distribution degenerates in the multiplication of the marginal PDFs. Likewise, the joint conditional probability density function degenerates in the multiplication of the marginal probability density functions. Mathematically:

\begin{equation}
    p(x_1, x_2, ..., x_n|y_k) = \prod_{i=1}^{n} p(x_i|y_k)
\end{equation}

Further, one should note that due to the logarithmic function being monotone, maximizing a function or the its logarithmic has the same maximizing argument. This is a way of transforming the above multiplication into $\sum_{i=1}^{n}\log(p(x_i|y_k))$.

One particular case where the Naive Bayes Estimator performs decently is language recognition. Not because the letters in the \ii{ngrams} considered are independent - that is certainly not the case, e.g `ã' is much more likely to be followed by a `o' in portuguese - but because the assumption of independence throughout the languages has somewhat of the same effect, not influencing the estimation too much. Therefore, assuming independence between characters in a case where there are so many characters and so many \ii{ngrams} combinations to derive our estimator from, doesn't return that bad results and is quite a good application of Naive Bayes, a very simple estimator.

\subsection{Support Vector Machines}

\subsubsection{Linear Classifiers}
We call linear methods the classifiers whose decision boundaries are linear, hyperplanes.

A way of classifying multiple classes is to assign to class i, a discriminant $f_i(x) = \left[1 x^\text{T}\right] \beta_i$.

The purpose of this function is to be 1 when the class of the input is class i, and be 0 when the class of the input is not $y_i$.

Therefore, decisions are made with $\hat{y}_i : i = \underset{i}{\operatorname{argmax}} f_i(x)$.

Logistic Regression is a Generalized Linear Model (GLM) which can perform prediction and inference while linear Perceptrons can only achieve predictions, and in this case will perform similarly as to logistic regression. Statistical Modelling versus Machine Learning in practice.

In logistic regression, since each output represents a probability, the maximisation of the correct probability is the aim. Therefore gradient ascent (the same apart from a signal) is used.

\subsubsection{SVM's}

This is a great tutorial on the vector math part of SVM.

\href{https://www.svm-tutorial.com/2014/11/svm-understanding-math-part-2/}{\ul{Vectors of SVMs}}

This is a very very good and complete tutorial about SVM:
\href{https://cling.csd.uwo.ca/cs860/papers/SVM_Explained.pdf}{\ul{Fletcher: SVM explained}}

SVMs can only separate between two classes, then is necessary to do strategies like 1 vs All to separate more classes. This is our $n$ sample dataset with $p$ features per sample.
\quickimage{AIML/AA-cap8-001.png}{0.6}

From Fletcher's:
\quickimage{AIML/AA-cap8-002.png}{0.6}

We'll now have a look to some of the whys. Note that this is for a Linearly Separable Binary Classification.
\begin{itemize}
    \item How about non-Binary? One vs All.
    \item What if it is not Linearly Separable? Kernels.
    \item What else, Linear Regression? Yes.
    \item But the SVM always separates with an hyperplane right, always linearly? Unless a Non-linear SVM is created.
\end{itemize}

\subsubsection{One vs All approach}

Applies to SMVs and all classification problems that can only distinguish between 2 classes.

It consists of training K classifiers that will make decisions between the K classes of option. Each classifier will consider the labels of that class against the ``other'' label that will be all the other samples.

The class with higher certainty is the correct one.

\quickimagesidebyside{AIML/AA-002.png}{0.6}{AIML/AA-001.png}{0.6}
\quickimagesidebyside{AIML/AA-003.png}{0.6}{AIML/AA-004.png}{0.6}

\subsubsection{Formulation of SVMs}

All the edge cases explained above will be constructed above this, usually being a simple modification.

We want to define a plane in an n-dimension space that divides our data in its classes.

Also, the best plane will be the one that has the highest margin of error, thus choosing correctly with higher probability.

One can define a plane in $\mathds{R}^n$ like:

\begin{equation}
    \vec{n} \cdot (\vec{r} - \vec{r}_o) = 0
\end{equation}

$\vec{n}$ is the vector normal to the plane, $\vec{r}$ is a random point and $\vec{r}_o$ is the vector that has the point $x_o = (x_1, x_2, \dots, x_n)$.

For instances, in 3D, calling now the normal to the plane $\vec{w} = (a,b,c)$ and $\vec{x} = (x,y,z)$ and representing vectors in bold:

\begin{gather}
    \vec{n} \cdot (\vec{r} - \vec{r}_o) = 0  \\
    \Leftrightarrow  \ a (x-x_o) + b (y-y_o) + c (z-z_o) = 0 \\
    \Leftrightarrow  \ a x + b y + c z + d = 0 \\
    \Leftrightarrow  \ \mathbf{w} \cdot \mathbf{x} + b = 0
\end{gather}

To compute the distance of a point to the hyperplane, we simply have to calculate the norm of the projection to the unitary normal vector.

\begin{equation}
    dist = \mathbf{x} \cdot \frac{\mathbf{w}}{||\mathbf{w}||}
\end{equation}

Note now that if we want the distance of the plane to the origin, calling $\mathbf{v}$ to our vector such that $\mathbf{v} = (x_1 - x_o, y_1 - y_o, z_1 - z_o)$, where $P_o$ belongs to the plane and $P_1$ is the point in question, and $\mathbf{n}$ to our unitary normal just so:

\begin{align}
    dist &= \left|v \cdot \frac{\mathbf{w}}{||\mathbf{w}||}\right| \\
    dist &= \left|(x_1 - x_o, y_1 - y_o, z_1 - z_o) \cdot \frac{\mathbf{w}}{||\mathbf{w}||}\right| \\
    dist &= \frac{\left|A(x_1 - x_o) + B(y_1 - y_o) + C(z_1 - z_o)\right|}{\sqrt{A^2 + B^2 + C^2}}
\end{align}

Note now that we need the point in the plane that is closer to the origin to calculate this distance. In fact is possible to find that point by intersecting with the plane a line directed with the normal to the place from the origin. This point however is not required for this analysis.
Also, note that if $P_1$ is the origin, and $P_o$ is the nearest point to the origin belonging to the plane, because it belongs to the plane $\mathbf{w} \cdot \mathbf{x_o} + b = 0$, so the expression degenerates in:

\begin{equation}
    dist = \frac{b}{||\mathbf{w}||}
\end{equation}

More generally:
\begin{enumerate}
    \item Is possible to calculate the distance of any point to the plane if we choose any point in the plane and get a vector from that point to the point the distance is required. This is because the plane is not defined uniquely by the orthogonal vector.
    \item A sign can be given to the distance $\frac{b}{||\mathbf{w}||}$. If it is below the plane, the distance is negative. If above it is positive. Before only the absolute value was considered, however if we consider w to always be the normal pointing up, we can choose this convention;
\end{enumerate}

This is where everything gets interesting:

\bb{This is called Support Vector Machines because they use Support vectors that are the points that are closer together.} The amount of support vectors required will depend on the dimensions, in 2 dimensions, 2 points are enough to define a plane of equal distance to both. In 3 dimensions 2 points are not enough because there's a line of points of equal distance and infinite planes can pass through that line. Therefore, 3 points are required. N dimensions, n points to define the problem. Actually, these points will be used to define $\mathbf{w}$. The value of $b$ will be set with the margin between the hyperplanes!

Two hyperplanes can be defined as:

\begin{equation}
    \mathbf{w} \cdot \mathbf{x} + b = \pm \delta
\end{equation}

The actual value of $\delta$ won't matter because it won't be more than a factor of scale.

The mid way between the two hyperplanes will be the boundary. And the decision will be based on what side of the boundary the point lies in.

\quickimage{AIML/AA-005.png}{0.6}

\bb{We want to maximise the distance between the two planes in order to have the boundary as far from the support vector as possible.}

Also, may be it is a good idea to take more than the $n$ closest points because those might generate a plane that doesn't divide the data properly. But this can be taken into account later. First, what is the margin between the two described planes and how to maximise it?

Note that since the hyperplanes are parallel, the constant will be the only thing moving them along the normal direction and the distance between them can certainly be deduced directly from that constant.

The constants of both planes are $b \pm \delta$. To calculate the distance between them, is doing $\frac{\left| D_1 - D_2 \right|}{||w||}$. This formula can be explained by the calculation of the difference of both planes' distance to the origin. Note now that the $b$ will be $b-\delta$.

Therefore, we get:

\begin{equation}
    d = \frac{2 \delta}{||w||}
\end{equation}

And to maximize this distance is necessary to minimize $||w||$, with the a constraint per training sample:
\quickimage{AIML/AA-010.png}{0.6}

Constraint optimization is a problem for the Lagrange multipliers. See Section \ref{sec:Lagrange} to see how to apply the method carefully.

The first application gets us to the primary Lagrangean function:

\quickimage{AIML/AA-011.png}{0.6}

Then, by doing the gradient and replacing it in the above expression:
\quickimage{AIML/AA-012.png}{0.6}

Note that this is a problem dual from the first one. We are still solving the same optimization problem but with this substitution, the final expression to optimize depends only on the dot inner product of $x^{(i)}$ and $x^{(j)}$ which will be very important for the \bb{kernel trick}. Also, with the formulation below (after some simplifications) it's possible to use a \ul{Converx Quadratic Programming (QP) Solver}. See the end of Section \ref{sec:Lagrange} for more info on this Duality.

\quickimage{AIML/AA-013.png}{0.6}

In conclusion, with $f(x) = x cdot w + b$, the SVM provides more than a decision, a score, a certainty associated with that decision.

\subsubsection{Soft Margin - Slack Variables}
This was performed for an hard margin in linear separable data. If the data is not linear separable, either some \bb{Kernel Trick} is performed or some slack is added to account for variables in the other side of the boundary. Slack variables are the penalties that will be added to points in the wrong side of the boundary and thieir sum should be minimised, i.e $C \sum_{i=1}^n \epsilon_i$, where C is the scale of the penalty.

\quickimage{AIML/AA-014.png}{0.6}

\subsubsection{Non-Linear SVM \& The Kernel Trick}

In higher dimensions the data may be separable, but in the current one it may not be.

There are mappings/transformations to increase the feature space dimensionality. With higher dimensions and more complex features that are the weird combinations of the current ones, is possible that there's a separation.

\begin{equation}
    \tilde{\mathbf{x}} = \phi(\mathbf{x}) =
    \begin{bmatrix}
    \phi_1(\mathbf{x})\\
    \phi_2(\mathbf{x})\\
    \vdots\\
    \phi_p(\mathbf{x})
    \end{bmatrix}
\end{equation}

The kernel function $\phi(\mathbf{x})$ will map n features to p, where (usually at least) $p \geq n$.

The biggest problem is that this high dimensionality may drive the problem infeasible since that each sample will have much more information. And the process of mapping all samples, apply the SVM and them map the boundary back to the first feature space is quite time consuming.

Instead of that, since the dual formulation just requires the inner product of features, is possible to define a kernel function that is the product of the two mappings and compute the inner products in the low dimension input space instead of really having to climb the dimensionality ladder.

\quickimage{AIML/AA-015.png}{0.6}

The typical kernels:
\quickimage{AIML/AA-016.png}{0.6}

\subsection{Decision Trees \& Random Forests}

Only used for classification problems.
It's a technique used with discrete features - this does not limit it's usage because continuous features can be discretized in a finite number of intervals.

It works by figuring out what is the best question to ask in each node (the question that separates better que labels) and when the sample reaches que bottom of the tree

A measure that is used to know which variable to ask about is the impurity. There are multiple impurity functions.

\quickimage{AIML/AA-017.png}{0.6}

The impurity drop is what is used to choose which feature to use at a certain node.

\subsubsection{ID3 algorithm}

Impurity criterion is entropy. And stops either when every leaf is pure or if all attributes (features) have been tested along the path to the root.

If the data or attributes are noisy (there are multiple instants with different results for supposedly the same label or attributes), then ID3 may overfit because it will ask enough questions to properly and thoroughly separate the data. One way of avoiding this is to \bb{stop early} (when the impurity drop gets below a threshold) or do \bb{post-processing pruning}.

Note that the first method may provide problematic solutions because the impurity droop in a certain iteration may be smaller than it will be in the next one, therefore the algorithm would stop before checking the tree properly.

\subsubsection{Pruning}

Simplifying the tree improves it's performance since ``less questions'' need to be asked on each sample evaluation. Additionally, pruning can lead to the exact same classification errors, therefore is very worth to do.

The actual way of implementing pruning is by testing splitting notes - not worth to check the leafs - in a bottom-up way. For each tested node, replace it by a leaf with the most likely result of the subtree. Then check if the number of classification errors improved. If it improved or stayed the same, then that simplification is viable.

\subsubsection{Boostrapping with aggregation (Bagging)}
To understand how random forests work, one needs to look at \bb{Bootstrapping aggregation} first, also called \bb{bagging}.
Given an $m$ sample training set, one samples it $m$ times with replacement, and repeats the process B times. This way, B training sets can be created from the original one.

\quickimage{AIML/AA-018.png}{0.6}

\bb{Bootstrapping is sampling with replacement}.

The ``with aggregation'' parts refers to final part of taking the average, i.e aggregating the results into one.

This allows us to reduce the uncertainty of $f(x)$.

\subsubsection{Random Forest}

A random forest is using multiple random trees, each with a subset of features, in order to have approaches as distinct as possible in order to avoid errors of noisy labels. Each tree is trained with one of B sets of training data achieved by Bootstrapping. Then each tree is used for the estimation, and the most popular final result is considered the correct one.

The random selection of features is called \bb{Random Subspace} and the actual percentage of features that will be used in each tree/each splitting node is a algorithm parameter.

\subsection{Leading with data anomalies - Data imbalances}

\href{https://towardsdatascience.com/handling-imbalanced-datasets-in-machine-learning-7a0e84220f28}{\ul{Good link}}

If the data is imbalanced, for instances: 99\% of data is class A and only one percent is class B, it is possible to find a trivial classifier that has 99\% accuracy in this dataset by only guessing A. This is very bad as the outliers will never be found.

There are two ways of proceeding:
\begin{itemize}
    \item Reworking the dataset;
    \item Reworking the problem - (spoiler: this is the solution)
\end{itemize}

First of all, one has to identify a naive behaviour. This can be done with a confusion matrix, computing precision, recall and F1 measure.

\bb{``}The precision of a class define how trustable is the result when the model answer that a point belongs to that class. The recall of a class expresses how well the model is able to detect that class. The F1 score of a class is given by the harmonic mean of precision and recall ($\frac{2\times precision \times recall}{precision + recall})$, it combines precision and recall of a class in one metric. For a given class, the different combinations of recall and precision have the following meanings :

\begin{itemize}
    \item high recall + high precision : the class is perfectly handled by the model
    \item low recall + high precision : the model can’t detect the class well but is highly trustable when it does
    \item high recall + low precision : the class is well detected but the model also include points of other classes in it
    \item low recall + low precision : the class is poorly handled by the model
\end{itemize}\bb{''}

Note: recall is also called True Positive Rate (TP or TPR). Precision can be obtained from TP and False Positive Rate. Mathematically:
\begin{itemize}
    \item Recall = True positive rate = $P(\hat{y} = T | y = T)$
    \item False positive rate = $P(\hat{y} = T | y = F)$
    \item Precision = $\frac{\text{TP}}{\text{TP}+\text{FP}}$
\end{itemize}

Additionally Received Operating Characteristic (ROC) is something common to compute for each class. The Area Under ROC is an objective measure how good the performance is - the closer to 1 the better, the closer to 0.5 the worst (a cube with side 1 has area 1 and the random guess is a diagonal, thus area 0.5).

\quickimage{AIML/roc.jpeg}{0.12}

\subsubsection{Working the problem}

The alternative to this would be to work the dataset. To undersample the most common (throw away some samples), oversample the least common one (replicate samples) or some more fancy stuff like SMOTe (search...). All of these try to rebalance the dataset which will show the wrong proportions of data to the classifier. Therefore it will be more likely to guess the least likely class but at a considerable cost of guessing correctly the common one, specially if the data is not separable.

A better way is enriching the dataset with additional features. If we find a new feature that helps distinguish the classes, the classifier can be more accurate.

\section{Machine Learning - Unsupervised Learning}

\section{Reinforcement Learning}

\subsection{Introduction}

An agent to make the wisest decision in its situation has to have knowledge on what state he is in, how the environment will evolve, how it will be like if he performs a certain action (taking into account stochastic environments) and a utility function to  know how much that state contributes to its happiness.

\quickimage{AIML/RL-001.png}{0.6}

\quickimage{AIML/RL-002.png}{0.6}

The state is the information used to determine what to do next.
The environment state $S_t^e$ is how the environment uses the history to choose the reward and the observation based on the action chosen by the agent.
The agent state $S_t^a$ is how the agent takes the history and chooses what action to do next.

\subsubsection{Markov Property}

When a state is a sufficient statistic for the future, i.e the next state only depends on the previous state and not on the complete history, then it's said to be a \bb{Markov State}. In other words, a Markov State is a State that has all useful information about the past.

\quickimage{AIML/RL-003.png}{0.6}

Therefore, we can thrown away all the previous states because the current state has all the information I need to know the probability of the state of the future.

Also, what our representation of state is will influence what the agent will think about the outcome. If the state is the last 3 items, then shock, if it's the count, then cheese. Or it may even be impossible to tell from our state what will happen next.
\quickimage{AIML/RL-004.png}{0.6}

\subsubsection{Observability}
When the agent fully observes the environment, then $O_t = S_t^e = S_t^a$, this is known as a \bb{Markov Decision Process}.

Partial observability is when the environment is more than what the agent can see. This is very often the case in the real world since it's hard to get the complete information about the environment. In this case, the agent must construct it's own representation of the state. That construct may be estimated with a \ul{Recurrent Neural Network}.

\subsubsection{Agent Components}

\quickimage{AIML/RL-005.png}{0.6}

The less obvious is the value function. It's a prediction of the future rewards, not necessary only in the next state but long term future can be included as well. It's used to evaluate how good a state is.

From the policy, it should choose an action $a = \pi(s)$:

\quickimage{AIML/RL-007.png}{0.6}

The value function will put a value in the state that each action will lead to:

\quickimage{AIML/RL-006.png}{0.6}

\quickimage{AIML/RL-008.png}{0.6}

The model is something that predicts the next state or the next immediate reward given a certain action.

\quickimage{AIML/RL-009.png}{0.6}

\subsubsection{Agent Categories}
\begin{itemize}
    \item Value Based - only has value function (the policy is implicit)
    \item Policy Based - only has policy
    \item Actor Critic - has both policy and value function
    \item Model Free -  Policy and/or Value function, no model
    \item Model Based - Policy and/or Value function, with model
\end{itemize}

In one image:
\quickimage{AIML/RL-010.png}{0.6}

\subsubsection{Exploration vs Exploitation}
One may set some rules to make the choice much more straightforward for the machine, but also letting it learn by itself can be interesting. This is the difference between Planning and Learning. One exploits useful information to maximise the reward when the other find more information about the environment.

\bb{Something important in RL is how to balance Exploration and Exploitation. If we know some rules, we can simply program something that heuristicly would give a good result. However, exploring unknown strategies and simply let the machine learn by itself can also provide a very good insight on the game and even return higher rewards.}

Therefore, this balance is key.

Is it worth to give up some reward only to know more about the environment? This tradeoff is unique to RL.

\subsection{Markov Decision Processes}


\section{Finite field in depth}

\subsection{Preliminaries}

\begin{frame}[t]{Group}

	An group \cite{Paar2010understanding} $(G, +)$ consists of a set $G$ and an operation defined on its elements, here denoted by $\circ$:
	\[\circ: G \times G \to G : (a, b) \to a \circ b = c, c \in G\]

	A group has the following properties:

	\begin{enumerate}[i.]
		\item \textbf{Closed}: $\forall a, b \in G: a \circ b \in G$
		\item \textbf{Associative}: $\forall a, b, c \in G: (a \circ b) \circ c = a \circ (b \circ c)$
		\item \textbf{Neutral element}: $\exists 0 \in G, \forall a \in G: 0 \circ a = a$
		\item \textbf{Inverse elements}: $\forall a \in G, \exists b \in G: a \circ b = 0$
	\end{enumerate}

\end{frame}

\begin{frame}[t]{Group Examples}

	Some easy examples of groups are:

	\medskip

	\begin{enumerate}[i.]
		\item $\left( \mathbb{Z}, +\right)$
		\item $\left( \mathbb{Z}_m, +\right)$ with:
			\begin{itemize}
				\item $Z_m = \left\{z \in \mathbb{Z}: z < m \right\}$
				\item $+: G \times G \to G$ is $+(a, b) = a + b \mod m$
			\end{itemize}
	\end{enumerate}

\end{frame}

\begin{frame}[t]{Abelian Group}

	A group \cite{Paar2010understanding, Menezes2012handbook} $G$ is abelian if the operation $\circ$ is abelian (commutative).

	\bigskip

	So, the properties would be written as:

	\medskip

	\begin{enumerate}[i.]
		\item \textbf{Closed commutative}: $\forall a, b \in G: a \circ b = b \circ a = c, c \in G$
		\item \textbf{Associative}: $\forall a, b, c \in G: (a \circ b) \circ c = a \circ (b \circ c)$
		\item \textbf{Neutral element}: $\exists 0 \in G, \forall a \in G: 0 \circ a = a \circ 0 = a$
		\item \textbf{Inverse elements}: $\forall a \in G, \exists b \in G: a \circ b = b \circ a = 0$
	\end{enumerate}

\end{frame}

\begin{frame}[t]{Ring}

	A ring \cite{Rijndael2020design} $(R, +, \times)$ consists of a set $R$ with two operations defined on its elements, here denoted by $+$ and $\times$. For $R$ to qualify as a ring, the operations have to fulfill the following conditions:

	\medskip

	\begin{enumerate}[i.]
		\item The structure $(R, +)$ is an Abelian group.
		\item The operation $\times$ is closed, and associative over $R$. There is a neutral element for $\times$ in $R$ usually denoted by $1$.
		\item The two operations $+$ and $\times$ are related by the distributive law: $\forall a, b, c \in R : (a + b) \times c = (a \times c) + (b \times c)$.
	\end{enumerate}

	\medskip

	A ring $(R, +, \times)$ is called a commutative ring if the operation $\times$ is commutative.
\end{frame}

\begin{frame}[t]{Field}
	A field \cite{Rijndael2020design, Paar2010understanding} $F$ is a set of elements with the following properties:

	\medskip

	\begin{enumerate}[i.]
		\item The structure $(F, +, \times)$ is a commutative ring:
			\begin{itemize}
				\item All elements of $F$ form an additive group with the group operation $+$ and the neutral element $0$.
				\item All elements of $F$ except for the $0$ form a multiplicative group with the operation $\times$ and the neutral element $1$.
			\end{itemize}
		\item For all elements of $F$, there is an inverse element in $F$ with respect to the operation $\times$, except for the element 0, the neutral element of $(F, +)$.
	\end{enumerate}

\end{frame}

\begin{frame}[t]{Finite fields}{Definition}
	A field $F$ is a \textbf{finite field} (or Galois Field) if the number of elements ($|F|$) is finite.

	\medskip

	\begin{itemize}
		\item The number of elements in the set is called the \textbf{order} of the field.
		\item A field with \textbf{order} $m$ exists iff $m$ is a prime power. This is, \[m = p^n\] with $n \in \mathbb{Z}$ and $p$ a prime number.
		\item $p$ is called the \textbf{characteristic} of the finite field.
	\end{itemize}

	\bigskip

	Finite fields used in the description of Rijndael have characteristic $2$.
\end{frame}

\begin{frame}[t]{Finite fields}{Remarks}
	\begin{itemize}
		\item The previous definition implies there are finite fields with $11$ elements or with $169$ elements because $169 = 13^2$. But not all $orders$ are possible. For example, there's no finite field with $24$ elements because $24=3*2^3$.
		\item The most intuitive examples of finite fields use $p=1$ and are called prime fields. In fact, $GF(2)$ plays an important role in Rijndael.
		\item The notation from now on for a Galois Field will be $GF(p^n)$.
	\end{itemize}
\end{frame}