\acresetall
\chapter{Artificial Intelligence}\label{ch:ml_intro}

\section{Overview}
\emph{Intelligence} as a concept has been a topic of exhaustive research in fields such as neurology, philosophy, neuroscience, neurobiology, and data science, among others. The Oxford dictionary defines intelligence as \emph{"the ability to acquire and apply knowledge and skills"} \cite{Oxforda}. The first part of this definition applies to what is known as "learning", which is according to the accepted definition of the term as well \cite{Oxford}, and that supports, from the etymology, the importance of the process of learning on intelligence.\\

Jeff Hawkins, a dedicated neuroscientist and author, has approached the subject from the engineering and medical flanks, analyzing the structure of the brain and having the perspective of the possibilities of replicating artificially the most sophisticated type of intelligence found on Earth: the human. In his book \emph{On Intelligence} \cite{HawkinsJeff2004}, he captures his findings after inspecting the brain cortex and making a parallel between humans and machines. According to Hawkins, \emph{"it is the ability to make predictions about the future that is the crux of intelligence"}, and these predictions are based on the experiences from which the intelligent being has learned, making decisions that lead it to the best possible known result. In order to create artificially a so-called \emph{intelligent agent}, scientist have put extensive efforts first on trying to replicate the known human intelligence \cite{Brooks1991}\cite{Reed2007}\cite{Hawkins}, taking the approach of generating a machine that is human-like and that behaves like one, being able to observe its surroundings, learn from stimulus that comes from the real world, adapt to changes, plan accordingly to foreseeable process (therefore, make predictions), make decisions and act appropriately. These are the characteristics that Mitola \cite{Mitola1999} described in the cognition cycle for \ac{CR}, which can be applied to any intelligent agent and, consequently, motivate the further research of \ac{AI}.\\

\ac{AI}, however, encircles a variety of disciplines that are in themselves a complete course of research, as it can be seen in Fig~\ref{fig:ai}. This work focuses only on the top branch: \ac{ML}. However, given the slight differences regarding implementation, a separate section will be dedicated solely to \ac{DL}.

\begin{figure}[htb]
    \centering
      \includestandalone[width=\textwidth]{figures/ai_tree}
      \caption{Artificial Intelligence}
      \label{fig:ai}
\end{figure}

\section{Machine Learning}
\ac{ML} encloses the process of taking a data set that represents any phenomena and learning from it. Any type of being that is capable of learning from previous experiences is showing a kind of intelligence, as it interiorizes the stimulus/data and reacts accordingly when it presents itself again. The vast majority of living beings have this capacity, being the humans who have the lead on its effectiveness. Identifying objects, speaking languages, and reacting to any sensorial stimulus is a result of a successful learning process. \\

Generally speaking, learning from data is done when there is no analytic solution to an encountered situation, but there is enough data to adapt to it, generating an empirical solution to a problem that cannot be mathematically a-priori described, but that follows a specific pattern\cite{Yaser}. Just as humans do, the idea of \ac{ML} is to generate intelligent agents computationally - teach computers to learn. The idea is as follows: a \ac{ML} algorithm is given a set of data from which it can extract specific information that tells it the specifics about the data. With enough information, the computer is able to make predictions about other data in a different point in time if this data presents the same characteristics.

Although there is no specific mathematical representation of the specific problem to solve, many \ac{ML} algorithms rely heavily on mathematical definitions and optimization theory. Further information regarding \ac{ML} algorithms can be found in section~\ref{ch:ml_algs}. The versatility provided by the fact of not needing to pin down the specific analytic description of the problem is what has allowed this methodology to be applied in several fields of knowledge, being nowadays used to solve problems in areas such as financial forecasting\cite{Bose2001}, medical diagnosis\cite{Kononenko2001}, entertainment\cite{Bennett2007} and communications systems (such as this thesis), among others. Examples of everyday problems that are suitable for \ac{ML} implementation are:

\begin{itemize}
    \item Ranking links and clicks for a better web search engine and advertisements.
    \item Custom user recommendations based on purchases/rents/views.
    \item Prediction of markets and stock exchange.
    \item Dating sites with a reevaluation of algorithms based on successful matches.
    \item Financial fraud detection.
    \item Supply chain optimization
    \item Biotechnology research acceleration by sequencing and screening of DNA and protein/compound structures.
    \item National security based on enormous surveillance data.
\end{itemize}

\subsection{The learning problem}
Learning from data is definitely a hot topic, which can be seen from the increasing amount of research and application that has been handed over this theory and methodology. Additionally, it is noticeable how the term has been capturing the mainstream interest and is somewhat heard-of, as it can be seen in the Fig.~\ref{fig:ml_trend}, where this trend over the past few years is clear. At this point, it is preeminent to clarify what is the purpose of \ac{ML}, and when it plays an important role. Although \ac{ML} has shown to perform outstandingly into solving many problems, it is not intended to move aside the many and well-designed analytic solutions for many of the scientific existing problems, but to come in handy when that analytic solution does not describe completely the problem, or does not exist. In his book \cite{Yaser}, Prof. Yaser nicely states that although many problems can be solved effectively using a learning approach or an analytic approach, the point of learning is not to compare itself and overcome the performance over the mathematical description of existing problems, but to be a complementary tool for scientist in their eagerness to solve complex problems without being stuck when facing the lack of a complete description of it.

\begin{figure}[htb]
    \centering
      \includegraphics[width=0.6\textwidth]{figures/ml_trend.png}
      \caption{'Machine learning' Google search trend \cite{GoogleInc.2017}}
      \label{fig:ml_trend}
\end{figure}

The learning problem is summarized in Fig~\ref{fig:learning_problem} and follows the theoretical description provided at \cite{Yaser}. The learning algorithm \( \mathcal{A} \) receives data of any form, and its sole purpose is to identify mechanisms that describe that dataset \( \mathcal{D} \) closely. This dataset is defined as input-output samples for the supervised learning, input-weights samples for reinforcement learning and only inputs for the unsupervised learning. Further information regarding supervised, reinforcement and unsupervised learning can be found in section~\ref{ch:learning_types}. For the sake of the explanation, let us take the supervised learning case, where the dataset  \( \mathcal{D} \) includes samples of the form \((\mathbf{x}_1, y_1),\cdots , (\mathbf{x}_N, y_N) \), where \(x\) is the input that belongs to the input space \( \mathcal{X} \), \(y\) is the corresponding output such that \(y_n = f(x_n)\), and belongs to the output space \( \mathcal{Y}\). Now, the learning algorithm \( \mathcal{A} \) needs to find that function \(f(x)\). For this, it employs a hypothesis set \( \mathcal{H} \), which are the mathematical representations that the algorithm uses as tools to accomplish its purpose. From \( \mathcal{H} \) the algorithm takes one hypothesis \( g:\mathcal{ X \rightarrow Y} \) that approximates \(f\). After a \(g\) has been selected, the process estimates how alike the outputs from \(g(x)\) are to \(f(x)\), and feedbacks an error measure \(E(g,f)\). This process is repeated iteratively until a hypothesis produces an acceptable minimum error. \\

\begin{figure}[htb]
    \centering
      \includegraphics[width=0.6\textwidth]{figures/learning_problem.png}
      \caption{The learning problem \cite{Yaser}}
      \label{fig:learning_problem}
\end{figure}

Now, it is imperative to determine quantitatively what exactly \emph{acceptable} means. Different applications can have different tolerances to error, and this affects directly the hypothesis \( g \) that is chosen at the end of the learning process. This means that this is an end-user parameter that has to be set as a requirement for the whole system.

After taking different samples from \( \mathcal{X} \), and reducing the error, we should take into account the samples that \emph{we did not take}, meaning the samples that are not in our input set, and that probably behave similarly to the \( \mathcal{X} \) set - i.e. we should be able to identify similar samples in order to be able to predict. Therefore, a probability distribution is added to both the input samples and the final hypothesis in order to infer from \( \mathcal{D} \) the behavior of samples that are not in \( \mathcal{D} \). Let us assume a binary classification problem, where \( \mathcal{D} \) contains two classes, A and B, in a possibly infinite number. From a random sample pick the probability that the input sample is of the A type will be denoted as \(\mu\) and, consequently, the probability that the input sample is of the B type is 1 - \(\mu\). The value of \(\mu\) is unknown and will continue to be unknown in the process. From a random pick of N samples from \( \mathcal{D} \), there is a proportion of \(\nu\) samples of the A-type, and we intend to determine how \(\nu\) relates to \(\mu\). In statistical jargon, we want to determine how our sample relates to the whole population. \\

From any point of view, the larger the sample that contributes to \(\nu\), the closer the relation it has with \(\mu\), but this relationship can be quantified using the \emph{Hoeffding Inequality} \cite{Hoeffding1963}, which states that for any sample size \(N\),

\begin{equation}
  \mathbb{P}[|\nu -\mu|>\epsilon ]\leq 2e^{-2\epsilon^{2}N} \quad \text{for any}\quad \epsilon>0
\end{equation}

Where \(\mathbb{P}[\cdot]\) denotes the probability with respect to the chosen sample, and \(\epsilon\) can be any positive value chosen by the data scientist, and represents the tolerance of error. This inequality says, simply put, that as the sample size \(N\) grows, it is \emph{exponentially unlikely} that the realization \(\nu\) deviates from \(\mu\) by more than the tolerance \(\epsilon\). It can be clearly seen that only the size of the sample \(N\) affects this bound. Consequently, to achieve a small tolerance \(\epsilon\), a large \(N\) has to be used.\\

Additionally, it is important to take into account the intrinsic noise of the systems, which can come from the nature of the input samples, i.e the samples are not product of a deterministic system, or are immersed into some stochastic variation that can be modeled by noise, and that implies that by having the same input into the system \emph{it is probable} to get a different output. This entails a change in the labels of the model. That is, instead of having \(y = f(x)\), we take \(y\) as a random variable resulting from a probability distribution \(P(y|x)\). Accordingly, the input data points are therefore generated by the joint distribution \(P(x,y) = P(x)P(y|x)\). With this description of the model, our target function (what \(\mathcal{A}\) wants to learn) becomes \(P(y|x)\), while \(P(x)\) quantifies the importance of the input \(x\) in our learning accuracy.

\subsection{Types of learning}\label{ch:learning_types}

There are three types of learning: supervised-, unsupervised-, and reinforcement learning. Each of them has specific characteristics, which are explained in the following subsections.

\subsubsection{Supervised learning}
Is the type of learning where, in addition to the input dataset, the explicit correct outputs for those given inputs are given to the \ac{ML} algorithm for training. There are two types of supervised learning:
\begin{itemize}
    \item \textbf{Classification:} consists in predicting a \emph{class label} from a determined set of choices. If the number of choices is two, the model corresponds to a \emph{binary classification}. As it has only two options, it is suitable for the problems whose expected answer is of the form "yes/no", "present/not present", "valid/invalid". For a greater number of classes, the model corresponds to a \emph{multiclass classification}. \cite{Andreas} Provides a simple list of examples of classification:
        \begin{itemize}
            \item Determining whether an email is spam or not constitutes a binary classification problem.
            \item Identifying the zipcode from handwritten digits on an envelope is a multiclass classification problem.
            \item Determine whether a tumor is benign based on size and shape data constitutes a binary classification problem
        \end{itemize}
    \item \textbf{Regression:} consists in predicting a continuous behavior, such as a trend, or a floating-number, and it is this continuity what sets it apart from the classification models. Examples of regression \cite{Andreas} are:
        \begin{itemize}
            \item Predict the value of the stock market
            \item Determine the expected amount of crops yield from a plantation based on data such as previous yields, weather history, etc.
        \end{itemize}
\end{itemize}

\subsubsection{Unsupervised learning}
Unlike supervised learning, in unsupervised learning the algorithms are feed with data but not with the expected outputs, which make this type of learning suitable for solving problems to which the output is unknown. The model is then in charge of extracting knowledge from the input data all by itself, without no further instructions. There are mainly two types of unsupervised learning that can be found in the literature \cite{Andreas}, which are the \emph{unsupervised transformations} and \emph{clustering}.

\begin{itemize}
    \item \textbf{Unsupervised transformations:} consists in creating new representations of the input data, so that it becomes easier to understand and/or to use than the original data. This functionality is used, for example, to reduce the dimensionality of data that consists of several features. In such a situation, the model transforms the data into a representation that summarizes the input with fewer features. Another important use of this type of models is finding the overall representation of the input data, such as the topic of a full text, or the sentiment in a short comment.
    \item \textbf{Clustering:} this kind of models collects the data into determinate groups that share similar characteristics. This is used, for example, to generate suggestions based on previous purchases/views, or to group pictures from a directory with several images to the ones that contain certain people (and suggest tagging the names on them).
\end{itemize}

As the models do not know beforehand what type of information they are intended to learn, one of the tasks of the data scientist is to assess that the model is indeed learning something useful. This creates the opportunity for this sort of models to be used for the same data scientist to help them identify certain characteristics of the data that were not obvious for the \emph{human-eye}, and certainly get a different perspective of the data from the \ac{ML} model point-of-view.

\subsubsection{Reinforcement learning}
This type of learning has a different approach to the previous two descriptions. Just like unsupervised learning, the model does not receive the expected outputs for the inputs it is given but, in contrast, it receives some \emph{possible} output along with a weight that states how good of an output it is. The idea behind this is that the model is then penalized when it provides a solution that is not according to the possible output and is rewarded when it is. Then, the model uses these penalizations and rewards to adjust the type of outputs it generates, and so it eventually learns the correct behavior for the situations it has been immersed into. This type of learning is similar to the way humans learn, in ways such as being penalized with pain when taking a sip of very hot coffee, or rewarded when winning a game of chess.\\

In that same manner, reinforcement learning comes in handy in teaching an intelligent agent how to play a game, where it is presented to a plethora of options (which makes it difficult to be modeled as a supervised learning problem) and has to choose the one that brings it near to victory. The most recent example is AlphaGo \cite{Fu2017}, an intelligent agent capable of winning against the World Champion in a Go game. A similar approach has been followed by IBM with the Deep Blue chess-playing machine \cite{Hsu1999}.

\subsection{Training \& testing Models}
Training and testing are concepts that, when applied to learning algorithms, are not far away from the definitions that are used in common language when applied to humans. In general, \emph{training} a learning model consists of supplying it with enough data from which it can extract information and create generalizations, and testing consists of providing similar data to the model and identify if the generalizations apply. "Generalization" in this context means inferences, decisions or choices that the model makes on data it has never seen before based solely on the data it has learned from, and clearly, the goal of \ac{ML} is to have models that are able to generalize as accurately as possible. However, all learning scenarios depend on data, and this data, being as extensive as it can be, does not contain all the information that describes its surroundings completely. Therefore, the question arises: how can it be determined if the model can indeed learn to generalize from data?\\

The way to tackle this problem is separating the data into two: A portion of it to be used for training, and the rest of it to be used for validation/testing. Being the training set and testing set generated from the same whole dataset, it is expected that they share enough characteristics for the model to perform well at generalizing. However, there are cases when the model is too complex, and instead of learning to generalize it \emph{memorizes} the training set, and then performs poorly on the training set and on any unknown data. This phenomenon is known as \emph{overfitting} because the model is fit intimately to the training set as is not able to generalize on new data. That being said, one might think that creating the simplest learning model will never end up in overfitting, which is absolutely true. However, in such cases, the model might not be able to even capture the descriptive characteristics of the data, and ends up being incapable of learning from it. This phenomenon is known as \emph{underfitting}. Fig~\ref{fig:model_complexity} relates the complexity of the model with its capability of generalizing from new data.

\begin{figure}[!htb]
    \centering
      \includegraphics[width=0.6\textwidth]{figures/model_complexity}
      \caption{Model complexity vs. Training and test accuracy \cite{Andreas}}
      \label{fig:model_complexity}
\end{figure}

Therefore, it is clear that the objective model is not too simple or too complex, but somewhere in between; what \cite{Andreas} legitimately calls "the sweet spot", where the model has learnt as much as it can from the provided data, and is still able to create accurate inferences from unknown samples, ergo make predictions.

\subsection{Influence of dataset size in the learning process}\label{ch:size}
It has been constantly repeated that having data is paramount, but "how much data is required?" is a question that will always be asked, and that, unfortunately, has no concrete answer. The logic answer to it is "use as much data as possible", which also follows the Hoeffding inequality presented earlier in this chapter. Data is limited and generally comes along with the description of the problem itself. What this means is that, in data science, it is common for the learning problem to be stated as "The problem to solve is X, and the data available for it is Y" \cite{Yaser}. Asking for more data is, in most cases, impossible, and the data scientist has to work with what it has. However, the way the scientist extracts information from the data could be the lever that turns a failing model into a successful one.

Moreover, the model complexity is tied to the amount of information that the dataset holds; i.e. if the dataset has considerable variety within, the model can allow itself to grow in complexity without overfitting. This variety usually comes intrinsically with larger datasets, but the clarification has to be made: it is not the size of the dataset what directly yields to variety, meaning that duplicating samples won't help, as repeated data provides no increase on entropy.

\subsection{Data preprocessing}
Possibly the data that is measured or handed over to solve a problem does not provide much information \emph{as is}. However, after some careful treatment, that data might be transformed in the specific characteristics that allow an accurate generalization. The process of extracting specific \emph{features} that describe the information contained in the \emph{raw} dataset is called \emph{feature extraction}. It has been said earlier that unsupervised learning can be used for this purpose, using techniques such as \emph{Principal Component Analyzis (PCA)}, which retrives uncorrelated data and transforms the dataset component-wise in order to determine the information contribution of this component to the whole dataset. Other uses for unsupervised learning in data preprocessing are multiclass visualization, dimensionality reduction and generally finding a representation of the data that is more informative for later processing.\\

Additionally, feature extraction can be done manually, where the data scientist analyzes the data and produces a new dataset with a more representative description of it. This \emph{manual} method is also called \emph{feature engineering}, and was used by the \ac{CEL} in the DySpan spectrum challenge. Those extracted features are used in this work. A description of the implemented feature extraction process can be found in section~\ref{ch:features}.

Other type of preprocessing the data is \emph{scaling}, which comes in handy when samples of different orders of magnitude, belonging to different classes, are compared. This can be done, for example, by removing the mean of all the samples and scaling them to unit variance. Another way of scaling is setting a minimum and a maximum value, and fit all the features inside that range.

\subsection{Model Evaluation}
There are multiple ways to evaluate the performance of \ac{ML} algorithms, and they are helpful to determine if the model is generalizing from data satisfactorily. In this work, we focus on multiclass classification, and one of the most thorough tools for model evaluation is the confusion matrix. When a confusion matrix is calculated for a multiclass classification problem, a square matrix with dimensions equal to the number of classes is generated. Fig~\ref{fig:conf_matrix} shows the confusion matrix configuration for a binary classification, where the classes are "positive" and "negative". Each cell of the matrix contains the number of samples from the testing set that was classified to that specific class. This means that the diagonal of the matrix states the number of correctly classified elements.

\begin{figure}[htb]
    \centering
      \includegraphics[width=0.4\textwidth]{figures/cm}
      \caption{Confusion Matrix}
      \label{fig:conf_matrix}
\end{figure}

Some applications have to regard specifically the type of misclassification that could happen in order to focus efforts into mitigating it. Prof. Yaser \cite{Yaser} exemplifies this with a fraudulent credit card transaction. If a fraudulent transaction is not happening, meaning that the rightful owner of the credit card is make use of the services he/she is entitled to, but the transaction is labeled as fraudulent, this has little negative consequences when compared with the situation in which a fraudulent transaction is indeed happening but is labeled as an honest one.

In this work, misclassification affects equaly the final output regardless of its type.
\subsection{\ac{ML} algorithms}\label{ch:ml_algs}
\subsubsection{K-nearest Neighbors}
This is one of the simplest \ac{ML} algorithms, and consists solely of making decisions based on the characteristics of the points that surround the point under test, i.e. the "nearest neighbors". It can be used for both supervised and unsupervised classification as well as for regression. The distance with which the "nearest" point is determined can be any metric measure such as:
\begin{itemize}
    \item Euclidian distance between any two points \( \sqrt{\sum (x - y)^2} \)
    \item Manhattan distance \( \sum {|x - y|} \)
    \item Chebyshev distance \( max(|x - y|) \)
    \item Minkowski distance \( \sum(|x - y|^p)^{1/p} \)
\end{itemize}
among others.

This algorithm is commonly called \emph{non-generalizing} because it remembers the location of its training data points and checks the proximity of its points to every other new data to predict. However, even though this procedure seems as "memorizing", this algorithm is not prone to overfitting and can lead to satisfactory results in datasets of various sizes, and because of its easiness to understand and apply, it is a good choice as a starting point for tackling a learning problem before applying more elaborated models. On very large datasets, or on datasets of very high dimensionality (several hundreds of classes) the amount of distances calculations escalates, which might lead to long prediction times.

\subsubsection{Decision trees}
This model is based on simple boolean-like bifurcations (if-then) that build a hierarchy tree that eventually leads to a decision, which makes it an uncomplicated model easy applicable in classification and regression. The concept can be easily explained with an example: assume that we have four common elements that can be found in a lab, such as an external monitor/screen, a laptop, an office chair and a USB stick. The classification of such can be as simple as it is shown in Fig~\ref{fig:dt_example}. In this example, the features are "has a screen?", "has wheels?" and "has a keyboard?". Clearly, this example model can be extended for further classification of lab elements.

\begin{figure}[!htb]
    \centering
      \includegraphics[width=0.7\textwidth]{figures/dt_example}
      \caption{Decision tree for classification of four lab elements}
      \label{fig:dt_example}
\end{figure}

The complexity of the model is given by the number of questions that have to be asked before reaching a final decision, i.e. the depth of the tree. With a long depth, the questions become more specific, and only a few samples are able to fit the tree, which leads to overfitting. In \ac{ML}, the tree is generated automatically, so a good way to avoid overfitting is stopping the tree from growing more than it should, in a process called \emph{pre-pruning}, or removing branches that provide little to no information after the tree has been created, which is called \emph{post-pruning}.

Given the nature of the decision tree, preprocessing the data (with methods such as scaling) does not influence its performance in terms of accuracy, because each feature is processed completely independent of the others. Because of this reason, this model is recommended when the features do not fall into a common range or have completely different characteristics (such as a mixture of discontinuous and continuous features).

\subsubsection{Support Vector Machines}
Support vector machines (SVM) are an extension of the linear models that serve as a base for classification systems, such as the perceptron, which is explained more in detail in section~\ref{ch:neural}. Basic linear models are not regarded in this work because they are not explicitly used in the implementation described in following chapters, and the specifics of the mathematic description for the linear models and \ac{SVM} models is extensive and out of the scope of this thesis. Prof. Yaser provides an extensive explanation of perceptrons and on its use for the learning process \cite{Yaser}. In summary, a linear model sets a classification boundary that separates different classes. \ac{SVM} extends this functionality by introducing the concept of \emph{hyperplanes}, which ensures an improved classification by maximizing the separation of points of different classes by the means of the hyperplanes. This classification boundaries are shown in Fig~\ref{fig:svm} for two different classes.

\begin{figure}[!htb]
    \centering
      \includegraphics[width=0.5\textwidth]{figures/svm}
      \caption{\ac{SVM} - Hyperplanes separating two classes. Functional margin in red.}
      \label{fig:svm}
\end{figure}
The hyperplanes that generate the largest separation between training data points of different classes form a so-called \emph{functional margin}, which provides the smallest generalization error for the classifier. These models are said to have exceptionally good performance in high dimensional spaces, even when the number of classes is greater than the number of available samples \cite{SKLEARN}.

\section{Deep Learning}
Additionally, to the approach of learning from extracted features, deep learning is an approach based on learning data representations. This is a subgroup of \ac{ML} and was developed on top of a technique called Neural Networks, which is described briefly below. The term "Deep learning" was coined in 1986 by Rina Dechter \cite{Dechter}, in a proposed work to overcome a way to learn the maximum amount of information from techniques using backpropagation, making a parallel of learning and recording, and so when a so-called \emph{dead end} is encountered, the recorded information could guide iteratively to make new decisions.

\subsection{Neural Networks}\label{ch:neural}
Before getting deeper into the specifics of neural networks, it is precise to first define one small unit that is used to build the networks: the perceptron. A perceptron is a node that implements a very simple binomial linear classification. It maps the output based on an activation function applied to the input:

\begin{equation}
    f(x) =
    \begin{cases}
        1 \text{ if } w*x + b > 0\\
        0 \text{ otherwise}
    \end{cases}
\end{equation}

It can clearly be seen that the output of the perceptron is based on a linear equation of the form \(y=mx+b\), where \(w\) is a vector of weights, \(x\) is the input of the perceptron, and \(b\) is the bias. The weights set certain restrictions to the input data, and the bias determinates the decision threshold for a binary classification. Assuming that values higher than the threshold are set to a positive decision, and below the threshold to a negative decision, the mathematic representation is set as:

\begin{equation}
    \text{positive decision if } \sum_{i=1}^{d} w_i\cdot x_i > \text{bias}
\end{equation}
\begin{equation}
    \text{negative decision if } \sum_{i=1}^{d} w_i\cdot x_i < \text{bias}
\end{equation}

which can be represented in the following simplified form:
\begin{equation}
    \text{decision } h(x) = \text{sign}\left(  \left( \sum_{i=1}^{d} w_i\cdot x_i \right )+ b \right )
\end{equation}

A basic representation of a perceptron is shown in Fig~\ref{fig:perceptron}.

\begin{figure}[!htb]
    \centering
      \includegraphics[width=0.4\textwidth]{figures/perceptron}
      \caption{Interconnection of perceptrons generating a simple neural network. \cite{Yaser}}
      \label{fig:perceptron}
\end{figure}

Perceptrons are also called \emph{neurons}, which are activated to specific types of input data. When more neurons are interconnected, each one with a different activation function (based on the weights and the bias), a neural network is created, which can then classify input data of higher complexity. An arbitrary amount of neurons can be added increasing as well the complexity of the network, but each of them classifies the data independently. Historically, the name \emph{neural network} is given because of the similarity of this behavior to the synapses of the neurons in a biological neural system. The book of professor Yaser \cite{Yaser} explains in further detail the perceptrons and the learning process using these units.

\begin{figure}[!htb]
    \centering
      \includegraphics[width=0.8\textwidth]{figures/neuralnetwork}
      \caption{Basic structure of a neural network.}
      \label{fig:neuralnetwork}
\end{figure}

\subsection{Convolutional Neural Networks}

\ac{ConvNet} are a special type of neural networks that share their parameters across space, making them suitable for image classification. The main idea is as follows: assume we have an image depicted in blue in Fig~\ref{fig:convolutionalnetwork} as the input of this system, whose dimensions are (height, width, and depth). The height and width are the numbers of pixels that determine the size of the picture, and the depth is 3 for color pictures (RGB) and 1 for grayscale. When this picture is inserted into the \ac{ConvNet}, only a portion of it is regarded as a so called patch, shown in red.

\begin{figure}[t!]
    \centering
      \includegraphics[width=\textwidth]{figures/convolutionalnetwork}
      \caption{Convolutional Network}
      \label{fig:convolutionalnetwork}
\end{figure}

This patch, also called kernel, is then swept along the picture with a given arbitrary stepsize, applying a neural network operation over it. As it is a smaller patch, each operation regards many fewer weights in comparison with the whole input, which is shared across space with the slide operation. As a result, a stack of convolutions is generated, representing a picture-like output, having not only a reduced height and width but an increased depth K that represents the non-linearities that describe the input, as if it increased the number of color channels that describe the input picture.\\

Generally, these convolutional layers are stacked together generating a pyramid, in a so-called sequential model, reducing the spatial dimensions of an input image (h,w), but increasing its depth. As a result, the depth corresponds to the \emph{semantic complexity} of the image representation, mapping the illustrative content of it. Inside of each layer, there is a number K of filters, in charge of picking different characteristics from the image, such as a specific shape or a determinate intensity value. The characteristic that the filter notices is not programmed, but learned by itself. This means that the network learns on its own what type of feature representation is going to learn and, consequently, which shape is going to activate it.


\section{Last lecture}
Finish off parton construction with quantum spin liquids and spend the last 20
minutes for an overview of what we did in the course and try to tie things
together a bit.
Fermionic parton construction.
\begin{align}
    \vec{S} &=
    \begin{cases}
        \frac{1}{2}f^\dagger \vec{\sigma} f\\
        \frac{1}{2}z^\dagger \vec{\sigma} z
    \end{cases}
\end{align}

We had spin operators with 2 states per site.
And two flavours of fermions per site.
We actually get 4 states per site for the fermions,
but we deemed two of them unphysical,
so that's how we describe the two initial states.

So we have $\ket{\uparrow}=\ket{01}$
and $\ket{\downarrow}=\ket{10}$
which are physical states but the states
$\ket{00}$ and $\ket{11}$ are not gauge invariant and hence unphysical.

There is a $U(1)$ gauge redundancy that keeps physical operators invariant.
\begin{align}
    f &\to e^{i\theta}f
\end{align}
Actually there is an $SU(2)$ gauge redundancy.

To implement this gauge constraint,
we need to enforce a constraint,
which is that the number of particles per site should always be 1.
\begin{align}
    n_1 + n_2 = 1
\end{align}
So we're in the $01$ and $10$ subspace.
If we decompose the operator this way and enforce the constraint,
that gives us new mean-field approximations that we didn't have access to
before.

We started with the Heisenberg model,
and rewrote it as just  model of fermions hopping.
\begin{align}
    H &=
    \sum_{ij} \vec{S}_i\cdot\vec{S}_j J_{ij}\\
    &=
    \sum_{ij}
    \left[
    -\frac{1}{2} J_{ij} f_{i\alpha}^\dagger
    f_{j\alpha}
    f_{j\beta}^\dagger
    f_{i\beta}
    +
    J_{ij}
    \left(
    \frac{1}{2}n_i
    -
    \frac{1}{4} n_i n_j
    \right)
    \right]
\end{align}
And, we can then write the ansatz that
\begin{align}
    \chi_{ij} &=
    \langle f_{i\alpha}^\dagger f_{j\alpha}\rangle
\end{align}
which if we put nito the Hamiltonian gives the mean-field Hamiltonian
\begin{align}
    H_{mf}
    &=
    \sum_{ij}
    \left[
    \left(
    -\frac{1}{2} J_{ij} f_{i\alpha}^\dagger f_{i\alpha}
    \chi_{ji}
    +\mathrm{h.c.}
    \right)
    -
    \left| \chi_{ij} \right|^2
    \right]
\end{align}
where we have replaced $f_{i\alpha}^\dagger f_{j\alpha}$
by the mean field.
And for consistency,
we want ot enforce the equation
$ \chi_{ij} = \langle f_{i\alpha}^\dagger f_{j\alpha}\rangle$.

The zeroth order thing to do is to not require this constraint exactly,
but require it on average.
So there is a zeroth order mean field which is to treat the constraint on
average.

One thing we can do is add an extra Lagrange mulitiplier in the Hamiltonian
which acts like a chemical potential that fixes this average
$\langle n_i \rangle = 1$.
So we have
\begin{align}
    H_{0-mf}
    &=
    H_{mf}
    +
    \sum_i a_0
    \left(
    f_{i\alpha}^\dagger f_{i\alpha}
    - 1
    \right)
\end{align}
so this is basically a site-dependent chemical potential to enforce
$\langle n_i \rangle = 1$.
We hvae this Hamiltonian that is afree-fermion Hamitonian,
and we hvae thse parameters $a_0$ and $\chi$ we can tune,
but we want to choose them to be self-consistent such.
Choose $a_0$ such that $\langle n_i\rangle = 1$.

We are assuming $\chi_{ij} &= \bar{\chi}_{ij}$
ad $a_0(i) = \bar{a}_0$
satisfy self-consistency equations.
The fermions carry spin half but are electrically neutral.
These excitations are electrically neutral spin-$\frac{1}{2}$ fermions.
We call these fermionic \emph{spinons},
which are excitations that carry spin-$\frac{1}{2}$.
Local excitaitons always carry integer spin,
because every spin flip operation changes the spin by one,
when it goes from $-\frac{1}{2}$ to $\frac{1}{2}$.

Any logcal opeator can only create integer spin excitations,
so a spin-$\frac{1}{2}$ is a non-trivial thing.
They also happen to be electrically neutral fermions.
The zeroth order mean field theory is useful because you can deduce you can have
fermionic spinon excitations,
but in some sense you can only treat the constrant on average,
but you should rally treat hte constaint exactly.

We don't know if hte meanf ifeld theory is stable to flutcuations either,
which could be in phase or amplitude of the $\chi_{ij}$ mean field.
Another way of arriving at the $\chi_{ij}$
is instead of doing a mean field theory is do a
Hubbard-Stratonvich transformation,
where you replace
$\phi^4 \to \phi^2 \phi^2 + \chi^2$
completing the square.
It's a common trick,
but if you're familiar wiht it,
just consider it a mean field hteory.

\subsection{First-order mean field}c:w
we consider a first-order mean field theory
to check the stability of the zeroth order mean field
and treat hte constraint exactly.

The first fluctuations to consider are phase fluctuations.
\begin{align}
    \chi_{ij} &=
    \bar{\chi}_{ij} e^{i a_{ij}}
\end{align}
The amplitude fluctuation is gapped due to the $|\chi_{ij}|^2$
term so we don't have to worry about it.

A useful way of thinking about this is the integral language.
\begin{align}
    Z &=
    \int \mathcal{D} f \mathcal{D} \bar{f}\,
    e^{iS}
    \prod_{i,t} \delta\left( f_i^\dagger f_i - 1 \right)
\end{align}
Every site and time has a delta function.

And we can introduce fields $a_0(i, t)$
\begin{align}
    Z &=
    \int \mathcal{D} f \mathcal{D} \bar{f} \mathcal{D} a_0\,
    e^{i \sum_{i, t} a_0 (i, t)} \left( f_9^\dagger f_i - 1 \right)
    e^{iS}
\end{align}
and hte Hamiltonian becomes
\begin{align}
    H_{1,mf}
    &=
    \sum_{ij}
    -\frac{1}{2} J_{ij}
    \left[
    \left(
    f_{i\alpha}^\dagger f_{j\alpha}
    \bar{\chi}_{ji} e^{-ia_{ij}}
    + \mathrm{h.c.}
    \right)
    -
    \left| \bar{\chi}_{ij} \right|^2
    \right]
    -
    \sum_{i} a_0\left( i \right) \left(
    f_{i\alpha}^\dagger f_{i\alpha} - 1
    \right)
\end{align}
and now $a_0$ becomes a dynamical field.
Then $a$ can be thought of as a $U(1)$ gauge field,
with the transformation
\begin{align}
    f_{i} &\to e^{i\theta_i} f_i\\
    a_{ij} &\to a_{ij} + \theta_j - \theta_i
\end{align}
keeping it invariant.
Then $a_0$ literally enters the theory as the time-componento f the gauge field.
\begin{align}
    \mathcal{L} &=
    i \sum_{i}
    f_i^\dagger \left( \partial_t - i a_0 \right) f_i
\end{align}
So basically we get to fermionic spinons plus $U(1)$ gauge fields.
We should emphases the gauge field is a dynamical emergent gauge field.
It wasn't there microscopically,
but once we do this decomposition,
it has to be there due to the gauge redundancy,
because we expanded the Hilbert space got expanded and we need to restrict it
back down.

The fluctuations of $\chi_{ij}$ correspond to the gauge field fluctuations.
If we want to understand the stability of the mean-field ansatz,
we need to understand the stability to fluctuations of this $U(1)$ gauge field.

The stability of mean field is related to the stability to $U(1)$ gauge field
fluctuations.
There's  lot to do in this direction,
and you could consider all kinds of mean field ansatzs and you can go back to
see that we have an $SU(2)$ gauge field fluctuations which are much larger,
and you can put those in the theory which requires you to havea few more
paramteres,
so you can analyse all sorts of mean field theories.

But now I want ot switch gears and not talk about hte Hamiltonian and the mean
field gauge theory,
but instead I want to talk about the wave functions.

This leads to a whole class of trial wave functions,
or variational wave functions,
which are basically projected wave functions.

This is a spatial component and this is a time component.

Whence have this mean field ansatz,
we have to look at the ground state
\begin{align}
    \ket{\Psi_{\mathrm{mean}}\left( \chi_{ij} \right)}
\end{align}
which is the mean field state of partons.
this mean field by itself is not a physical state because it doesn't live in the
spin Hilbert space.
Because it's a ground state of the zeroth order mean field theory,
it contains unphysical states where there are 0 or 2 fermions per site.
To get back into the original Hilbert space,
we can take this mean field ground state and project it into the physical
Hilbert space.
\begin{align}
    \ket{\Psi_{\mathrm{phys}}
    &=
    P\ket{\Psi_{\mathrm{mean}}(\chi_{ij})}
\end{align}
where $P$ is the projection onto 1 particle-per-site states.
This is called the Gutewillen projection.

Start with the Fock vacuum $\ket{0}$ and add fermions,
and take that inner product with the state.
\begin{align}
    \Psi_{\mathrm{spin}}^{\left( \chi_{ij} \right)}
    \left( \left\{ \alpha_i \right\} \right)
    &=
    \bra{0}
    \prod_{i} f_{\alpha i}
    \Ket{\Psi_{\mathrm{mean}^{(\chi_{ij})}}
\end{align}
Consider a different ansatz
\begin{align}
    \tilde{\chi}_{Ij}
    &=
    \chi_{ij} e^{i\left( \theta_i - \theta_j \right)}
\end{align}
which is the same as taking $f_i \to e^{i\theta_i} f_i$,
so then
\begin{align}
    \Psi_{\mathrm{spin}}^{\left( \tilde{\chi} \right)}
    &=
    e^{i\sum_i \theta_i} \Psi_{\mathrm{spin}}^{\left( \chi \right)}
\end{align}
and the only difference is the overall phase which is the same physical state.
It's incredible that this class of wave functions is so incredibly rich.
Fractional quantum hall states,
quantum spin liquids,
any exotic state you can write down parton construction,
and then do mean field and then do a projection.

We can not just get ground states,
but also variational states for excited states.
I can put some winding or fluctuation of $\chi$,
then do some projection and get some candidate excited wave function.

To do it more explicitly,
I can do a spinon wave function.

The spinon wave functions
\begin{align}
    \Psi_{\mathrm{spin}}^{\mathrm{spinon}}
    \left( i_1, \lambda_1; i_2, \lambda_2 \right)
    &=
    \bra{0}
    \left(
    \prod_{i} f_{i,\alpha_i}
    \right)
    f_{i_1\lambda_1}^{\dagger}
    f_{i_2 \lambda_2}
    \ket{\Psi_{\mathrm{mean}}^{\chi}}
\end{align}
There's been a lot of work trying to write down new fancy wave functions using
neural networks and tensor networks and machine learning,
and PEPS and so on,
but these are the ones that actually work,
and describe everything.
All the popular ones people talk about but are not nearly as powerful as this
class of wave functions.
Those other classes of wave functions are fun to think about they have not been
as powerful so far as these wave functions.

You set up this mean field theory,
and you set up spins with spinons,
that gets yo to $U(1)$ gauge field,
and then you get wave functions by projection to the physical Hilbert space.
Now you consider all sorts of mean field theories.
$\chi$ and $a_0$ are the variational parameter.

\begin{question}
    Have there improvmentes by including gague flucationas?
\end{question}
You can improve the wave functions by including gague fluctuations.
It is an interesting reearch direction but I have seen very littel work.
One thing cold be to get better wave functions by doing exactly that.

At this point the name of hte game is to consider all sorts of mean field
ansatzs and describe all sorts of phases.

\section{$\ZZ_2$ spin liquid}
There's another we of arriving at very similar states in terms of projected wave
functions and mean field theory approach.
One way is to assume the spinons form an $s$-wave superconductor.
Assume when spinons form $s$-wave superconductor
\begin{align}
    \langle f_{i\uparrow} f_{i\downarrow}\rangle
    &\ne 0
\end{align}
The mean field wave function si going to be a mean field wave function for an
$s$-wave superconductor,
and then sou do a projection onto one particle per site.
This gives a physical wave function in the Hilbert space,
which si a $\ZZ_2$ spin liquid wave function.
When this parton model acquires a non-zero expectation,
there is an Anderson-Higgs gauge symmetry
and what's left behind is this $\ZZ_2$ gauge theory.
\begin{align}
    \ket{\Psi_{\ZZ_2 \mathrm{QSL}}}
    &=
    P
    \ket{\Psi_{\Psi_{mf,s\text{-wave s.c.}}}}
\end{align}

In parton mean field theory,
$U(1)$ gauge field plus $f_{i,\alpha}$.
And $U(1)$ breaks by Higgs to $\ZZ_2$.
The $\ZZ_2$ gauge field coupled $f_{i,\alpha}$.

Then we have topological excitations.
The first kind of quasiparticle $f$ is BdG quasi-particles of $s$-wave conductor
created by breaking ``cooper pairs`` of the neutral spin-half spinons.
Each one is fermionic spin-$\frac{1}{2}$ excitations.

The second kind.
Instead of writing the mean field state you insert a vortex nito this
$s$-wave superconductor.
The $\pi$-flux vortex of an $s$-wave superconductor after projection becomes a
vison,
which we talked about in the context of the quantum dimer model,
and in the context of the $\ZZ_2$ toric code vortex where this was the $m$
particle.

This $f$ is a fermion and this $m$ particle is a boson.
Every other topological class of excitations we could get from here.
We could consider composite $f$ and $m$ particles,
which gives what we would normally call the $e$ particle in the toric code
state.

The third kind of $f\times m = e$
When $f$ goes around $m$,
that picks up a minus sign,
I would pick up the spin of the $f$ which is minus $-1/2$,
and I would pick up a spin of the $m$ which is $+1/2$ and then the composite is
a boson.

The fourth kind is local excitations.
Even numbers of spinons.
You need more work to show that two visons coming together is local,
but I'll leave that for you to think about.
The whole point is that this is another way of arriving at this $\ZZ_2$ spin
liquid,
but there is something which didn't come up previously.

We see the fermion $f$ is a spinon that carries spin-$\frac{1}{2}$,
whereas the vison does not carry spin-$\frac{1}{2}$.
All particles in the toric code were spin-less,
so this differs from the quantum dimer model discussion and $\ZZ_2$ toric code
discussion due to the quantum numbers these excitations have under the $SO(3)$
rotations.
This didn't exist in the previous discussions.

It could have existed if we demanded the dimness had spin-$\frac{1}{2}$.
It's distinct from the toric code in that the spinons carry
spin-$\frac{1}{2}$,
so this whole state carries symmetry fractionalization.
All these particles carry fractional spin.

You can think of a spin-liquid as a superconducting state of spinons.
That's one way of thinking about a quantum $\ZZ_2$ spin liquid.

There is another important class of states is fractional quantum Hall states,
also known as fractional Chern insulators.
We can do it in terms of spins but we don't have to.
In terms of spins,
you can think of the boson $b$.
We don't necessarily there is full SO(3) full spin rotation symmetry.
We can write
\begin{align}
    b &=
    f_1 f_2
\end{align}
which si basically the same thing as writing the spins as
$S^\dagger &= f_1^\dagger f_2 = \tilde{f}_1 \tilde{f}_2$.

In this description $U(1)$ gauge theory goes
\begin{align}
    f_1 &\to e^{i\theta} f_1\\
    f_2 &\to e^{-i\theta} f_2
\end{align}
Each of these form some kind of insulating phase with Chern number 1.
Consider mean field state where $f_1,f_2$
are each in Chern insulator with $C_1=C_2=1$.
I want ot track the background external gauge field under which $b$ carries
charge 1.
So let $A$ be a background $U(1)$ gauge field,
not to be confused with the internal emergent gauge field that emerges from the
$U(1)$ gauge transformation.

\begin{table}[h]
    \centering
    \begin{tabular}{ccc}
        & $q_a$ & $q_A$\\
        $f_1$ & + 1 &  1\\
        $f_2$ & -1 & 0
    \end{tabular}
    \caption{z2spin}
    \label{tab:z2spin}
\end{table}
Imagine we have some effective Lagrangian
\begin{align}
    \mathcal{L}_b
    &=
    \mathcal{L}\left( f_1, a + A \right)
    +
    \mathcal{L}\left( f_2, a \right)
\end{align}
$f_2$ is a sate which si a Chern insulator with Chern number 1.
And it's coupled to this $a$ gague field.
What's an effective Lagrangian for such a csitaution.
You may not remember,
but we covered this before,
to introduce a $U(1)$ level Chern-Simons theory.
We can write the effective theory as
\begin{align}
    \mathcal{L}\left( f_2, a \right)
    &=
    -\frac{1}{4\pi}
    \alpha_{\mu} \partial_{\nu} \alpha_{\lambda}
    \epsilon^{\mu\nu\lambda}
    -
    \frac{1}{2\pi}
    \epsilon^{\mu\nu\lambda}
    a_{\mu} \partial_{\nu} a_{\lambda}
\end{align}
so then the current is
\begin{align}
    j_{\mu}^{\left( f_2 \right)}
    &=
    \frac{1}{2\pi}
    \epsilon^{\mu\nu\lambda}
    \partial_\nu
    \alpha_{\lambda}
\end{align}
So this is also a Chern-insulator 1 state.
\begin{align}
    \mathcal{L}\left( f_1, a + A \right)
    &=
    -\frac{1}{4\pi}
    \beta \, d\beta
    +
    \frac{1}{2\pi}
    \left( a + A \right)
    d\beta
\end{align}
If we only keep leading order terms in these gauge fields,
\begin{align}
    \mathcal{L}_b
    &=
    -\frac{1}{4\pi}
    \alpha\, d\alpha
    -
    \frac{1}{4\pi}
    \beta\, d\beta
    +
    \frac{1}{2\pi}
    a\, d\left( \beta - \alpha \right)
    +
    \frac{1}{2\pi} A\, d\beta
\end{align}
The whole point of hte parton mean field ansatz is that you can treat all the
gauge fluctuations perturbabitvely.
If you summarised in one sentence the parton constution.
Decomose the physcial operaotrs in temrso f p
Then aassume some mean field state,
then assume fluctuiatons are weak.

We assume the gague fluctuiaotns are greated perturbabilty,
and all subleadng terms are irrelevant.
And then we can just integrate out the $a$.
And because thsi si a free theory,
we can just solve the equation of motion for $a$.
Either way you're going to get the same thing.
\begin{align}
    \epsilon^{\mu\nu\lambda}
    \partial_\nu \beta_{\lambda}
    =
    \epsilon^{\mu\nu\lambda}
    \partial_\nu a_\lambda
\end{align}
which assuming it's not topologically weird space,
\begin{align}
    \beta_{\mu} &=
    a_\mu + \partial_\mu \phi
\end{align}
and we can just ignore the second term,
and we arrive at
\begin{align}
    \mathcal{L}_b
    &=
    -\frac{2}{4\pi}
    \alpha\, d\alpha + \frac{1}{2\pi}A\, d\alpha
\end{align}
and we saw this before.
This is $U(1)$ level 2 CS theory or $U(1)_2$.
And you will find the Hall conducatnce is
\begin{align}
    \sigma_H
    &=
    \frac{1}{2} \frac{1}{2\pi}
\end{align}
and so this ansatz describes a fractional quantum Hall state.

\begin{question}
    Why is that $1$ and $0$ is a choice in the charge table?
\end{question}
In ternral you could have $a_{A1}$ and $q_{A2}$,
but all you need is $q_{A1}+q_{A2}=1$.
In that case your Lagrangian will be
\begin{align}
    \mathcal{L}_b
    &=
    -\frac{2}{4\pi}
    \alpha\, d\alpha + \frac{1}{2\pi}A\, d\alpha
    +
    \frac{1}{2\pi}
    A\, \alpha
    \left(
    q_{A1} \beta +
    q_{A2} \alpha
    \right)
\end{align}
But in consdensed matter physics,
people are find not worrying about global issues,
and you could just wirte $\frac{1}{2}$ and $\frac{1}{2}$,
but field theorists yell at me to write $1$ and $0$.
But the correct thing to do is do the field throeeists integer charge $1$ and
$0$.
You'll get the same answer for the Hall conducatnce eithere way.
They mess aorund with the global compactness of the gauge field.
If you don't worry about non-trivial topologies you won't worry about this
issue.

Now you can also write down the wave funciton.
Each mena field state is in some Chern number 1.

Now the projection.
We have two flavours of fermions.
\begin{align}
    \Psi\left( \left\{ r_i \right\} \right)
    &=
    P_{n_1=n_2}
    \Phi_{C=1}\left( \left\{ r_i^2 \right\} \right)
    \Phi_{C=1}\left( \left\{ r_i^1 \right\} \right)\\
    &=
    \left( \Phi_{c=1}\left( \left\{ r_i \right\} \right) \right)^2
\end{align}
If you're familiar with this,
this is the $\frac{1}{2}$-Laughlin FQH wave function.

\begin{question}
    ???
\end{question}
It's again the same thing,
you identify
$\ket{0}_b=\ket{00}$ and $\ket{1}_b=\ket{11}$ sa the physical states
and $\ket{10}$ and $\ket{01}$ as non-phsyical.

This parton construction is much better than the other approaches,
because it unifies quantum spin liquids and quanutm Hall states.

\begin{question}
    How did you get to the fact I need to consider 2 Chern nisulators?
\end{question}
In principle,
there's an infinite numbero f mean field ansatz options,
but why did I decide on the ansatz $b=f_1f_2$?
The parton construciton is a way of describing any phase with an emergent gauge
theory,
even if gapless.
When you do MF,
you can consider many ansatzs.
You can just pick one,
see what its energetics looks like,
and the one with least energy is the most likely.
You can consdier all sorts of mean field ansatz whcih gives you many variational
wave functions,a
nd that gives you many.

How did Laughlin know it's a square or cube of the free fermion wave function?
It was just his guess based on his experience.

\begin{question}
    How general is this?
    If you write $b=$ 3 fermions?
\end{question}
You could do in general $b=f_1f_2\cdots f_k$.
the gauge symmetry is different,
but you can still do it.
You end up with $U(1)_k$ CS theory,
if you assume all are in $C=1$ Chern-insulator states.

\begin{question}
    Math procedure for free fermion model and mean field ansatz to output some
    topolgocial orderd satte.
\end{question}
There probably is one,
but it hasn't been elucidated.
Perhaps there's something more formal that can be developed.

\section{Summary}
I want to give you an overview of what we covered.
A brief review of what we've done and why it all fits together.
There's a lot more things to cover for another course.
There are important things we didn't cover,
like the algebraic theory of anyons,
this modular tensor category.
I can give another lecture about modular tensor categories,
but since the lectures for the course are over we don't technically have to do
that.
Are you interested in a bonus lecture on Wednesday?

Let me try to take 10 minutes to give you a summary or overview.

We start off by talking about general htings,
like locality in quantu mssytesm,
how gapped implies correlation length.
We didn't directly do naything wiht these,
but we talked about Hilbert spaces decompsoing into local tensor products.
We did use finite correlation lneght in some places.

Then we described TQFTs with path integrals $Z\left( M^{d+1} \right)$,
but the important point is that topologicla phases of matter are believed to be
in 1-1 correspondtance to deformaiton clases of TQFTs.
An important distinction was the notion of invertible vs non-invertible.
Invertible in this contextz means that $|Z|=1$
and non-invertible menas $|Z|\ne 1$.
Invertible means unique ground state,
but non-invertible means degenerate ground states.

We did some examples of TQFTs.
They came in two flavours.
We talked about this,
but there was $U(1)_k$ CS theory,
which came in two flavours.
We could think of this as a response theory,
meaning that we have a background gauge field
$Z\left( M^3, A \right) = Z\left( M^3, 0 \right) e^{i k S_{CS}[A]}$.
This can describe Chern insulator integer quantum Hall states,
in other words invertible topological phase.

Then the way of thinking about it is that we have a dynamical gauge theory.
Here,
we're actually integrating over all gauge field configurations.
\begin{align}
    Z\left( M^3 \right) &=
    \int \mathcal{D} a\,
    e^{ikS_{CS}(a)}
\end{align}
and this describes non-invertible topological phases,
like fractional quantum Hall states,
quantum spin liquids.
One of the problems is thinking about the toric code in terms of CS theories
with 2 by 2 matrices.

It depends on whether you treat the gauge field as background or dynamical.

Another example is SPT path integrals, or Digraph-Witten theory,
where you pick a cohomology class inside a cohomology group
\begin{align}
    \left[ \nu_{d + 1} \right]
    \in
    H^{d + 1}
    \left( G, U(1) \right)
\end{align}
You can think of it as an invertible TQFT
\begin{align}
    Z\left( M^{d+1}, A \right)
    &=
    e^{i\int_M A^* \nu_{d+1}}
\end{align}
We triangulated our manifold,
and defined group elements on links,
and the path integral value was always a phase.
This describes SPTs.
One utility of this construction was that it gave exact wave functions,
and it gives parent Hamiltonians,
a full solvable model to describe a huge class of SPTs.
And remember SPTs are a subset of invertible phases.
An invertible phase is a subset that is invertible on all manifolds.
In a ground states of an invertible topological state,
it has an inverse state that if you stack them can adiabatically go to a trivial
state.
If you don't care about the symmetry,
you can always adiabatically connect it to a trivial state,
but not if you want to respect the symmetry.

Then here what we can do is consider non-invertible TQFTs,
\begin{align}
    Z(M) \propto
    \sum_{[a]} e^{i \int a^* \nu}
\end{align}
where the sum of over all flat gauge fields.
If you take $G=\ZZ_2$ and $[\nu] = [0]$,
then this describes the $\ZZ_2$ spin liquid,
which is the toric code.

There's one other example that we did,
which was the TQFT for the $(1+1$-D Majorana.
We had a path integral over a 2D manifold with a 2-dimensional spin structure.
\begin{align}
    Z\left( M^2, S \right) &=
    \left( -1 \right)^{\Arf(S)}
\end{align}

And then we talked about concrete examples of topological phases.

We had the $(1+1)$D Majorana.
Then we talked about the $(2+1)D$ Chern insulator,
TRI TI and $p+ip$ superconductor.

The invertible free fermion model.
Time-reversal invariant topological insulator.

We talked about the $(3+1)$D TRI TI.

Then we tlked about boson SPTs,
including the AKLT construction,
and group cohomology models.

Then we talked about non-invertible topological phases,
which includes quantum spin liquids,
including quantum dimer models,
parton construction,
lattice gauge thoery and toric code.

FQH state $U(1)_k$ CS theory by parton construction.

Thep oint of the anyon pahses was topological excitations wiht fractional
statistics.
We had topological ground state degeneracies that depended on the topology of
the manifold and so on.
In practice,
this class is a survey of the huge classes of topological phaes.
It's a deep and rich subject.
In the next bonus lecture,
I'll tell you about non-Abelian topological phaes and the algebraic theory.


\chapter{Introduction}
Modern communication systems often incorporate numerous, uncoordinated users competing for a limited resource. In this work, a game theoretic approach considers a wireless communication network with uncoordinated macrocell users and femtocell base stations. The goal of this approach is to find Nash equilibrium (NE) in which femtocell base stations optimize individual utility while ensuring interference tolerances are not violated for macrocell users.
Previous work has shown solutions for systems in which femtocell base stations transmit over a single input, single output (SISO) channel; therefore the multiple input, multiple ouput (MIMO) case is investigated here.
\par
In the communications context, distributed optimization is often preferable in order to minimize the overhead of information passing required by central optimization.
This work focuses on achieving NE using distributed solutions.

\section{Notation}
The following notation is used throughout the remainder.
$E\{\cdot\}$ is the expectation operator.
Vectors are denoted by bold font, lower-case letters ($\mathbf{x}$) and are assumed to be column vectors.
Matrices are denoted by bold font, upper-case letters ($\mathbf{X}$). The transpose and hermetian of $\mathbf{X}$ are denoted by $\mathbf{X}^T$ and $\mathbf{X}^H$ respectively.
$\mathbf{I}$ denotes the identity matrix. $\mathrm{diag}(\mathbf{x})$ denotes the diagonal matrix created by placeing the elements of the vector $\mathbf{x}$ along the diagonal. $\mathrm{trace}(\cdot)$ is the trace operator. The operator $(\cdot)^+$ yields the argument if greater than 0, and 0 otherwise.

\section{Relevant Tools/Theory}\label{tools}
It is useful to first review relevant game theoretic tools. Together, these tools provide a series of steps to follow in order to determine Nash Equilibrium via distributed solutions.
\par
We consider a game in which $F$ players each choose a strategy $\mathbf{b}_{f}$ from a set of possible strategies $\mathcal{B}_f$. The vector $\mathbf{b}=[\mathbf{b}_{1}^T, \mathbf{b}_2^T... \mathbf{b}_{f}^T]^T \in \mathcal{B}$ represents the strategies of all players. Each player has a utility function $U_f(\mathbf{b})$ which it attempts to maximize through the choice of $\mathbf{b}_{f}$. The set of strategies available to a player, $\mathbf{b}_{f} \in\mathcal{B}_f$, may be dependent on the strategies of other players $\mathbf{b}_{j}, j \neq f $.

\begin{enumerate}
\item The potential function: a central optimization problem used to identify Nash equilibrium of a game.
\begin{itemize}
\item
\textbf{Definition:}  A potential function is any function
$ \Psi(\mathbf{b})$ which satisfies:
\begin{equation}\label{potential_game_condition}
\frac{\partial \Psi(\mathbf{b})}{\partial \mathbf{b}_{f}}
 =
 \frac{\partial U_f(\mathbf{b})}{\partial \mathbf{b}_{f}}.
\end{equation}

In words, $ \Psi(\mathbf{b})$ is a function whose gradient with respect to the strategy $\mathbf{b}_{f}$ of a player, is equal to the gradient of that player's utility function with respect to $\mathbf{b}_{f}$.

\item \textbf{Result:} Global optima of a potential function are Nash Equilibria \cite{monderer1996potential}.
\item \textbf{Note:} Not all games admit a potential function.

\end{itemize}

\item Normalized Nash equilibrium: if we consider the case in which the set $\mathcal{B}$ is convex and $U_f(\mathbf{b})$ is concave, non-decreasing and has continuous first derivative in $\mathbf{b}_{f}$, the resulting optimal strategy for each player (and thus any Nash equilibrium) will satisfy the necessary and sufficient KKT conditions. A normalized Nash equilibrium (NNE) is a Nash equilibrium for which the dual variables of all players are equal up to a constant.
Intuitively, this can be thought of as a Nash equilibrium for which each players is "charged" an equal amount (up to a constant) for any change in strategy.

\item The Concave N-Person game: A framework for proving existence and uniqueness of normalized Nash equilibrium.

\begin{itemize}
\item
\textbf{Definition:} A game in which all $U_f(\mathbf{b})$ are concave with respect to $\mathbf{b}_f$ and the set of possible $\textbf{b}\in\mathcal{B}$ is convex.
\item
\textbf{Existence of Nash Equilibrium:} Pure Strategy NEs exist for Concave N-Person games \cite[Thm1]{rosen1964existence}.
\item
\textbf{Uniqueness of Nash Equilibrium:} In general, multiple NNEs may exist for a game. However, sufficient conditions for uniqueness are shown in \cite[Thm4]{rosen1964existence} and introduced in the following.
Taking the gradient of each $U_f(\mathbf{b})$ with respect to the chosen strategy $\textbf{b}_f$ we first form the vector
\begin{equation}
g(\mathbf{b})=
\begin{bmatrix}
\nabla U_{1}(\mathbf{b})
\\
\nabla U_{2}(\mathbf{b})
\\
\vdots\\
\nabla U_{F}(\mathbf{b})
\end{bmatrix}.
\end{equation}
We define the Jacobian of $g(\mathbf{b}) $ as the matrix valued function $G(\mathbf{b}) $.
Negative definiteness of the matrix $[G(\mathbf{b})+G^{T}(\mathbf{b})] $ is a sufficient condition for uniqueness of a NNE.
\item \textbf{Result 1:} For a Concave N-Person game admitting a potential function, the potential function is concave, allowing for use of convex optimization tools to find a NE.

\end{itemize}

\item Distributed Optimization: Solving a central optimization problem using multiple processes in parallel.
In particular, the dual ascent method described in \cite{boyd2011distributed} is used in this work.

\textbf{Distributed Dual Ascent}
\begin{itemize}
\item For a central optimization problem (obtained using a potential function in this work), determine the dual problem to the central, primal problem.
\item Evaluate if the dual problem can be separated into sub-problems, each with independent optimization variables.
\item Perform an ascent of the dual problem by iterating the following:
\begin{enumerate}
\item Solve the separate dual function of the sub-problems.
\item Broadcast these solutions and perform the gradient step of the dual problem.
\end{enumerate}
\end{itemize}

If the problem is convex with strong duality, the dual ascent will converge to the optimal (in this case a Nash Equilibrium).

\end{enumerate}

\section{Outline}
The above tools are used in the steps outlined below.
\begin{itemize}
\item
\ref{genmodel}: Introduce the system model for the general game.
\item
\ref{genproblem}: Analyze the general setup to see why further refinements are useful in arriving at a distributed solution.
\item
\ref{conmodel}: Introduce refinements to the system model.
\item
\ref{conproblem}: Develop a distributed solution using the refined system model.
\item
\ref{numerical}: Discuss numerical results of the proposed solution.
\end{itemize}
\chapter{System Model}

\section{General System Model}\label{genmodel}

\subsection{Players: Femtocell Base Stations}

Each player of the game, a femtocell base station (FBS) $f \in \{1 ... F\}$ , is characterized by the following.
\begin{itemize}
\item
$T_{f}$ antennas to transmit to $K_{f}$ femtocell users (we assumed that $T_{f} \geq K_{f}$). Each femtocell user $i = [1... \;K_{f}]$ has a signal to interference plus noise ratio (SINR) $\gamma_{f,i}$. The vector of user SINRs for FBS $f$ is $\boldsymbol{\gamma}_{f} = [\gamma_{f,1}...\; \gamma_{,K_{f}}]$
\\
\item
	A beamforming matrix $\mathbf{U}_{f} \in \mathbb{C}_{T_{f} \times K_{f}}$ such that the transmitted
	signal is $\mathbf{s}_{f
	}= \mathbf{U}_{f}\mathbf{x}_{f}$. The
	vector of symbols for users of FBS $f$, $\mathbf{x}_{f}$, is  normalized such that $E\{\|x_{f,i}
	\|_2^2\}=1$ and $E\{\mathbf{x}_{f}\mathbf{x}_{f}^H\}=\mathbf{I}_{K_{f} \times K_{f}}$.
\\
\item
	An average power constraint $E\{trace(\mathbf{U}_{f}^H\mathbf{U}_{f})\} \leq P^{Total}_{f} $.

\item
	A cost function $U_{f}(\boldsymbol{\gamma}_{f}) =
	\sum_{i=1}^{K_{f}}
    	 U_{f,i}(\gamma_{f,i}) $
    	in which $U_{f,i}(\cdot)$ is a non-decreasing function describing the utility obtained by each user, $i$, of FBS $f$.

\item
	Perfect knowledge of the downlink channel to the users of FBS $f$ given by the matrix $\mathbf{H}_f \in \mathbb{C}_{K_{f} \times T_{f}} $ .
% TODO(Simulate degradation with incomplete CSI solution?)
\\
\item
	 FBSs are assumed to be spaced far apart in distance such that they cause no interference to one another.
\end{itemize}

\subsection{Macrocell Users}
Macrocell users $m \in \{1 ... M\}$ introduce constraints into the game. These users are characterized by the following.

\begin{itemize}
\item
	Received interference constraint
	$E\{\sum^F_{f=1} \tilde{\mathbf{h}}_{m,f}^T  \mathbf{U}_{f}
	\mathbf{U}_{f}^{H} \tilde{\mathbf{h}}_{m,f}^*\} \leq I^{Threshold}
	_{m} $. In which $\tilde{\mathbf{h}}_{m,f}$ is the channel from FBS $f$ to macro user $m$.

\item
	FBS $f$ is assumed to know the downlink channel matrix $\tilde{\mathbf{H}}_{f} \in \mathbb{C}_{M \times T_{f}}$ to all $M$ macrocell users.
\\
\end{itemize}

\subsection{Femtocell Users}
\begin{itemize}

\item User $i$ of FBS $f$ has signal to interference plus noise ratio (SINR)
	\begin{equation*}
	\gamma_{f,i} = \frac{\|\mathbf{h}^H_{f,i}\mathbf{u}_{f,i}\|^2}
	{\sigma^2_{\text{noise}}   +
	\underbrace{
	 \sum_{\tilde{f}=1, \tilde{f}\neq f}^{f} \sum_{u=1}^{K_{\tilde{f}}}
	\|\mathbf{h}^H_{\tilde{f},u}\mathbf{u}_{\tilde{f},i}\|^2}_{\mathrm{inter-cell}}
	 +
	 \underbrace{
	 \sum_{\tilde{k}=1, \tilde{k}\neq i}^{K_f}
	 \|\mathbf{h}^H_{f,\tilde{k}}\mathbf{u}_{f,\tilde{k}}\|^2}_{\mathrm{intra-cell}}},
	  \; i \in \{1 ... K_f\}
	  \end{equation*}
\\
with noise power $\sigma^2_{\text{noise}}.$
\\

By the FBS spacing assumption above, this reduces to
	\begin{equation*}
	\gamma_{f,i} = \frac{\|\mathbf{h}^H_{f,i}\mathbf{u}_{f,i}\|^2}
	{\sigma^2_{\text{noise}}
	 + \sum_{\tilde{k}=1, \tilde{k}\neq i}^{K_f}
	  \|\mathbf{h}^H_{f,\tilde{k}}\mathbf{u}_{f,\tilde{k}}\|^2},
	  \; i \in \{1 ... K_f\}
	\end{equation*}

\end{itemize}

\subsection{General Optimization Problem}\label{genproblem}

Each FBS $f$ attempts to maximize utility function $U_{f}(\cdot)$ while satisfying the interference constraints imposed by the macrocell users and a transmission power constraint. This results in the following optimization problem at each FBS.
\par

	\begin{subequations}
	\label{optim}
	\begin{align}
	    \underset{\mathbf{U}_{f} }{\text{minimize  }} \;
	    & - \sum_{i=1}^{K_f}
    	U_{f,i}(\gamma_{f,i}) \label{player_opt} \\
	    \text{subject to  } \; &
	   \sum^F_{f=1} \tilde{\mathbf{h}}_{m,f}^H  \mathbf{U}_{f}^*
	\mathbf{U}_{f}^{T} \tilde{\mathbf{h}}_{m,f} \leq I^{Threshold}
	_{m} & m \in \{1 ...M\}
		\label{interference_const_gen}\\
        & trace(\mathbf{U}_{f}^H\mathbf{U}_{f}) \leq P^{Total}_{f} \label{power_const_gen}\\
        & \mathbf{h}_{f,i}^H\mathbf{u}_{f,i} =0\ & \; \forall \; j \in \{1... K_{f}\}\backslash i,\; \forall \; i\in \{1 ... K_{f}\} \label{zf_const_gen}
	\end{align}
	\end{subequations}

The constraints in \eqref{zf_const_gen} imply that $\mathbf{U}_{f}$ is chosen such that
	\begin{equation}\label{zf_snr}
	\gamma_{f,i} = \frac{\|\mathbf{h}^H_{f,i}\mathbf{u}_{f,i}\|^2}
	{\sigma^2_{noise}
	}
	=
	\frac{\mathbf{u}^H_{f,i}\mathbf{h}_{f,i}\mathbf{h}^H_{f,i}\mathbf{u}_{f,i}}
	{\sigma^2_{noise}
	},
	\end{equation}
	which is a convex function with respect to $\mathbf{u}_{f,i}$.
%	As a result, $U_{f,i}(\gamma_{f,i}) $ is concave with respect to $\gamma_{f,i}$.

\subsection{Concave N-Person game analysis of general setup}
In this section, the convexity of the resulting problem is analyzed to investigate if the system satisfies the conditions of the concave N-person game described \ref{tools}.

\begin{enumerate}
\item
First, we check that constraints \eqref{interference_const_gen}-\eqref{zf_const_gen} form a convex, closed and bounded set.

\begin{itemize}

\item
	Constaint \eqref{interference_const_gen} contains $M$ quadratic constraints on $\mathbf{U}_f$, each of which
	can be rewritten  as

\begin{gather*}
	\sum_{f=1}^{f}
	trace(\mathbf{U}_{f}^H \tilde{\mathbf{h}}_{m,f} \tilde{\mathbf{h}}_{m,f}^H \mathbf{U}_f )\leq
	I^{Threshold}_{m}.
\end{gather*}
This can be further decomposed into a sum of independent, quadratic form terms
	\begin{gather*}
	\sum_{f=1}^{f}
	\sum_{i=1}^{f_i}
	\mathbf{u}_{f,i}^H\tilde{\mathbf{h}}_{m,f} \tilde{\mathbf{h}}_{m,f}^H
	\mathbf{u}_{f,i} \leq I^{Threshold}_{m}
	\end{gather*}
in which each matrix $\tilde{\mathbf{h}}_{m,f} \tilde{\mathbf{h}}_{m,f}^H$ is positive semi-definite \cite[p.~8,9]{BoV:04}. The union of independent convex sets forms a convex set \cite{BoV:04}, therefore this constraint is convex.
%This is essentially high dimensional ellipsoid.

\item \
	Constraint \eqref{power_const_gen} can be similarly decomposed into the sum
	\begin{gather*}
		\sum_{i=1}^{K_{f}}\mathbf{u}_{f,i}^{H} \mathbf{I}
		\mathbf{u}_{f,i} \leq  P^{Total}_{f}
	\end{gather*}
	in which $\mathbf{I}$ is positive definite, creating a strictly convex set.

\item
	Constaint \eqref{zf_const_gen} is an affine constraint.
		\begin{gather*}
		\mathbf{h}_{f,j}^T \mathbf{u}_{f,i} =0
		\end{gather*}
%Note that affine constaints to not have to satisfy Slater's condition
\end{itemize}

\item Second, we check if the utility function is concave.
\begin{itemize}
\item
Let us consider the SINR at femto-user i of FBS f given by
	\begin{equation*}\label{zf_snr}
	\gamma_{f,i} =
	\frac{\mathbf{u}^H_{f,i}\mathbf{h}_{f,i}\mathbf{h}^H_{f,i}\mathbf{u}_{f,i}}
	{\sigma^2_{noise}
	}.
	\end{equation*}
	Since
	 $\mathbf{h}_{f,i}\mathbf{h}^H_{f,i}$ is positive semi-definite, $\gamma_{f,i}$ is convex in ${\mathbf{u}_{f,i}}$.
	 The resulting composition $U_{f,i}(\gamma_{f,i}) $ is concave only if $U_{f,i}(\cdot) $ is concave and non-increasing; violating the non-decreasing definition of $U_{f,i}(\cdot) $.
If $U_{f,i}(\cdot) $ is a general, non-decreasing function, the result is quasiconvex \cite[p.~102]{BoV:04}. If $U_{f,i}(\cdot) $ is  non-decreasing and convex, the resulting
   function is convex which can be solved using concave programming to find a \emph{maximum}.
\end{itemize}

\end{enumerate}

\subsection{Potential Game for the General Problem}
Under the current system model, the game is not a concave N-person game. Nevertheless, since the utility function of individual FBSs are independent of the strategy chosen by other FBSs, due to the spacing assumption, the game still admits a potential function. The investigation of distributed concave programming techniques for solving this general formulation of the problem is left for future work.

\section{Concave System Model}\label{conmodel}

The general setup of the game proposed in the previous section does not enable the use of the N-person concave game framework to find normalized Nash equilibrium. We propose a more restrictive model characterized by a unique, normalized Nash equilibrium. This NNE can be be found via a single convex optimization problem which may be solved in a distributed manner across FBSs and macro-users in the network.
\par
Rather than jointly optimizing the power allocation and the zero-forcing beamformers, we now pre-select normalized, zero-forcing beamformers independently, \emph{before} the allocation of powers.
As we will see later, one result of this restriction is that the degrees of freedom available at the FBS cannot be fully exploited during power allocation.

\subsection{Players: Femtocell Base Stations}\label{conmodel_fbs}
Femtocell base stations are adapted from the general setup to the restricted setup by the following.
\begin{itemize}
\item
	FBSs with multiple antennas ($T_f \geq 1$) can beamform their transmission using the precoding
	matrix $\mathbf{U}_{f} \in \mathbb{C}_{T_{f} \times K_{f}}$ .
	The columns of $\mathbf{U}_{f}$ are now \emph{normalized} such that
	 $\|\mathbf{u}_{f,i}\|^2 =1 \;\forall i \in \{1 ... K_{f}\}$.
\\

\item
$\mathbf{U}_f$ is selected as a pseudo-inverse to $\mathbf{H}_f$.
Such that
\begin{gather*}
\mathbf{H}_{f}  \mathbf{U}_{f} = \mathbf{I}.
\end{gather*}

\item
	FBS $f$ allocates the transmit powers using the diagonal, power allocation
	matrix $\mathrm{diag}(\mathbf{p}_{f})$ with $p_{f,i} \geq 0, \forall i \in \{1 ... K_{f}\}$
such that the transmitted
	signal is
	$\mathbf{s}_{f	}= \mathbf{U}_{f}
	\mathrm{diag}(\mathbf{p}_{f})^{\frac{1}{2}}
	\mathbf{x}_{f}$. We define the vector variable $\mathbf{p}^T= [\mathbf{p}_1^T, \mathbf{p}_2^T...\mathbf{p}_{f}^T]$ containing the transmit power vectors of all FBSs.
\\
\item
	FBS $f$ enforces the power constraint given by
	\begin{gather*}
	\text{trace}(E[\mathbf{s}_f\mathbf{s}_f^H]) =
	\sum_{i=1}^{K_{f}} p_{f,i}
	  \leq P^{Total}_{f}.
	  	\end{gather*}

% TODO(Simulate degradation with incomplete CSI solution?)

\item
	Utility function of FBS f is given by $U_{f}(\boldsymbol{\gamma}_f) =
	\sum_{i=1}^{K_{f}}
    	U_{f,i}(\gamma_{f,i}) $
    	where all $U_{f,i}(\cdot)$ are non-decreasing and
    	\emph{strictly} concave.
\item The SINR of the femto-user $i$ in FBS $f$ is given by
\begin{equation}\label{zf_snr}
	\gamma_{f,i} =\frac{\sum_{i=i}^{K_{f}}
 p_{f,i}\|h_{f,i}u_{f,i}\|^2}
	{\sigma^2_{noise}
	}.
	\end{equation}

\end{itemize}

\subsection{Macrocell Users}
No change with respect to the general setup.

\subsection{Femtocell Users}\label{conmodel_fuser}
No change with respect to the general setup.

\subsection{Optimization Problem of Player $f$}\label{conproblem}
The optimization problem at FBS $f$ for the system described in \ref{conmodel_fbs}-\ref{conmodel_fuser} is given by

	\begin{subequations}
	\label{optim}
	\begin{align}
	    \underset{\mathbf{p}_{f} }{\text{minimize  }} \;
	    & - \sum_{i=1}^{K_f}
    	U_{f,i}(\gamma_{f,i}) \label{player_opt_c} \\
	    \text{subject to  } \; &
	  \sum^F_{f=1} E\{ \tilde{\mathbf{h}}_{m,f}^T  \mathbf{s}_{f}
	\mathbf{s}_{f}^{H} \tilde{\mathbf{h}}_{m,f}^* \}
	=
	\sum_{f=1}^{f}	\sum_{i=1}^{K_f}
	 p_{f,i}\|\tilde{\mathbf{h}}_{m,f}^T \mathbf{u}_{f,i}\|^2_2
	\leq I^{Threshold}
	_{m} & m \in \{1 ...m\}
		\label{interference_const_c}\\
        &
        	\sum_{i=1}^{K_{f}} p_{f,i}
	   \leq P_{f}^{\text{Total}}  \label{power_const_c}\\
        & p_{f,i} \geq 0 &  i\in \{1 ...K_{f}\} \label{pos_power_const_c}
	\end{align}
	\end{subequations}

Note that because $\mathbf{U}_{f}$ is pre-selected as a pseudo-inverse to  $\mathbf{H}_f$, the zero-forcing constraint \eqref{zf_const_gen} has been removed.

\subsection{Concave N-Person game analysis of concave setup}
The convexity of the resulting problem is analyzed to investigate if this system admits the concave N-person game framework.

\begin{enumerate}

\item
First, check that the constraints form a convex, closed and bounded set.

\begin{itemize}

\item
	Note that \eqref{interference_const_c} includes $m$ affine constraints in $\mathbf{p}_{f}$.

\begin{gather*}
	  \sum^F_{f=1} \tilde{\mathbf{h}}_{m,f}^T  \mathbf{s}_{f}
	\mathbf{s_{f}^{H}} \tilde{\mathbf{h}_{m,f}^*}
	=
	\sum_{f=1}^{f}	\sum_{i=1}^{K_f}
	p_{f,i}\|\tilde{\mathbf{h}}_{m,f}^T \mathbf{u}_{f,i}\|
	\leq I^{Threshold}_{m}
\end{gather*}

\item \
	Constraint \eqref{power_const_c} is  affine in $\mathbf{p}_{f}$.

\item \
	Constraint \eqref{pos_power_const_c} is affine in $\mathbf{p}_{f}$.
\end{itemize}

%Note that affine constaints to not have to satisfy Slater's condition

\item The utility function was defined to be concave in $p_{f,i}$.

\end{enumerate}

This problem satisfies the conditions for the concave N-person game and therefore a  normalized Nash equilibrium exists due to
\cite[Thm1]{rosen1964existence}.

Assuming continuously differentiable $U_{f,i}(\gamma_{f,i})$ and $\gamma_{f,i}(p_{f,i})$, the diagonal elements of $[G(b,r)+G^{T}(b,r)] $ correspond to
\begin{equation}
\underbrace{U^{''}_{f,i}(\gamma_{f,i}(p_{f,i}))}_{<0}\underbrace{\gamma^{'}_{f,i}(p_{f,i})\gamma^{'}_{f,i}(p_{f,i})}_{>0}
+
\underbrace{U^{'}_{f,i}(\gamma_{f,i}(p_{f,i}))}_{>0}\underbrace{\gamma^{''}_{f,i}(p_{f,i})}_{0};
\end{equation}
satisfying the  negative definite condition on $[G(b,r)+G^{T}(b,r)] $ for a unique Normalized Nash equilibrium \cite[Thm4]{rosen1964existence}.

\subsection{Potential Game for Concave Problem}
Noting that during power allocation, $\boldsymbol{\gamma}_{f}$ is only dependent upon $\mathbf{p}_f$, the potential function given by

\begin{gather*} \label{Potential_Function}
\Psi(\mathbf{p}) = \sum_{f = 1}^{F} U_{f}(\mathbf{p}_{f})
\end{gather*}
and satisfying \eqref{potential_game_condition} is proposed to state the following central optimization problem

		\begin{subequations}
	\label{optim}
	\begin{align}
	    \underset{\mathbf{p}}{\text{minimize  }}
	    & \; \Psi(\mathbf{p}) \label{potential_game} \\
	    \text{subject to  } \; &
	  \sum^F_{f=1} E\{\tilde{\mathbf{h}}_{m,f}^T  \mathbf{s}_{f}
	\mathbf{s}_{f}^{H} \tilde{\mathbf{h}}_{m,f}^* \}\leq I^{Threshold}
	_{m} & m \in \{1 ...m\}
		\label{interference_const}\\
        & E\{trace(\mathbf{s}_f\mathbf{s}_f^H)\}  \leq P_{f}^{\text{Total}}  \label{power_const}
        & \forall f \in \{1 ... f\}\\
        & p_{f,i} \geq 0 &  \forall i \in \{1 ...K_{f}\} \; \forall f \in \{1 ... F\}\label{pos_power_const}
	\end{align}
	\end{subequations}.

%\begin{theorem}\label{distributed}
%\cite{ghosh2015normalized}
%If a game's potential function is strictly concave and the derivative of the function with respect to the individual players variables are independent of the other player variables, then there exists a distributed solution.
%\end{theorem}

\subsection{Distributed Solution to the Potential Game}
A distributed solution to \eqref{optim} is now proposed using the steps from Section \ref{tools}.
\subsubsection{Central Problem Resulting from the Potential Game}\label{central_solution}
The Lagrangian of the potential game (the primal problem) is
\begin{multline}
L(\mathbf{\mathbf{p},\boldsymbol{\lambda},\boldsymbol{\chi},\boldsymbol{\nu}}) =
\;
\sum_{f=1}^F U_{f}(\mathbf{p}_{f})
+\\
\sum_{m=1}^M \lambda_{m}
(	  \sum^F_{f=1} \tilde{\mathbf{h}}_{m,f}^T \mathbf{U}_{f}
	\mathrm{diag}(\mathbf{p}_{f})^{\frac{1}{2}}
	E\{\mathbf{x}_{f}\mathbf{x}_{f}^H\}\mathrm{diag}(\mathbf{p}_{f})^{\frac{1}{2}}\mathbf{U}_{f} ^H
	 \tilde{\mathbf{h}}_{m,f}^* - I^{Threshold}
	_{m} )+
\\
\sum_{f=1}^f
\chi_{f}(trace(\mathbf{U}_{f}
	\mathrm{diag}(\mathbf{p}_{f})^{\frac{1}{2}}
	E\{\mathbf{x}_{f}\mathbf{x}_{f}^H\}\mathrm{diag}(\mathbf{p}_{f})^{\frac{1}{2}}\mathbf{U}_{f} ^H	)-P^{Total}_{f} ) +
\sum_{f=1}^f
\sum_{i=1}^{K_{f}}
\nu_{f,i}(-p_{f,i}).
\end{multline}
Using $\mathbf{p}= [\mathbf{p}_1, \mathbf{p}_2...\mathbf{p}_{f}],$ the corresponding dual problem is given by
\begin{gather*}
\underset{\boldsymbol{\lambda},\boldsymbol{\chi},\boldsymbol{\nu}}{\mathrm{maximize}}\;\underset{\mathbf{p}}{\mathrm{argmin}}\;L(\mathbf{\mathbf{p},\lambda}).
\end{gather*}
		\begin{subequations}
	\label{optim}
	\begin{align}
\underset{\boldsymbol{\lambda},\boldsymbol{\chi},\boldsymbol{\nu}}{\mathrm{maximize:}}&&\underset{\mathbf{p}}{\mathrm{argmin}}\;L(\mathbf{\mathbf{p},\lambda})\\
\text{subject to }&& \boldsymbol{\lambda},\boldsymbol{\chi},\boldsymbol{\nu} \geq \boldsymbol{0}
	\end{align}
	\end{subequations}.
This dual function can be decomposed into F component functions

\begin{multline}
g_f(\boldsymbol{\lambda},\boldsymbol{\chi},\boldsymbol{\nu}) = \underset{p_{f}}{\mathrm{argmin}}
\{
\;
U_{f}(\gamma_{f})
\\+
\sum_{m=1}^M \lambda_{m}
(\tilde{\mathbf{h}}_{m,f}^T  \mathbf{U}_{f}
	\mathrm{diag}(\mathbf{p}_{f})^{\frac{1}{2}}
	\mathrm{diag}(\mathbf{p}_{f})^{\frac{1}{2}}\mathbf{U}_{f} ^H \tilde{\mathbf{h}}_{m,f}^* - \frac{I^{Threshold}_{m}}{F})
\\
+
\chi_{f}(trace(\mathbf{U}_{f}
	\mathrm{diag}(\mathbf{p}_{f})^{\frac{1}{2}}
	\mathrm{diag}(\mathbf{p}_{f})^{\frac{1}{2}}\mathbf{U}_{f} ^H)-P^{Total}_{f} )
+
\sum_{i=1}^{K_f}
\nu_{f,i}(-p_{f,i})\}
\end{multline}
\\

The problem can now be solved using a distributed version of dual ascent to reach the unique NE of the potential game.
\subsubsection{Distributed Algorithm}\label{algo1}
The non-decreasing, strictly concave utility function $U_{f,i}(\gamma_{f,i}) = log(1+\gamma_{f,i})$
is used in the following algorithms and simulation results.

\begin{enumerate}
\item
Individual players (FBSs) can solve $ g_f(\boldsymbol{\lambda},\boldsymbol{\chi},\boldsymbol{\nu})$ independently and analytically by setting $\frac{\partial g_f(\boldsymbol{\lambda},\boldsymbol{\chi},\boldsymbol{\nu})}{\partial p_{f,i}} = 0$
and solving for the power of user $i$ of FBS $f$, $p_{f,i}$, using

\begin{gather}
p_{f,i} = \frac{1}{(\sum_{m=1}^{m}\lambda_{m}\|\tilde{\mathbf{h}}_{m,f}^T \mathbf{u}_{f,i}\|^2_2
+\chi_{f} \|\mathbf{u}_{f,i}\|^2_2
-\nu_{f,i}
 )}-
  \frac{\sigma^2_{\text{noise}}}{\|\mathbf{h}_{f,i}^T \mathbf{u}_{f,i}\|^2_2}.
\end{gather}

\item
With $g(\boldsymbol{\lambda},\boldsymbol{\chi},\boldsymbol{\nu}) = \sum_{f=1}^{F}g_f(\boldsymbol{\lambda},\boldsymbol{\chi},\boldsymbol{\nu})$ the dual variables are updated independently in the direction of the positive gradient of the dual function using

\begin{gather}
\lambda_{m}^{k+1} = (
\lambda_{m}^{k}
+
\alpha^{k}*
(
\underbrace{
\sum _{f=1}^{f}
\sum _{i=1}^{K_{f}}
p_{f,i}
\|\tilde{\mathbf{h}}_{m,f}^T \mathbf{u}_{f,i}\|^2_2}_{\text{Interference created}}
- I_{m}
))^+
\end{gather}
 at the macrocell users,

\begin{gather}
\chi_{f}^{k+1} = (
\chi_{f}^{k}
+
\alpha^{k}*
(\underbrace{\sum_{i=1}^{K_{f}} p_{f,i}}_{\text{Used power}} - P_{f}^{Total}) )^+
\end{gather}
 at individual FBSs, and

\begin{gather}
\nu_{f,i}^{k+1} = (
\nu_{f,i}^{k}
+
\alpha^{k}*
(-p_{f,i}))^+
\end{gather}
also at individual FBSs.
The algorithm directly enforces non-negativity using $(\cdot)^+$.
The step size $\alpha^{k}$ is predefined and constant .
 % must satisfy  $\sum_{\mathrm{k=1}}^{\infty} \alpha^{k}$ (cite).

\end{enumerate}
These local updates impose additional information passing in the network. In particular, macrocell users must monitor their incoming interference then broadcast the updated dual variable $\lambda_{m}^{k+1}$ with which FBSs can update  transmit power. In contrast, however, a central solution would require the same information plus addition information from FBSs to be passed to a central location, a much higher overhead on the system than the proposed distributed solution.
(Note also that the dual ascent algorithm uses the dual and thus cannot guarantee a feasible solution to the primal problem originally posed.)

\subsection{Beamformer Optimization for Convex Setup}\label{beamformer_optimization}
When additional degrees of freedom for the choice of $\mathbf{U}_{f}$ are available after satisfying the zero-forcing constraint (i.e. $T_{f}> K_{f}$ ), a preliminary optimization can be performed. In the following, variations of an objective function are considered. Note that because the beamformer $\mathbf{U}_{f}$ is chosen prior to the power allocation, the final values of $p_{f,i}$ are not yet known for this optimization.
\par
The first objective function considered corresponds to the Moore-Penrose pseudoinverse of $\mathbf{H}_{f}$.
    \begin{equation}
    \begin{array}{ll}
    \underset{\mathbf{U}_{f} }{\text{minimize }}   & tr(\sigma_n\mathbf{U}_{f}  \mathbf{U}_{f}^H)
    \\
    \mbox{subject to } & \mathbf{H}_{f}  \mathbf{U}_{f} = \mathbf{I}
    \end{array}
    \label{mp_opt}
    \end{equation}
    in which $\sigma_n$ is the constant receiver noise variance.
    This problem is convex and therefore can be found easily at individual FBSs.

    \par
Alternatively, we can optimize to minimize the correlation of the chosen beam-former $\mathbf{U}_{f}$ with the known macrocell user channels $\tilde{\mathbf{h}}_{f}$.

    \begin{equation}
    \begin{array}{ll}
    \underset{\mathbf{U}_{f} }{\text{minimize }}   & \sum^F_{f=1} \sum^{K_f}_{i=1}\|\tilde{\mathbf{h}}_{f}  \mathbf{u}_{f,i}\|^2
    \\
    \mbox{subject to } & \mathbf{H}_{f} \mathbf{U}_{f} = \mathbf{I}
    \end{array}
    \label{correlation_opt}
    \end{equation}
This is problem also convex in $\mathbf{U}_{f}$ and can be found at individual FBSs.

\par
Lastly, we consider the case in which FBSs minimize correlation with macrocell users while also maximizing correlation with the channels of its own users. The relative importance of these two terms is weighted by $\alpha.$
    \begin{equation}
    \begin{array}{ll}
    \underset{\mathbf{U}_{f} }{\text{minimize }}   & \alpha\sum^F_{f=1} \|\tilde{\mathbf{h}}_{f}  \mathbf{U}_{f}\|^2_2
    -
    (1-\alpha)\sum^F_{f=1} \|\mathbf{H}_{f}  \mathbf{U}_{f}\|^2, \; 0\leq \alpha \leq 1
    \\
    \mbox{subject to } & \mathbf{H}_{f}  \mathbf{U}_{f} = \mathbf{I}
    \end{array}
    \label{e-opt-prob}
    \end{equation}
    This problem is not convex in $\mathbf{U}_{f}$ since the first term of the utility function is convex and the second, subtracted term is concave. Combined with the convex zero-forcing constraint, this type of problem, known as a difference of convex functions problem (DC), has been extensively studied and can be solved (at least to determine local optima) via DC programming methods. Note that as this optimization is performed locally at a FBS, there is no restriction to distributed methods.

\chapter{Simulation and Results}\label{numerical}
\subsection{Simulation System Description}
The heterogeneous network is simulated by randomly placing $F$ FCBSs and $M$ macrocell users within a predefined area. $K_{f}$ femto users are served by each FBS $f$ and channels between FBSs and users (both femto and macro) are modeled as Rayleigh fading with a pathloss attenuation coefficient $d^{- \beta}$ with $\beta=2$ and $d$ as the distance between user and base-station. Case-specific parameters of the simulation environment are detailed in the following.

\subsection{Simulation Results: Central Solution}
As shown in Section \ref{central_solution} the game has a unique NE. This ensures the equivalence of any central or distributed solution to the system game. Here we consider simulation results from the central solution. These results confirm the expected behavior of the system and offer insight into further optimization. In order to compare the performance of different systems (e.g. transmit power or number of antennas) we define the social utility as the sum of the utilities of all FCBSs. In addition to the social utility, we also compare the slack in the system constraints. By observing the constraint slack, we can compare when the FCBSs are limited by either macro user interference or transmission power. In the following, the systems with parameters defined in Table \ref{table1} are investigated empirically.

\begin{table}[H]
\centering
\begin{tabular}{ | m{8cm} | m{5cm} | }
\hline
Simulation Area & $30 \times 30$\\
\hline
Number of macrocell users & \ref{central_power}, \ref{central_antenna} : 10, \ref{central_beamformer}: 20\\
\hline
Number of FBSs & 5\\
\hline
Number of users for each FBSs & 5\\
\hline
Number of antennas for each FBSs & \ref{central_power}: 5, \ref{central_antenna}: variable, \ref{central_beamformer}: 15 \\
\hline
FBS transmit power constraint & \ref{central_power}: variable , \ref{central_antenna} - $20$ dB, \ref{central_beamformer}: variable   \\
\hline
Noise power at femto cell users & $0$ dB ($1$)\\
\hline
Macro user interference threshold & \ref{central_power}, \ref{central_antenna} : $0$ dB, \ref{central_beamformer}: $-1$ dB\\
\hline
\end{tabular}
\caption{\label{table1} System simulation parameters for Figures \ref{central_power}, \ref{central_antenna} and \ref{central_beamformer}}
\end{table}

In Figure \ref{central_power}, it is seen that with a constant number of FCBS antennas, increasing the available transmission power can increase the social utility only up to the point at which the macro user interference constraints are tight. This result motivates the results in Figure \ref{central_antenna} comparing the case in which the number of antennas is increased while all FCBSs are provided with excess transmission power. Figure \ref{central_antenna} indicates that increasing the number of antennas allows FCBSs to transmit with more power while still satisfying the interference constraints.
\par
In Figures \ref{central_power} and \ref{central_antenna} the beamforming matrix at each FCBS was chosen according to \ref{mp_opt} (the Moore-Pensrose pseudo-inverse). We now compare the system when FCBSs choose their beamformer, $\mathbf{U}_f$, according to  \eqref{mp_opt} or \eqref{correlation_opt}. In Figure \ref{central_beamformer} it is observed that from a social utility perspective, the optimal choise for $\mathbf{U}_f$ depends on whether transmission power or macro user interference is the limiting constraint. This observation motivates future investigation of a FCBS control loop for choosing $\mathbf{U}_f$ based on \eqref{mp_opt} or \eqref{correlation_opt} (or any other choice of the pseudo-inverse) by identifying the limiting constraint for each FCBS.

\begin{figure}[H]
	\includegraphics[width= 15cm,height = 10cm]{figures/central_power}
	  \caption{Increasing Power}
	  \label{central_power}
\end{figure}

\begin{figure}[H]
	\includegraphics[width= 15cm,height = 10cm]{figures/central_antenna}
	  \caption{Increasing Antenna}
	  \label{central_antenna}
\end{figure}
\begin{figure}[H]
	\includegraphics[width= 15cm,height = 10cm]{figures/central_beamformer}
	  \caption{Beamformer Comparison}
	  \label{central_beamformer}
\end{figure}

\chapter{Conclusion}
In this work, the power allocation problem for heterogeneous networks with multi-antenna femto cell base stations has been analyzed. The most general setup of this problem poses difficulties  due to the non-concavity of the resulting utility functions. It is shown that under additional beamforming constraints on base stations, a unique, normalized Nash equilibrium can be achieved. Simulations confirmed these results and motivated the need for further optimization of the beamformers used at femtocell base stations.
\par
Future work should extend the system presented here to further generalizations including interference between the femtocell base stations and solutions which enforce solution fairness between players.