\chapter{Attack Design}

In this section, we describe the overall design of our automatic feature generation models, outline the attack strategy and explain certain design decisions.
Most of this section will be split into two different parts.
First, we describe the feature generation processes and then the overall attack.

\section{Stacked Autoencoder}

A stacked autoencoder takes a fixed-length input vector and tries to learn a function that compresses that vector.
Thus if we were to use it for a fingerprinting extraction task, we will need to preprocess the variable-length traces into a fixed-length input.
There are various different manners of doing this.
One of the most naive ones is to find the longest length trace and pad all of the other traces up to that length.
However, in Greschbach et al.'s dataset, this length is around $250,000$ \cite{greschbach2016effect}, which means that our network will need to be incredibly deep to extract a short trace.

Instead, we can pick the average or median length of the traces and cut or pad traces which are longer or shorter.
In this work we will investigate the average length, which is around $3,000$.
But the median length might be slightly more appropriate, as the length is positively skewed.
On top of that, we deal with the fact that each packet is represented by a tuple, by just multiplying the time by the direction.
Therefore, all of the outgoing packets are positive whilst the incoming ones will be negative.

Finally, since the output of the activation functions used is either between 0 and 1 or -1 and 1, we will scale the inputs accordingly.

\section{Sequence-to-Sequence Model}

As described in section \ref{sec:seq2seq}, a sequence-to-sequence model is able to learn how to construct a fixed-length representation from a variable-length sequence, which means that we will not have to perform as much preprocessing as for our autoencoder.

Although we already know the overall structure, there are still a couple of design decisions that had to be made specifically for our attack.
These decisions are outlined in the following section.

One of the first parameters which we will examine is the \textit{amount of hidden states} in the RNN cells.
This number affects the amount of neurons for each layer with the cells.
For instance if the amount of hidden states is set to $100$, the state of an LSTM cell is represented by vector of length $200$ (two vectors of length $100$).
The higher this number, the easier it should be to learn a representation, as the compression factor is lower.
But we also need to consider the fact that the higher the amount of hidden neurons, the more variables the model needs to learn and thus the more complex the model will be are.

Each value in a trace can be represented by a vector of length two (timestamp and direction), as seen in table \ref{table:cell-extract}.
Therefore, if the amount of hidden cells is not equal to two, we will also need to \textit{project} the input and output vectors into the necessary dimensions.

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.7\textwidth]{lstm-projection}
  \caption{Example of projection within a LSTM cell with 4 hidden states.}
  \label{fig:lstm-projection}
\end{figure}

\newpage

Some of the traces can be particularly long and therefore the network needs to be unrolled to extreme lengths \cite{greschbach2016effect}.
In fact, given memory constraints, this becomes a major problem.
But it can be solved by cutting the traces after a couple seconds since it has been shown that the first part of a trace carries more information than the latter.

\begin{figure}[ht]
  \centering
  \includegraphics[width=\textwidth]{full-seq2seq}
  \caption{Overal structure of our sequence-to-sequence model.}
  \label{fig:full-seq2seq}
\end{figure}

Based on the above, we can construct our \textit{computational graph} in Tensorflow as outlined in figure \ref{fig:full-seq2seq}.

\section{Attack Strategy}

Here we consider an adversary that relies on deep learning to extract fingerprints for a website fingerprinting attack.
This adversary can have two different goals in mind, as previously stated in section \ref{sec:threat-model}.
The full attack, however, can be split up into four different stages, namely \textit{data collection}, \textit{fingerprint extraction training}, \textit{classifier training} and \textit{the attack}.

\noindent
\textbf{Data Collection}
\begin{enumerate}
  \item Choose web pages that the attacker wishes to monitor.
  \item Collect traffic for a set of monitored and unmonitored sites.
  \item Convert the raw TCP data into Tor cells.
  \item Remove SENDMEs and other noise.
\end{enumerate}

\noindent
\textbf{Fingerprint Extraction Training}
\begin{enumerate}[resume]
  \item Further process the data into batches and perform any other preprocessing required by the model such as cutting or padding the traces.
  \item Prepare the fingerprint extraction model.
  \item Train the fingerprint extraction model on a copy task for monitored and some unmonitored web pages.
  \item Extract fingerprints from data by using the trained model.
\end{enumerate}

\noindent
\textbf{Classifier Training}
\begin{enumerate}[resume]
  \item Given a classifier, train it using the extracted fingerprints.
  \item Measure performance of classifier.
\end{enumerate}

\noindent
\textbf{The Attack}
\begin{enumerate}[resume]
  \item Passively capture traffic from Tor users.
  \item Pre-process the collected data.
  \item Extract fingerprints using the trained fingerprint extraction model.
  \item Classification via the trained classifier.
\end{enumerate}

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.7\textwidth]{overall-structure}
  \caption{Attack strategy.}
  \label{fig:attack-strategy}
\end{figure}

\subsection{Data Collection} \label{sec:data_collection1}

As previously mentioned, the data collection process first requires the adversary to choose a set of $n$ websites to monitor.
Next, the adversary crawls these pages a total of $i$ iterations to ensure that a classifier has enough data to generalize.
As suggested by Wang et al. when performing page loads, browser caching should be disabled since Tor does not allow caching to disk and therefore the browser cache should be cleared every time a web page is loaded \cite{wang_goldberg_2013}.
After the collection, the TCP data is converted into Tor cells and probabilistic algorithms, designed by Wang et al. \cite{wang_goldberg_2013}, are used to remove SENDMEs from the data.
This data can be processed further, however we chose not to since the model should be able to learn how to perform this processing.

Most of our analysis will be done using the dataset provided by Greschbach et al. \cite{greschbach2016effect}, which can be used for open-world analysis since it provides us with $100$ samples for Alexa's top $9000$ websites and one sample of one sample for $909,000$ unmonitored sites.
For the rest of this paper, we will refer to this dataset as \texttt{GRESCHBACH}.
Next, to see how our model performs on data, which was recorded at a different time and under different circumstances, we will also be using the dataset provided by Wang et al. \cite{wang_cai_johnson_nithyanand_goldberg_2014}, which we will call \texttt{WANG14} \cite{panchenko2}.
This set is slightly smaller with $100$ monitored websites with $90$ instances each and $8400$ unmonitored sites.

\subsection{Fingerprint Extraction Training} \label{sec:fingerprint-extraction-training}

In order to truly evaluate the model, we need to split the data up into a training and a validation set.
We do not train the model on any data in the validation set but instead use it to see how well the model performs on unseen data.
For this split, we use a \textit{stratified shuffle split}, meaning that we shuffle the data and then perform the split, whilst preserving the class distributions.
On top of training the feature extractor with monitored pages, we also train it on unmonitored pages, as it needs to be able to extract features effectively from both sets.

During training and extracting the fingerprints, \textit{mini-batch processing} will always be used.
This will allow us to gain a performance boost and perhaps even have a faster convergence.
When dividing the data up into these batches, we also need to determine how big they will be.
The bigger they are, the larger the performance gain will be but the lower the accuracy might be.
Additionally, the size of the batches also depend on the amount of available memory since we cannot have the VM run out of memory whilst training.

On top of determining the batch size, the individual models require different preprocessing steps and tuning of different parameters.

\subsubsection{Stacked Autoencoder}

As previously mentioned, after dividing the data up into batches, we either need to cut or pad the traces such that they all are of a fixed-length.
Next, we know that the first layer needs to have the same amount of neurons than the length of the input.
But after that, there are a variety of different architectures that need to be chosen.
Some of which are outlined below:

\newpage

\begin{itemize}
  \item How many hidden layers we want.
    The more there are, the more complex the function that can be modeled but also the harder it becomes to train the model.

  \item The amount of hidden neurons in each layer.
    This number should gradually decrease to the number of features we would like to extract.

  \item The activation function of the neurons.
    The most popular ones being \textit{sigmoid}, \textit{ReLu} and an \textit{atan}.

  \item Whether or not to include batch normalization at every step.
\end{itemize}

After that the model has been constructed, there are still several learning parameters that need to be tuned:

\begin{itemize}
    \item The optimizer to use (\textit{adam}, \textit{gradient descent} or \textit{RMSProp}) \cite{tensorflow}.
    \item Learning rate ($\gamma$) for the previously chosen optimizer.
    \item Amount of traces within a single mini-batch ($m$).
    \item Cost, or loss function ($f$) to minimize (\textit{mean squared error} (MSE), \textit{absolute loss} (AL) or \textit{cross-entropy}) \cite{tensorflow}.
\end{itemize}

\subsubsection{Sequence-to-Sequence Model}

Each batch can either be presented in \textit{batch-major} or \textit{time-major} form.
Although time-major is slightly more efficient \cite{tensorflow}, we opt for a batch-major form, since it makes the fingerprint extraction process easier.
Next, after the data has been divided into mini-batches, we perform some further processing such as cutting the traces after several seconds.
Finally, since all of the traces within a batch need to be of the same length, padding is performed as a final preprocessing step.

After collecting and fully preprocessing the data, the adversary can start to construct the sequence-to-sequence model.
However, in order to do so, there are a variety of different architectures that need to be considered, some of which are outlined below:

\begin{itemize}
  \item Which sort or RNN cells to use. The most popular ones being GRU or LSTM cells.
    We could also potentially investigate the usefulness of multilayered RNN cells but we expect the performance gain to be limited, which is why we do not consider them in our evaluation.

  \item Whether or not to use a bidirectional encoder to ensure that the output at time $t$ is not only affected by past information but also on future information.

  \item The amount of hidden states within a RNN cell, which affects the size of the fingerprints.
\end{itemize}

Now that the model has been constructed, the adversary again has to chose various learning parameters such as:

\newpage

\begin{itemize}
  \item The optimizer to use (\textit{adam}, \textit{gradient descent} or \textit{RMSProp}) \cite{tensorflow}.
  \item Learning rate ($\gamma$) for the previously chosen optimizer.
  \item Amount of traces within a single mini-batch ($m$).
  \item Cost, or loss function ($f$) to minimize (\textit{mean squared error} (MSE), \textit{absolute loss} (AL) or \textit{cross-entropy}) \cite{tensorflow}.
  \item After how much time the traces are cut.
\end{itemize}

After these parameters have been tuned, the computational graph can be constructed in Tensorflow and the model can be trained.
When this training has been completed, fingerprints can finally be extracted for all the traces in the test set.

\subsection{Classifier Training} \label{sec:classifier-training}

When the adversary has extracted the fingerprints for websites within the test set, they need to train a classifier on those fingerprints.
Most works so far rely on some sort of \textit{supervised machine learning} techniques such as \textit{support vector classifiers} (SVC), \textit{k-nearest neighbours} (kNN), \textit{random forests} (RF) or \textit{naive bayes} (NB) \cite{panchenko1,panchenko2,wang_cai_johnson_nithyanand_goldberg_2014,kfingerprinting,naivebayes}.
All of these algorithms rely on different techniques but an explanation of their inner workings is outside the scope of this paper.
Instead, we will consider them as \textit{black box models}.
This means that all we know is that we can apply a \texttt{fit} function to the models, which causes them to learn how to classify the fingerprints and a \texttt{predict} function, which predicts the classes of given inputs.

To measure the performance of our black-box models, we use a similar technique as we did in the previous section.
We split our test set up into two more sets, a \textit{classifier training set} and a \textit{classifier test set}.
But since training a classifier, requires less time, we can use another technique, called \textit{stratified k-fold validation}.
Here we split our original test set up into $k$, mutually exclusive, folds.
Next, one of the folds is chosen to be the classifier test set and all of the other folds form the classifier training set.
This process is repeated for $k$ iterations, where at each iteration, a different fold is chosen to be the test set.
But again, we preserve the class distributions within all of the folds.

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.6\textwidth]{kfold}
  \caption{Example of one iterations in a k-fold validation $(k = 7)$.}
  \label{fig:kfold}
\end{figure}

For each iteration of the validation process, several statistics can be recorded and then averaged over all iterations.
The k-fold validation process ensures that every data point will be in the test set at least once and therefore giving us an accurate measure of these statistics.

Some of the performance measures that we will use in the evaluation stage are outlined below.
Please note that we describe them within the context of a WF attack:
\begin{itemize}
  \item \textbf{True Positive Rate (TPR)} is the probability that a monitored page is classified as the correct monitored page \cite{kfingerprinting}.

  \item \textbf{False Positive Rate (FPR)} is the probability that an unmonitored page is incorrectly classified as a monitored page \cite{kfingerprinting}

  \item \textbf{Bayesian Detection Rate (BDR)} is the probability that a page corresponds to the correct monitored page, given that the classifier recognized it as that monitored page \cite{kfingerprinting}.
    This can be calculated as follows:
    $$\text{BDR} = \frac{\textit{TPR} \times \Pr(M)}{\textit{TPR} \times \Pr(M) + \textit{FPR} \times \Pr(U)}$$
    where
    $$\Pr(M) = \frac{|\text{Monitored}|}{|\text{Total Pages}|}, \quad \Pr(U) = 1 - \Pr(M)$$

    This measure essentially indicates the practical feasibility of the attack, as the adversary is mainly concerned with this specific measure \cite{kfingerprinting}.

  \item \textbf{Accuracy (A)} is the percentage of correctly classified instances.
    Although it can be used as a rough indicator, it will not be used in the final conclusions because of the \textit{accuracy paradox}, which arises due to class imbalance.

  \item \textbf{F1-Score (F1)} measures the harmonic mean between precision and recall \cite{scikitlearn}.
    \begin{align*}
      \text{F1} &= 2 \times \frac{\text{precision} \times \text{recall}}{\text{precision} + \text{recall}}\\
                &= \frac{2 \times \textit{TP}}{2 \times \textit{TP} + \textit{FP} + \textit{FN}}
    \end{align*}
    where
    $$\text{recall} = \frac{\textit{TP}}{\textit{TP} + \textit{FN}}, \quad \text{precision} = \frac{\textit{TP}}{\textit{TP} + \textit{FP}}$$

    This measure is particularly useful since it is not affected by class imbalance.
\end{itemize}

Rather than using another classifier, we could potentially add a \textit{softmax layer} on top of our encoder, like V. Rimmer uses on top of her stacked autoencoder \cite{deeplearningthesis}.
We could use this idea for both our autoencoder and the sequence-to-sequence model.

This process would involve first training the sequence-to-sequence model, then stacking the softmax layer on top of each unrolled cell in the encoder and using it for classification.
What would be interesting about this approach is that the adversary can analyse how certain the classifier is, as it analyses packets more and more packets from the trace.
But unfortunately this is outside the scope of this paper, as we are solely focusing on the feature extraction process.

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.55\textwidth]{softmax}
  \caption{Example of encoder with softmax layer for 3 web pages.}
  \label{fig:softmax}
\end{figure}

\newpage

\subsection{The Attack}

Finally, after the adversary has trained the required models, the real WF attack can start.
First, the adversary starts capturing web traffic data between the user and the entry guard, as shown in figure \ref{fig:threat_model}.
Next, the data is preprocessed for the fingerprint extraction process, as described in section \ref{sec:fingerprint-extraction-training}.
After all the preprocessing has finished, fingerprints are extracted using the previously trained model.
Finally, those fingerprints are used as features for a classifier, which classifies the traffic into web pages.

The time between data collection for training and performing the WF has to be kept as small as possible since Juarez et al.'s experiments show that web pages' content changes greatly over time, therefore affecting the accuracy of the attack \cite{wfpevaluation}.

\section{Code Structure} \label{sec:code-structure}

In this work, we will not be conducting the final stage of a WF attack.
Instead, we will be reporting the results on the test sets.
All of this is reflected in the overall structure of the code, which consists of four main components, as can be seen in figure \ref{fig:code-structure}.

\begin{figure}[ht]
  \centering
  \includegraphics[width=\textwidth]{code-structure}
  \caption{Diagram of how different components are related.}
  \label{fig:code-structure}
\end{figure}

All of the data that is used within this work has already been preprocessed into Tor cells with SENDMEs removed, as described in section \ref{sec:data_collection1}.
The \texttt{feature\_generation} module contains all of the code to further preprocess the data, train the fingerprint extraction models and to extract the fingerprints using the respective model.

The classifiers that will be used for the attacks are defined in the \texttt{attacks} module.
We tried to pick a variety of existing models to measure how well our extracted fingerprints work on different classifiers.
The code to actually test the attacks is defined in the \texttt{run\_models} module, which also does some data preprocessing, defines the logic for the stratified k-fold validation and the different scoring methods.

\newpage

Finally, there is also the \texttt{feature\_extraction} module, which is used to extract hand-picked features from the raw data.

As can be seen, all of the code is written in Python, due to the wide availability of machine learning tools.
Except for the \texttt{kNN.go} file, in the \texttt{feature\_extraction} module, which is adapted from Pylls and written in \textit{Golang} to gain a performance boost \cite{gokNN}.


\newpage
\chapter{Conclusion}

The recognition of musical notes played in a song is a complex problem to deal with because of the polyphonic aspect of music and it's still an unsolved problem. The neural network has to be able to distinguish the notes from different instruments. Because of this, a Constant-Q transform was applied. This removes instruments from the equation, normalizing the data and obtains valuable information from the song. The MAPS dataset~\cite{maps} provides raw labels for each song but they don't fit the expected output of the network. Because of this, each MIDI provided needs to be analyzed and one-shot encodings of size 128 are extracted for each $\frac{1}{6}$ of a second window of the song to determine what notes are played during this time frame. The windows are then converted into wav format and then into spectrograms using the Librosa~\cite{librosa} library. The spectrograms are then fed to the network's learning process. The learning set is split $80\%$ for training, $20\%$ for testing and $10\%$ of the training set is used for validation.

\par
The output is a one-shot encoding of each window. These are then glued together and a MIDI file is created. From this MIDI file, a wav file is exported for comparison and the music sheet is generated using the Sheet freeware~\cite{sheet}.

\par
The proposed model reached a 99.9\% accuracy for the monophonic part of the problem. With a combination of one, two and three overlapping notes with no more than 50\% and 10\% overlapping time for two and three notes respectively the system achieved an accuracy of 70\% which is the state of the art method from Google Magenta. That accuracy is achieved for songs way more complicated, but their system is far more complex than the one proposed in this thesis. Going into more complicated songs with the proposed model drops the accuracy significantly as it can't give a strong enough prediction about any playing notes at a certain time if there's too much noise and overlapping sounds.

\par

The proposed solution for automatic music transcription consists of creative and heuristic methods to overcome the lack of digital processing knowledge necessary to solve this problem. There are ways to improve accuracy further:

\begin{itemize}
	\item A long short-term memory network could be added to aid with the classification and detect the length of the notes
	\item Genre detection could also aid with the classification as it will tell if a note makes sense in a progression
	\item Further improvements in the preprocessing. By changing the spectrogram from a STFT to a Constant-Q the accuracy improved significantly. There are more ways the make the input clearer by reducing the noise and improve the window splitting. For this task more knowledge of digital signal processing is needed.
	\item Train on more data. Converting one window takes ~0.3 seconds. A song is split into $16*n$ windows, where n is the song length in seconds. More than two billion MIDI windows were created but only 160 thousand were actually converted into spectrograms due to time constraints.
\end{itemize}

The final goal for Theia7 is to be integrated into a mobile music teaching and music sheet database application, where people could share their ideas and music sheets with everyone. Theia7 would aid people with no musical theory knowledge by creating the musical sheets for them. This would disrupt the market as many people ask for money in exchange for their music sheets, a common practice for YouTube cover channels.


\chapter{\label{chap:Appendix A}Model parameters: Chapters 1--4}

For each of the basic exemplar models for which simulations were run,
the following parameter values were used:

\begin{table}[H]\footnotesize
\caption{Simulation parameter values}
\begin{tabular}{lS[table-format=1.1]cS[table-format=1.1]S[table-format=1.1]S[table-format=1.1]cc}
\lsptoprule
 & {$\varepsilon$} & {$\sigma_{\text{error}}$} & $\alpha$ & {p} & {$\beta$} & $\sigma$ & {N}\tabularnewline
\midrule
Baseline Model (\chapref{ch:The-Exemplar-Model}) & .3 & $\sigma$ & .1 & – & – & 2 & –\tabularnewline
Model 1: Context-Free (\sectref{subsec:Model-1:-Context-Free}) & .3 & $\sigma$ & .5$\sigma$ & – & – & 2 & –\tabularnewline
Model 2: Context-Dep. (Gradient) (\sectref{subsec:Phrase-Final Lengthening}) & .3 & $\sigma$ & .1 & .25 & – & 2 & –\tabularnewline
Model 3: Context-Dep. (Discrete) (\sectref{subsec:Model-3:-Categorical}) & .3 & $\sigma$ & – & .5 & – & 2 & –\tabularnewline
Soft-Target Model (\sectref{subsec:Soft-Targets}) & .3 & $\sigma$ & .1 & .25 & .6 & 2 & 50\tabularnewline
\lspbottomrule
\end{tabular}

\end{table}

\chapter{\label{chap:Appendix B}The frequency effect}

This material is supplemental to Chapters \ref{subsec:Model-1:-Context-Free}
and \ref{subsec:Word-Frequency} of the main text.

The iterative model implies that the frequency effect must arise in
the lifetime of the speaker, and only after they have had sufficient
exposure to a given (high frequency) category. This may happen very
quickly. However, the less time it takes, the more opportunities there
will be for lower-frequency categories to “catch up”. Therefore,
in order to give the best chance to the basic model, we will assume
the largest possible time period in which the effect could arise:
the age of the experimental population for which frequency effects
are found. As the pool of participants for psychology and linguistics
experiments is most often university undergraduates, we will take
20 years to be the maximum amount of time necessary to produce a reduction
in duration comparable to what has been reported in the literature.

We don't know how many model iterations correspond to 20 years. But
we will define the number of productions during this time, for a word
of frequency \emph{f}, as $n_{f}$, and the proportion by which it
is reduced, as $\delta_{n_{f}}$, from an initial average duration
of $\overline{d_{0}}$. This period of time will be called an epoch
(\emph{e}).
\begin{equation}
\overline{d_{n_{f}}}=\overline{d_{0}}-\delta_{n_{f}}\overline{d_{0}}\label{eq:epoch-dur}
\end{equation}
To simplify the problem, we will consider a scenario in which there
is only a single token belonging to each category, located at the
category mean, which is replaced, each time production occurs, by
a token reduced by a fixed proportion of the current duration. With
this simplification all categories will reduce faster, since it is
always the most reduced token that is chosen in production. However,
since all measures are comparisons between categories of different
frequencies (rather than absolute values), this should not affect
the result. Low and high frequency categories are also of exactly
the same size token-wise in this simplified scenario, and only update-rate
differentiates them. Equalizing low- and high -frequency categories
in this way does affect the outcome, as we saw in \sectref{subsec:Model-1:-Context-Free},
but it advantages the basic model by ensuring that higher-frequency
words are always shorter than lower-frequency ones.

In the simplified scenario, each generation is exponentially more
reduced than the last. From Eq (\ref{eq:linear bias}): $x_{o(+n)}=x_{o}\left(1-\alpha\right)^{n}$,
we can derive Eq. (\ref{eq:Reduction}), which expresses the duration,
after 1 epoch, for a word category of frequency \emph{f}, and an
initial average duration of $\overline{d_{0}}$. Rewriting Eq. (\ref{eq:linear bias})
in terms of these variables:
\begin{equation}
\overline{d_{n_{f}}}=\overline{d_{0}}(1-\alpha)^{n_{f}}
\end{equation}
Substituting in from Eq. (\ref{eq:epoch-dur}):
\begin{equation}
\overline{d_{0}}-\delta_{n_{f}}\overline{d_{0}}=\overline{d_{0}}(1-\alpha)^{n_{f}}
\end{equation}
And,
\begin{equation}
\delta_{n_{f}}=1-(1-\alpha)^{n_{f}}\label{eq:Reduction}
\end{equation}

We don't know what the amount of reduction over 1 epoch is. But we
do have an idea of the size of the frequency effect: word duration
as a function of frequency (log frequency is typically what is plotted
in order to make the frequency distribution closer to Normal (see
e.g. \citet{gahl2012reduce})). If we assume a linear relation between
word duration and log frequency, then for each unit change in log
frequency, the difference in word duration should be equal to a constant
value (\emph{b}). Thus, the predicted difference in duration between
a low frequency and high frequency word is related to the difference
in frequencies by the following formula:
\begin{equation}
\frac{\Delta d_{e}}{\log(f_{L})-\log(f_{H})}=b\label{eq:log-linear}
\end{equation}
If speakers/listeners begin at birth with equal experience of all words
– meaning, none – then the differences in duration that accrue over
the course of an epoch will be due entirely to the amount of reduction
that occurs over that epoch. By the time that one epoch has passed,
the higher frequency word of any pair will have reduced more than
its counterpart. Assuming that the two words in question are otherwise
identical, for our purposes, that they have the same original duration,
then the difference in absolute duration at that time will be given
by:
\begin{equation}
\Delta d_{e}=\delta_{n_{L}}-\delta_{n_{H}}\label{eq:duration-diff}
\end{equation}
Combining (\ref{eq:log-linear}) and (\ref{eq:duration-diff}),
\begin{equation}
\delta_{n_{L}}-\delta_{n_{H}}=b\left[\log\left(\frac{f_{L}}{f_{H}}\right)\right]
\end{equation}
Substituting in Eq. (\ref{eq:Reduction}):
\begin{equation}
1-(1-\alpha)^{n_{L}}-[1-(1-\alpha)^{n_{H}}]=b\left[\log\left(\frac{f_{L}}{f_{H}}\right)\right]
\end{equation}
Simplifying:
\begin{equation}
(1-\alpha)^{n_{H}}-(1-\alpha)^{n_{L}}=b\left[\log\left(\frac{f_{L}}{f_{H}}\right)\right]\label{eq:reduction-to-freq}
\end{equation}

The higher the frequency of a given word, the more times it should
be produced within a given time period. And if reduction is proportional
to the log frequency, with every production resulting in a given amount
of reduction, then the number of productions should also be proportional
to log frequency.
\begin{equation}
n_{f}=\rlog(f)\label{eq:n-productions}
\end{equation}
Substituting (\ref{eq:n-productions}) into (\ref{eq:reduction-to-freq}):
\begin{equation}
(1-\alpha)^{\rlog(f_{H})}-(1-\alpha)^{\rlog(f_{L})}=b\left[\log\left(\frac{f_{L}}{f_{H}}\right)\right]\label{eq:boundary cond}
\end{equation}
Assuming that it is possible to find values for $\alpha$ and \emph{r}
that satisfy Eq. (\ref{eq:boundary cond}) for all frequencies, the
additional reduction that will occur over the lifetime of the speaker
can then be determined.

If 1 epoch corresponds to about 20 years, then there will be about
4 over the lifetime of an individual. If we assume a constant rate
of production for each category proportional to its frequency, then
lifetime (\emph{E}) average reduction is given by $\delta_{E_{f}}=1-(1-\alpha)^{4n_{f}}$,
which can be rewritten as:
\begin{equation}
\delta_{E_{f}}=1-(1-\alpha)^{4\rlog(f)}
\end{equation}

With the necessary constants, we can now determine the difference
in reduction between the same two word categories after 4 epochs.
If we assume that there exists a floor beyond which words cannot reduce
further, then we will need to determine if any words are predicted
to reach floor in the lifetime of the speaker, and what effect that
will have on the behavior of the frequency dependence – either entirely
neutralizing the duration difference between certain words, or decreasing
that difference to some extent.

The exact predictions of the linearly biased frequency model will
depend on a host of implementational details. As already discussed
in the text, the choice of whether lower-frequency categories should
have proportionally fewer tokens than higher-frequency categories
will affect the outcome. Other parameters that have the potential
to alter the outcome include whether or not each individual experience
is automatically added to memory – or only a certain minimum number,
or some average of recent experience – and how quickly older memories
decay, being replaced by new experiences. It may be possible, if unlikely,
that at least one set of parameter values exists that will prevent
any words reaching floor within the lifetime of the speaker. However,
under any parameter settings, all words are predicted to continue
reducing over the lifetime of the speaker. This prediction is empirically
testable.

\chapter{\label{chap:Appendix C}Derivation of State Model}

This material is supplemental to \sectref{subsec:Lengthening-as-State}
of the main text.

For the Pure State Model (G), with 2-targets, each sub-category is
subject to two forces: entrenchment, and inertia. Under the simplifying
assumption that each sub-category can be treated as a Normal distribution
with constant variance, the equilibrium locations of the sub-category
means can be derived in the following way. At equilibrium the entrenchment
force is balanced by the inertial force due to each sub-category's
attractor. The location of the sub-category mean is the location at
which the displacement that would occur due to the entrenchment force
is exactly counteracted by the displacement that would occur due
to the inertia force. For the non-biased sub-category this equilibrium
occurs under the following conditions:
\begin{equation}
\beta\left(\overline{x_{E}^{NB}}-N\right)=\varepsilon\left(\overline{x_{E}}-\overline{x_{E}^{NB}}\right)\label{eq:xNB-state}
\end{equation}
For the biased sub-category, equilibrium occurs when:
\begin{equation}
\alpha\left(\overline{x_{E}^{B}}-L\right)=\varepsilon\left(\overline{x_{E}}-\overline{x_{E}^{B}}\right)\label{eq:xB-State}
\end{equation}
Because the entrenchment force depends on the global mean, so too
do the two equilibrium equations. In turn, the global mean can be
expressed as a function of the sub-category means (where the proportion
of biased tokens is given by \emph{p}):
\begin{equation}
\overline{x_{E}}=(1-p)\overline{x_{E}^{NB}}+p\overline{x_{E}^{B}}\label{eq:weighted-means}
\end{equation}
With three equations, we can solve for the three distribution means.
Solving for $\overline{x_{E}^{NB}}$ in Eq. (\ref{eq:xNB-state}):
\begin{equation}
\overline{x_{E}^{NB}}=\frac{\beta N+\varepsilon\overline{x_{E}}}{\beta+\varepsilon}
\end{equation}
Solving for $\overline{x_{E}^{B}}$ in Eq. (\ref{eq:xB-State}):
\begin{equation}
\overline{x_{E}^{B}}=\frac{\alpha L+\varepsilon\overline{x_{E}}}{\alpha+\varepsilon}
\end{equation}
Substituting these two values into Eq. (\ref{eq:weighted-means}):
\begin{equation}
\overline{x_{E}}=(1-p)\frac{\beta N+\varepsilon\overline{x_{E}}}{\beta+\varepsilon}+p\frac{\alpha L+\varepsilon\overline{x_{E}}}{\alpha+\varepsilon}
\end{equation}
Solving for $\overline{x_{E}}$ as a function of \emph{p, }and collecting
terms:
\begin{equation}
\overline{x_{E}}=\frac{(1-p)\beta N}{\beta+\varepsilon}+\frac{(1-p)\varepsilon\overline{x_{E}}}{\beta+\varepsilon}+\frac{p\alpha L}{\alpha+\varepsilon}+\frac{p\varepsilon\overline{x_{E}}}{\alpha+\varepsilon}
\end{equation}

\begin{equation}
\overline{x_{E}}-\frac{(1-p)\varepsilon\overline{x_{E}}}{\beta+\varepsilon}-\frac{p\varepsilon\overline{x_{E}}}{\alpha+\varepsilon}=\frac{(1-p)\beta N}{\beta+\varepsilon}+\frac{p\alpha L}{\alpha+\varepsilon}
\end{equation}

\begin{equation}
\frac{\overline{x_{E}}(\beta+\varepsilon)(\alpha+\varepsilon)-(\alpha+\varepsilon)(1-p)\varepsilon\overline{x_{E}}-(\beta+\varepsilon)p\varepsilon\overline{x_{E}}}{(\beta+\varepsilon)(\alpha+\varepsilon)}=\frac{(1-p)\beta N}{\beta+\varepsilon}+\frac{p\alpha L}{\alpha+\varepsilon}
\end{equation}

\begin{equation}
\begin{aligned}[t]
& \frac{\overline{x_{E}}(\beta+\varepsilon)(\alpha+\varepsilon)-(\alpha+\varepsilon)(1-p)\varepsilon\overline{x_{E}}-(\beta+\varepsilon)p\varepsilon\overline{x_{E}}}{(\beta+\varepsilon)(\alpha+\varepsilon)}\\
= &\frac{(1-p)\beta N(\alpha+\varepsilon)+p\alpha L(\beta+\varepsilon)}{(\alpha+\varepsilon)(\beta+\varepsilon)}
\end{aligned}
\end{equation}

\begin{equation}
\overline{x_{E}}[(\beta+\varepsilon)(\alpha+\varepsilon)-(\alpha+\varepsilon)(1-p)\varepsilon-(\beta+\varepsilon)p\varepsilon]=(1-p)\beta N(\alpha+\varepsilon)+p\alpha L(\beta+\varepsilon)
\end{equation}

\begin{equation}
\overline{x_{E}}=\frac{(1-p)\beta N(\alpha+\varepsilon)+p\alpha L(\beta+\varepsilon)}{(\beta+\varepsilon)(\alpha+\varepsilon)-(\alpha+\varepsilon)(1-p)\varepsilon-(\beta+\varepsilon)p\varepsilon}\label{eq:Model G-eq}
\end{equation}
Eq. (\ref{eq:Model G-eq}) is a complex function of $\alpha,\beta,\varepsilon,N,L$,
and \emph{p}, the derivative of which is not trivially calculated.
For known values of $\alpha,\beta,\varepsilon,N$, and \emph{L} ,
$\overline{x_{E}}(p)$ can be determined exactly. The general behavior
of this function, however, can be understood via the following chain
of reasoning.

For a given $p=p_{i}$ (for $p_{i}<1$), the equilibrium location
of the global mean can be found using Eq. (\ref{eq:Model G-eq}).
Now imagine that \emph{p} increases from $p_{i}$ to $p_{j}$. This will
result in the global mean moving closer to the biased sub-category
(Eq. (\ref{eq:weighted-means})). A change in the global mean will
cause a change in the entrenchment force for both sub-categories.
It will increase for the non-biased sub-category, which is now farther
from the global mean; and it will decrease in exactly the same degree
for the biased sub-category, which is now closer to the global mean.

Because inertia does not depend on \emph{p}, the lefthand sides of
Eqs. (\ref{eq:xNB-state}) and (\ref{eq:xB-State}) will remain constant.
Thus, the non-biased sub-category will shift in the direction of the
mean – rightward – as a result of the increase in \emph{p}. The decrease
in the entrenchment force on the biased sub-category, conversely,
will cause a shift away from the mean, and towards the attractor at
\emph{L}. This is also a rightward shift, however. The net effect
will be to perturb the sub-categories from their former equilibrium
locations to points farther to the right, and closer to \emph{L}.
As \emph{p} increases, $\overline{x_{E}}$ will always increase (as
long as both sub-categories are located between \emph{N} and \emph{L}).

The distance between the means of the two sub-categories can also
be written as a function of \emph{p}. Once equilibrium has been reached,
the separation can be derived from Eqs. (\ref{eq:xNB-state}) and
(\ref{eq:xB-State}):
\begin{equation}
\Delta\overline{x_{E}}\equiv\overline{x_{E}^{B}}-\overline{x_{E}^{NB}}=\frac{\alpha L+\varepsilon\overline{x_{E}}}{\alpha+\varepsilon}-\frac{\beta N+\varepsilon\overline{x_{E}}}{\beta+\varepsilon}
\end{equation}
Collecting terms and simplifying:

\begin{equation}
=\frac{\alpha L}{\alpha+\varepsilon}-\frac{\beta N}{\beta+\varepsilon}+\frac{\varepsilon\overline{x_{E}}}{\alpha+\varepsilon}-\frac{\varepsilon\overline{x_{E}}}{\beta+\varepsilon}
\end{equation}

\begin{equation}
=\frac{\alpha L}{\alpha+\varepsilon}-\frac{\beta N}{\beta+\varepsilon}+\varepsilon\overline{x_{E}}\left[\frac{1}{\alpha+\varepsilon}-\frac{1}{\beta+\varepsilon}\right]
\end{equation}
The change in sub-category separation as a function of changing \emph{p}
is thus given by:
\begin{equation}
\frac{\partial\Delta\overline{x_{E}}}{\partial p}=\frac{\partial\overline{x_{E}}}{\partial p}\varepsilon\left[\frac{1}{\alpha+\varepsilon}-\frac{1}{\beta+\varepsilon}\right]\label{eq:Model G-sep}
\end{equation}
In order to determine $\frac{\partial\Delta\overline{x_{E}}}{\partial p}$,
we must be able to calculate $\frac{\partial\overline{x_{E}}}{\partial p}$.
For the special case in which all forces have the same strength ($\alpha=\beta=\varepsilon$),
it is straightforward to calculate the derivative of Eq. (\ref{eq:Model G-eq}):
\begin{equation}
\overline{x_{E}}=\frac{2\alpha^{2}N+p(2\alpha^{2}L-2\alpha^{2}N)}{4\alpha^{2}-2\alpha^{2}}
\end{equation}
Collecting terms and simplifying:
\begin{equation}
\overline{x_{E}}=\frac{2\alpha^{2}[N+pL-pN]}{2\alpha^{2}[2-1]}
\end{equation}

\begin{equation}
\overline{x_{E}}=N+p(L-N)\label{eq: State-special case}
\end{equation}
This gives the expected behavior; for $p=0$, there is only the non-biased
distribution, which is stable at \emph{N}, and for $p=1$, there is
only the biased distribution, which is stable at \emph{L}. For equal
numbers of biased and non-biased variants, each sub-category stabilizes
at the same distance from its attractor, and the global mean is halfway
between the two. The change in the global category mean as a function
of \emph{p} is a positive, fixed value: $L-N$, the derivative of
(\ref{eq: State-special case}). Plugging this value for $\frac{\partial\overline{x_{E}}}{\partial p}$
into Eq. (\ref{eq:Model G-sep}) gives:
\begin{equation}
\frac{\partial\Delta\overline{x_{E}}}{\partial p}=(L-N)\alpha\left[\frac{1}{2\alpha}-\frac{1}{2\alpha}\right]=0\label{eq:separation-special case}
\end{equation}
Thus, while the overall category mean gets larger as \emph{p} increases,
the separation between the categories remains constant.

In the general case, the separation between the two sub-categories
will show different behavior for different parameter values. Because
${\partial\overline{x_{E}}}/{\partial p}>0$, the sign of ${\partial\Delta\overline{x_{E}}}/{\partial p}$
depends on the $\varepsilon[{1}/({\alpha+\varepsilon})-{1}/({\beta+\varepsilon})]$
term. When $\alpha<\beta$, the separation increases with increasing
\emph{p}. This follows from the fact that ${\partial\Delta\overline{x_{E}}}/{\partial p}$
is positive only when $\varepsilon[{1}/({\alpha+\varepsilon})-{1}/({\beta+\varepsilon})]>0$.
For $\varepsilon[{1}/({\alpha+\varepsilon})-{1}/({\beta+\varepsilon})]$
to be greater than zero it must be the case that ${1}/({\alpha+\varepsilon})>{1}/({\beta+\varepsilon})$.
This, in turn, requires that $\alpha<\beta$. By the same reasoning,
the separation decreases as a function of increasing \emph{p} when
$\alpha>\beta$. Finally, the separation remains constant when $\alpha=\beta$,
because this entails that $\varepsilon[{1}/({\alpha+\varepsilon})-{1}/({\beta+\varepsilon})]=0$,
verifying the result in Eq. (\ref{eq:separation-special case}).

\chapter{\label{chap:Appendix D}Derivation of Process Model}

This material is supplemental to \sectref{subsec:Model-B:-Lengthening}
of the main text.

For the Pure Process Model, there is a single category, and all tokens
are subject to the same inertial force, in proportion to their distance
from the single attractor at \emph{N}. Additionally, a proportion
\emph{p} of randomly selected tokens undergo a lengthening process,
moving away from the rest of the distribution during production. The
simplifying assumption, that each sub-distribution can be treated
as a Normal distribution with constant variance, is adopted. To derive
the model behavior we will look at the contribution of the different
forces in stages. This derivation references the stages depicted in
Figure \ref{fig:Derivation}.

First we apply the lengthening process, at time $t$, to tokens drawn
from a distribution with a global mean of $\overline{x_{t}}$. These
tokens are simultaneously subjected to an inertial force. Eq. (\ref{eq:Model-B B-Prime})
gives the mean of the biased sub-distribution at time \emph{t,}
\begin{equation}
\overline{x_{t}^{B}}^{\prime}=\overline{x_{t}}(1+\alpha)+\beta(N-\overline{x_{t}})\label{eq:Model-B B-Prime}
\end{equation}
and Eq. (\ref{eq:Model-B NB-Prime}) gives the means of the non-biased
sub-distribution at time \emph{t}.

\begin{equation}
\overline{x_{t}^{NB}}^{\prime}=\overline{x_{t}}+\beta(N-\overline{x_{t}})\label{eq:Model-B NB-Prime}
\end{equation}
On average, a proportion \emph{p} of the distribution will be lengthened,
thus the location of the global mean, after lengthening and inertia
apply, can be expressed as
\begin{equation}
\overline{x_{t}}^{\prime}=(1-p)\overline{x_{t}^{NB}}^{\prime}+p\overline{x_{t}^{B}}^{\prime}\label{eq:Model-B weight mean}
\end{equation}
Entrenchment must also be applied in order to determine the final
outcome, but entrenchment does not affect the location of the global
mean, only the locations of the sub-distribution means, and their
separation. To see this, we can compare the global mean before and
after entrenchment applies. After entrenchment, the means of each
sub-distribution are given by:
\begin{equation}
\overline{x_{t}^{B}}^{\prime\prime}=\overline{x_{t}^{B}}^{\prime}-\varepsilon\left(\overline{x_{t}}^{\prime}-\overline{x_{t}^{B}}^{\prime}\right)
\end{equation}

\begin{equation}
\overline{x_{t}^{NB}}^{\prime\prime}=\overline{x_{t}^{NB}}^{\prime}-\varepsilon\left(\overline{x_{t}}^{\prime}-\overline{x_{t}^{NB}}^{\prime}\right)
\end{equation}
Substituting into Eq. (\ref{eq:Model-B weight mean}), gives
\begin{equation}
\overline{x}_{t}^{''}=(1-p)\left[\overline{x_{t}^{NB}}^{\prime}-\varepsilon\left(\overline{x_{t}}^{\prime}-\overline{x_{t}^{NB}}^{\prime}\right)\right]+p\left[\overline{x_{t}^{B}}^{\prime}-\varepsilon\left(\overline{x}_{t}^{\prime}-\overline{x_{t}^{B}}^{\prime}\right)\right]
\end{equation}
Simplifying and collecting terms:
\begin{equation}
=\overline{x_{t}}^{\prime}-\varepsilon(1-p)\left(\overline{x_{t}}^{\prime}-\overline{x_{t}^{NB}}^{\prime}\right)-p\varepsilon\left(\overline{x_{t}}^{\prime}-\overline{x_{t}^{B}}^{\prime}\right)
\end{equation}

\begin{equation}
=\overline{x_{t}}^{\prime}+\varepsilon(1-p)\overline{x_{t}^{NB}}^{\prime}-\left[\varepsilon(1-p)+p\varepsilon\right]\overline{x_{t}}^{\prime}+p\varepsilon\overline{x_{t}^{B}}^{\prime}
\end{equation}

\begin{equation}
=\overline{x_{t}}^{\prime}-\varepsilon\overline{x_{t}}^{\prime}+\varepsilon\left[(1-p)\overline{x_{t}^{NB}}^{\prime}+p\overline{x_{t}^{B}}^{\prime}\right]
\end{equation}
The term $(1-p)\overline{x_{t}^{NB}}^{\prime}+p\overline{x_{t}^{B}}^{\prime}$
is equivalent to $\overline{x_{t}}^{\prime}$ by Eq. (\ref{eq:Model-B weight mean}).
Therefore
\begin{equation}
\overline{x_{t}}^{\prime\prime}=\overline{x_{t}}^{\prime}-\varepsilon\overline{x_{t}}^{\prime}+\varepsilon\overline{x_{t}}^{\prime}=\overline{x_{t}}^{\prime}
\end{equation}
Because it does not depend on entrenchment, the global mean at equilibrium
can be determined directly from (\ref{eq:Model-B B-Prime}) and (\ref{eq:Model-B NB-Prime}).
Equilibrium occurs when the two sub-distributions are also at equilibrium,
and the global mean stops changing: $\overline{x_{E}}=\overline{x_{E}}^{\prime}$,
$\overline{x_{E}^{NB}}^{\prime}=\overline{x_{E}^{NB}}$, and $\overline{x_{E}^{B}}^{\prime}=\overline{x_{E}^{B}}$.
Therefore,
\begin{equation}
\overline{x_{E}}=(1-p)\overline{x_{E}^{NB}}^{\prime}+p\overline{x_{E}^{B}}^{\prime}
\end{equation}
Substituting in Eqs. (\ref{eq:Model-B B-Prime}) and (\ref{eq:Model-B NB-Prime}):

\begin{equation}
\overline{x_{E}}=[\overline{x_{E}}+\beta(N-\overline{x_{E}})]-p[\overline{x_{E}}+\beta(N-\overline{x_{E}})]+p[\overline{x_{E}}+\overline{x_{E}}\alpha+\beta(N-\overline{x_{E}})]
\end{equation}
Simplifying and collecting terms:

\begin{equation}
\overline{x_{E}}=\overline{x_{E}}+\beta(N-\overline{x_{E}})+p\overline{x_{E}}\alpha-p[\overline{x_{E}}+\beta(N-\overline{x_{E}})]+p[\overline{x_{E}}-\beta(N-\overline{x_{E}})]
\end{equation}

\begin{equation}
\overline{x_{E}}=\overline{x_{E}}+\beta(N-\overline{x_{E}})+p\overline{x_{E}}\alpha
\end{equation}

\begin{equation}
\overline{x_{E}}=\overline{x_{E}}(1-\beta+p\alpha)+\beta N
\end{equation}

\begin{equation}
\overline{x_{E}}(1-1+\beta-p\alpha)=\beta N
\end{equation}

\begin{equation}
\overline{x_{E}}=\frac{\beta N}{\beta-p\alpha}\label{eq:equ-mean-state}
\end{equation}
For the case when $p\alpha<\beta$, the denominator in (\ref{eq:equ-mean-state})
is positive. As \emph{p} increases (but $p\alpha$ remains smaller
than $\beta$), the denominator decreases, and the global mean increases.
As $p\alpha$ approaches $\beta$, the global mean goes to infinity;
lengthening is unbounded. For $p\alpha>\beta$ the only stable point
is negative, and thus there is no well-defined equilibrium. The \noun{process}
model is thus only stable if the lengthening strength is not too great,
and the percentage of biasing contexts is not too large.

To calculate the dependence of the sub-distribution separation on
\emph{p,} the effect of entrenchment must be included. The equilibrium
separation is defined as:
\begin{equation}
\Delta\overline{x_{E}}^{\prime\prime}\equiv\overline{x_{E}^{B}}^{\prime\prime}-\overline{x_{E}^{NB}}^{\prime\prime}
\end{equation}
And

\begin{equation}
\overline{x_{E}^{B}}^{\prime\prime}=\overline{x_{E}^{B}}^{\prime}+\varepsilon\left(\overline{x_{E}}^{\prime}-\overline{x_{E}^{B}}^{\prime}\right)
\end{equation}

\begin{equation}
\overline{x_{E}^{NB}}^{\prime\prime}=\overline{x_{E}^{NB}}^{\prime}+\varepsilon\left(\overline{x_{E}}^{\prime}-\overline{x_{E}^{NB}}^{\prime}\right)
\end{equation}
Therefore,
\begin{equation}
\overline{x_{E}^{B}}^{\prime\prime}-\overline{x_{E}^{NB}}^{\prime\prime}=\overline{x_{E}^{B}}^{\prime}-\overline{x_{E}^{NB}}^{\prime}+\varepsilon\left(\overline{x_{E}}^{\prime}-\overline{x_{E}^{B}}^{\prime}\right)-\varepsilon\left(\overline{x_{E}}^{\prime}-\overline{x_{E}^{NB}}^{\prime}\right)
\end{equation}
Collecting terms:
\begin{equation}
=\overline{x_{E}^{B}}^{\prime}-\overline{x_{E}^{NB}}^{\prime}-\varepsilon\left(\overline{x_{E}^{B}}^{\prime}-\overline{x_{E}^{NB}}^{\prime}\right)
\end{equation}
and
\begin{equation}
\Delta\overline{x_{E}}^{\prime\prime}=(1-\varepsilon)\left(\overline{x_{E}^{B}}^{\prime}-\overline{x_{E}^{NB}}^{\prime}\right)
\end{equation}
The observed separation at equilibrium depends on the separation due
to prior model forces. From Eqs. (\ref{eq:Model-B B-Prime}) and (\ref{eq:Model-B NB-Prime}),
\begin{equation}
\overline{x_{E}^{B}}^{\prime}-\overline{x_{E}^{NB}}^{\prime}=\overline{x_{E}}(1+\alpha)+\beta(N-\overline{x_{E}})-[\overline{x_{E}}+\beta(N-\overline{x_{E}})]
\end{equation}
This reduces to $\alpha\overline{x_{E}}$. Note that this is exactly
the amount that biased tokens are shifted away from the mean at equilibrium.
Because this is a \noun{process} model, the separation created by
the lengthening bias only exists transiently, and it is not possible
for any specific subset of tokens to continue to increase their separation
from the rest of the distribution. Therefore, the prior separation
between the sub-distribution means is always given by the lengthening
bias applied to that mean. And the total separation, by
\begin{equation}
\Delta\overline{x_{E}}=(1-\epsilon)(\alpha\overline{x_{E}}).\label{eq:Cat Sep-1}
\end{equation}
In the stable parameter range, where $\overline{x_{E}}$ increases
as \emph{p} increases, the separation of the sub-distributions also
increases, but more slowly, by a factor of $\alpha(1-\epsilon)$.

\chapter{\label{chap:Appendix E}Nasalization model parameters}

The following parameters were identical for the two models:
\begin{itemize}
\item The entrenchment strength is set to $\varepsilon=.2$
\item The production error on each articulatory dimension is drawn from
the distribution $\mathcal{\mathscr{N}}\left(0,.25\sigma_{x^{Z}}\right)$,
where $\sigma_{x^{Z}}$ indicates the standard deviation of the current
distribution of stored tokens on dimension $x^{Z}$
\item Speaking Rate:
\begin{itemize}
\item Expansion force (\emph{E}) is a random variable distributed according
to $\mathcal{\mathscr{N}\left(\mathrm{0,.25}\right)}$.
\item The speaking rate transformation lengthens or shortens a given duration
parameter, according to the following dependence on \emph{E:}
\begin{equation}
x_{i}^{O^{\prime}}=\frac{2x_{i}^{O}}{(1+e^{k_{O}E})}\label{eq:Speaking rate transform-1}
\end{equation}

\begin{equation}
x_{i}^{V^{\prime}}=\frac{2x_{i}^{V}}{(1+e^{-k_{V}E})}\label{eq:Speaking rate transform-1-1}
\end{equation}

\begin{equation}
x_{i}^{N^\prime}=\frac{2x_{i}^{N}}{(1+e^{-k_{N}E})}\label{eq:Speaking rate transform-1-1-1}
\end{equation}

\end{itemize}
For these simulations all gestures are set to the same elasticity
($k_{O}=k_{N}=k_{V}=1$).
\item Model outputs are reported after 10,000 iterations
\item $x_{i}^{O}$ is never allowed to fall below 0, or to exceed the shorter
of the two values $(x_{i}^{N},x_{i}^{V})$
\item The duration of $x_{i}^{V}$ is never allowed to fall below 50 ms,
or to exceed 600 ms
\item The duration of $x_{i}^{N}$ is never allowed to fall below 25 ms,
or to exceed 500 ms
\end{itemize}

\section{No-Phoneme Model}
The fluency attractor affects overlap duration according to the following
formula:

\begin{equation}
x_{i}^{O^{\prime}}=x_{i}^{O}+\beta(T-x_{i}^{O})\label{eq:Frequency attractor-1}
\end{equation}
The target overlap duration for these simulations is set at $T=x_{i}^{N}$.
$\beta$ parameterizes frequency on a scale between 0 and 1.

\section{Multiple-Parse Model}
\begin{itemize}
\item Resting activation acts as a perturbation to the expansion force,
\emph{E}. The mean of the expansion function is shifted ${1}/{4}$
of a standard deviation for each unit of \emph{f}, where \emph{f} parameterizes
frequency:
\end{itemize}
\[
\overline{E}^{\prime}=\overline{E}-f(.25\sigma_{E})
\]

\begin{itemize}
\item The overlap duration for Analysis 2 tokens is a random variable distributed
according to $\mathcal{\mathscr{N}}\left(.25\overline{x^{N}},\sigma_{x^{N}}\right)$
\item The probability of Analysis 1 is given by:
\begin{equation}
P(a=1)=Ae^{-b(1-Q)}-C\label{eq:segmentation-1-1}
\end{equation}
where $Q={x_{i}^{O}}/{x_{i}^{N}}$\emph{ }. For all simulations,
the constants are set to: $A=1$, $b=2$, and $C=0$.
\end{itemize}


\clearpage
\subsection{C Boolean Data} % (fold)
\label{sub:c_boolean_data}

C has very flexible support for Boolean values. In C a \texttt{0} value is considered to be false, and any other value is true. Modern C compilers have now added support for an explicit Boolean type, \texttt{bool}. This type requires the \texttt{stdbool.h} header file, which defines the \texttt{bool} type as well as the values \texttt{true} and \texttt{false}.

\begin{table}[h]
\begin{minipage}{\textwidth}
\centering
\begin{tabular}{|l|c|c|}
\hline
\multicolumn{3}{|c|}{\textbf{Boolean Type}} \\
\hline
\emph{Name} & \emph{Size} & \emph{Values} \\
\hline
\texttt{bool} & 1 byte/8 bits & \texttt{true} or \texttt{false} \\
\hline
\end{tabular}
\caption{C Boolean Type}\label{tbl:c-boolean}
\end{minipage}
\end{table}

\begin{table}[h]
  \centering
  \begin{tabular}{|c|c|c|}
    \hline
     & \textbf{Description} & \textbf{C} \\
    \hline
    \textbf{Equal} & Are the values the same? & \texttt{a == b} \\
    \hline
    \textbf{Not Equal} & Are the values different? & \texttt{a != b} \\
    \hline
    \textbf{Larger Than} & Is the left value larger than the right? & \texttt{a > b}  \\
    \hline
    \textbf{Less Than} & Is the left value smaller than the right? & \texttt{a < b}  \\
    \hline
    \textbf{Larger Or Equal} & Is the left value equal or larger than the right? & \texttt{a >= b}  \\
    \hline
    \textbf{Less Or Equal} & Is the left value smaller or equal to the right? & \texttt{a <= b}  \\
    \hline
  \end{tabular}
  \caption{C Comparison Operators}
  \label{tbl:c_comparison_op}
\end{table}

\begin{table}[h]
  \centering
  \begin{tabular}{|c|c|c|}
    \hline
     & \textbf{Description} & \textbf{C} \\
    \hline
    \textbf{And} & Are both values True? & \texttt{a \&\& b} \\
    \hline
    \textbf{Or} & Is at least one value True? & \texttt{a || b} \\
    \hline
    \textbf{Xor} & Is one value True, and the other False? & \texttt{a \^{} b} \\
    \hline
    \textbf{Not} & Is the value False? & \texttt{!a} \\
    \hline
  \end{tabular}
  \caption{Logical Operators}
  \label{tbl:c-logical-operators}
\end{table}

\mynote{
\begin{itemize}
  \item To use the Boolean type in C you must include the \texttt{stdbool.h} header.
  \item In C false is the value \texttt{0}, and true is any other value.
  \item The \texttt{stdbool.h} header gives you access to two value, \texttt{true} and \texttt{false}. \texttt{true} has the value 1, \texttt{false} has the value 0.
\end{itemize}
}

\clearpage

\csection{\ccode{clst:bool-test}{C Boolean Test Code}{code/c/control-flow/test-bools.c}}

% subsection c_boolean_data (end)


\chapter{Machine Learning}
Machine learning is playing a fundamental role in natural language processing
and computational linguistics. The impact is so significant that one of key note
speakers of ACL 2012, Mark Johnson, predicted that in 50 years from now NLP/CL
will not exist as a research field and will be emerged as two different
fields:(1) machine learning and (2) logic. This chapter is dedicated  to subset
of models in machine learning that can play a role of bridge between logic and
machine learning. This chapter provides the building material for a new
direction of research so called \textit{Representation Learning} which enables
us to transfer information from logical forms such as predicate-argument form to
vector space (beside many other advantages which we will discuss).
Most of current models in machine learning are designed to work in vector space
therefore it is an important achievement to be able to work with type of
information which is in other forms or representations by first transferring
them to vector space. Working with logical forms in vector space helps us to use
many available knowledge bases and lexicons with available machine learning
models which is crucial for relation discovery. I will skip discussing classical
machine learning problems to emphasis more on new relevant topics.
I will first briefly review artificial neural networks and then motivate and
define the task of representation learning. In next chapters, we will
need the materials provided in this chapter to understand the undergoing
research about relation discovery.
\section{Artificial Neural Networks}
\label{sec:ml-ann}
Before diving into different methods of representation learning, it would be
useful to briefly review the architecture, applications and learning algorithms
of artificial neural networks (ANN). ANNs can be used for variety of
applications in machine learning: classification, clustering, dimensionality
reduction,\ldots ~. What makes ANN important for us is their ability to do many
of mentioned tasks jointly. Its layer-wised architecture very well fits to the
idea of multi-task learning \cite{Bengio2012a}. Different layers of an ANN can
have different objectives and can share information with other layers through parameter
sharing. First I discuss about elements and architecture of ANNs and then we
will very briefly review \textit{backpropagation} and \textit{stochastic
gradient descent} as we need them to understand the mechanism of learning in
ANNs.

Artificial neural networks have old history in the domain of machine learning.
They are inspired from the mechanism humans learn via their neural networks. The
basic element of this network is a neuron which takes a weighted input signals and by applying an activation function
on this input, the neuron will output a signal.
 Neurons are usually arranged in
layers and receive input from the neurons from the previous layer and send their
outputs to the next layer. The combination of different layers and using
different activation function with different degree of non-linearity is enabling
ANNs to learn non-linear functions.\cite{Braspenning1995}
A common mathematical model for neuron in ANNs is shown in
\eqref{eq:neuron} and visually in \autoref{fig:neuron-model}.
\begin{equation}
\label{eq:neuron}
\begin{split}
x^{l-1} \mbox{~: input vector of size~}n  \mbox{~and layer~}l-1
\\
w^{l,l-1} \mbox{~: weight vector of size}~n~ \text{from layer}~l-1 \text{~to
layer}~l
\\
y_{j}^{l} \mbox{~: output value of neuron~}j  \mbox{~at layer}~l\\
y_{j}^{l} = f(\sum_{i=0}^{n} w_{i,j}x_{i}^{l-1})\\
\end{split}
\end{equation}
Activation functions can be
usually either of functions in the list below:
\begin{description}
\item[linear function] ~~ $f(x)=ax+b \mbox{~where}~ a,b \in \mathbb{R}$
\item[step function] ~~ $ f(x)=1 \mbox{~if~} x > \theta \mbox{~else~} f(x)=0
\mbox{~where~}\theta \in \mathbb{~R} \mbox{~is a constant threshold} $
\item[tangent hyperbolic] ~~ $f(x)=tanh(x)$
\item[Log-sigmoid] ~~ $f(x)=\frac{1}{1+e^{-x}}$
\end{description}

    \begin{figure}[h!]
  \caption{A model of neuron}
  \centering
    \includegraphics[width=0.8\textwidth]{neuron-model.eps}
    \label{fig:neuron-model}
\end{figure}

The type of ANNs matters mostly for us in this research are
networks which neurons are arranged in ordered layers and there is no feedback
from a deeper layer  to the layers in the back. The first layer is usually for
unsupervised pre-training of objects, then a layer for adding non-linearity
comes and finally we have an axillary classification or ranking task.\cite{Bengio2003}
 \begin{figure}[h!]
  \caption{A Neural Language Model proposed by Bengio et al.}
  \centering
    \includegraphics[width=1\textwidth]{ann.eps}
    \label{fig:ann}
\end{figure}
 In \autoref{fig:ann}
 we can see a neural architecture proposed by Bengio et al. \cite{Bengio2003}
 which works as language model. This model takes n-grams as input and predict
 the probability of the next word. The first layer is hot-one representation of
 sequence words, $w_{t-n+1}, \ldots ,w_{t-2}, w_{t-1}$ and then the layer for
 unsupervised pre-training of words comes.
 Learned features of words will be combined together with \textit{tanh} layer
 and finally we will have a \textit{softmax} layer which outputs probability of
 all words as the next word, $w_t$, in the sequence. Later we will see more details about
 different layers of this ANN but
 the main question that arises here is how to learn parameters of such a network
 in order to make good predictions and induce meaningful features for words.

There are two main approaches for learning parameters of a single supervised
layer. By supervised layer we mean that true outputs/predictions are available
for training the parameter of the layer. :
\begin{description}
  \item[Online learning] \hfil \\
    To optimize the parameters, we
    iterate over all points in the training dataset and calculate the layer
    predictions for input. In each iteration, we update the weights in order to
    decrease the difference between the predicted output and the true output for
    each point. This forms an objective function (error function) which can be
    minimized by gradient descent, by calculating error derivatives and update
    weights to decrease them.\cite{Bengio2012}
    Since each data point can change the parameters, we call this algorithm an online learning algorithm.
  \item[Batch learning] \hfil \\
  In this approach, unlike the previous approach, first we calculate the
  predicted outputs of whole dataset and form the error function and try to
  minimize the error derivative for whole dataset. Since we update the
  parameters after observing all points in the training dataset we call this
  approach batch learning.\cite{Braspenning1995}

  \item[Stochastic Gradient Descent] \hfil \\
  The idea of SGD \cite{Bottou2010},~\cite{Bengio2012} is to make best out of both
  previous ideas.
  Neither updating after each datapoint nor after all points, SGD makes small batches in each
  iteration, each usually contains only 100 datapoints, and calculate error
  derivatives and update parameters for each batch.

\end{description}

Since we now know how to train a single supervised layer, other layers of a
network can be trained using \textit{backpropagation}. Parameters of the network
will be randomly initialized and then by choosing one of the approaches above we
can start by learning the last layer, since we have the output for that. After
updating the last layer, it works as the output of the layer before it and we
can use the same strategy to train its parameters. Likewise, we continue
training of other layers. The criteria for stopping the algorithm can be either
maximum number of iterations or when parameters don't change from after some
iterations.

After introducing ANNs we will focus more on the idea of unsupervised
pre-training in the next section. I show the generalization of this idea using
other methods and motivates it as a very crucial step in many NLP tasks.

\section{Representation Learning}
\label {sec:repr-learning}

In this chapter, we will define and justify the task of
\emph{Representation Learning} and we will see different families of methods for
inducing word representation and its application in NLP.

In machine learning specially in industry, most of the labor is dedicated to
\emph{Feature Engineering}. Extracting informative features is the crucial part
of most supervised methods and it is done mostly manually. While many different
applications share common learning models and classifiers, the difference in
performance of competing methods mostly goes to the data representation and
hand-crafted features that they use. This observation reveals an important
weakness in current models, namely their inability to extract and organize
discriminative features from data. Representation learning is an umbrella term
for a family of unsupervised methods to learn features from data. Most of recent
works on the application of this idea in NLP focus on inducing word
representations. \emph{Word representation} is a mathematical object, usually a
vector, which each dimension in this vector represents a grammatical or
semantical feature to identify this word and is induced automatically from data
\cite{Turian2010b}. Recently, it has been shown in \cite{Turian2010b} and \cite{Collobert2011} that using
 induced features can be helpful to improve state-of-the-art methods in
different NLP tasks. It seems that relation extraction can also benefit from such features since similar tasks like
semantic role labeling has been shown to benefit from
induced word representations. In ~\autoref{ch:text-kb} and
~\autoref{ch:ent-link} I will show two applications of it, (1) direct usage for relation discovery and
(2) word feature generation which can be used in current relation discovery
systems as well as other NLP applications.
In the next two sections, two major families of representation learning methods
will be shortly reviewed.

\subsection{Distributional Representation}
\label{sec:distl-repr}
In distributional semantics, the meaning of a word is expressed by the context
that it appears in it \cite{Harris1981}. Features that are used to represent the
meaning of a word are other words in its neighborhood as it is so called the
context. In some approaches like LDA and latent semantic analysis (LSA),
the context is defined in the scope of a document rather than a window around a
word. To represent word meanings in via distributional approach, one should
start from count matrix (or zero-one co-occurrence matrix) which each row
represents a word and each column is a context. The representation can be
limited to raw usage of the very same matrix or some transforms like
\emph{tf-idf} will be applied first. A further analysis over this matrix to
extract more meaningful features is applying dimensionality reduction methods or
clustering models to induce latent distributional representations. A similar
clustering method to k-means is used in \cite{Lin2009} to represent phrase and
word meanings and brown clustering algorithm \cite{Brown1992} has been shown to
have impact on near to state-of-the-art NLP tasks \cite{Turian2010b}.

\subsection{Distributed Representation}
\label{sec:disted-repr}
Distributed representation has been introduced in the literature for the first
time in \cite{Bengio2003} where Bengio et al. introduced a first language
model based on deep learning methods\cite{Bengio2009b}. Deep learning is
learning through several layers of neural networks which each layer is
responsible to learn a different concept and each concept is built over other
more abstract concepts. In the deep learning society, any word representation
that is induced with a neural network is called \emph{Word Embedding}.
In contrast to raw count matrix in distributional representations, word embeddings are low-dimensional, dense and real-valued vectors.
 The term, \textbf{`Distributed'}, in this context refers to the fact that
 exponential number of objects (clusters) can be modeled by word embeddings.
 Here we will see two famous models to induce for such representations. One
 family will use n-grams to learn word representation jointly with a language
 model and the other family learns the embedding from structured resources.
In \cite{Collobert2008a}, Weston and Collobert use a non-probabilistic and
discriminative model to jointly learn word embeddings and a language model that
can separate plausible n-grams from noisy ones. For each word in a n-gram, they
combine the word embeddings and use it as positive example. They put noise in
the n-gram to make negative examples and then train a neural network to learn to
classify positive labels from negative ones. The parameters of neural network
(neural language model) and word embedding values will be learned jointly by an
optimization method called \emph{Stochastic Gradient Descent} \cite{Bottou2010}.

A hierarchical distributed language model (HLBL) proposed by Mnih and
Hinton in \cite{Mnih2009} is another influential work on word embeddings. In
this model a probabilistic linear neural network(LBL) will be trained to
combine word embeddings in first $n-1$ words of a n-gram to predict the $n_{th}$
word.

Weston-Collobert model and HLBL by Mnih and Hinton are evaluated in
\cite{Turian2010b} in two NLP tasks: chunking and named entity recognition. With
using word embeddings from these models combined with hand-crafted features, the
performance of both tasks are shown to be improved.

\subsection{Representation Learning of Knowledge Bases}
\label{sec: repr-learning-kb}
Bordes et al. in \cite{Bordes2011} and \cite{Bordes2012} have attempted to use
a neural distributed model to induce word representations from lexical resources
such as WordNet ~\cite{Fellbaum1998} and knowledge bases (KB) like Freebase
~\cite{Bollacker2008} .
In Freebase for example, each named entity is related to another entity by an instance of a specific type of relation. In
\cite{Bordes2011}, each entity is represented as a vector and each relation is decomposed to two
matrices. Each of these matrices transform left and right-hand-side entities
to a semantic space. Similarity of transformed entities indicates that the
relation holds between the entities.  A prediction task is defined to evaluate
the embeddings. Given a relation and one of the entities, the task is to predict
the missing entity. The high accuracy (99.2\%) of the model on prediciton
of training data shows that learned representation highly captures attributes of
the entities and relations in Freebase.

Two major models are proposed in \cite{Bordes2011} and \cite{Bordes2012} to
learn features in continuous  vector space from a Knowledge Bases(KB) which information is
usually represented in form of triples of $(e_{i},r_{k} , e_{j} )$ where $e_{i}$ and $e_{j}$ are $i_{th}$ and $j_{th}$ entities related
 by a binary relation of type $r_{k}$. The purpose of the models is to induce a vector space and associate
  each entity or relation to an embedding vector or a matrix.
  The dimensions of such an embedding vector are supposed to reflect a set of informative features of entities and relations.

   In the first model, \textbf{structured embeddings(SE)}, entities are modeled as \textit{d}-dimensional vectors.
    An associated vector to the $i_{th}$ entity, $e_{i}$, is $E_{i} \in \mathbb{R}^{d}$. Each relation $r_{k}$
    is decomposed to two operators each represented as $d \times d$ matrix, $ R_k = (R_{k}^{left}, R_{k}^{right})$.
    These operators transform the left and right entities to a new space induced by each relation and by using
    a $p$-norm measure  (L1 norm in this work) they associate a similarity value or a score to each triple.
    This similarity value is being calculated by
    Equation ~\eqref{eq:sim}.
    \begin{equation}
    \label{eq:sim}Sim(E_{i}, E_{j}, R) = ||R_{k}^{left}E_{i} - R_{k}^{right}E_{j} ||_{1}
    \end{equation}
    The similarity between transformed entities works as a score to measure the strength of a relation holds between two entities.

    Using the idea of contrastive learning , the model will be trained to increase similarity of
    embeddings for a positive triple (a triple which exists in the KB) or lowering its rank among other training samples
    and decrease the similarity of embeddings when the relation doesn't hold (negative triple) or raising its rank .
    For each positive triple, two negative triples will be generated by randomly alternating the right entity or left entity with other entities.
    Inspired from large margin methods a constraint is introduced on the model
    that forces negative triples to have lower associated similarity value  than correspondent
    positive triples by a large margin. In \autoref{fig:bordes2011} a schematic
    view of model is presented.

    \begin{figure}[h!]
  \caption{Neural Distributed Model to Learn Structured Embeddings (SE)}
  \centering
    \includegraphics[width=0.5\textwidth]{bordes2011.eps}
    \label{fig:bordes2011}
\end{figure}
    The second model, \textbf{Semantic Matching Energy using Bilinear layers(SME-Bil)},
    is using a different representation for relations,weighted bilinear
    transformation of embeddings and  dot product similarity function instead of L1 norm.
    In this model, each relation is represented by a \textit{d}-dimensional vector $R_{k}$ same as entities.
    For triple $(e_{i},r_{k} , e_{j} )$ , the model combines the weighted transformation of each entity embedding with
    the weighted embedding of relation using element-wise vector product. as it is shown in Equation ~\eqref{eq:bil}.
    \begin{equation}
    \label{eq:bil} E'_{left} = (W_{i} E_{i}) \odot (W_{k} R_{k}) + b_{left}
    \end{equation}
    $W_{i}$ and $W_{k}$ are $d \times d$ weight matrices and $b_{left}$ is a $d$-dimensional bias vector.
    The same equation holds for transforming the right entity embeddings to $E'_{right}$. Finally, the associated score for the triple
     can be calculated by dot product of $E'_{left}$ and $E'_{right}$ which is
     shown in Equation ~\ref{eq:dot}. \autoref{fig:bordes2012} is used by Bordes
     et al. to sketch SME-Bil model. Similar constraints to the first model are also applied to this model and
     both models can be trained by stochastic gradient descent (SGD) which we
     introduced in \autoref{sec:ml-ann}.

   \begin{equation}
    \label{eq:dot} Sim(E_{i}, E_{j}, R_{k}) = -E'_{left}E'_{right}
   \end{equation}
     \begin{figure}[h!]
  \caption{Neural Distributed Model to Learn Structured Embeddings (SME)}
  \centering
    \includegraphics[width=0.5\textwidth]{bordes2012.eps}
    \label{fig:bordes2012}
\end{figure}

\iffalse
\section{Bayesian Non-Parametric Models}
\label {sec:bnp}

\subsection{Introduction}
\label{ssec:intro}
One of the most important contributions of Machine Learning to Natural Language
Processing and Computational Linguistics is to provide variety of statistical frameworks for modeling
 different aspects of language. Modeling makes the data more interpretable and is trying to represent
 it in a compact way. From language modeling to syntax and semantics,
 computational models are needed to first describe and then predict the certain phenomena in language.
 Here we address a family of models which are trying to be less dependent on human choice or
 trial and error schema for choosing best parameters and are called Bayesian nonparametric models (BNP).

\subsection{Model Selection with BNP Models}
\label{ssec:model-sel}
A traditional approach for model selection is to have a set of models and then try to fit
these models on the linguistic data by tuning and inferring the parameters of them [2].
It is more like a search in possible space of models and their parameters to find the best setting.
Therefore, it is necessary to have a criterion to select the models to see which one is better
explaining the data and have the prediction power. There are two main features that any criterion
should consider necessarily. First is the accuracy of prediction (model fitness) and it ensures that
the model is well-defined and second is the complexity of model [1,2].
Let us define these terms in a mathematical language so we can introduce a Bayesian framework for modeling.
The data is often a set of data points which could be represented as vectors or is in more
complex structure like a graph or an ordered sequence. For simplicity we first start with
data point or vector representation of data. Let us define the data as:
\begin{equation}
\label{eq:data}
D = y_1,y_2,\ldots,y_n
\end{equation}

The model could express predictions by the likelihood which is in this form:
\begin{equation}
P(D|\theta,m)
\end{equation}

You see that the predictions are conditioned on the model and a specific parameter in this way.
In a fully Bayesian model, one should specify the range of parameters or more precisely
a distribution over possible parameters should be determined beforehand [2].
This could be done by adjusting the prior and then we can incorporate the role of all possible parameters
to predictions weighted by their importance. Here one can notice that the set of parameters are fixed.
\begin{equation}
P(D|m) = \int \! P(D,\theta|m) \, \mathrm{d}\theta = \int \!
P(D|\theta,m)P(\theta|m) \, \mathrm{d}\theta.
\end{equation}

One can see in this formula that the prediction is average over all parameters weighted by prior.
Each model is a generative process of data which gives us a joint distribution of hidden and observable variables.
Each data point is drawn from a distribution of observable variables conditioned on
hidden variables (parameters of model) and hidden variables are drawn from a prior distribution [1].
To infer the posterior distribution (distribution of hidden variables conditioned on observable variables)
we can reverse the generative process.
Instead of generating a data point from the distribution we look for the model that more likely explains
the generation of the data. [1,4]
The second important aspect of model selection is measuring the complexity of the model.
If we have two models with about same level of fitting, we favor the one that is simpler since it
performs better generalization for unseen data. Knowingly as Occam’s razor rule having
less complex model helps avoiding overfitting to the observed data. Bayesian
framework naturally favors simpler models by putting prior over parameters
and penalizing unnecessary complexity of the model [2].
One can cleverly ask how much complexity is required for a particular data and how can
we adapt our modeling to change predictions in the case of having new unseen data?
The answer of Bayesian nonparametric modelling (BPN) to this problem differs to all previous models
in a way that instead of comparing different models with different complexities it tries to fit
a single model on the data which changes its complexity by seeing new data.
While previous methods have fixed set of parameters, BNP assumes an infinite number of parameters.
It is like that BNP has infinite amount of money (parameters) in the bank and whenever
the data is more expensive (more complex) it spends more money to model the data.
Actually \textit{nonparametric} could be misleading  since they have infinite number of parameters
but only activate a finite subset of them in the learning phase.
Another point of view is to see a BNP model as information channel which parameters are bandwidth of this channel.
As data grows bigger, more information should be captured by the model so it increase its bandwidth
(uses more parameters) [2].
There are several families of BNP models available due to different tasks and structure of data and
hidden variables. In following, we enumerate these families shortly and then we
go more in depth for one of them.

These groups of models are [1,4,2]:
\begin{enumerate}
  \item Classification and Regression task (Gaussian processes)
\item Clustering (Dirichlet processes and Chinese restaurant processes)
\item Density estimation (closed link to clustering)
\item Ordered discrete sequences data (infinite HMM)
\item Tree structured data or Hierarchical data (Adaptor grammar, infinite PCFG,
Kingman’s coalescent, HDP)
\item Overlapping clusters and factor analysis (Indian Buffet processes)
\end{enumerate}

\subsection{Clustering (Dirichlet processes and Chinese restaurant processes)}
\label{ssec:bnp-chinese}

\fi


\iffalse
This is a big-ish template that I've been using for years now.
A lot of the includes are redundant, so if you're reading this and have no idea why
I've included most things, don't worry - I don't either.
Also, compiling this (to PDF, for instance) relies on CSS files present on my
computer (coming from the packages). So it might not look as nice if you
do it on yours (especially if you don't have all the packages).

Ya'll've been warned.
\fi

\documentclass{article}
    \usepackage{subcaption}
    \usepackage{amsmath}
    \usepackage{amssymb}
    \usepackage{color}
    \usepackage[dvipsnames]{xcolor}
    \usepackage{graphicx}
    \usepackage{caption}
    \usepackage{float}
    \usepackage[hidelinks]{hyperref}
    \usepackage{enumitem}
    \usepackage[bottom]{footmisc}
    \usepackage{flexisym}
    \usepackage{cancel}
    \usepackage[braket]{qcircuit}
    \usepackage[margin=.8in, tmargin=.3in]{geometry}
    \renewcommand{\baselinestretch}{1.2}
    \newcommand{\eps}{\epsilon}
    % \newcommand{\tens}{\otimes}

    % Fancy comments from Gutmann
\newcommand{\comment}[1][]{#1} % Turn on color comments
% \newcommand{\comment}[1]{} % Turn off color comments
\newcommand{\todo}[1]{\comment{\textcolor{Red}{\textit{\lbrack TODO: #1 \rbrack}}}}
\newcommand{\note}[1]{\comment{\textcolor{Blue}{\textit{\lbrack NOTE: #1 \rbrack}}}}

\newcommand{\tens}[1]{%
  \mathbin{\mathop{\otimes}\limits_{#1}}%
}

    \setlength{\parskip}{\baselineskip}
    % \newcommand{\l}{\left(}

    \usepackage{titlesec}
    \usepackage{physics}

    \setlength\parindent{0pt}
    \captionsetup{justification=centering}

    \title{Quantum Computation and Universal Quantum Computing}
    \date{\today}
    \author{Traiko Dinev \textless traiko.dinev@gmail.com\textgreater}

\begin{document}
\maketitle
\textit{NOTE: This partially follows Introduction to Quantum Computing, a masters level course at the University of Edinburgh.}

\textit{NOTE: Note this "summary" is NOT a reproduction of the course materials nor is it copied from the corresponding courses. It was entirely written and typeset from scratch.}

\textit{License: Creative Commons public license; See README.md of repository}

\newcommand{\epsplus}{\left( \frac{1}{2} + \eps \right)^{1/2}}
\newcommand{\epsminus}{\left( \frac{1}{2} - \eps \right)^{1/2}}

\section{Quantum Systems}
The state of a binary quantum system is described by a vector (a ket) in 2 dimensions:
\begin{gather}
    \ket{0} = \begin{pmatrix}1 \\ 0\end{pmatrix} \qquad
    \ket{1} = \begin{pmatrix}0 \\ 1\end{pmatrix} \\
\end{gather}
%
Physically these can be any differentiable states. In digital computing we often use thresholding to distinguish the "1"s from the "0"s. Here we use naturally quantized states such as energy levels, spin or (equivalently) polarization. We can extend this to continuous spaces where the discrete quantum states become continuous and sums become integrals. This is the basis of infinite-dimensional Hilbert spaces, which are discussed at the end of this [todo].

\section{Double Slit Experiment and Superposition}
\todo{Motivate why the system collapses when a measurement is performed, etc.} \todo{Motivate superposition, motivate probabilistic view of QM}
\todo{normalization of wavefunctions}

\section{Postulates of Quantum Mechanics in Bra-Ket Notation}
\textbf{System State}
The state of a system is captured by its wavefunction $\psi(x)$. A wavefuncton describes the state of the system as a probability distribution over possible states.
\vskip 0.15in

\textbf{Observables and Wavefunction Collapse}
Every observable in classical mechanics (e.g. position, velocity, moment) has a corresponding \textbf{operator} in quantum mechanics. The operators correspond to observables. Observables are matrices and are Hermitian.
\vskip 0.15in

The most basic observable will be for given state $\ket{\alpha}$. If a system is in state $\ket{\psi}$ the inner product $\bra{\alpha}\ket{\psi}$ yields the probability of obtaining $\bra{\alpha}$ when measuing in a binary basis. The observable is the matrix that has eigenvalues $\ket{\alpha}$ and $\ket{\alpha}^\top$, i.e. not-$\ket{\alpha}$.

When we perform a measurement, the wavefunction is \textbf{collapsed} to one of the underlying states we are measuring. We will only consider orthogonal states in this note. So if we measure in the $\ket{0}$, $\ket{1}$ basis, we will collapse \textbf{every quantum state} to either $\ket{0}$ or $\ket{1}$. Different states only differ in the probability between the basis vectors. \textit{This is the essence of quantum mechanics.}

For example, position vectors are orthogonal states, which means we can measure (infinitely-small) the position of a particle. An operator, the observable, will have for eigenvectors (eigen-"states" in QM) each separate position.

\textbf{Superposition} Given the above observation, we can treat quantum states as a superposition of all possible \textbf{orthogonal} eigenstates. We can do all sorts of magic by measuring in clever states. Think of it as de-composing a signal into basic frequencies. \textit{Note: the following is my interpretation}. It's not that the signal itself \textbf{is built this way by nature}, it's a by-product of how we perceive and measure the world. \textit{If we can only treat an object as there and not there, we can treat it as being there AND not there at the same time with some probability.}

\textbf{Time-evolution} Schr{\"o}dinger \todo{add}
Note this is a wave equation. C.f. "the" wave equation (\todo{add rope wave equation}):

\begin{equation}
    H(t) \ket{\psi(t)} = i \hbar \frac{\partial}{\partial t} \ket{\psi(t)}
\end{equation}

$H$ is the hamiltonian, providing the total energy in the system. This is what we will have to find ourselves for different systems.

\subsection{Example in a discrete system}
Let's assume we have the state:
\begin{equation}
    \ket{\psi} = \alpha_0 \ket{\phi_0} + \dots \alpha_{k - 1} \ket{\phi_{K - 1}}
\end{equation}

This is a superposition of $K$ different states with varying amplitudes. This corresponds to the state having a distribution according to these amplitudes. Now let's say we have a device that can measure each of those states $\phi_i$. This assumes the states are orthogonal, because otherwise we wouldn't be able to perfectly distinguish those states. Then our \textbf{observable} is:

\begin{equation*}
    \textbf{M} = \sum_{j = 0}^{K - 1} \lambda_j
        \ket{\phi_j}\bra{\phi_j} = \text{diag}(\lambda_1, \dots, \lambda_{K - 1})
\end{equation*}

We take the outer product of the vectors, giving us a diagonal matrix over all states. The $\lambda_j$ correspond to the outcomes, i.e. the
\textbf{numbers we read off of the measurement device}.

Let's consider an example. Set $K = 3$ and assume the following quantum state:

\begin{equation*}
    \ket{\psi} = \frac{1}{\sqrt{2}}\ket{\phi_0} + \frac{1}{\sqrt{2}} \ket{\phi_2}
\end{equation*}

The operator for the four states could be:
\begin{gather*}
    \textbf{M} = \begin{pmatrix}
        1 & 0 & 0 & 0 \\
        0 & -1 & 0 & 0 \\
        0 & 0 & 2 & 0
    \end{pmatrix} = \\ 1 \times \ket{\phi_0}\bra{\phi_0} - 1 \times \ket{\phi_1}\bra{\phi_1} + 2 \times \ket{\phi_2}\bra{\phi_2}
\end{gather*}

The average measurement result would be $\bra{\psi} M \ket{\psi}$:
\begin{gather*}
    (\frac{1}{\sqrt{2}}\bra{\phi_0} + \frac{1}{\sqrt{2}} \bra{\phi_2}) \times 1 \times \ket{\phi_0}\bra{\phi_0} - 1 \times \ket{\phi_1}\bra{\phi_1} + 2 \times \ket{\phi_2}\bra{\phi_2} \times (\frac{1}{\sqrt{2}}\ket{\phi_0} + \frac{1}{\sqrt{2}} \ket{\phi_2}) \\
    = \frac{1}{2} (\ket{\phi_0} \bra{\phi_0} + 2 \times \ket{\phi_2}\bra{\phi_2}) = \frac{3}{2}
\end{gather*}

To understand why this makes sense, consider that $1/2$ of the time we will observe $\ket{\phi_0}$, i.e. the number (measurement) $1$. The other half we will observe $\ket{\phi_2}$ and $2$. Which adds up to the same figure.

\subsection{Projective Measurements}
\subsection{Density Matrices and Mixed Quantum Probabilitic States}
\textit{NOTE: This is a combination of quantum uncertainty and classical uncertainty (noise)}
\subsection{Tensors and Tensor Products (Systems of Multiple Quantum States)}

\section{No-cloning Theorem}
Assume there exists an operator $U$ such that:
\begin{gather*}
    U \ket{\psi} \tens{} \ket{0} = \ket{\psi} \tens{} \ket{\psi} \\
    U \ket{\phi} \tens{} \ket{0} = \ket{\phi} \tens{} \ket{\phi}
\end{gather*}

Then:
\begin{gather*}
    \ket{00} \xrightarrow{U} \ket{00} \\
    \ket{10} \xrightarrow{U} \ket{11}
\end{gather*}

Hence:
\begin{gather*}
    \alpha \ket{00} + \beta \ket{10} \xrightarrow{U}
        \alpha U \ket{00} + \beta U \ket{10} \\
        = \alpha \ket{00} + \beta \ket{11}
\end{gather*}

However, at the same time:
\begin{gather*}
    \alpha \ket{00} + \beta \ket{10} = (\alpha \ket{0} + \beta \ket{1}) \tens{} \ket{0} \xrightarrow{U}
        (\alpha \ket{0} + \beta \ket{1}) \tens{} (\alpha \ket{0} + \beta \ket{1})
\end{gather*}

Which are trivialy not the same on the right side, but are the same on the left side. Q.e.d.

\section{Entanglement and Bell States}
Entanglement is a phenomenon that exists only in quantum systems and it's at the core of quantum computing. Mathematically, consider the following operation on two quantum states:

\begin{align*}
    \ket{+} &= H \ket{0} =
        \frac{1}{\sqrt{2}}
        \begin{pmatrix}
            1 & 1 \\
            1 & -1
        \end{pmatrix} \begin{pmatrix} 1 \\ 0 \end{pmatrix} =
        \frac{1}{\sqrt{2}} \begin{pmatrix} 1 \\ 1 \end{pmatrix}
            = \frac{1}{\sqrt{2}} (\ket{0} + \ket{1})  \\
    \ket{-} &= H \ket{1} = \frac{1}{\sqrt{2}} (\ket{0} - \ket{1})
\end{align*}

where $H$ is the hadamard gate. These are a superposition of states. Now let's use $\ket{+}$ as a control for a CNOT on a $\ket{0}$. I.e. this circuit:

\[
    \Qcircuit @C=1em @R=.7em {
        & \lstick{\ket{0}} & \gate{H} & \ctrl{1} \\
        & \lstick{\ket{0}} &  \qw     & \targ
    }
\]

The resulting quantum state (of both qubits) is:
\begin{align*}
    \frac{1}{\sqrt{2}}(\ket{0} + \ket{1})\ \text{CNOT}\ \ket{0} =
        \frac{1}{\sqrt{2}} (\ket{00} + \ket{11})
\end{align*}

where $\ket{00} = \ket{0} \tens{} \ket{1}$ is shorthand for a tensor product. This is known as a \textbf{maximally entangled} state. This system cannot be separated into two distinct systems (i.e. two tensor-ed quantum states $\ket{\alpha} \tens{} \ket{\beta}$). Another way to look at it is that if these qubits were separated (to Alice and Bob's labs), then measurements depend on each other. If Alice measures $\ket{0}$, then the state collapses to $\ket{00}$ and Bob also measures $\ket{0}$ (and vice-versa).

This is a profound result that holds true in reality. In fact, observation even after the fact causes the system to collapse. Key words are \textbf{quantum eraser} and \textbf{delayed-choice quantum eraser} experiments.

\section{Superdense Coding}
\[
  \Qcircuit @C=1em @R=.7em {
    &                  &          &          & b_1 \qquad \cwx[1]   & b_2 \qquad \cwx[1] & \\
    & \lstick{\ket{0}} & \gate{H} & \ctrl{1} & \targ                & \targ              & \ctrl{1} & \gate{H} & \meter & \rstick{b_1} \cw\\
    & \lstick{\ket{0}} &  \qw     & \targ    & \qw                  &  \qw               & \targ    & \qw      & \meter & \rstick{b_2} \cw
  }
\]

\begin{align*}
    b_1 b_2 ; \quad
            & \text{if}\ b_1 = 1, \text{then}\ Z \\
            & \text{if}\ b_2 = 1, \text{then}\ X
\end{align*}

\end{document}