\section{Conclusions}
\label{section:conclusion}

We present a statistical framework for measuring precise ages of MS stars and
subgiants by combining observables that relate, via different evolutionary
processes, to stellar age.
Specifically, we combined HRD/CMD placement with rotation periods, in a
hierarchical Bayesian model, to age-date stars based on both their hydrogen
burning and magnetic braking history.
The two methods of isochrone fitting and gyrochronology were combined by
taking the product of two likelihoods: one that contains an isochronal model
and the other a gyrochronal one.
We used the MIST stellar evolution models and computed isochronal ages and
likelihoods using the {\tt isochrones} {\it Python} package.
We fit a new broken power law gyrochronology model to the Praesepe cluster
and included a modification recommended by \citet{vansaders2016} that accounts
for weakened magnetic braking at Rossby numbers larger than 2.
The rotation periods of hot stars, cool stars, evolved stars, young stars and
metal poor and rich stars were modeled with a broad log-normal distribution.
We tested \sd\ on simulated data and cluster stars and demonstrated that
combining gyrochronology with isochrone fitting improves the precision of age
estimates for FGK dwarfs by a factor of 3 over isochrone fitting alone,
assuming 5\% measurement uncertainties on rotation periods.
Incorporating rotation periods into stellar evolution models also improves the
precision of the equivalent evolutionary phase (EEP) parameter and, since EEP,
combined with age and metallicity, determines the mass, radius and \logg\ of a
star, this means rotation periods can improve the precision of {\it all}
stellar parameters.
Although V-band extinction is marginalized over during inference, correcting
photometry for dust-extinction before analysis, or including it as a
prior can improve the accuracy of stellar ages measured with \sd.
\racomment{
We also tested \sd\ on a set of 21 \kepler\ asteroseismic stars
\citep{vansaders2016}.
We found that discrepancies between ages measured with \sd\ and ages measured
with asteroseismology are likely produced by differences between the MIST and
BaSTI stellar evolution models.
% In general, the new Praesepe-based gyrochronology relation is not a good model
% for all asteroseismic stars.
% Ultimately, these empirical gyrochronology relations need further, and
% more detailed calibration.
Asteroseismic and cluster stars provide an opportunity for calibration but
given the high-dimensionality of the gyrochronology relations (\ie\ rotation
period depends on age, mass, metallicity, surface gravity, etc), many stars
with precise ages, spanning a range of properties, are still needed to
reliably calibrate them.
}

In cases where gyrochronology predicts inaccurate stellar ages it is either
because models are not correctly calibrated, because the rotation periods or
rotation period uncertainties are themselves inaccurate, or because of
rotational outliers.
For example, \sd\ may predict inaccurate ages for stars in close binaries
whose interactions influence their rotation period evolution.
Rotational outliers are often seen in clusters \citep[see \eg][]{douglas2016,
rebull2016, douglas2017, rebull2017} and many of these fall above the main
sequence on a CMD, indicating that they are binaries.
In addition, measured rotation periods may not always be accurate and can, in
many cases, be a harmonic of the true rotation period.
For example, a common rotation period measurement failure mode is to measure
half the true rotation period.
The best way to prevent an erroneous or outlying rotation period from
resulting in an erroneous age measurement is to {\it allow} for outlying
rotation periods using a mixture model, a feature that could be built into
\sd\ in the future.

% Motivation
The optimal way to age-date stars is by combining {\it all} their available
age-related observables.
This could ultimately include activity dating via flare rates and
chromospheric activity indices, kinematic dating and chemical dating.
Of all the established age-dating methods, gyrochronology and isochrone
fitting are two of the most complementary.
The two methods are optimal in different parts of the HRD:
gyrochronology works well for FGK dwarfs and isochrone fitting works well for
subgiants and hot stars, so combining the two methods results in consistently
precise ages across a range of masses, ages and evolutionary stages.
In addition, using both methods at once circumvents the need to decide which
method to use {\it a priori}.
It eliminates the circular process of classifying a star based on its CMD
position (M dwarf, subgiant, etc), then deciding which age-dating method to
use, then inferring an age which itself depends on the classification that was
made.
It is important to infer all stellar properties at once since they all depend
on each other.
\sd\ is applicable to a large number of stars: FGKM dwarfs and subgiants with
a rotation period and broad-band photometry.
This already includes tens-of-thousands of \kepler\ and \ktwo\ stars and could
include millions more from \tess, \lsst, \wfirst, \plato, \gaia, and others in
the future.

The code used in this project is available as a {\it Python} package called
\sd, DOI: \url{ https://doi.org/10.5281/zenodo.2712419}.
It is available for download via Github\footnote{git clone
https://github.com/RuthAngus/stardate.git} or through
PyPI\footnote{pip install stardate\_code}.
Documentation is available at \url{https://stardate.readthedocs.io/en/latest/}.
All code used to produce the figures in this paper is available at
\url{https://github.com/RuthAngus/stardate}.
This paper is based on code with the following Git hash:
f739562c1546e117e9bb217e1732c62b41be8061.
% git log --pretty=format:'%H' -n 1
% git log --pretty=format:'%h' -n 1


\documentclass[10pt]{article}
\usepackage[margin=1in]{geometry}
\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage{amsfonts}
\usepackage{amsthm}
\usepackage{amsmath}
\usepackage{amscd}
\usepackage{amsmath,amsthm,amssymb}
\usepackage[mathscr]{eucal}
\usepackage{indentfirst}  % indent first paragraph after headers
\usepackage{graphicx}
\usepackage{graphics}
\usepackage{fancyhdr}  % headers and footers
\usepackage{authoraftertitle}  % making author, title variables available
\usepackage{mathrsfs}
\usepackage{siunitx}
\usepackage{braket}  % bra-ket notation
\usepackage{array}

% Defining mathematical paper sections

\numberwithin{equation}{section}

\theoremstyle{plain}
\newtheorem{Th}{Theorem}[section]
\newtheorem{Cor}[Th]{Corollary}

\theoremstyle{definition}
\newtheorem{Def}[Th]{Definition}
\newtheorem{Ex}[Th]{Example}
\newtheorem{Rem}[Th]{Remark}

\title{Title}  % Title appearing on title page and page headers
\author{Elijah Sheridan} % Author appearing on title page

\begin{document}

% Title Page

\begin{titlepage}
	\centering
	{\scshape\Large Project Name \par}  % Project name
	\vspace{0.75cm}
	{\scshape\Large Course Num \\Course Name \par}  % Course name and number
	\vspace{1cm}
	\includegraphics[width=0.3\textwidth]{example-image-a}\par\vspace{1cm}
	{\scshape\LARGE \MyTitle \par}
	\vspace{1.25cm}
	{\Large\itshape \MyAuthor \par}
	\vfill
	supervised by\par
	Professor~First \textsc{Last}  % Professor Name
	\vfill
	{\large \today\par}
\end{titlepage}

\pagestyle{fancy}
\lhead{\MyTitle}  % Title number in header
\rhead{\thepage}  % Page number in header

% Table of Contents

\tableofcontents

\newpage

% Paper contents

\section{First Section}

\subsection{First Subsection}

This is a citation: \cite{test}.

\begin{Th}[Example Theorem]
This is a theorem.
\end{Th}

\begin{Cor}[Example Corollary]
This is a corollary.
\end{Cor}

\begin{Def}[Example Definition]
This is a definition.
\end{Def}

\begin{Ex}[Example Example]
This is an example.
\end{Ex}

\begin{Rem}[Example Remark]
This is a remark.
\end{Rem}

\newpage

% Bibliography

\bibliographystyle{plain}
% if you don't want in-text citations:
%\nocite{*}
\bibliography{ref.bib}

\end{document}


\documentclass[anon]{CI}

% The following packages will be automatically loaded:
% amsmath, amssymb, natbib, graphicx, url, algorithm2e
\usepackage{float}
%\usepackage{amsthm}

\newtheorem{assumption}{Assumption}

\title[A Practical Comparison Between Hopfield Networks and Restricted Boltzmann Machines]{A Practical Comparison Between Hopfield Networks and Restricted Boltzmann Machines as Content-Addressable Autoassociative Memories}

 % Use \Name{Author Name} to specify the name.
 % If the surname contains spaces, enclose the surname
 % in braces, e.g. \Name{John {Smith Jones}} similarly
 % if the name has a "von" part, e.g \Name{Jane {de Winter}}.
 % If the first letter in the forenames is a diacritic
 % enclose the diacritic in braces, e.g. \Name{{\'E}louise Smith}

 % Two authors with the same address
  % \coltauthor{\Name{Author Name1} \Email{abc@sample.com}\and
  %  \Name{Author Name2} \Email{xyz@sample.com}\\
  %  \addr Address}

 % Three or more authors with the same address:
 % \coltauthor{\Name{Author Name1} \Email{an1@sample.com}\\
 %  \Name{Author Name2} \Email{an2@sample.com}\\
 %  \Name{Author Name3} \Email{an3@sample.com}\\
 %  \addr Address}

 % Authors with different addresses:
 \author{
\Name{Javier Beltr{\'a}n} \Email{javierbeltranj@gmail.com}\\
 %\addr Academic address (optional)
 \AND
 \Name{Guillermo Bern{\'a}rdez} \Email{gbg1441@gmail.com}\\
 %\addr Academic address (optional)
 \AND
 \Name{Juan Lao-Tebar} \Email{nitzing@gmail.com}\\
 %\addr Academic address (optional)
 \AND
 \Name{Jorge Rodr{\'i}guez} \Email{j.rodriguez.molinuevo@gmail.com}\\
 %\addr Academic address (optional)
 }

\begin{document}

\maketitle

\begin{abstract}
In this paper we analyze the performance of Hopfield Networks and Restricted Boltzmann Machines when they are used as autoassociative memories for several kinds of addressable binary content. The aim of this work is to recommend to the reader a model that offers the best results for a specific given pattern data set, based on the results obtained after testing several parameterized models against different random pattern data sets that satisfy the same statistical properties.
\end{abstract}

\begin{keywords}
 Artificial Neural Networks, Recurrent Neural Networks, Hopfield Networks, Restricted Boltzmann Machine, Autoassociative Memory, Addressable Content
\end{keywords}

\section{Introduction}

Attractor networks such as Hopfield Networks [\cite{hopfield1982neural}] and Restricted Boltzmann Machines [\cite{Smolensky87}] are widely used as binary content-addressable memory systems, [\cite{DAI1998159}][\cite{SCHULZ1995145}][\cite{KrizhevskyH11}] and the theory behind them has been deeply studied in the past years. [\cite{80232}][\cite{298706}][\cite{6889573}] Both models have been compared on specific problems [\cite{552753}] and mathematical relations of equivalence have been demonstrated. [\cite{Barra20121}][\cite{Agliari201352}]

In this paper we analyze the behaviour of both models after being trained with several random pattern data sets that satisfy certain statistical properties. These properties can be extracted from any kind of binary data set and, in conjunction with the result tables presented in this paper, the reader can have an approximate idea about what model offers the best results for his specific problem. In addition, if the reader has a sample of expected input for the system, it is possible to extract other statistical properties from the sample and compare them with the statistical properties of the tested random input pattern data sets.

Hopfield Networks are known to have a limited number of patterns that can be successfully stored, a.k.a. \emph{capacity}, as well as to be very sensitive to correlations between the training samples, which aggravates their ability to generate correct attractors and also reduces the capacity of the network[\cite{hopfield1982neural}][\cite{Storkey1997}]. For this reason, we expect bad performances of these networks both when considering training samples very correlated and when considering large amounts of patterns to store.

Regarding Restricted Boltzmann Machines, we are not aware of any limitation, We presumably expect this networks to have a higher ability to memorize large amounts of patterns than Hopfield Networks, but perhaps at the expense of a smaller ability to recall these memories from similar ones.

%\paragraph{What we expected:}
%\begin{itemize}
%	\item A higher ability to memorize patterns/capacity of Hopfield Networks trained with the Storkey rule vs Hebbian rule.
%	\item A worse performance of Hopfield networks when:
%	\begin{itemize}
%		\item Incrementing the number of patterns to memorize.
%		\item Incrementing the correlation between the patterns to memorize.
%	\end{itemize}
%	\item Nothing special about RBMs (perhaps the possibility to adapt to the data by tuning its parameters? Able to memorize whatever thanks to overfitting?).
%\end{itemize}

\section{Analyzed Models}

Hopfield Networks are well-known content-addressable memories for which the Hebbian learning rule has been the traditional training approach. [\cite{hopfield1982neural}][\cite{Hebb1950}] This paper analyzes the Storkey learning rule [\cite{Storkey1997}] as well as the previously mentioned Hebbian. Storkey is being considered because it has been proved to provide a great increase in the capacity of the network, that is, it is able to recall more patterns.

Additionally, units in the Hopfield Network may be updated either synchronously or asynchronously. This paper only contains an analysis of the asynchronous method, since the synchronous is considered less realistic based on the absence of observed global clock influencing analogous biological or physical systems of interest. [\cite{MacKay2003}]

Boltzmann Machines are usually considered a stochastic analogue of the Hopfield Networks. [\cite{Ackley1985147}] Despite their theoretical usefulness, it is well known that in practice they cannot learn properly for large enough problems. That’s why we are considering its constrained version, the Restricted Boltzmann Machine, that permits an efficient training using the Contrastive Divergence algorithm. [\cite{carreira2005contrastive}]

Learning in a Restricted Boltzmann Machine is dependent on several parameters that have to be tuned appropriately. Our experiments try to follow Hinton’s recommendations [\cite{Hinton2012}] on the following issues, although some details have been implemented in a different way:
\begin{itemize}
	\item An appropriate estimation of the number of hidden units required is necessary to prevent overfitting, but hard to define. Our decisions are inspired in Hinton’s recommendations.
	\item A mini-batch containing one element of each class is proposed by Hinton’s, but we find that smaller mini-batches work better in our study.  In fact, we use an online strategy for training.
	\item  Stopping criteria of the training algorithm. Given the norm of the delta for the weights in each epoch, the training algorithm stops when the summation of the norm of the delta weights for the last 1000 epochs do not decrease during other 1000 epochs. At this point it is supposed that the weights are oscillating around their target value.
	\item The weights and offsets initialization follows the recommended values: small random values for the weights chosen from a normal $N(0,0.01)$ distribution, hidden offsets equal to 0, and visible offsets set to $\log{\frac{p_i}{(1-p_i)}}$, where $p_i$ is the proportion of training vectors in which unit $i$ is on.
	\item The learning rate parameter is also crucial for a good performance of the network. We fix it to 0.001 in all our models since this value provides better results than any other checked.
	\item Momentum is an improvement on the speed of the learning rule. We follow the recommendations, setting its value to 0.5 during the first stages of the learning process (5 epochs) and then incrementing it to 0.9.
	\item Theoretically, weight decay helps learning and improves generalization. However, this seems not to apply in our case, as the obtained results are worse when we use it. For this reason, we decide to disable this parameter in our final models.

\end{itemize}

Appendix \ref{AppB} presents a table of the analyzed models in this experiment, with explicit values for each of its parameters.

\section{Pattern Data Set Statistical Properties}

In abstract, a set of statistical properties can be extracted from any kind of binary pattern data set. For this experiment we generated random pattern data sets that have a predefined set of statistical property values.

Let $d(A, B)$ be the distance\footnote{For every pair $A,B \in \{0,1\}^n$, we define the distance $d$ between $A$ and $B$, $d(A,B)$, as the number of different components between both vectors; formally, $d(A,B) = \sum_{i=1}^{n} |A_i - B_i|$. Note that, with this definition, $d$ is the restriction of the well-known $L_1$ distance to the space $\{0,1\}^n$.} between the couple of patterns $A$, $B$. If we choose randomly (with a uniform distribution) a couple of 2 different patterns $A$, $B$ from a pattern data set $P$ and we define $D_P=d(A, B)$, we conclude that $D_P$ is a random variable that follows a normal distribution, with mean $\mu_P$ and standard deviation $\sigma_P$.

Basically, the following statistical properties define a pattern data set $P$:
\begin{itemize}
	\item $\left|P\right|$, the number of patterns in the set.
	\item $n$, the dimension of the patterns in $P$.
	\item $\frac{\mu_P}{n}$
	\item $\frac{\sigma_P}{n}$
\end{itemize}

We point out that we use this mean distance between two random patterns of the Pattern Data Set as a direct measure of their correlation; the smaller the distance is, the higher the number of components shared, i.e. the higher the correlation.

The reader of this paper can extract these statistical properties from any desired pattern data set, and use as a reference the most similar pattern data set analyzed in this experiment. Appendix \ref{AppA} presents a table of the generated and analyzed pattern data sets, with explicit values for their statistical properties.

\section{Input Data Set Statistical Properties}

Input data sets have a strong relation with the original pattern data set used to train the model. It is possible to extract some statistical properties from this relation, and they can be used to have an idea about the performance of the model for input data sets with the same properties.

For each input vector $v$, let $minD(v,P)=d(v,p)$, where $p$ is the pattern from the pattern data set $P$ such that $d(v,p)\leq d(v,p'),\;\forall p'\in P,\;p'\neq p$.

For a given pattern data set $P$, if we choose randomly (with a uniform distribution) an input vector $v$ from an input data set $I$ and we define $D_I=minD(v,P)$, we conclude that $D_I$ is a random variable that follows a normal distribution, with mean $\mu_I$ and standard deviation $\sigma_I$.

Basically, the only proposed statistical property that defines an input data set is $\frac{\mu_I}{\mu_P}$, the mean minimum distance between a random input $v$ and the closest pattern to $v$, expressed as a proportion of the mean distance between 2 random patterns of the pattern data set. We consider that $\sigma_I$ has not huge relevance for our analysis.

The reader of this paper can extract these statistical properties from any sample input data set passed to the system, and use as a reference the most similar input data set analyzed in this experiment. Appendix \ref{AppC} presents a table of the generated and analyzed input data sets, with explicit values for their statistical property.

\section{Experiment}

In this experiment we generated stochastically 28 different pattern data sets (with different statistical properties), 6 different models (with different determined combinations of tuned parameters) and 5 different random input data sets (with different statistical properties). The characteristics of each of these items are defined in appendices \ref{AppA}, \ref{AppB}, \ref{AppC}.

The next step of the experiment consisted in training each of the 6 models with each of the 28 pattern data sets, obtaining 168 training and validation results that are presented in appendix \ref{AppD}.

Finally, we tested each of the 168 trained models with each of the 5 input data sets, obtaining 840 test results that are presented in appendix \ref{AppE}.

This process has been executed 8 times, and the tables presented in appendices \ref{AppA}, \ref{AppB}, \ref{AppC} contain the mean and standard deviation of the generated and obtained values.

This experiment can be reproduced following the steps explained in appendix \ref{AppF}.

\section{Analysis of the Results}

Before presenting the obtained results after training and testing the models, we need to make an explanation, in regards to the dimensionality of the pattern data sets:

\begin{assumption} We are computing many measures as a proportion of the dimension of the patterns, and we assume that the obtained results may not depend on that pattern dimension. Some tests that we have performed considering both lower and higher pattern dimensions suggest that the behaviour does not change when the dimension varies, but this should be checked more rigorously in Future Work.
\end{assumption}

We have separated the results for both phases mentioned above, first we will start with the training phase results.

\subsection{Training and Validation Results}
Appendix \ref{AppD} presents the training and validation results we obtained for the analyzed models. The first thing we need to note is the possible variations in the learning rules. Hopfield Networks trained with the Storkey rule clearly outperformed the ability to memorize patterns of those trained with the Hebbian rule, independently of the correlation (or distance) between the patterns of the pattern data set. The results suggest an increase of the capacity using the Storkey rule, as we expected.

An issue that has been commented before and is presented in several publications is the correlation between patterns, which clearly affects the capacity of Hopfield Networks. As the mean distance between two random patterns of the pattern data set is increased (i.e. the correlation is reduced), we observe that these networks are able to successfully store a higher number of patterns

Furthermore, results show a much higher robustness of the RBM with respect to both the correlation of the patterns of the pattern data set and the number of patterns that must be stored. In some configurations, such as M3 and M4, RBM is able to remember almost all patterns even when they are numerous and highly correlated. Hence, for pattern data sets whose $\mu_P$ is lower or equal than $ 30\% $ of the dimension of the patterns (i.e. whose patterns exhibit a strong correlation), Restricted Boltzmann Machines properly configured provide much better ratios of stored patterns than any Hopfield Network.

When dealing with pattern data sets with less correlated patterns (i.e. with mean distances between patterns higher than $30\%$ of the dimension of the patterns), Hopfield Networks trained with the Storkey rule achieve similar ratios of successfully stored patterns than those of RBM, all of this as long as the capacity of the Hopfield Network is not largely exceeded by the number of patterns to store [\cite{Storkey1997}]. The increase of the capacity using the Storkey rule allows us to obtain very good results with Hopfield Networks for a considerably wide range of numbers of patterns to be memorized.

Finally, we also notice that RBM properly configured seem to have a very large capacity compared to Hopfield Networks. Therefore, if the considered pattern data set contains a higher number of patterns to be stored than the relative fractions analyzed in this paper, it will be reasonable that the reader considers the use of an RBM instead of a Hopfield Network.

\subsection{Testing results}

Appendix \ref{AppE} presents the testing results we obtained for the analyzed models. These results can be summarized as follows.

To begin with, as we have described in the previous subsection, Hopfield Networks trained with the Hebbian rule show very bad performances in terms of memorizing patterns when they are compared to those trained with the Storkey rule, mainly because of its lower capacity. The difficulties of the learning process are obviously inherited in these testing results, where we test the ability of the models to recall the original patterns: they provide the worst performance by far.

We have observed a general trend shared by all models with respect to the input properties: the ability to recall the original patterns decreases as the inputs differ more from them. This behaviour was expected.

In our specific case, both RBM configurations considered ($M_3$ and $M_4$) only differ from the number of hidden neurons. We note that they obtain similar results in the majority of cases, though the model with the higher number of hidden neurons provides a slightly better overall performance.

We also observe that the ability of RBM to recall the training patterns given similar input vectors now depends on both the number of stored patterns ---more patterns imply lower ability--- and the correlation between the training patterns ---the more decorrelated the patterns are, the higher ability to recall them---. The latter dependency seems stronger.

However, as happened in training, with the most correlated pattern data sets RBM still performs better than Hopfield Networks. Since Hopfield can store a lower number of patterns than RBM, this behaviour was expected. Nevertheless, as the correlation between patterns decreases, results suggest that Hopfield Networks using the Storkey rule outperform RBM when restoring patterns that highly differ from the original ones.

It seems then that Hopfield Networks generate deeper attractors than RBM. It can be observed that, given a pattern data set, the ability of Hopfield Networks to recall patterns remains approximately constant when augmenting the variability of the input patterns with respect to the original stored patterns. Yet, in RBM there is a clear loss of restoration power as the variability of the inputs increases. Hence, if the patterns of the pattern data set are sufficiently decorrelated and it is expected to receive highly variable inputs, Hopfield Networks appear to be more robust for recalling, as long as the number of patterns to be stored does not overtake the capacity of the network.

\section{Conclusions and future work}

In this project we do a practical analysis on the performance of two models used as autoassociative memories for several kinds of addressable binary content: Hopfield Networks and Restricted Boltzmann Machines. For this purpose we generate random pattern data sets ---with different properties---, we train the models and then perform several tests in order to extract some conclusions from the results.

As we observed in the training and validation process, in all the cases, Hopfield Networks trained with the Storkey learning rule are able to store a higher number of patterns than those trained with the Hebbian rule. Additionally, as the distance between patterns increases (i.e. the correlation decreases), Hopfield Networks are able to store a larger amount of patterns. We also observed that RBM is able to store a large amount of patterns (in comparison with Hopfield Networks), independently of the correlation between them.

On the testing process, we observed that Hopfield Networks are able to recall almost all remembered patterns, independently on the distance between the given input and the pattern. On the other hand, RBM recall performance is inversely proportional to the distance between the input and the pattern, not being able to recall it in several cases. However, a larger number of hidden neurons provides a slightly better overall performance. RBM recall performance also depends on the number of stored patterns, as more stored patterns imply a slight reduction of successful recalls.

More testing cases are still pending (e.g. several combinations of pattern properties), but due to the limited resources of our team, we could not perform a larger experiment. We assume that our conclusions are valid for any pattern dimension, but more research is needed so as to validate it.

It is also possible to tune several parameters of RBM to offer better results. The optimal tuning of RBM is far from the scope of this paper, but in the future it could be done automatically using an optimization algorithm, such as simplex or genetic algorithms. In the past years some techniques for improving Hopfield Networks appeared, [\cite{Paik1992}] and it would be interesting to test them too.

Additionally, although Hinton’s guide for training RBM [\cite{Hinton2012}] recommends the usage of a weight decay, we were unable to train any model using it, so we have not included it in this paper. It is possible that this part of our RBM implementation is not correct, causing this problem.

Taking everything into account, it would be interesting to reinforce the conclusions of this paper in a real scenario, such as classifying the MNIST database of handwritten digits. [\cite{lecun1998mnist}]

%\section*{References}

\bibliography{bibfile}

\pagebreak

\appendix

\section{Annalyzed Pattern Data Sets}\label{AppA}

% PATTERN DATA SETS TABLE

\begin{table}[H]
	\centering
	%\label{my-label}
	\def\arraystretch{1.5}
	%\resizebox{\textwidth}{!}{%
	\begin{tabular}{cccccccc}

		& $P_{1}$  & $P_{2}$  & $P_{3}$  & $P_{4}$  & $P_{5}$  & $P_{6}$  & $P_{7}$  \\ \cline{2-8}
		$\left|P_i\right|$  & 5  & 10  & 15  & 20  & 25  & 30  & 35 \\
		$n$  & 100  & 100  & 100  & 100  & 100  & 100  & 100 \\
		$\frac{\mu_{P_i}}{n}$  & $0.20$  & $0.20$  & $0.20$  & $0.20$  & $0.20$  & $0.20$  & $0.20$ \\
		$\frac{\sigma_{P_i}}{n}$  & $0.02$  & $0.02$  & $0.02$  & $0.02$  & $0.02$  & $0.02$  & $0.02$
	\end{tabular}
	%\end{tabular}}
	\caption{Analyzed pattern data sets (1 of 4)}
\end{table}

\begin{table}[H]
	\centering
	%\label{my-label}
	\def\arraystretch{1.5}
	%\resizebox{\textwidth}{!}{%
	\begin{tabular}{cccccccc}

		& $P_{8}$  & $P_{9}$  & $P_{10}$  & $P_{11}$  & $P_{12}$  & $P_{13}$  & $P_{14}$  \\ \cline{2-8}
		$\left|P_i\right|$  & 5  & 10  & 15  & 20  & 25  & 30  & 35 \\
		$n$  & 100  & 100  & 100  & 100  & 100  & 100  & 100 \\
		$\frac{\mu_{P_i}}{n}$  & $0.30$  & $0.30$  & $0.30$  & $0.30$  & $0.30$  & $0.30$  & $0.30$ \\
		$\frac{\sigma_{P_i}}{n}$  & $0.03$  & $0.03$  & $0.03$  & $0.03$  & $0.03$  & $0.03$  & $0.03$
	\end{tabular}
	%\end{tabular}}
	\caption{Analyzed pattern data sets (2 of 4)}
\end{table}

\begin{table}[H]
	\centering
	%\label{my-label}
	\def\arraystretch{1.5}
	%\resizebox{\textwidth}{!}{%
	\begin{tabular}{cccccccc}

		& $P_{15}$  & $P_{16}$  & $P_{17}$  & $P_{18}$  & $P_{19}$  & $P_{20}$  & $P_{21}$  \\ \cline{2-8}
		$\left|P_i\right|$  & 5  & 10  & 15  & 20  & 25  & 30  & 35 \\
		$n$  & 100  & 100  & 100  & 100  & 100  & 100  & 100 \\
		$\frac{\mu_{P_i}}{n}$  & $0.40$  & $0.40$  & $0.40$  & $0.40$  & $0.40$  & $0.40$  & $0.40$ \\
		$\frac{\sigma_{P_i}}{n}$  & $0.04$  & $0.04$  & $0.04$  & $0.04$  & $0.04$  & $0.04$  & $0.04$
	\end{tabular}
	%\end{tabular}}
	\caption{Analyzed pattern data sets (3 of 4)}
\end{table}

\begin{table}[H]
	\centering
	%\label{my-label}
	\def\arraystretch{1.5}
	%\resizebox{\textwidth}{!}{%
	\begin{tabular}{cccccccc}

		& $P_{22}$  & $P_{23}$  & $P_{24}$  & $P_{25}$  & $P_{26}$  & $P_{27}$  & $P_{28}$  \\ \cline{2-8}
		$\left|P_i\right|$  & 5  & 10  & 15  & 20  & 25  & 30  & 35 \\
		$n$  & 100  & 100  & 100  & 100  & 100  & 100  & 100 \\
		$\frac{\mu_{P_i}}{n}$  & $0.50$  & $0.50$  & $0.50$  & $0.50$  & $0.50$  & $0.50$  & $0.50$ \\
		$\frac{\sigma_{P_i}}{n}$  & $0.05$  & $0.05$  & $0.05$  & $0.05$  & $0.05$  & $0.05$  & $0.05$
	\end{tabular}
	%\end{tabular}}
	\caption{Analyzed pattern data sets (4 of 4)}
\end{table}

\section{Analyzed Models}\label{AppB}

% MODELS TABLE

\begin{table}[H]
	\centering
	%\label{my-label}
	\def\arraystretch{1.5}
	%\resizebox{\textwidth}{!}{%
	\begin{tabular}{ccccc}

		& $M_{1}$  & $M_{2}$  & $M_{3}$  & $M_{4}$  \\ \cline{2-5}
		(1)  & Hopfield  & Hopfield  & RBM  & RBM \\
		(2)  & Hebbian  & Storkey  & CD  & CD \\
		(3)  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & 50  & 100 \\
		(4)  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & 1  & 1
	\end{tabular}
	%\end{tabular}}
	\caption{Analyzed models}
\end{table}

\begin{description}
	\item [(1)] Model
	\item [(2)] Learning rule.
	\item [(3)] Number of hidden neurons.
	\item [(4)] Patterns per batch
	\end {description}

	\section{Tested Input Data Sets}\label{AppC}

	% INPUT DATA SETS TABLE

	\begin{table}[H]
		\centering
		%\label{my-label}
		\def\arraystretch{1.5}
		%\resizebox{\textwidth}{!}{%
		\begin{tabular}{cccccc}

			& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$  \\ \cline{2-6}
			(1)  & 10  & 10  & 10  & 10  & 10 \\
			$\frac{\mu_{I_i}}{\mu_{P_j}}$  & $0.1\pm0.0$  & $0.2\pm0.0$  & $0.3\pm0.0$  & $0.4\pm0.0$  & $0.5\pm0.0$
		\end{tabular}
		%\end{tabular}}
		\caption{Randomly generated and tested input data sets}
	\end{table}

	\begin{description}
		\item [(1)] Number of generated inputs per each pattern of the pattern data set, $\frac{\left|I_i\right|}{\left|P_j\right|}$
		\end {description}

		\section{Training and Validation Results}\label{AppD}

		% TRAINING EPOCHS TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\scriptsize
			\begin{tabular}{ccccc}

				& $M_{1}$  & $M_{2}$  & $M_{3}$  & $M_{4}$ \\ \cline{2-5}
				$P_{1}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $18124\pm7069$  & $14541\pm2420$ \\
				$P_{2}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $29057\pm8731$  & $11242\pm2781$ \\
				$P_{3}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $33583\pm11031$  & $12570\pm3410$ \\
				$P_{4}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $28940\pm16282$  & $12377\pm5766$ \\
				$P_{5}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $35530\pm14682$  & $9834\pm2972$ \\
				$P_{6}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $25162\pm9013$  & $12035\pm2854$ \\
				$P_{7}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $22319\pm8792$  & $12006\pm3489$ \\
				$P_{8}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $31690\pm8718$  & $21964\pm4678$ \\
				$P_{9}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $35074\pm8181$  & $18915\pm3348$ \\
				$P_{10}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $42173\pm21376$  & $23287\pm9620$ \\
				$P_{11}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $42339\pm25028$  & $26494\pm15153$ \\
				$P_{12}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $64959\pm39081$  & $36511\pm14325$ \\
				$P_{13}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $38354\pm22668$  & $24570\pm11618$ \\
				$P_{14}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $36352\pm23636$  & $30642\pm14164$ \\
				$P_{15}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $44274\pm14970$  & $44308\pm13092$ \\
				$P_{16}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $60494\pm12280$  & $37955\pm7551$ \\
				$P_{17}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $86497\pm46676$  & $50112\pm22391$ \\
				$P_{18}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $64698\pm33553$  & $37774\pm12024$ \\
				$P_{19}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $95758\pm71621$  & $60481\pm30881$ \\
				$P_{20}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $76866\pm60918$  & $98700\pm16541$ \\
				$P_{21}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $58147\pm41255$  & $74267\pm29266$ \\
				$P_{22}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $74411\pm38141$  & $59317\pm11266$ \\
				$P_{23}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $69858\pm33073$  & $66048\pm10678$ \\
				$P_{24}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $132268\pm57657$  & $82277\pm23651$ \\
				$P_{25}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $217777\pm271439$  & $93176\pm38089$ \\
				$P_{26}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $53481\pm55669$  & $114696\pm31592$ \\
				$P_{27}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $43289\pm35583$  & $131611\pm65733$ \\
				$P_{28}$  & \textcolor{gray}{n/a}  & \textcolor{gray}{n/a}  & $56949\pm69490$  & $151782\pm41599$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of epochs needed to train model $M_i$ with pattern data set $P_j$}
		\end{table}

		% TRAINING STORED PATTERNS TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{ccccc}

				& $M_{1}$  & $M_{2}$  & $M_{3}$  & $M_{4}$ \\ \cline{2-5}
				$P_{1}$  & $0.00\pm0.00$  & $0.42\pm0.12$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{2}$  & $0.00\pm0.00$  & $0.17\pm0.04$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{3}$  & $0.00\pm0.00$  & $0.07\pm0.02$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{4}$  & $0.00\pm0.00$  & $0.05\pm0.00$  & $0.87\pm0.33$  & $1.00\pm0.00$ \\
				$P_{5}$  & $0.00\pm0.00$  & $0.04\pm0.00$  & $0.95\pm0.09$  & $1.00\pm0.01$ \\
				$P_{6}$  & $0.00\pm0.00$  & $0.04\pm0.01$  & $0.99\pm0.01$  & $0.99\pm0.01$ \\
				$P_{7}$  & $0.00\pm0.00$  & $0.03\pm0.01$  & $0.80\pm0.33$  & $1.00\pm0.01$ \\
				$P_{8}$  & $0.00\pm0.00$  & $1.00\pm0.00$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{9}$  & $0.00\pm0.00$  & $0.47\pm0.10$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{10}$  & $0.00\pm0.00$  & $0.30\pm0.06$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{11}$  & $0.00\pm0.00$  & $0.17\pm0.04$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{12}$  & $0.00\pm0.00$  & $0.10\pm0.03$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{13}$  & $0.00\pm0.00$  & $0.10\pm0.02$  & $0.98\pm0.04$  & $1.00\pm0.00$ \\
				$P_{14}$  & $0.00\pm0.00$  & $0.05\pm0.02$  & $1.00\pm0.01$  & $1.00\pm0.00$ \\
				$P_{15}$  & $0.82\pm0.16$  & $1.00\pm0.00$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{16}$  & $0.00\pm0.00$  & $1.00\pm0.00$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{17}$  & $0.00\pm0.00$  & $0.93\pm0.03$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{18}$  & $0.00\pm0.00$  & $0.81\pm0.09$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{19}$  & $0.00\pm0.00$  & $0.61\pm0.06$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{20}$  & $0.00\pm0.00$  & $0.44\pm0.07$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{21}$  & $0.00\pm0.00$  & $0.25\pm0.04$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{22}$  & $1.00\pm0.00$  & $1.00\pm0.00$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{23}$  & $0.99\pm0.03$  & $1.00\pm0.00$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{24}$  & $0.72\pm0.12$  & $1.00\pm0.00$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{25}$  & $0.42\pm0.09$  & $1.00\pm0.00$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{26}$  & $0.08\pm0.03$  & $1.00\pm0.00$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{27}$  & $0.03\pm0.03$  & $0.99\pm0.02$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\
				$P_{28}$  & $0.01\pm0.02$  & $0.96\pm0.03$  & $1.00\pm0.00$  & $1.00\pm0.00$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of stored patterns, proportional to $\left|P_i\right|$}
		\end{table}

		\section{Testing Results}\label{AppE}
		\subsection{Testing Results for Pattern Data Set $P_{1}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.370\pm0.066$  & $0.375\pm0.071$  & $0.305\pm0.046$  & $0.273\pm0.042$  & $0.258\pm0.045$ \\
				$M_{3}$  & $1.000\pm0.000$  & $0.998\pm0.007$  & $0.995\pm0.009$  & $0.948\pm0.041$  & $0.855\pm0.060$ \\
				$M_{4}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.998\pm0.007$  & $0.972\pm0.026$  & $0.840\pm0.036$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{1}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{2}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.164\pm0.040$  & $0.149\pm0.042$  & $0.140\pm0.035$  & $0.128\pm0.025$  & $0.115\pm0.022$ \\
				$M_{3}$  & $0.991\pm0.014$  & $0.938\pm0.055$  & $0.752\pm0.101$  & $0.490\pm0.104$  & $0.254\pm0.064$ \\
				$M_{4}$  & $0.998\pm0.007$  & $0.966\pm0.017$  & $0.801\pm0.065$  & $0.528\pm0.049$  & $0.230\pm0.040$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{2}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{3}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.068\pm0.004$  & $0.068\pm0.002$  & $0.067\pm0.000$  & $0.067\pm0.000$  & $0.067\pm0.003$ \\
				$M_{3}$  & $0.930\pm0.053$  & $0.700\pm0.091$  & $0.383\pm0.085$  & $0.162\pm0.036$  & $0.063\pm0.022$ \\
				$M_{4}$  & $0.981\pm0.008$  & $0.833\pm0.067$  & $0.494\pm0.061$  & $0.195\pm0.049$  & $0.054\pm0.018$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{3}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{4}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.050\pm0.000$  & $0.050\pm0.000$  & $0.050\pm0.000$  & $0.050\pm0.000$  & $0.050\pm0.000$ \\
				$M_{3}$  & $0.746\pm0.288$  & $0.443\pm0.185$  & $0.179\pm0.077$  & $0.053\pm0.024$  & $0.007\pm0.006$ \\
				$M_{4}$  & $0.907\pm0.044$  & $0.586\pm0.065$  & $0.219\pm0.047$  & $0.048\pm0.020$  & $0.008\pm0.004$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{4}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{5}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.040\pm0.000$  & $0.040\pm0.000$  & $0.040\pm0.000$  & $0.040\pm0.001$  & $0.040\pm0.001$ \\
				$M_{3}$  & $0.662\pm0.103$  & $0.283\pm0.059$  & $0.075\pm0.017$  & $0.018\pm0.008$  & $0.003\pm0.003$ \\
				$M_{4}$  & $0.829\pm0.049$  & $0.378\pm0.074$  & $0.087\pm0.024$  & $0.011\pm0.011$  & $0.002\pm0.002$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{5}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{6}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.033\pm0.000$  & $0.033\pm0.002$  & $0.033\pm0.000$  & $0.032\pm0.002$  & $0.032\pm0.003$ \\
				$M_{3}$  & $0.609\pm0.064$  & $0.175\pm0.042$  & $0.042\pm0.013$  & $0.006\pm0.005$  & $0.001\pm0.002$ \\
				$M_{4}$  & $0.746\pm0.086$  & $0.248\pm0.056$  & $0.035\pm0.011$  & $0.004\pm0.003$  & $0.000\pm0.000$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{6}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{7}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.032\pm0.009$  & $0.031\pm0.006$  & $0.029\pm0.001$  & $0.029\pm0.000$  & $0.026\pm0.003$ \\
				$M_{3}$  & $0.388\pm0.162$  & $0.112\pm0.049$  & $0.020\pm0.011$  & $0.003\pm0.004$  & $0.000\pm0.000$ \\
				$M_{4}$  & $0.672\pm0.097$  & $0.185\pm0.041$  & $0.018\pm0.007$  & $0.000\pm0.001$  & $0.000\pm0.000$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{7}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{8}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.998\pm0.007$  & $0.968\pm0.042$  & $0.890\pm0.087$  & $0.843\pm0.087$  & $0.735\pm0.086$ \\
				$M_{3}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.985\pm0.017$  & $0.902\pm0.064$ \\
				$M_{4}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.995\pm0.009$  & $0.950\pm0.035$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{8}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{9}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.440\pm0.077$  & $0.395\pm0.072$  & $0.345\pm0.034$  & $0.295\pm0.036$  & $0.273\pm0.038$ \\
				$M_{3}$  & $1.000\pm0.000$  & $0.996\pm0.005$  & $0.921\pm0.033$  & $0.736\pm0.074$  & $0.471\pm0.073$ \\
				$M_{4}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.966\pm0.015$  & $0.820\pm0.039$  & $0.474\pm0.037$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{9}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{10}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.264\pm0.041$  & $0.213\pm0.043$  & $0.172\pm0.027$  & $0.132\pm0.026$  & $0.109\pm0.017$ \\
				$M_{3}$  & $0.994\pm0.006$  & $0.928\pm0.027$  & $0.709\pm0.059$  & $0.377\pm0.079$  & $0.133\pm0.042$ \\
				$M_{4}$  & $0.999\pm0.002$  & $0.978\pm0.014$  & $0.822\pm0.038$  & $0.470\pm0.076$  & $0.152\pm0.033$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{10}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{11}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.153\pm0.030$  & $0.127\pm0.028$  & $0.108\pm0.026$  & $0.089\pm0.025$  & $0.078\pm0.023$ \\
				$M_{3}$  & $0.981\pm0.013$  & $0.814\pm0.049$  & $0.473\pm0.045$  & $0.166\pm0.024$  & $0.037\pm0.009$ \\
				$M_{4}$  & $0.992\pm0.018$  & $0.918\pm0.049$  & $0.587\pm0.084$  & $0.210\pm0.043$  & $0.031\pm0.012$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{11}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{12}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.083\pm0.026$  & $0.074\pm0.023$  & $0.060\pm0.017$  & $0.049\pm0.009$  & $0.042\pm0.010$ \\
				$M_{3}$  & $0.948\pm0.029$  & $0.611\pm0.071$  & $0.232\pm0.027$  & $0.054\pm0.017$  & $0.007\pm0.004$ \\
				$M_{4}$  & $0.988\pm0.008$  & $0.820\pm0.046$  & $0.365\pm0.041$  & $0.079\pm0.017$  & $0.006\pm0.002$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{12}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{13}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.082\pm0.013$  & $0.074\pm0.008$  & $0.051\pm0.009$  & $0.041\pm0.007$  & $0.028\pm0.009$ \\
				$M_{3}$  & $0.823\pm0.063$  & $0.436\pm0.080$  & $0.096\pm0.024$  & $0.015\pm0.010$  & $0.003\pm0.003$ \\
				$M_{4}$  & $0.969\pm0.015$  & $0.650\pm0.064$  & $0.185\pm0.028$  & $0.023\pm0.010$  & $0.001\pm0.001$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{13}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{14}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.048\pm0.012$  & $0.042\pm0.011$  & $0.036\pm0.009$  & $0.028\pm0.004$  & $0.023\pm0.004$ \\
				$M_{3}$  & $0.754\pm0.053$  & $0.285\pm0.052$  & $0.051\pm0.013$  & $0.006\pm0.005$  & $0.000\pm0.001$ \\
				$M_{4}$  & $0.934\pm0.032$  & $0.479\pm0.078$  & $0.098\pm0.019$  & $0.005\pm0.003$  & $0.000\pm0.001$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{14}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{15}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.723\pm0.103$  & $0.583\pm0.053$  & $0.425\pm0.060$  & $0.272\pm0.055$  & $0.170\pm0.070$ \\
				$M_{2}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.998\pm0.007$  & $0.990\pm0.014$ \\
				$M_{3}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.998\pm0.007$  & $0.970\pm0.032$  & $0.912\pm0.077$ \\
				$M_{4}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.995\pm0.009$  & $0.960\pm0.032$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{15}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{16}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.988\pm0.029$  & $0.981\pm0.033$  & $0.970\pm0.033$  & $0.925\pm0.046$  & $0.847\pm0.033$ \\
				$M_{3}$  & $1.000\pm0.000$  & $0.994\pm0.013$  & $0.954\pm0.033$  & $0.810\pm0.108$  & $0.529\pm0.110$ \\
				$M_{4}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.993\pm0.008$  & $0.883\pm0.061$  & $0.631\pm0.105$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{16}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{17}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.912\pm0.033$  & $0.880\pm0.040$  & $0.802\pm0.047$  & $0.696\pm0.058$  & $0.537\pm0.054$ \\
				$M_{3}$  & $0.997\pm0.003$  & $0.955\pm0.024$  & $0.785\pm0.084$  & $0.466\pm0.085$  & $0.214\pm0.045$ \\
				$M_{4}$  & $1.000\pm0.000$  & $0.998\pm0.005$  & $0.943\pm0.021$  & $0.645\pm0.048$  & $0.287\pm0.028$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{17}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{18}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.748\pm0.074$  & $0.689\pm0.068$  & $0.571\pm0.057$  & $0.445\pm0.037$  & $0.292\pm0.034$ \\
				$M_{3}$  & $0.989\pm0.009$  & $0.868\pm0.067$  & $0.577\pm0.070$  & $0.251\pm0.046$  & $0.067\pm0.025$ \\
				$M_{4}$  & $0.999\pm0.002$  & $0.976\pm0.011$  & $0.790\pm0.046$  & $0.372\pm0.038$  & $0.090\pm0.020$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{18}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{19}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.518\pm0.049$  & $0.420\pm0.044$  & $0.325\pm0.030$  & $0.205\pm0.039$  & $0.112\pm0.022$ \\
				$M_{3}$  & $0.969\pm0.014$  & $0.728\pm0.036$  & $0.344\pm0.054$  & $0.099\pm0.030$  & $0.014\pm0.010$ \\
				$M_{4}$  & $0.998\pm0.003$  & $0.933\pm0.026$  & $0.607\pm0.046$  & $0.201\pm0.030$  & $0.023\pm0.013$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{19}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{20}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.326\pm0.045$  & $0.253\pm0.027$  & $0.173\pm0.029$  & $0.104\pm0.025$  & $0.048\pm0.013$ \\
				$M_{3}$  & $0.929\pm0.028$  & $0.559\pm0.063$  & $0.188\pm0.033$  & $0.041\pm0.009$  & $0.005\pm0.004$ \\
				$M_{4}$  & $0.996\pm0.006$  & $0.877\pm0.028$  & $0.420\pm0.027$  & $0.079\pm0.016$  & $0.005\pm0.003$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{20}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{21}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.204\pm0.039$  & $0.156\pm0.028$  & $0.095\pm0.020$  & $0.055\pm0.015$  & $0.023\pm0.010$ \\
				$M_{3}$  & $0.876\pm0.037$  & $0.437\pm0.060$  & $0.101\pm0.029$  & $0.011\pm0.007$  & $0.001\pm0.002$ \\
				$M_{4}$  & $0.987\pm0.008$  & $0.742\pm0.038$  & $0.258\pm0.034$  & $0.029\pm0.008$  & $0.002\pm0.002$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{21}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{22}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.988\pm0.010$ \\
				$M_{2}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.995\pm0.009$ \\
				$M_{3}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.998\pm0.007$  & $0.950\pm0.020$ \\
				$M_{4}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.970\pm0.026$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{22}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{23}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.968\pm0.045$  & $0.953\pm0.065$  & $0.926\pm0.088$  & $0.884\pm0.089$  & $0.802\pm0.129$ \\
				$M_{2}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.994\pm0.007$ \\
				$M_{3}$  & $1.000\pm0.000$  & $0.998\pm0.004$  & $0.981\pm0.013$  & $0.881\pm0.061$  & $0.635\pm0.056$ \\
				$M_{4}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.996\pm0.005$  & $0.924\pm0.044$  & $0.726\pm0.067$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{23}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{24}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.648\pm0.102$  & $0.603\pm0.122$  & $0.537\pm0.110$  & $0.439\pm0.117$  & $0.307\pm0.087$ \\
				$M_{2}$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $1.000\pm0.000$  & $0.991\pm0.007$  & $0.924\pm0.018$ \\
				$M_{3}$  & $1.000\pm0.000$  & $0.947\pm0.027$  & $0.791\pm0.062$  & $0.481\pm0.095$  & $0.220\pm0.049$ \\
				$M_{4}$  & $1.000\pm0.000$  & $0.997\pm0.005$  & $0.939\pm0.031$  & $0.708\pm0.058$  & $0.338\pm0.053$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{24}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{25}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.339\pm0.083$  & $0.268\pm0.070$  & $0.192\pm0.058$  & $0.134\pm0.035$  & $0.064\pm0.024$ \\
				$M_{2}$  & $1.000\pm0.000$  & $0.999\pm0.002$  & $0.994\pm0.008$  & $0.926\pm0.013$  & $0.761\pm0.036$ \\
				$M_{3}$  & $0.995\pm0.004$  & $0.902\pm0.053$  & $0.639\pm0.097$  & $0.297\pm0.055$  & $0.089\pm0.030$ \\
				$M_{4}$  & $0.999\pm0.002$  & $0.984\pm0.012$  & $0.839\pm0.057$  & $0.480\pm0.072$  & $0.161\pm0.027$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{25}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{26}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.042\pm0.025$  & $0.025\pm0.018$  & $0.009\pm0.008$  & $0.008\pm0.006$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.998\pm0.003$  & $0.988\pm0.011$  & $0.933\pm0.011$  & $0.748\pm0.031$  & $0.488\pm0.026$ \\
				$M_{3}$  & $0.972\pm0.025$  & $0.759\pm0.070$  & $0.389\pm0.079$  & $0.123\pm0.049$  & $0.024\pm0.010$ \\
				$M_{4}$  & $0.998\pm0.004$  & $0.952\pm0.026$  & $0.675\pm0.093$  & $0.285\pm0.063$  & $0.055\pm0.022$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{26}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{27}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.011\pm0.013$  & $0.007\pm0.010$  & $0.003\pm0.004$  & $0.001\pm0.002$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.970\pm0.020$  & $0.912\pm0.023$  & $0.772\pm0.027$  & $0.519\pm0.038$  & $0.275\pm0.040$ \\
				$M_{3}$  & $0.925\pm0.022$  & $0.611\pm0.071$  & $0.223\pm0.049$  & $0.053\pm0.015$  & $0.006\pm0.004$ \\
				$M_{4}$  & $0.999\pm0.002$  & $0.890\pm0.034$  & $0.477\pm0.056$  & $0.127\pm0.020$  & $0.014\pm0.006$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{27}$, proportional to $\left|I_i\right|$}
		\end{table}

		\subsection{Testing Results for Pattern Data Set $P_{28}$}

		% TESTING TABLE

		\begin{table}[H]
			\centering
			%\label{my-label}
			\def\arraystretch{1.5}
			%\resizebox{\textwidth}{!}{%
			\footnotesize
			\begin{tabular}{cccccc}

				& $I_{1}$  & $I_{2}$  & $I_{3}$  & $I_{4}$  & $I_{5}$ \\ \cline{2-6}
				$M_{1}$  & $0.005\pm0.007$  & $0.001\pm0.001$  & $0.001\pm0.002$  & $0.000\pm0.000$  & $0.000\pm0.000$ \\
				$M_{2}$  & $0.914\pm0.031$  & $0.798\pm0.016$  & $0.572\pm0.014$  & $0.310\pm0.032$  & $0.124\pm0.022$ \\
				$M_{3}$  & $0.874\pm0.048$  & $0.475\pm0.060$  & $0.152\pm0.023$  & $0.020\pm0.006$  & $0.003\pm0.003$ \\
				$M_{4}$  & $0.991\pm0.006$  & $0.789\pm0.052$  & $0.336\pm0.055$  & $0.050\pm0.012$  & $0.005\pm0.003$ \\

			\end{tabular}
			%\end{tabular}}
			\caption{Number of successful recalls when input data set $I_i$ is given to model $M_j$, trained with pattern data set $P_{28}$, proportional to $\left|I_i\right|$}
		\end{table}

	\section{Reproducibility}\label{AppF}
	The source code of this project is public on github\footnote{https://github.com/juanlao7/bblr-hopfield-boltzmann}. The default configured random seed produces the same exact results as the ones published in this paper.
	\subsection{Requirements}
	\begin{itemize}
		\item Python $ \geq \ 2.7.0$
		\item NumPy $ \geq \  1.12.0$
		\item UNIX-like system shell.
	\end{itemize}

	\subsection{Execution}
	Just execute the following command in the root of the project to generate data sets and test the models:

	$$\texttt{\$\ sh\ generate\_results.sh}$$

	This will create several JSON files in $ \texttt{out/results/} $. When it finishes, execute the following command:

	$$\texttt{\$\ sh\ generate\_latex.sh\ >\ out/tables.tex}$$

	This will create a TeX file in $ \texttt{out/tables.tex} $ containing the result tables used in this paper.

\end{document}


\section*{Standard Extensions}

\subsection*{RV32M Multiply Extension}
\begin{center}
\begin{tabular}
{T | l | c | T | T | T | C } \hline
\rm Inst & Name              & FMT & \rm Opcode & \rm F3 & \rm F7 & \rm Description (C)     \\ \hline
mul      & MUL               & R   & 0110011    & 0x0    & 0x01   & rd = (rs1 * rs2)[31:0]  \\
mulh     & MUL High          & R   & 0110011    & 0x1    & 0x01   & rd = (rs1 * rs2)[63:32] \\
mulsu    & MUL High (S) (U)  & R   & 0110011    & 0x2    & 0x01   & rd = (rs1 * rs2)[63:32] \\
mulu     & MUL High (U)      & R   & 0110011    & 0x3    & 0x01   & rd = (rs1 * rs2)[63:32] \\
div      & DIV               & R   & 0110011    & 0x4    & 0x01   & rd = rs1 / rs2          \\
divu     & DIV (U)           & R   & 0110011    & 0x5    & 0x01   & rd = rs1 / rs2          \\
rem      & Remainder         & R   & 0110011    & 0x6    & 0x01   & rd = rs1 \% rs2         \\
remu     & Remainder (U)     & R   & 0110011    & 0x7    & 0x01   & rd = rs1 \% rs2         \\
\hline
\end{tabular}
\end{center}

\subsection*{RV32A Atomic Extension}

\begin{itemize}
    \item \texttt{aq}: acquire access bit -- this operation must occur before later memory ops
    \item \texttt{rl}: release access bit -- this operation must occur after earlier memory ops
\end{itemize}

\begin{center}
\begin{tabular}{R@{}W@{}W@{}R@{}R@{}F@{}R@{}O}
\instbitrange{31}{27} &
\instbit{26} &
\instbit{25} &
\instbitrange{24}{20} &
\instbitrange{19}{15} &
\instbitrange{14}{12} &
\instbitrange{11}{7} &
\instbitrange{6}{0} \\
\hline
\multicolumn{1}{|c|}{funct5} &
\multicolumn{1}{c|}{aq} &
\multicolumn{1}{c|}{rl} &
\multicolumn{1}{c|}{rs2} &
\multicolumn{1}{c|}{rs1} &
\multicolumn{1}{c|}{funct3} &
\multicolumn{1}{c|}{rd} &
\multicolumn{1}{c|}{opcode} \\
\hline
5 & 1 & 1 & 5 & 5 & 3 & 5 & 7 \\
\end{tabular}

\begin{tabular}
{T | l | c | T | T | T | T } \hline
\rm Inst  & Name              & FMT   & \rm Opcode & \rm F3 & \rm F5 & \rm Description (C)         \\ \hline
lr.w      & Load Reserved     & R     & 0101111    & 0x2    & 0x02   & rd = M[rs1], reserve M[rs1] \\
sc.w      & Store Conditional & R     & 0101111    & 0x2    & 0x03   & \parbox[t]{2.5in}{ if (reserved) \{ M[rs1] = rs2; rd = 0 \} \\
                                                                        else \{ rd = 1 \}}   \\
amoswap.w & Atomic Swap       & R     & 0101111    & 0x2    & 0x01   & rd = M[rs1]; swap(rd, rs2); M[rs1] = rd \\
amoadd.w  & Atomic ADD        & R     & 0101111    & 0x2    & 0x00   & rd = M[rs1] + rs2; M[rs1] = rd \\
amoand.w  & Atomic AND        & R     & 0101111    & 0x2    & 0x0C   & rd = M[rs1] \& rs2; M[rs1] = rd \\
amoor.w   & Atomic OR         & R     & 0101111    & 0x2    & 0x0A   & rd = M[rs1] | rs2; M[rs1] = rd \\
amoxor.w  & Atomix XOR        & R     & 0101111    & 0x2    & 0x04   & rd = M[rs1] \^{} rs2; M[rs1] = rd \\
amomax.w  & Atomic MAX        & R     & 0101111    & 0x2    & 0x14   & rd = max(M[rs1], rs2); M[rs1] = rd \\
amomin.w  & Atomic MIN        & R     & 0101111    & 0x2    & 0x10   & rd = min(M[rs1], rs2); M[rs1] = rd \\
\hline
\end{tabular}
\end{center}

\subsection*{RV32F / D Floating-Point Extensions}

\begin{center}
\begin{tabular}
{T | l | c | T | T | T | T } \hline
\rm Inst  & Name                  & FMT   & \rm Opcode & \rm F3 & \rm F5 & \rm Description (C)         \\ \hline
flw       & Flt Load Word         & *     &            &        &        & rd = M[rs1] + imm    \\
fsw       & Flt Store Word        & *     &            &        &        & M[rs1 + imm] = rs2   \\
fmadd.s   & Flt Fused Mul-Add     & *     &            &        &        & rd = rs1 * rs2 + rs3 \\
fmsub.s   & Flt Fused Mul-Sub     & *     &            &        &        & rd = rs1 * rs2 - rs3 \\
fnmadd.s  & Flt Neg Fused Mul-Add & *     &            &        &        & rd = -rs1 * rs2 + rs3 \\
fnmsub.s  & Flt Neg Fused Mul-Sub & *     &            &        &        & rd = -rs1 * rs2 - rs3 \\
fadd.s    & Flt Add               & *     &            &        &        & rd = rs1 + rs2       \\
fsub.s    & Flt Sub               & *     &            &        &        & rd = rs1 - rs2       \\
fmul.s    & Flt Mul               & *     &            &        &        & rd = rs1 * rs2       \\
fdiv.s    & Flt Div               & *     &            &        &        & rd = rs1 / rs2       \\
fsqrt.s   & Flt Square Root       & *     &            &        &        & rd = sqrt(rs1)       \\
fsgnj.s   & Flt Sign Injection    & *     &            &        &        & rd = abs(rs1) * sgn(rs2) \\
fsgnjn.s  & Flt Sign Neg Injection& *     &            &        &        & rd = abs(rs1) * -sgn(rs2) \\
fsgnjx.s  & Flt Sign Xor Injection& *     &            &        &        & rd = rs1 * sgn(rs2)  \\
fmin.s    & Flt Minimum           & *     &            &        &        & rd = min(rs1, rs2)   \\
fmax.s    & Flt Maximum           & *     &            &        &        & rd = max(rs1, rs2)   \\
fcvt.s.w  & Flt Conv from Sign Int& *     &            &        &        & rd = (float) rs1 \\
fcvt.s.wu & Flt Conv from Uns Int & *     &            &        &        & rd = (float) rs1 \\
fcvt.w.s  & Flt Convert to Int    & *     &            &        &        & rd = (int32\_t) rs1 \\
fcvt.wu.s & Flt Convert to Int    & *     &            &        &        & rd = (uint32\_t) rs1 \\
fmv.x.w   & Move Float to Int     & *     &            &        &        & rd = *((int*) \&rs1) \\
fmv.w.x   & Move Int to Float     & *     &            &        &        & rd = *((float*) \&rs1) \\
feq.s     & Float Equality        & *     &            &        &        & rd = (rs1 == rs2) ? 1 : 0 \\
flt.s     & Float Less Than       & *     &            &        &        & rd = (rs1 <  rs2) ? 1 : 0 \\
fle.s     & Float Less / Equal    & *     &            &        &        & rd = (rs1 <= rs2) ? 1 : 0\\
fclass.s  & Float Classify        & *     &            &        &        & rd = 0..9 \\
\hline
\end{tabular}
\end{center}

\subsection*{RV32C Compressed Extension}

\small
\begin{center}
\begin{tabular}
{p{0in}p{0.05in}p{0.05in}p{0.05in}p{0.05in}p{0.05in}p{0.05in}p{0.05in}p{0.05in}p{0.05in}p{0.05in}p{0.05in}p{0.05in}p{0.05in}p{0.05in}p{0.05in}p{0.05in}l}
%&
%\multicolumn{1}{l}{\instbit{31}} &
%\multicolumn{1}{r}{\instbit{27}} &
%\instbit{26} &
%\instbit{25} &
%\multicolumn{1}{l}{\instbit{24}} &
%\multicolumn{1}{r}{\instbit{20}} &
%\instbitrange{19}{15} &
%\instbitrange{14}{12} &
%\instbitrange{11}{7} &
%\instbitrange{6}{0} \\
%\cline{2-11}

&
\instbit{15} &
\instbit{14} &
\instbit{13} &
\multicolumn{1}{c}{\instbit{12}} &
\instbit{11} &
\instbit{10} &
\instbit{9} &
\instbit{8} &
\instbit{7} &
\instbit{6} &
\instbit{5} &
\instbit{4} &
\instbit{3} &
\instbit{2} &
\instbit{1} &
\instbit{0} \\
\cline{2-17}

&
\multicolumn{4}{|c|}{funct4} &
\multicolumn{5}{c|}{rd/rs1} &
\multicolumn{5}{c|}{rs2} &
\multicolumn{2}{c|}{op} &
CR-type \\
\cline{2-17}

&
\multicolumn{3}{|c|}{funct3} &
\multicolumn{1}{c|}{imm} &
\multicolumn{5}{c|}{rd/rs1} &
\multicolumn{5}{c|}{imm} &
\multicolumn{2}{c|}{op} &
CI-type \\
\cline{2-17}

&
\multicolumn{3}{|c|}{funct3} &
\multicolumn{6}{c|}{imm} &
\multicolumn{5}{c|}{rs2} &
\multicolumn{2}{c|}{op} &
CSS-type \\
\cline{2-17}

&
\multicolumn{3}{|c|}{funct3} &
\multicolumn{8}{c|}{imm} &
\multicolumn{3}{c|}{rd'} &
\multicolumn{2}{c|}{op} &
CIW-type \\
\cline{2-17}

&
\multicolumn{3}{|c|}{funct3} &
\multicolumn{3}{c|}{imm} &
\multicolumn{3}{c|}{rs1'} &
\multicolumn{2}{c|}{imm} &
\multicolumn{3}{c|}{rd'} &
\multicolumn{2}{c|}{op} &
CL-type \\
\cline{2-17}

&
\multicolumn{3}{|c|}{funct3} &
\multicolumn{3}{c|}{imm} &
\multicolumn{3}{c|}{rd'/rs1'} &
\multicolumn{2}{c|}{imm} &
\multicolumn{3}{c|}{rs2'} &
\multicolumn{2}{c|}{op} &
CS-type \\
\cline{2-17}

&
\multicolumn{3}{|c|}{funct3} &
\multicolumn{3}{c|}{imm} &
\multicolumn{3}{c|}{rs1'} &
\multicolumn{5}{c|}{imm} &
\multicolumn{2}{c|}{op} &
CB-type \\
\cline{2-17}

&
\multicolumn{3}{|c|}{funct3} &
\multicolumn{11}{c|}{offset} &
\multicolumn{2}{c|}{op} &
CJ-type \\
\cline{2-17}

\end{tabular}
\end{center}

\begin{center}
\begin{tabular}
{T | l | c | T | T | T } \hline
\rm Inst   & Name                    & FMT & \rm OP & \rm Funct & \rm Description          \\ \hline
c.lwsp     & Load Word from SP       & CI  & 10     & 010       & lw rd, (4*imm)(sp)       \\
%c.ldsp     & Load Double from SP     & CI  & 10     & 011       & ld rd, (8*imm)(sp)       \\
%c.lqsp     & Load Quad from SP       & CI  & 10     & 001       & lq rd, (16*imm)(sp)      \\
%c.flwsp    & Flt Load Word from SP   & CI  & 10     & 011       & flw rd, (4*imm)(sp)      \\
%c.fldsp    & Flt Load Double from SP & CI  & 10     & 001       & flw rd, (8*imm)(sp)      \\
c.swsp     & Store Word to SP        & CSS & 10     & 110       & sw rs2, (4*imm)(sp)      \\
%c.sdsp     & Store Double to SP      & CSS & 10     & 111       & sd rs2, (8*imm)(sp)      \\
%c.sqsp     & Store Quad to SP        & CSS & 10     & 101       & sq rs2, (16*imm)(sp)     \\
%c.fswsp    & Flt Store Word to SP    & CSS & 10     & 111       & fsw rs2, (4*imm)(sp)     \\
%c.fsdsp    & Flt Store Double to SP  & CSS & 10     & 101       & fsd rs2, (8*imm)(sp)     \\
c.lw       & Load Word               & CL  & 00     & 010       & lw rd', (4*imm)(rs1')    \\
%c.ld       & Load Double             & CL  & 00     & 011       & ld rd', (8*imm)(rs1')    \\
%c.lq       & Load Quad               & CL  & 00     & 001       & lq rd', (16*imm)(rs1')   \\
%c.flw      & Flt Load Word           & CL  & 00     & 011       & flw rd', (4*imm)(rs1')   \\
%c.fld      & Flt Load Double         & CL  & 00     & 001       & flw rd', (8*imm)(rs1')   \\
c.sw       & Store Word              & CS  & 00     & 110       & sw rs1', (4*imm)(rs2')    \\
%c.sd       & Store Double            & CS  & 00     & 111       & sd rs1', (8*imm)(rs2')   \\
%c.sq       & Store Quad              & CS  & 00     & 101       & sq rs1', (16*imm)(rs2')  \\
%c.fsw      & Flt Store Word          & CS  & 00     & 111       & fsw rs1', (4*imm)(rs2')  \\
%c.fsd      & Flt Store Double        & CS  & 00     & 101       & fsd rs1', (8*imm)(rs2')  \\
c.j        & Jump                    & CJ  & 01     & 101       & jal x0, 2*offset         \\
c.jal      & Jump And Link           & CJ  & 01     & 001       & jal ra, 2*offset         \\
c.jr       & Jump Reg                & CR  & 10     & 1000      & jalr x0, rs1, 0          \\
c.jalr     & Jump And Link Reg       & CR  & 10     & 1001      & jalr ra, rs1, 0          \\
c.beqz     & Branch == 0             & CB  & 01     & 110       & beq rs', x0, 2*imm       \\
c.bnez     & Branch != 0             & CB  & 01     & 111       & bne rs', x0, 2*imm       \\
c.li       & Load Immediate          & CI  & 01     & 010       & addi rd, x0, imm         \\
c.lui      & Load Upper Imm          & CI  & 01     & 011       & lui rd, imm              \\
c.addi     & ADD Immediate           & CI  & 01     & 000       & addi rd, rd, imm         \\
%c.addiw    & ADD Immediate Word      & CI  & 01     & 001       & addiw rd, rd, imm        \\
c.addi16sp & ADD Imm * 16 to SP      & CI  & 01     & 011       & addi sp, sp, 16*imm      \\
c.addi4spn & ADD Imm * 4 + SP        & CIW & 00     & 000       & addi rd', sp, 4*imm      \\
c.slli     & Shift Left Logical Imm  & CI  & 10     & 000       & slli rd, rd, imm         \\
c.srli     & Shift Right Logical Imm & CB  & 01     & 100x00    & srli rd', rd', imm       \\
c.srai     & Shift Right Arith Imm   & CB  & 01     & 100x01    & srai rd', rd', imm       \\
c.andi     & AND Imm                 & CB  & 01     & 100x10    & andi rd', rd', imm       \\
c.mv       & MoVe                    & CR  & 10     & 1000      & add rd, x0, rs2          \\
c.add      & ADD                     & CR  & 10     & 1001      & add rd, rd, rs2          \\
c.and      & AND                     & CS  & 01     & 10001111  & and rd', rd', rs2'       \\
c.or       & OR                      & CS  & 01     & 10001110  & or rd', rd', rs2'        \\
c.xor      & XOR                     & CS  & 01     & 10001101  & xor rd', rd', rs2'       \\
c.sub      & SUB                     & CS  & 01     & 10001100  & sub rd', rd', rs2'       \\
%c.addw     & ADD Word                & CS  & 01     & 10011101  & addw rd', rd', rs2'      \\
%c.subw     & SUB Word                & CS  & 01     & 10011100  & subw rd', rd', rs2'      \\
c.nop      & No OPeration            & CI  & 01     & 000       & addi x0, x0, 0           \\
c.ebreak   & Environment BREAK       & CR  & 10     & 1001      & ebreak                   \\
\hline
\end{tabular}
\end{center}


\chapter*{Preface}
This monograph is devoted to mathematical descriptions of deep neural
networks.

This set of notes are based on three courses that I have taught at Penn
State:
\begin{enumerate}
\item Introduction to Deep Learning, Fall 2017 (?)
\item Deep Learning, Spring 2019
\item Introduction to Deep Learning, Summer 2019
\end{enumerate}
These courses are mainly accessible to undergraduate students from
different colleges and department.

The audience of this notes are those who are knowledgeable for
\begin{enumerate}
\item Multivariable calculus
\item Linear algebra
\end{enumerate}

who have
taken course on multivariable


% language=uk

\startcomponent mk-calcmath

\environment mk-environment

\chapter{An example: CalcMath}

\usemodule[calcmath]

\subject{introduction}

For a long time \TEX's way of coding math has dominated the
typesetting world. However, this kind of coding is not that well
suited for non academics, like schoolkids. Often kids do know how
to key in math because they use advanced calculators. So, when a
couple of years ago we were implementing a workflow where kids
could fill in their math workbooks (with exercises) on||line, it
made sense to support so called Texas Instruments math input.
Because we had to parse the form data anyway, we could use a \type
{[[} and \type {]]} as math delimiters instead of \type {$}. The
conversion too place right after the form was received by the web
server.

\start

\def\test#1%
  {\NC \vrule height 2ex depth 2ex width 0pt\type{#1}
   \NC \calcmath{#1}
   \NC \NR}

\setupinterlinespace[1.5]

\starttabulate
\test{sin(x) + x^2 + x^(1+x) + 1/x^2}
\test{mean(x+mean(y))}
\test{int(a,b,c)}
\test{(1+x)/(1+x) + (1+x)/(1+(1+x)/(1+x))}
\test{10E-2}
\test{(1+x)/x}
\test{(1+x)/12}
\test{(1+x)/-12}
\test{1/-12}
\test{12x/(1+x)}
\test{exp(x+exp(x+1))}
\test{abs(x+abs(x+1)) + pi + inf}
\test{Dx Dy}
\test{D(x+D(y))}
\test{Df(x)}
\test{g(x)}
\test{sqrt(sin^2(x)+cos^2(x))}
\stoptabulate

\stop

By combining \LUA\ with \TEX, we can do the conversion from
calculator math to \TEX\ immediately, without auxiliary programs
or complex parsing using \TEX\ macros.

\subject{tex}

In a \CONTEXT\ source one can use the \type {\calcmath} command, as
in:

\starttyping
The strange formula \calcmath {sqrt(sin^2(x)+cos^2(x))} boils
down to ...
\stoptyping

One needs to load the module first, using:

\starttyping
\usemodule[calcmath]
\stoptyping

Because the amount of code involved is rather small, eventually we
may decide to add this support to the \MKIV\ kernel.

\subject{xml}

Coding math in \TEX\ is rather efficient. In \XML\ one needs way
more code. Presentation \MATHML\ provides a few basic constructs
and boils down to combining those building blocks. Content \MATHML\
is better, especially from the perspective of applications that
need to do interpret the formulas. It permits for instance the
\CONTEXT\ content \MATHML\ handler to adapt the rendering to
cultural driven needs. The \OPENMATH\ way of coding is like
content \MATHML, but more verbose with less tags. Calculator math
is more restrictive than \TEX\ math and less verbose than any of
the \XML\ variants. It looks like:

\starttyping
<icm>sqrt(sin^2(x)+cos^2(x))</icm> test
\stoptyping

And in display mode:

\starttyping
<dcm>sqrt(sin^2(x)+cos^2(x))</dcm> test
\stoptyping

\subject{speed}

This script (which you can find in the \CONTEXT\ distribution as
soon as the \MKIV\ code variants are added) is the first real
\TEX\ related \LUA\ code that I wrote; so far I had only written
some wrapping and spell checking code for the \SCITE\ editor. It
also made a nice demo for a couple of talks that I held at
usergroup meetings. The script has a lot of expressions. These
convert one string into another. They are less powerful than
regular expressions, but pretty fast and adequate. The feature I
miss most is alternation like \type {(l|st)uck} but it's a small
price to pay. As the \LUA\ manual explains: adding a \POSIX\
compliant regexp parser would take more lines of code than \LUA\
currently does.

On my machine, running this first version took 3.5 seconds for 2500
times typesetting the previously shown square root of sine and
cosine. Of this, 2.1 seconds were spent on typesetting and 1.4 seconds
on converting. After optimizing the code, 0.8 seconds were used for
conversion. A stand alone \LUA\ takes .65 seconds, which includes loading
the interpreter. On a test of 25.000 sample conversions, we could gain
some 20\% conversion time using the \LUAJIT\ just in time compiler.

% \usemodule[narrowtt] \typefile[ntyping]{m-calcmath.lua} % narrow, no color

% \def\inlinecalcmath    #1{\mathematics{\ctxlua{calcmath.tex("#1",1)}}}
% \def\displaycalcmath   #1{\startformula\ctxlua{calcmath.tex("#1",2)}\stopformula}

\stopcomponent