\vssub
\subsection{~Air-sea processes}
\vsssub
\subsubsection{~General concepts}
\vsssub

Additional subroutines are provided within \ws\ for use as part of coupled ocean-wave or ocean-atmosphere systems.
These subroutines are designed to compute additional quantities related to the surface wave field which are intended to be passed to external models (e.g. ocean models).
The motivation for these subroutines is to allow the external model to include the impact of waves on quantities such as the wind stress and the upper ocean turbulence.

\paragraph{Sea-state dependent air-sea fluxes}

The air-sea momentum flux, or the total wind stress, is the sum of the momentum flux into both surface waves and subsurface currents.
Coupled atmosphere-ocean models that do not consider the impact of the surface gravity wave field typically compute the total wind stress based on an empirical relationship between the wind speed and the wind stress (via a drag coefficient, $C_d$).
The provided {\it FLD} subroutines allows the computation of the total wind stress based on the \ws\ wavenumber-direction spectrum for use in coupled numerical models.

To the leading order, the total wind stress is equal to the sum of the momentum flux into surface waves (form drag of surface waves) and the momentum flux directly into the subsurface currents (through viscous stress).
The momentum flux into the waves may be expressed as an integral of the wave variance spectrum multiplied by the wave growth rate (momentum-uptake rate).
A few assumptions are needed to calculate the wave form drag.
First, the wave form drag is sensitive at the leading order to the level of the high frequency waves (or the spectral tail).
This part of the wave spectrum contains a great deal of uncertainty within the wave model, and therefore may need to be separately parameterized for computing the wind stress.
An assumption must therefore be made to parameterize the high frequency, which is not constrained by observational data and wind speeds above 15 m/s.
Second, assumptions of the wave growth-rate function are needed since it has historically been parameterized from either the wind speed or the wind stress.
In either case, empirical coefficients are needed within the growth-rate function based on wavelength and wave direction relative to the wind and/or stress.
Third, there is feedback due to the wave form drag on the turbulence profile and the wind profile within the wave boundary layer (roughly the upper 10 meters above the air-sea interface).
How important this feedback is on determining the wind stress and the mean wind profile is not entirely understood.
Finally, the growth rate is known to be different over breaking and non-breaking waves.
However, there are no simple methods for explicitly including the breaking wave impact within wind-stress calculation models.
Therefore, no separation is made in either of the present {\it FLD} subroutines between breaking and non-breaking wave growth-rates.

The total air-sea momentum flux can be expressed (to the leading order) as:
\begin{equation}
\vec{\tau}=\vec{\tau}_{\nu}+\vec{\tau}_{f},
\end{equation}
where $\vec{\tau}_{\nu}$ is the viscous stress vector and $\vec{\tau}_{f}$ is the wave form drag.
At the air-sea interface, the wave form drag can be computed as the contribution of the momentum flux into all waves:
\begin{equation}\label{formdrag}
\vec{\tau}_{f}=\rho_w\int_{k_{min}}^{k_{max}}\int_{-\pi}^{\pi} \beta_g(k,\theta) \sigma F (k,\theta) d\theta \vec{k} dk,
\end{equation}
where $\rho_w$ is the water density, $k$ is the wavenumber, $\theta$ is the wave direction, $\sigma$ is the angular frequency, $\beta_g (k,\theta)$ is the growth rate,
$F (k,\theta)$ is the wave variance spectrum, and $k_{min}$ and $k_{max}$ are the minimum and maximum wavenumbers of contributing waves.
The expression for the growth rate varies based on the theory applied in the model, and will be described separately for each theory in their following descriptions.

The spectral tail at wind speeds above 15 m/s is not well constrained observationally or theoretically.
Therefore, the spectral tail level in the {\it FLD} subroutines has been empirically parameterized such that the mean drag coefficient corresponds to the standard bulk drag coefficient used within the modeling system.
In this way, the mean value of the wind stress will not be modified by using any explicit sea state dependent wind stress formulation, but the stress will deviate from the mean based on the sea-state.
It is assumed that the tail level is a function of a wind speed only and is independent of sea states.


\documentclass[12pt]{IEEEtran}%
\usepackage{amsmath}
\usepackage{cite}
\usepackage{graphicx}
\usepackage{graphics}
%\usepackage{geometry}
\usepackage{setspace}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{hyperref}
\usepackage{comment}
\usepackage{algorithm}
\usepackage{algpseudocode}%
\setcounter{MaxMatrixCols}{30}
%TCIDATA{OutputFilter=latex2.dll}
%TCIDATA{Version=5.50.0.2953}
%TCIDATA{CSTFile=IEEEtran.cst}
%TCIDATA{Created=Monday, February 05, 2001 16:11:19}
%TCIDATA{LastRevised=Saturday, September 29, 2018 22:28:54}
%TCIDATA{<META NAME="GraphicsSave" CONTENT="32">}
%TCIDATA{<META NAME="SaveForMode" CONTENT="1">}
%TCIDATA{BibliographyScheme=BibTeX}
%TCIDATA{Language=American English}
%BeginMSIPreambleData
\providecommand{\U}[1]{\protect\rule{.1in}{.1in}}
%EndMSIPreambleData
\hypersetup{
    colorlinks=true,
    linkcolor=black,
    filecolor=magenta,
    urlcolor=black,
    citecolor=black
}
%\geometry{left=1in,right=1in,top=1in,bottom=1in}
\begin{document}
%
%TCIMACRO{\QSubDoc{Include IEEEHeading}{\input{IEEEHeading.tex}}}%
%BeginExpansion
\input{IEEEHeading.tex}
%EndExpansion

\section{Introduction}

\label{sec:rul_intro}

Traditionally, maintenance of mechanical systems has been carried out based on
scheduling strategies. Such strategies are often costly and less capable of
meeting the increasing demand of efficiency and reliability
\cite{Gebraeel2005, Zaidan2013}. Condition based maintenance (CBM) also known
as intelligent prognostics and health management (PHM) allows for maintenance
based on the current health of the system, thus cutting down the costs and increasing
the reliability of the system \cite{Zhao2017}. Here, we refer to prognostics
as the estimation of remaining useful life of a system. The remaining useful life (RUL) of
the system can be estimated based on the historical data. This data-driven
approach can help optimize maintenance schedules to avoid engineering failures
and to save the costs \cite{Lee2014}.

The existing PHM methods can be grouped into three different categories:
model-based \cite{Yu2001}, data-driven \cite{Liu2009, Mosallam2013} and
hybrid approaches \cite{Pecht2010, Liu2012}. Model-based approaches attempt to
incorporate physical models of the system into the estimation of the RUL. If
the system degradation is modeled precisely, model-based approaches usually
exhibit better performance than data-driven approaches \cite{Qian2017}. This
comes at the expense of having extensive a priori knowledge of the underlying
system and having a fine-grained model of the system, which can involve
expensive computations. On the other hand, data-driven approaches use pattern
recognition to detect changes in system states. Data-driven approaches are
appropriate when the understanding of the first principles of the system
dynamics is not comprehensive or when the system is sufficiently complex such as
jet engines, car engines and complex machineries, for which it is prohibitively difficult
to develop an accurate model.

Common disadvantages for the data-driven approaches are that they usually
exhibit wider confidence intervals than model-based approaches and that a fair
amount of data is required for training. Many data-driven algorithms have been
proposed. Good prognostics results have been achieved. Among the most popular
algorithms we can find artificial neural networks (ANNs) \cite{Gebraeel2004},
support vector machine (SVM) \cite{Benkedjouh2013}, Markov hidden chains (MHC)
\cite{Dong2007} and so on. Over the past few years, data-driven approaches
have gained more attention in the PHM community. A number of machine learning
techniques, especially neural networks, have been applied successfully to
estimate the RUL of diverse mechanical systems. ANNs have demonstrated good
performance in modeling highly nonlinear, complex, multi-dimensional systems
without any prior knowledge on the system behavior \cite{Li2018}. While the
confidence limits for the RUL predictions cannot be analytically provided
\cite{Sikorska2011}, the neural network approaches are promising for
prognostic problems.

Neural networks for estimating the RUL of jet engines have been previously
explored in \cite{Lim2016} where the authors propose a multi-layer perceptron
(MLP) coupled with a feature extraction (FE) method and a time window for the
generation of the features for the MLP. In the publication, the authors
demonstrate that a moving window combined with a suitable feature extractor
can improve the RUL prediction as compared with the studies with other similar
methods in the literature. In \cite{Li2018}, the authors explore a deep
learning ANN architecture, the so-called convolutional neural networks (CNNs),
where they demonstrate that by using a CNN without any pooling layers coupled
with a time window, the predicted RUL is further improved.

In this paper we propose a novel framework for estimating the RUL of complex
mechanical systems. The framework consists of a MLP to estimate the RUL of the
system, coupled with an evolutionary algorithm for the fine tuning of
data-related parameters, i.e. parameters that define the shape and quality of
the features used by the MLP. The publicly available NASA CMAPS dataset
\cite{CMAPS2008} is used to assess the efficiency and reliability of the
proposed framework. This approach allows for a simple and small MLP to obtain
better results than those reported in the current literature while using less
computing power.

The remainder of this paper is organized as follows. The CMAPS dataset is
presented in Section \ref{sec:rul_dataset}. The framework and its components
are thoroughly reviewed in Section \ref{sec:method}. The method is evaluated
using the CMAPS dataset in Section \ref{sec:rul_eval}. A comparison with the
state-of-the-art is also provided. Finally, the conclusions are presented in
Section \ref{sec:conclusions}.

\section{NASA C-MAPSS Dataset}

\label{sec:rul_dataset}

The NASA CMAPS dataset is used to evaluate performance of the proposed method
\cite{CMAPS2008}. The CMAPS dataset contains simulated data produced using a
model based simulation program developed by NASA. The dataset is further
divided into 4 subsets composed of multi-variate temporal data obtained from
21 sensors.

For each of the 4 subsets, a training and a test set are provided. The training
sets include run-to-failure sensor records of multiple aero-engines collected
under different operational conditions and fault modes as described in Table
\ref{TabCMAPSS}.

The data is arranged in an $n\times26$ matrix where $n$ is the number of data
points in each subset. The first two variables represent the engine and cycle
numbers, respectively. The following three variables are operational settings
which correspond to the conditions in Table \ref{TabCMAPSS} and have a
substantial effect on the engine performance. The remaining variables
represent the 21 sensor readings that contain the information about the engine
degradation over time.

Each trajectory within the training and test sets represents the life cycles
of the engine. Each engine is simulated with different initial health
conditions, i.e. no initial faults. For each trajectory of an engine the last
data entry corresponds to the cycle at which the engine is found faulty. On
the other hand, the trajectories of the test sets terminate at some point
prior to failure, hence the need to predict the remaining useful life. The aim
of the MLP NN model is to predict the RUL of each engine in the test set. The
actual RUL values of test trajectories are also included in the dataset for
verification. Further discussions of the dataset and details on how the data
is generated can be found in \cite{Saxena2008}.

\subsection{Performance Metrics}

\label{sec:rul_metrics}

To evaluate the performance of the proposed approach on the CMAPS dataset, we
make use of two scoring indicators, namely the Root Mean Squared Error (RMSE)
denoted as $e_{rms}(d)$ and a score proposed in \cite{Saxena2008} which we
refer as the RUL Health Score (RHS) denoted as $s_{rh}(d)$. The two scores are
defined as follows,
\begin{equation}
e_{rms} = \sqrt{ \frac{1}{N} \sum_{i=1}^{N}{d_{i}^{2}}} \label{eq:rmse}%
\end{equation}
%

\begin{align}
s_{rh}  &  = \frac{1}{N} \sum_{i=1}^{N}{s_{i}}\nonumber\\
s_{i}  &  =
\begin{cases}
e^{-\frac{d_{i}}{13}} - 1, & d_{i} < 0\\
e^{\frac{d_{i}}{10}} - 1, & d_{i} \geq0,
\end{cases}
\label{eq:rhs}%
\end{align}
where $N$ is the total number of samples in the test set and $d = \hat{y} - y$
is the error between the estimated RUL values $\hat{y}$, and the actual RUL
values $y$ for each engine within the test set. It is important to note that
$s_{rh}(d)$ penalizes late predictions more than early predictions since
usually late predictions lead to more severe consequences in fields such as aerospace.

\section{Framework Description}

\label{sec:method}

In this section, the proposed ANN-EA based method for prognostics is
presented. The model consists of a multi-layer perceptron (MLP) as the main
regressor for estimating the RUL of the engines in the CMAPS dataset. For the
training sets, the feature vectors are generated by using a moving time window
while a label vector is generated with the RUL of the engine. The label has a
constant RUL for the early cycles of the simulation, and becomes a linearly
decreasing function of the cycle in the remaining cycles. This is the
so-called piecewise linear degradation model \cite{Ramasso2014}. For the test
set, a time window is taken from the last sensor readings of the engine. The
data of the test set is used to predict the RUL of the engine.

The window-size $n_{w}$, window-stride $n_{s}$, and early-RUL $R_{e}$ are
data-related parameters, which for the sake of clarity and formalism in this
study, form a vector $v \in\mathbb{Z}^{3}$ such that $v = (n_{w}, n_{s},
R_{e})$. The vector $v$ has a considerable impact on the quality of the
predictions by the regressor. It is computationally intensive to find the best
parameters of $v$ given the search space inherent to these parameters. In this
paper, we propose an evolutionary algorithm to optimize the data-related
parameters $v$. The optimized parameter set $v$ allows the use of a simple
neural network architecture while attaining better results in terms of the
quality of the predictions compared with the results by other methods in the literature.

\subsection{The Network Architecture}

After careful examinations of the CMAPS dataset, we propose to use a rather
simple MLP architecture for all the four subsets of the data. The
implementations are done in Python using the Keras/Tensorflow environment. The
source code is publicly available at the git repository \url{https://github.com/dlaredo/NASA_RUL_-CMAPS-}.

The choice of the network architecture is made by following an iterative
process: comparing 6 different architectures, training each for $100$
iterations using a mini-batch size of $512$ and averaging their results over
$10$ different runs. Also, L1 (Lasso) and L2 regularization (Ridge)
\cite{Buhlmann2011} are used to prevent overfitting. L1 regularization
penalizes the sum of the absolute value of the weights and biases of the networks,
while L2 regularization penalizes the sum of the squared value of the weights and biases.
Two objectives are pursued during the iterations: 1) the architecture must be minimal in
terms of layers and neurons in each layer; and 2) the performance indicators must be minimized.

The process for choosing the network architecture is as follows. First, choose
a $v$ for the experiment, say $v= (30, 1, 140)$. Next, six different ANN
architectures are defined. These architectures are provided in
the appendix. For each of the six different architectures,
its performance is assessed using a cross-validation set from subset 1 of
CMAPS. Table \ref{table:tested_architectures_100} summarizes the results for
each tested architecture, while Table \ref{table:proposed_nn} presents the
architecture chosen for the remainder of this work. The chosen architecture
provides the best compromise between compactness and performance among the
tested architectures.

\subsection{Shaping the Data}

This section covers the data preprocessing applied to the raw sensor readings
in each of the datasets. Although the original datasets contain $21$ different
sensor readings, some of the sensors do not present much variance or convey
redundant information. These sensors are therefore discarded. In the end, only
$14$ sensor readings out of the $21$ are considered for this study. Their
indices are $\left\lbrace 2, 3, 4, 7, 8, 9, 11, 12, 13, 14, 15, 17, 20, 21
\right\rbrace $. The raw measurements are then used to create the strided time
windows with window size $n_{w}$ and window stride $n_{s}$. For the training
labels, $R_{e}$ is used at the early stages and then the RUL is linearly
decreased. The data is normalized to be within the range $\left[  -1,1
\right]  $ using the min-max normalization.
\begin{equation}
\hat{x}_{i} = 2* \frac{x_{i} - min(x_{i})}{max(x_{i}) - min(x_{i})} - 1,
\label{eq:min_max_norm}%
\end{equation}
where $x_{i}$ denotes the $m$-dimensional vector whose components are all the
readings for the \textit{i-th} sensor and $\hat{x}_{i}$ is the normalized
$x_{i}$ vector.

\subsubsection{Time Window and Stride}

In multivariate time-series problems such as RUL, more information can be
generally obtained from the temporal sequence of the data as compared with the
multivariate data point at a single time stamp. For a time window of size
$n_{w}$ with a stride $n_{s}=1$, all the sensor readings in the time window
form a feature vector $\mathbf{x}$. This approach has successfully been tested
in \cite{Li2018,Lim2016} where the authors propose the use of a moving window
with sizes ranging from 20 to 30. In this paper, we propose not only the use of
a moving time window, but also a \textit{strided} time window that updates
more than one elements ($n_{s}>1$) at the time. A graphical depiction of the
strided time window is shown in Figure \ref{FigWindow}.

The use of a strided time window allows for the regressor to take advantage
not only of the previous information, but also to control the ratio at which
the algorithm is fed with new information. With the usual time window
approach, only one point is updated for every new time window. The strided
time window considered in this study allows for updating more than one point
at the time for the algorithm to make use of the new information with less iterations. It
is believed that the information contained in the time window with stride size
$n_{s}>1$ is likely richer than the one contained in a time window with stride
size $n_{s}=1$.

\subsubsection{Piecewise Linear Degradation Model}

Different from common regression problems, the desired output value of the
input data is difficult to determine for a RUL problem. It is usually
impossible to evaluate the precise health condition and estimate the RUL of
the system at each time step without an accurate physics based model. For this
popular dataset, a piece-wise linear degradation model has been proposed in
\cite{Ramasso2014}. The model assumes that the engines have a constant RUL
label in the early cycles, and then the RUL starts degrading linearly until it
reaches 0 as shown in Figure \ref{FigRULinear}. The piecewise linear
degradation assumption is used in this work. We denote the value of the RUL in
the early cycles as $R_{e}$. Initially,  $R_{e}$ is randomly chosen between 95 and 140
cycles. When the difference between the cycle count in the time window and the terminating cycle
of the training data is less than the initial value of $R_{e}$, $R_{e}$ begins the linear
descent toward the terminating cycle.

\subsection{Optimal Data Parameters}

\label{sec:otimal_data_params}

As mentioned in the previous sections the choice of the data-related
parameters $v$ has a large impact on the performance of the regressor. In this
section, we present the framework for picking the optimal combination of the
data-related parameters $n_{w}$, $n_{s}$ and $R_{e}$ while being
computationally efficient.

Recall that $v = (n_{w}, n_{s}, R_{e})$ specific to the CMAPS dataset are
bounded such that $n_{w} \in\left[  1, b\right]  $, $n_{s} \in\left[  1,
10\right]  $, and $R_{e} \in\left[  90, 140 \right]  $, where all the
variables are integer. The value of $b$ is different for different subsets of
the data, Table \ref{table:b_values} shows the different values of $b$ for
each subset.

Let $X(v)$ be the training/cross-validate/test sets parametrized by $v$ and
used by the MLP to perform the RUL estimation. Finally, let $f(v)=e_{rms}%
(X(v))$. Recall from Equation (\ref{eq:rmse}) that $d = \hat{y} - y$ and that
$\hat{y}$ depends on $X(v)$. Note that one function evaluation of $f(v)$
implies training the MLP and computing the result of Equation (\ref{eq:rmse}).
Here we propose to optimize $v$ such that
\begin{equation}
\underset{v \in\mathbb{Z}^{3}}{\mathrm{min}} f(v)
\label{eq:optimization_problem}%
\end{equation}

The problem to find optimal data-related parameters has no analytical
descriptions. Therefore, no gradient information is available. An evolutionary
algorithm is the natural choice for this optimization problem.

\subsubsection{True Optimal Data Parameters}

The finite size of CMAPS dataset and finite search space
of $v$ allow an exhaustive
search to be performed in order to find the true optimal data-related
parameters. We would like to emphasize that although exhaustive search is a
possibility for CMAPS dataset, it is in no way a possibility in a more general
setting. Nevertheless, the possibility to perform exhaustive search on the
CMAPS dataset can be exploited to demonstrate the accuracy of the chosen EA
and of the framework overall. In the following studies, we use the results and
computational efforts of the exhaustive search as benchmarks to examine the
accuracy and efficiency of the proposed approach.

We should note that the subsets of the data FD001 and FD003 have similar
features and that the subsets FD002 and FD004 have similar features. Because
of this, we have decided to just optimize the data-related parameters by
considering the subsets FD001 and FD002 only. An exhaustive search is
performed to find the true optimal values for $v$.
The MLP is only trained for $20$ epochs. Table
\ref{table:true_optimal_data_params} shows the optimal as well as the worst
combinations of data-related parameters and the total number of function
evaluations used by the exhaustive search. It is important to notice that for
this experiment the window size is limited to be larger than or equal to $15$.

\subsubsection{Evolutionary Algorithm for Optimal Data Parameters}

\label{sec:ea_optimization_process}

Evolutionary algorithms (EAs) are a family of methods for optimization
problems. The methods do not make any assumptions about the problem, treating
it as a black box that merely provides a measure of quality given a candidate
solution. Furthermore, EAs do not require the gradient when searching for
optimal solutions, making them very suitable for applications such as neural networks.

For the current application, the differential evolution (DE) method is chosen
as the optimization algorithm \cite{Storn1997}. Though other meta-heuristic
algorithms may also be suitable for this application, the DE has been
stablished itself as one of the most reliable, robust and easy to use EAs.
Furthermore, a ready to use Python implementation is available through the
scipy package \cite{scipy}. Although the DE method does not have special
operators for treating integer variables, a very simple modification to the
algorithm, i.e. rounding every component of a candidate solution to its
nearest integer, is used for this work.

As mentioned earlier, evolutionary algorithms such as the DE use several
function evaluations when searching for the optimal solutions. Recall that for
this application, one function evaluation implies retraining the neural
network from scratch. This is not a desirable scenario, as obtaining the
optimal data-related parameters would entail an extensive computational
effort. Instead of running the DE for several iterations and with a large
population size, we propose to run it just for $30$ iterations, i.e. the
generations in the literature of evolutionary computation, with a population
size of $12$, which seems reasonable given the size of the search space of $v$.

During the optimization, the MLP is trained only $20$ epochs. The small number
of epochs of training the MLP is reasonable in this case because a small batch
of data is used in the training, because we only look for the trend of the scoring indicators.
Furthermore, it is common to observe that the parameters leading to lower
score values in the early stages of the training are more likely to provide
better performance after more epochs of trainings. The settings of
the DE algorithm to find the optimal data-related parameters are listed in
Table \ref{table:de_hyperparams}.

The optimal data-related parameters for the subsets FD001 and FD002 found by
the DE algorithm are listed in Table \ref{table:optimal_data_params}. As can be
observed, the results are in fact very close to the true optimal ones
in Table \ref{table:true_optimal_data_params} for both the subsets of the
data. The computational effort is reduced by one order of magnitude when using
the DE method as compared to the exhaustive search for the true optimal
parameters. From the results in Table \ref{table:optimal_data_params}, it can
be observed that the maximum allowable time window is always preferred while,
on the other hand, small window strides yield better results. For the case of
early RUL, it can be observed that larger values of $R_{e}$ are favored.

\subsection{The Estimation Algorithm}

Having described the major building blocks of the proposed method, we now
introduce the complete framework in the form of Algorithm \ref{alg:rul_framework}.

\setcounter{algorithm}{0}
\begin{algorithm}[H]
\caption{\newline ANN-EA RUL Estimation Framework}\label{alg:rul_framework}
\textbf{Input:} Initial set of data-related parameters $v \in \mathbb{Z}^n$,
Raw training/testing data $X$ and training labels $y$\\
\textbf{Output:} Optimal set of data-related parameters $v^*$
\begin{algorithmic}[1]
\State Choose regressor architecture (ANN, SVM, linear/logistic regression, etc).
\State Define $f(v)$ as in Section\ref{sec:otimal_data_params}.
\State Optimize $f(v)$ using the preferred evolutionary algorithm, i.e.
differential evolution, evolutionary strategies, genetic algorithm, etc, using the proposed
guidelines from Section \ref{sec:ea_optimization_process}.
\State Use $v^*$ to train the regressor for as many epochs as needed.
\end{algorithmic}
\end{algorithm}

\section{Evaluation of the Proposed Method}

\label{sec:rul_eval}

In this section, we evaluate the performance of the proposed method. The
architecture of the MLP is described in Table \ref{table:proposed_nn}.
The MLP was trained $10$ times for $200$
epochs each and tested in each subset of the CMAPS dataset.
The combinations of the optimal window size $n_{w}$, window
stride $n_{s}$ and early RUL $R_{e}$ are presented in Table
\ref{table:data_params_de}.

The obtained results for $f(v)$ using the above setting are presented in Table
\ref{table:results_ann_de}. Notice that the performances obtained for datasets
FD001 and FD002 are improved as compared with the results
in Table \ref{table:optimal_data_params}. This is due to the fact that the MLP is trained
for more epochs, thus obtaining better results.

%Next, the possibility of using a single set of data-related parameters for all
%the subsets is explored. For this experiment, the $n_{w}$ is fixed for all of
%the four datasets, given that the maximum allowable window size for all
%datasets is $18$. Hence, the data-related parameters are chosen as $v=(17, 1,
%139)$. The results of predictions are shown in Table \ref{table:results_ann_1}.

%As can be observed, the performance is decreased for the subsets FD001 and
%FD003. This indicates that larger window sizes are beneficial for this
%regression problem. Figures \ref{FigRMSEcomparison} and \ref{FigRHScomparison}
%show a comparison of the scores for each dataset by changing the data-related
%parameters.

%\subsection{Comparison with Published Works}

The performance of the proposed method is compared against
other state-of-the-art methods. Most of the methods chosen to compare here
have only reported the results on the test set FD001 in terms of
$e_{rms}$. The results are shown in Table \ref{table:results_comparison}. The
$e_{rms}$ value of the proposed method in Table \ref{table:results_comparison}
is the mean value of 10 independent runs. The values of other methods are
identical to those reported in their respective original papers.

From the comparison studies of the prediction results, we can conclude that
the proposed method performs better than the majority of the chosen methods
when taking into consideration the whole dataset FD001. Two existing methods
come close to the performance of the proposed approach in this paper, namely
the time window ANN \cite{Lim2016} and the Networks Ensemble \cite{Zhang2016}.
While the performance of these two methods comes close to the results of the
proposed method in this paper, the proposed method is more computationally
efficient. Furthermore, the framework proposed herein is simple to understand
and implement, robust, generic and light-weight. These are the features
important to highlight when comparing the proposed method against other
state-of-the-art approaches.

\section{Conclusions}

\label{sec:conclusions}

We have presented a novel framework for predicting the RUL of mechanical
components. While the method has been tested on the jet-engine dataset
CMAPS, the method is general enough that it can be applied to other similar
systems. The framework makes use of a strided moving time window to generate
the training and test records. A shallow MLP to make the predictions of the RUL
has been found to be sufficient for the current dataset. The evolutionary
algorithm DE needs to be run just once to find the best data-related
parameters that optimize the scoring functions. The results presented in this
paper demonstrate that the proposed framework is accurate and computationally
efficient, which makes this framework suitable for applications that have
limited computational resources such as embedded systems. Furthermore, the
comparison with other state-of-the-art methods has shown that the proposed
method is the best overall performer.

Two major features of the proposed framework are its generality and
scalability. While for this study, specific regressors and evolutionary
algorithms are chosen, many other combinations are possible and may be more
suitable for different applications. Furthermore, the framework can, in
principle, be used for model-construction, i.e. generating the best possible
neural network architecture tailored to a specific application.

\bibliographystyle{IEEEtran}
\bibliography{reference_rul_paper}
%

%TCIMACRO{\QSubDoc{Include IEEE Bio}{\input{IEEEBios.tex}}}%
%BeginExpansion
%\input{IEEEBios.tex}
%EndExpansion

\clearpage

\onecolumn%

%TCIMACRO{\TeXButton{Begin+Table+Center+Cap}{\begin{table}
%\begin{center}
%\caption{C-MAPSS Dataset details.}}}%
%BeginExpansion
\begin{table}
\begin{center}
\caption{C-MAPSS Dataset details.}%
%EndExpansion
%

\begin{tabular}
[c]{l|cccc}\hline
& \multicolumn{4}{c}{C-MAPSS}\\
Dataset & FD001 & FD002 & FD003 & FD004\\\hline\hline
Training Trajectories & 100 & 260 & 100 & 248\\
Test Trajectories & 100 & 259 & 100 & 248\\
Operating Conditions & 1 & 6 & 1 & 6\\
Fault Modes & 1 & 1 & 2 & 2\\\hline
\end{tabular}
\label{TabCMAPSS}%

%TCIMACRO{\TeXButton{End+Center+Table}{\end{center}
%\end{table}
%}}%
%BeginExpansion
\end{center}
\end{table}
%EndExpansion
%

%TCIMACRO{\TeXButton{Begin+Table+Center+Cap}{\begin{table}
%\begin{center}
%\caption{Results for different architectures for subset 1, 100 epochs.}}}%
%BeginExpansion
\begin{table}
\begin{center}
\caption{Results for different architectures for subset 1, 100 epochs.}%
%EndExpansion
%

\begin{tabular}
[c]{l|cccc|cccc}\hline
& \multicolumn{4}{|c}{RMSE} & \multicolumn{4}{|c}{RHS}\\
Tested Architecture & Min. & Max. & Avg. & STD & Min. & Max. & Avg. &
STD\\\hline\hline
Architecture 1 & 15.51 & 17.15 & 16.22 & 0.49 & 4.60 & 7.66 & 5.98 & 0.91\\
Architecture 2 & 15.24 & 16.46 & 15.87 & 0.47 & 4.07 & 6.26 & 5.29 & 0.82\\
Architecture 3 & 15.77 & 17.27 & 16.15 & 0.45 & 5.11 & 8.25 & 5.93 & 0.94\\
Architecture 4 & 15.13 & 17.01 & 15.97 & 0.47 & 3.90 & 7.54 & 5.65 & 1.2\\
Architecture 5 & 16.39 & 17.14 & 16.81 & 0.23 & 5.19 & 6.58 & 5.98 & 0.42\\
Architecture 6 & 16.42 & 17.36 & 16.87 & 0.30 & 5.15 & 7.09 & 6.12 &
0.62\\\hline
\end{tabular}
\label{table:tested_architectures_100}%

%TCIMACRO{\TeXButton{End+Center+Table}{\end{center}
%\end{table}}}%
%BeginExpansion
\end{center}
\end{table}%
%EndExpansion
%

%TCIMACRO{\TeXButton{Begin+Table+Center+Cap}{\begin{table}
%\begin{center}
%\caption{Proposed Neural Network architecture}.}}%
%BeginExpansion
\begin{table}
\begin{center}
\caption{Proposed Neural Network architecture}.%
%EndExpansion
%

\begin{tabular}
[c]{llll}\hline
Layer & Shape & Activation & Additional Information\\\hline\hline
Fully connected & \multicolumn{1}{c}{20} & \multicolumn{1}{c}{ReLU} &
$L1=0.1,L2=0.2$\\
Fully connected & \multicolumn{1}{c}{20} & \multicolumn{1}{c}{ReLU} &
$L1=0.1,L2=0.2$\\
Fully connected & \multicolumn{1}{c}{1} & \multicolumn{1}{c}{Linear} &
$L1=0.1,L2=0.2$\\\hline
\end{tabular}
\label{table:proposed_nn}%

%TCIMACRO{\TeXButton{End+Center+Table}{\end{center}
%\end{table}}}%
%BeginExpansion
\end{center}
\end{table}%
%EndExpansion
%

%TCIMACRO{\TeXButton{Begin+Table+Center+Cap}{\begin{table}
%\begin{center}
%\caption{Allowed values for $b$ per subset.}}}%
%BeginExpansion
\begin{table}
\begin{center}
\caption{Allowed values for $b$ per subset.}%
%EndExpansion
%

\begin{tabular}
[c]{c|cccc}\hline
& FD001 & FD002 & FD003 & FD004\\\hline
$b$ & 30 & 20 & 30 & 18\\\hline
\end{tabular}
\label{table:b_values}%

%TCIMACRO{\TeXButton{End+Center+Table}{\end{center}
%\end{table}}}%
%BeginExpansion
\end{center}
\end{table}%
%EndExpansion
%

%TCIMACRO{\TeXButton{Begin+Table+Center+Cap}{\begin{table}
%\begin{center}
%\caption{Exhaustive search results for subsets FD001 and F002.}}}%
%BeginExpansion
\begin{table}
\begin{center}
\caption{Exhaustive search results for subsets FD001 and F002.}%
%EndExpansion
%

\begin{tabular}
[c]{l|crcrr}\hline
Dataset & argmin $v$ & min $f(v)$ & argmax $v$ & max $f(v)$ & Function
evals.\\\hline\hline
FD001 & $\left[  24,1,127\right]  $ & \multicolumn{1}{c}{$15.11$} & $\left[
25,10,94\right]  $ & \multicolumn{1}{c}{$85.19$} & \multicolumn{1}{c}{8160}\\
FD002 & $\left[  16,1,138\right]  $ & \multicolumn{1}{c}{$30.93$} & $\left[
17,10,99\right]  $ & \multicolumn{1}{c}{$59.78$} & \multicolumn{1}{c}{3060}%
\\\hline
\end{tabular}
\label{table:true_optimal_data_params}%

%TCIMACRO{\TeXButton{End+Center+Table}{\end{center}
%\end{table}}}%
%BeginExpansion
\end{center}
\end{table}%
%EndExpansion

\bigskip%
%TCIMACRO{\TeXButton{Begin+Table+Center+Cap}{\begin{table}
%\begin{center}
%\caption{Differential evolution hyper-parameters.}}}%
%BeginExpansion
\begin{table}
\begin{center}
\caption{Differential evolution hyper-parameters.}%
%EndExpansion
%

\begin{tabular}
[c]{llll}\hline
Population Size & Generations & Strategy & MLP epochs\\\hline
\multicolumn{1}{c}{12} & \multicolumn{1}{c}{30} & \multicolumn{1}{c}{Best1Bin \cite{Engelbrecht2007}}
& \multicolumn{1}{c}{20}\\\hline
\end{tabular}
\label{table:de_hyperparams}%

%TCIMACRO{\TeXButton{End+Center+Table}{\end{center}
%\end{table}}}%
%BeginExpansion
\end{center}
\end{table}%
%EndExpansion
%

%TCIMACRO{\TeXButton{Begin+Table+Center+Cap}{\begin{table}
%\begin{center}
%\caption
%{Data-related parameters for each subset obtained with differential evolution.}%
%}}%
%BeginExpansion
\begin{table}
\begin{center}
\caption
{Data-related parameters for each subset obtained with differential evolution.}%
%EndExpansion
%

\begin{tabular}
[c]{l|crr}\hline
Dataset & argmin $v$ & min $f(v)$ & Function evals.\\\hline\hline
FD001 & $\left[  24,1,129\right]  $ & \multicolumn{1}{c}{$15.24$} &
\multicolumn{1}{c}{372}\\
FD002 & $\left[  17,1,139\right]  $ & \multicolumn{1}{c}{$30.95$} &
\multicolumn{1}{c}{372}\\\hline
\end{tabular}
\label{table:optimal_data_params}%

%TCIMACRO{\TeXButton{End+Center+Table}{\end{center}
%\end{table}}}%
%BeginExpansion
\end{center}
\end{table}%
%EndExpansion
%

%TCIMACRO{\TeXButton{Begin+Table+Center+Cap}{\begin{table}
%\begin{center}
%\caption{Data-related parameters for each subset as obtained by DE.}}}%
%BeginExpansion
\begin{table}
\begin{center}
\caption{Data-related parameters for each subset as obtained by DE.}%
%EndExpansion
%

\begin{tabular}
[c]{lrrr}\hline
Dataset & $n_{w}$ & $n_{s}$ & $R_{e}$\\\hline\hline
FD001 & 24 & 1 & 129\\
FD002 & 17 & 1 & 139\\
FD003 & 24 & 1 & 129\\
FD004 & 17 & 1 & 139\\\hline
\end{tabular}
\label{table:data_params_de}%

%TCIMACRO{\TeXButton{End+Center+Table}{\end{center}
%\end{table}}}%
%BeginExpansion
\end{center}
\end{table}%
%EndExpansion
%

%TCIMACRO{\TeXButton{Begin+Table+Center+Cap}{\begin{table}
%\begin{center}
%\caption
%{Scores for each dataset using the data-related parameters obtained by DE (Second architecture).}%
%}}%
%BeginExpansion
\begin{table}
\begin{center}
\caption
{Scores for each dataset using the data-related parameters obtained by DE (Second architecture).}%
%EndExpansion
%

\begin{tabular}
[c]{l|cccc|cccc}\hline
& \multicolumn{4}{|c}{RMSE} & \multicolumn{4}{|c}{RHS}\\
Data Subset & min & max & avg & STD & min & max & avg & STD\\\hline\hline
FD001 & 14.24 & 14.57 & 14.39 & 0.11 & 3.25 & 3.58 & 3.37 & 0.11\\
FD002 & 28.90 & 29.23 & 29.09 & 0.11 & 45.99 & 53.90 & 50.69 & 2.17\\
FD003 & 14.74 & 16.18 & 15.42 & 0.50 & 4.36 & 6.85 & 5.33 & 0.95\\
FD004 & 33.25 & 35.10 & 34.74 & 0.53 & 58.52 & 78.62 & 74.77 & 5.88\\\hline
\end{tabular}
\label{table:results_ann_de}%

%TCIMACRO{\TeXButton{End+Center+Table}{\end{center}
%\end{table}}}%
%BeginExpansion
\end{center}
\end{table}%
%EndExpansion
%

%%TCIMACRO{\TeXButton{Begin+Table+Center+Cap}{\begin{table}
%%\begin{center}
%%\caption
%%{Scores for each dataset using the single set of data-related parameters.}}}%
%%BeginExpansion
%\begin{table}
%\begin{center}
%\caption
%{Scores for each dataset using the single set of data-related parameters.}%
%%EndExpansion
%%
%
%\begin{tabular}
%[c]{l|rrrr|rrrr}\hline
%& \multicolumn{4}{|c}{RMSE} & \multicolumn{4}{|c}{RHS}\\
%Data Subset & min & max & avg & STD & min & max & avg & STD\\\hline\hline
%FD001 & 16.74 & 17.23 & 17.06 & 0.18 & 8.08 & 9.26 & 8.64 & 0.45\\
%FD002 & 29.77 & 30.12 & 29.94 & 0.11 & 54.90 & 63.53 & 58.82 & 2.39\\
%FD003 & 16.94 & 18.52 & 17.77 & 0.39 & 6.02 & 10.18 & 7.64 & 1.37\\
%FD004 & 33.95 & 35.46 & 35.14 & 0.44 & 58.71 & 67.91 & 65.78 & 2.64\\\hline
%\end{tabular}
%\label{table:results_ann_1}%
%%TCIMACRO{\TeXButton{End+Center+Table}{\end{center}
%%\end{table}}}%
%%BeginExpansion
%\end{center}
%\end{table}%
%%EndExpansion
%%

%TCIMACRO{\TeXButton{Begin+Table+Center+Cap}{\begin{table}
%\begin{center}
%\caption
%{Performance comparison of the proposed method and the latest related papers on the CMAPS dataset.}%
%}}%
%BeginExpansion
\begin{table}
\begin{center}
\caption
{Performance comparison of the proposed method and the latest related papers on the CMAPS dataset.}%
%EndExpansion
%

\begin{tabular}
[c]{l|r}\hline
Method & $e_{rms}$\\\hline
ESN trained by Kalman Filter \cite{Peng2012} & 63.45\\
Support Vector Machine Classifier \cite{Louen2013} & 29.82\\
Time Window Neural Network \cite{Lim2016} & 15.16\\
Multi-objective deep belief networks ensemble \cite{Zhang2016} & 15.04\\
Deep Convolutional Neural Network \cite{Babu2016} & 18.45\\
\textbf{Proposed method with $n_{w}=30$, $n_{s}=1$ and $R_{e}=128$} &
14.39\\\hline
\end{tabular}
\label{table:results_comparison}%

%TCIMACRO{\TeXButton{End+Center+Table}{\end{center}
%\end{table}}}%
%BeginExpansion
\end{center}
\end{table}%
%EndExpansion
%

%TCIMACRO{\TeXButton{Clearpage}{\clearpage}}%
%BeginExpansion
\clearpage
%EndExpansion
%

%TCIMACRO{\FRAME{ftbpFU}{4.0421in}{1.5212in}{0pt}{\Qcb{Graphical depiction of
%the time window used in this framework.}}{\Qlb{FigWindow}}{time_window.png}%
%{\special{ language "Scientific Word";  type "GRAPHIC";
%maintain-aspect-ratio TRUE;  display "USEDEF";  valid_file "F";
%width 4.0421in;  height 1.5212in;  depth 0pt;  original-width 10.933in;
%original-height 4.0672in;  cropleft "0";  croptop "1";  cropright "1";
%cropbottom "0";  filename 'Figures/time_window.png';file-properties "XNPEU";}}
%}%
%BeginExpansion
\begin{figure}
[ptb]
\begin{center}
\includegraphics[
%natheight=4.067200in,
%natwidth=10.933000in,
%height=1.5212in,
width=4.0421in
]%
{Figures/time_window.png}%
\caption{Graphical depiction of the time window used in this framework.}%
\label{FigWindow}%
\end{center}
\end{figure}
%EndExpansion

%

%TCIMACRO{\FRAME{ftbpFU}{3.2897in}{2.3661in}{0pt}{\Qcb{Piecewise linear
%degradation for RUL.}}{\Qlb{FigRULinear}}{test_engine.png}%
%{\special{ language "Scientific Word";  type "GRAPHIC";
%maintain-aspect-ratio TRUE;  display "USEDEF";  valid_file "F";
%width 3.2897in;  height 2.3661in;  depth 0pt;  original-width 5.3463in;
%original-height 3.8337in;  cropleft "0";  croptop "1";  cropright "1";
%cropbottom "0";  filename 'Figures/test_engine.png';file-properties "XNPEU";}}
%}%
%BeginExpansion
\begin{figure}
[ptb]
\begin{center}
\includegraphics[
natheight=3.833700in,
natwidth=5.346300in,
height=2.3661in,
width=3.2897in
]%
{Figures/test_engine.png}%
\caption{Piecewise linear degradation for RUL.}%
\label{FigRULinear}%
\end{center}
\end{figure}
%EndExpansion
%

%%TCIMACRO{\FRAME{ftbpFU}{3.5405in}{2.2995in}{0pt}{\Qcb{Comparison of RMSE
%%results for different sets of data-related parameters.}}%
%%{\Qlb{FigRMSEcomparison}}{rmse_comparisson.png}%
%%{\special{ language "Scientific Word";  type "GRAPHIC";
%%maintain-aspect-ratio TRUE;  display "USEDEF";  valid_file "F";
%%width 3.5405in;  height 2.2995in;  depth 0pt;  original-width 5.6593in;
%%original-height 3.6599in;  cropleft "0";  croptop "1";  cropright "1";
%%cropbottom "0";
%%filename 'Figures/rmse_comparisson.png';file-properties "XNPEU";}} }%
%%BeginExpansion
%\begin{figure}
%[ptb]
%\begin{center}
%\includegraphics[
%natheight=3.659900in,
%natwidth=5.659300in,
%height=2.2995in,
%width=3.5405in
%]%
%{Figures/rmse_comparisson.png}%
%\caption{Comparison of RMSE results for different sets of data-related
%parameters.}%
%\label{FigRMSEcomparison}%
%\end{center}
%\end{figure}
%%EndExpansion
%%
%
%%TCIMACRO{\FRAME{ftbpFU}{3.5405in}{2.3013in}{0pt}{\Qcb{Comparison of RHS
%%results for different sets of data-related parameters.}}%
%%{\Qlb{FigRHScomparison}}{rhs_comparisson.png}%
%%{\special{ language "Scientific Word";  type "GRAPHIC";
%%maintain-aspect-ratio TRUE;  display "USEDEF";  valid_file "F";
%%width 3.5405in;  height 2.3013in;  depth 0pt;  original-width 5.6559in;
%%original-height 3.6599in;  cropleft "0";  croptop "1";  cropright "1";
%%cropbottom "0";
%%filename 'Figures/rhs_comparisson.png';file-properties "XNPEU";}} }%
%%BeginExpansion
%\begin{figure}
%[ptb]
%\begin{center}
%\includegraphics[
%natheight=3.659900in,
%natwidth=5.655900in,
%height=2.3013in,
%width=3.5405in
%]%
%{Figures/rhs_comparisson.png}%
%\caption{Comparison of RHS results for different sets of data-related
%parameters.}%
%\label{FigRHScomparison}%
%\end{center}
%\end{figure}
%%EndExpansion
%

\clearpage

\appendix

\section*{Tested Neural Network Architectures}

%\label{sec:appendices}
\bigskip

\begin{table}[tbh]
\centering
\caption{Proposed Neural Network architecture 1.}%
\begin{tabular}
[c]{llll}\hline
Layer & Shape & Activation & Additional Information\\\hline\hline
Fully connected & 20 & ReLU & L1 = 0.1, L2 = 0.2\\
Fully connected & 20 & ReLU & L1 = 0.1, L2 = 0.2\\
Fully connected & 1 & Linear & L1 = 0.1, L2 = 0.2\\\hline
\end{tabular}
\label{table:proposed_nn_1}%
\end{table}

\begin{table}[tbh]
\centering
\caption{Proposed Neural Network architecture 2.}%
\begin{tabular}
[c]{llll}\hline
Layer & Shape & Activation & Additional Information\\\hline\hline
Fully connected & 50 & ReLU & L1 = 0.1, L2 = 0.2\\
Fully connected & 20 & ReLU & L1 = 0.1, L2 = 0.2\\
Fully connected & 1 & Linear & L1 = 0.1, L2 = 0.2\\\hline
\end{tabular}
\label{table:proposed_nn_2}%
\end{table}

\begin{table}[tbh]
\centering
\caption{Proposed Neural Network architecture 3.}%
\begin{tabular}
[c]{llll}\hline
Layer & Shape & Activation & Additional Information\\\hline
Fully connected & 100 & ReLU & L1 = 0.1, L2 = 0.2\\
Fully connected & 50 & ReLU & L1 = 0.1, L2 = 0.2\\
Fully connected & 1 & Linear & L1 = 0.1, L2 = 0.2\\\hline
\end{tabular}
\label{table:proposed_nn_3}%
\end{table}

\begin{table}[tbh]
\centering
\caption{Proposed Neural Network architecture 4.}%
\begin{tabular}
[c]{llll}\hline
Layer & Shape & Activation & Additional Information\\\hline
Fully connected & 250 & ReLU & L1 = 0.1, L2 = 0.2\\
Fully connected & 50 & ReLU & L1 = 0.1, L2 = 0.2\\
Fully connected & 1 & Linear & L1 = 0.1, L2 = 0.2\\\hline
\end{tabular}
\label{table:proposed_nn_4}%
\end{table}

\begin{table}[tbh]
\centering
\caption{Proposed Neural Network architecture 5.}%
\begin{tabular}
[c]{llll}\hline
Layer & Shape & Activation & Additional Information\\\hline\hline
Fully connected & 20 & ReLU & L1 = 0.1, L2 = 0.2\\
Fully connected & 1 & Linear & L1 = 0.1, L2 = 0.2\\\hline
\end{tabular}
\label{table:proposed_nn_5}%
\end{table}

\begin{table}[tbh]
\centering
\caption{Proposed Neural Network architecture 6.}%
\begin{tabular}
[c]{llll}\hline
Layer & Shape & Activation & Additional Information\\\hline\hline
Fully connected & 10 & ReLU & L1 = 0.1, L2 = 0.2\\
Fully connected & 1 & Linear & L1 = 0.1, L2 = 0.2\\\hline
\end{tabular}
\label{table:proposed_nn_6}%
\end{table}
\end{document}


\documentclass[12pt]{article}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{geometry}
\usepackage{enumerate}
\usepackage{natbib}
\usepackage{float}%稳定图片位置
\usepackage{graphicx}%画图
\usepackage[english]{babel}
\usepackage{a4wide}
\usepackage{indentfirst}%缩进
\usepackage{enumerate}%加序号
\usepackage{multirow}%合并行
\title{Final Report}

\begin{document}
\maketitle
\newpage
\section{Search Algorithm}
\subsection{Room Segment}
The general function of the base board is moving to your appointed destination and return. To realize it, we cut the room into many small squares with coordinates. Then, we can tell  the coordinates of the destination. In addition, since the coordinate system is absolute, Metbin also should know the coordinates of its initial place to calculate its relative position about the destination.
\begin{figure}[H]
\centering
\includegraphics[scale=0.8]{P1.jpg}
\caption{Room segment with coordinates}
\end{figure}
In the figure 1, the room is cut into $4\times4$ squares with coordinates, and there are x and y axis. So we can know the initial position of Metbin is (4,4) and the destination's coordinates are (4,1).
\subsection{Direction of Metbin}
Since we use a search algorithm, Metbin must know its direction relative to x and y axis, so it can know towards which square it's moving. Since the room is cut into squares, so we introduce a direction variable \textbf{d} of four values to represent robot's current direction.
\begin{figure}[H]
\centering
\includegraphics[scale=1]{P2.jpg}
\caption{Direction variable}
\end{figure}
The figure shows the direction in the room. Since its also absolute, it should be input to the Metbin in advance. We set the direction parallel to y axis is 1, and it will become plus one after Metbin turn right 90 degrees, and it will minus 1 after it turn left. It is worth to mention that \textbf{d} only has 4 values (1$\sim$4), so when its reaches 5, it will minus 4.
Similarly, when \textbf(d) is smaller than 1, it will plus 4.
\par In the figrue 1, it's clear that Metbin can only move against the direction of x axis, so its initial \textbf{d} is 4.
\subsection{Square Information and its Variables}
To realize the search function, we introduce ten variables to describe each square.
\begin{table}[H]
\centering
\begin{tabular}{|c|c|c|c|}
\hline
variable name & meaning  & value range  \\ \hline
\textbf{E} &The order of the square first reached by Metbin  & 1$\sim$16   \\ \hline
\textbf{X[E]} &The x coordinate of square \textbf{E}  &  1$\sim$4  \\ \hline
\textbf{Y[E]} &The y coordinate of square \textbf{E}  &  1$\sim$4  \\ \hline
\textbf{D[E]} &Metbin's \textbf{d} when it first reaches square \textbf{E}  &1$\sim$4    \\ \hline
\textbf{L[E]} &The left obstacle situation when Metbin first reaches square \textbf{E}  &-1$\sim$16  \\ \hline
\textbf{F[E]} &The front obstacle situation when Metbin first reaches square \textbf{E}  &-1$\sim$16     \\ \hline
\textbf{R[E]} &The right obstacle situation when Metbin first reaches square \textbf{E}  &-1$\sim$16     \\ \hline
\textbf{l[E]} &The left priority when Metbin reach square \textbf{E}  &0$\sim$3     \\ \hline
\textbf{f[E]} &The front priority when Metbin reach square \textbf{E}  &0$\sim$3     \\ \hline
\textbf{r[E]} &The right priority when Metbin reach square \textbf{E}  &0$\sim$3    \\ \hline
\end{tabular}
\caption{Variables describing a square}
\end{table}
\subsubsection{Square Order \textbf{E} and Direction \textbf{D[E]}}
As mentioned in table 1, \textbf{E} represents the order of the square first reached by Metbin. For example, in Figure 1, Metbin's initial position's \textbf{E} is 1 and the square with coordinates (3,4) is 2. Every square only has one corresponding \textbf{E}. If Metbin return a square which already has \textbf{E}, it won't change. Since there are 16 squares in the field, $E_{max}=16$.
\par Additionally, when Metbin first reaches the square, its \textbf{d} will be recorded as square \textbf{E}'s \textbf{D[E]}. It also won't change later.
\subsubsection{Obstacle situation \textbf{L[E]}, \textbf{F[E]}, and \textbf{R[E]}}
As mentioned the before, the three variable describe the obstacle situation in three sides when Metbin is in the square with direction \textbf{D[E]}. If there is an obstacle on right side, then \textbf{R[E]} will be 0. If there is no obstacle then, if there is a reached square \textbf{E'}, then \textbf{R[E]}=\textbf{E'}, else ]\textbf{R[E]}=-1. Since there are 16 squares in the field, the obstacle variable can't be bigger than 16.
\par If every square's obstacle situation is bigger than -1, than Metbin has already reached every squares.
\subsubsection{Priority Variable \textbf{l[E]}, \textbf{f[E]}, and \textbf{r[E]}}
The three variable in lower case describe the order of directions to turn when Metbin reach \textbf{E} with \textbf{D[E]}. These variable value is smaller than 3 because there is only 3 directions Metbin can advance. Metbin will choose the biggest variable and go its corresponding direction.
\par For example , if Metbin reaches square \textbf{5} and \textbf{l[5]}=0, \textbf{f[E]}=1, and \textbf{r[5]}=3, then it will turn right keep going. Then \textbf{r[5]} will become 0 after Metbin go back.
\par When Metbin reaches a new square, the three variables will generated according to Metbin's relative location to the destination and obstacle situation variables. If there is an obstacle on the left side (\textbf{L[E]}=0) then \textbf{l[E]} will also be 0 because there can't go. If there is no wall, then if the obstacle is in front of Metbin, then \textbf{f[E]} will be 3. Then if the obstacle is on the right, \textbf{r[E]} will be 2 and \textbf{l[E]}=1. If the obstacle is on the back of Metbin, then \textbf{f[E]} will be 1 and other two variables will plus one correspondingly. Otherwise, Metbin will go at the order of left, front, then right.
\subsubsection{An Example without return from Figure 1}
\begin{figure}[H]
\centering
\includegraphics[scale=0.4]{P3.jpg}
\caption{Direction variable}
\end{figure}
The first 5 squares' \textbf{E} is already shown on the picture and the following table shows the information recorder when Metbin first reaches square 5.
\begin{table}[H]
\centering
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}
\hline
\textbf{E}&\textbf{X[E]}&\textbf{Y[E]}&\textbf{D[E]}&\textbf{L[E]}&\textbf{F[E]}&\textbf{R[E]}&\textbf{l[E]}&\textbf{f[E]}&\textbf{r[E]}\\ \hline
1 &4  &4  &4  &0 &2 &0 &0 &2 &0  \\ \hline
2 &3  &4  &4  &0 &3 &0 &0 &1 &0  \\ \hline
3 &2  &4  &4  &4 &-1&0 &3 &2 &0  \\ \hline
4 &2  &3  &3  &-1&5&-1 &2 &3 &1 \\ \hline
5 &2  &2  &3  &-1&0&0  &2 &0 &0 \\ \hline
\end{tabular}
\caption{Information variables of the first 5 squares.}
\end{table}
\begin{figure}[H]
\centering
\includegraphics[scale=0.5]{P4.jpg}
\caption{Flow chart between square 1 and square 2}
\end{figure}
If there are only few obstacles, Metbin can go to the destination according to the above flow chart.
\begin{figure}[H]
\centering
\includegraphics[scale=0.5]{P5.jpg}
\caption{Metbin's route to destination}
\end{figure}
\subsection{Realization of Return Function}
However, if there are many obstacles in the room, Metbin may reach a square \textbf{E} which has three walls around it (\textbf{L[E]}=0,\textbf{F[E]}=0, and \textbf{R[E]}=0). Then Metbin must return to another square \textbf{E'} who must have one obstacle situation variable equals to \textbf{E}. If there are many \textbf{E'} has such variable, then Metbin will go to the square with smallest \textbf{E'}.
\begin{figure}[H]
\centering
\includegraphics[scale=0.7]{P6.jpg}
\caption{Metbin's whole route in example 2}
\end{figure}
In the Figure 6, Metbin's initial square's coordinates are \textbf{X[1]}=1, \textbf{Y[1]}=1, and \textbf{D[1]}=1. The destination's coordinates are \textbf{X[E]}=4 and \textbf{Y[E]}=3.
We draw the route of Metbin according to our algorithm, the black lines represent Metbin's route to a new square and the white lines represent its route of returning. Some square's \textbf{E} are already shown on the figure. This will be recorded when Metbin's moving. In this field, square \textbf{3}, \textbf{7}, and \textbf{8} has three obstacles around it, so Metbin will return after reached these squares. In addition, since the destination's coordinates has already been input to Metbin and it keeps recording its own coordinates. When these coordinates are same, Metbin knows that it has reached the destination and return.
\section{Computer Simulation}
Since the algorithm is hard to realize, we decide to simulate it on computer first. In this procedure, we draw a field and Metbin's route. Then we input the obstacle situation variable according to Metbin's route. This will be detected by infra-red sensors in real field. We only input 0 or -1 as every square's obstacle situation variable's value. (-1 means there is no obstacle and 0 represents there is an obstacle.)
\par for example in figure 6, we can input the following list
\begin{table}[H]
\centering
\begin{tabular}{|c|c|c|c|}
\hline
\textbf{E}  & \textbf{L[E]} & \textbf{F[E]} & \textbf{R[E]} \\ \hline
1  & 0        & -1       & -1       \\ \hline
2  & 0        & -1       & 0        \\ \hline
3  & 0        & 0        & 0        \\ \hline
4  & -1       & -1       & 0        \\ \hline
5  & -1       & -1       & 0        \\ \hline
6  & -1       & 0        & 0        \\ \hline
7  & 0        & 0        & 0        \\ \hline
8  & 0        & 0        & 0        \\ \hline
9  & 0        & -1       & 0        \\ \hline
10 & 0        & -1       & -1       \\ \hline
11 & -1       & -1       & 0        \\ \hline
12 & -1       & 0        & 0        \\ \hline
\end{tabular}
\caption{Input of Obstacle Situation}
\end{table}
Then the Metbin can generate every square's information variable according table 3
\begin{table}[H]
\centering
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}
\hline
\textbf{E}  & \textbf{X[E]}  & \textbf{Y[E]} & \textbf{D[E]} & \textbf{L[E]} & \textbf{F[E]} & \textbf{R[E]} & \textbf{l[E]} & \textbf{f[E]} & \textbf{r[E]} \\ \hline
1  & 1        & 1        & 1        & 0        & 2        & 4        & 0        & 3        & 2        \\ \hline
2  & 1        & 2        & 1        & 0        & 3        & 0        & 0        & 3        & 0        \\ \hline
3  & 1        & 3        & 1        & 0        & 0        & 0        & 0        & 0        & 0        \\ \hline
4  & 2        & 1        & 2        & 9        & 5        & 0        & 2        & 3        & 0        \\ \hline
5  & 3        & 1        & 2        & 8        & 6        & 0        & 2        & 3        & 0        \\ \hline
6  & 4        & 1        & 2        & 7        & 0        & 0        & 3        & 0        & 0        \\ \hline
7  & 4        & 2        & 1        & 0        & 0        & 0        & 0        & 0        & 0        \\ \hline
8  & 3        & 2        & 1        & 0        & 0        & 0        & 0        & 0        & 0        \\ \hline
9  & 2        & 2        & 1        & 0        & 10       & 0        & 0        & 3        & 0        \\ \hline
10 & 2        & 3        & 1        & 0        & -1       & 11       & 0        & 2        & 3        \\ \hline
11 & 3        & 3        & 2        & -1       & 12       & 0        & 2        & 3        & 0        \\ \hline
12 & 4        & 3        & 2        & -1       & 0        & 0        & 0        & 0        & 0        \\ \hline
\end{tabular}
\caption{Information variable}
\end{table}
To test whether Metbin will move according to the correct route, we introduce a array \textbf{A[i]} of four values from $1\sim3$.
\begin{table}[H]
\centering
\begin{tabular}{|c|c|c|c|}
\hline
\textbf{A[i]} & 1                           & 2                    & 3                     \\ \hline
         & move forward to next square & turn left 90 degrees & turn right 90 degrees \\ \hline
\end{tabular}
\caption{Array \textbf{A}}
\end{table}
So Metbin should generate \textbf{A[i]=$\lbrace 11133111333111212213133212213133211311221121131\rbrace$}
if the Metbin can generate the correct \textbf{A[i]} according to our input obstacle information, we just need to let Metbin read the array and make the correct motion
\section{Realization of \textbf{A[i]}}
Since the environment is of great randomness, we must use multiple sensors to ensure Metbin is in the correct position in the field. We decide to use posture sensors, encoders and many infra-red sensors.
\subsection{Posture Sensors}
The posture sensors have many functions such as detect the magnetic field and its angle. It can calculate its deflection angle by angular acceleration integral and this is the only function required. so I can set it on computer.
\begin{figure}[H]
\centering
\includegraphics[scale=0.5]{P7.jpg}
\caption{Posture Sensors' setting surface on computer}
\end{figure}
The posture sensors can return a value \textbf{z} describing the angle deflected. It's value range is $0\sim360$. If the Metbin needs to turning right 90 degrees, it will only turn until \textbf{z}-\textbf{$\mathbf{z_{initial}}$}$\approx90^o$. Similarly to \textbf{d} in Search Algorithm, $\mathbf{\bigtriangleup_z}$ may be small than 0 sometime, so it need to be dealt with.
\subsection{Encoders}
We use encoders to recorded how far does Metbin move on the field. To eliminate accumulative, it will be set to 0 every time Metbin meet an obstacle. The encoders can return 1 and 0 when it receive a change in Motor's voltage. So we can record the number of pulse to calculate how far does Metbin move. After many experiments, we think when Metbin will move 55cm when 1500 pulse are returned.
\subsection{Infra-red Sensors}
Infra-red sensors can return its distance from obstacle.
\begin{figure}[H]
\centering
\includegraphics[scale=0.5]{P8.jpg}
\caption{Relation between the distance and return value. }
\end{figure}
When infra-red sensors' return value is bigger than 100, we assume there is an obstacle before it. We install five infra-red sensors on Metbin, one is installed at the head and two are installed at left and right sides respectively. In addition ,infra-red sensors can be used to detect whether Metbin is parallel to the wall. Since there are two infra-red sensors on one side. If $\frac{v_1-v_2}{v_1+v_2}<0.01$ then Metbin is parallel to the wall. The posture's accumulative error will be eliminate at the time.
\end{document}


\section{Introduction}\label{sec:introduction}

A bunch of different characteristics in complex systems can be tracked down to
non-stationarity \cite{non_stat_1,non_stat_2}. These systems lack any kind of
equilibrium \cite{comp_sys_1,comp_sys_2,comp_sys_3,comp_sys_4}. Financial
markets are perfect examples of non-stationarity as they fluctuate considerably
in time. In general, the business relations between companies and agents can
change due to market expectations. During a state of crisis, non-stationarity
becomes dramatic
\cite{state_crisis_1,state_crisis_2,state_crisis_3,state_crisis_4,state_crisis_5,state_crisis_6,state_crisis_7}.

The fluctuation of the correlations induces generic features in financial time
series, where we showed that these fluctuations lift the tails of the
multivariate amplitude distributions, making them heavy-tailed
\cite{non_stationarity_fin_guhr,exact_distributions_guhr}.

Our goal is to use the analytical results for the multivariate distributions of
amplitudes, measured as time series in correlated, non-stationary financial
markets and provide quantitative measures for the degree of non-stationarity in
the correlations using the methodology first proposed in
\cite{non_stationarity_fin_guhr} and extended in
\cite{exact_distributions_guhr}. These amplitudes refer to the stock price
changes for the entire market.

We carry out a detailed data analysis that exposes generic features. Then we
use a random matrix model to explain them. We show that non-stationarity of the
correlations leads to heavy tails in the multivariate return distribution and
finally, we use the approach in
\cite{non_stationarity_fin_guhr,exact_distributions_guhr} to map a
non-invariant situation to an effectively invariant one.

A remarkable feature of the multivariate distributions we use to compare with
the financial data, is that eventually, they are of closed form or involve only
single integrals. Moreover, they use a low number of free parameters: one
measuring how strongly the non-stationary correlations fluctuate, and one or
two shape parameters for the tails. All the other parameters can be directly
measured from the data \cite{exact_distributions_guhr}.

Random matrix models \cite{random_matrix_1,random_matrix_2} fall into two
classes: (I) The ensemble is fictitious. It comes into play via an ergodicity
argument only. (II) The ensemble really exists and can be identified in the
system. The issue of ergodicity does not arise. It is conceptually important
that we here use a random matrix model in class (II) which may be seen as a new
interpretation of the Wishart model and generalizations thereof for random
covariance or correlation matrices \cite{wishart}. In finance there are
numerous random matrix applications
\cite{matrix_fin_01,matrix_fin_02,matrix_fin_03,matrix_fin_04,matrix_fin_05,matrix_fin_06,matrix_fin_07,matrix_fin_08,matrix_fin_09,matrix_fin_10,matrix_fin_11,matrix_fin_12,matrix_fin_13}
including non-Gaussian ensembles. To the best of our knowledge, all of them
fall into class (I) and focus on other observables. The distributions we use
arrive at rather universal and generic results, supporting the view that
non-stationarities can lead to universal features. To facilitate the
reproduction of our results, the source code for the data analysis is available
in Ref. \cite{code}.

The paper is organized as follows: in Sect. \ref{sec:data_set} we present our
data set of stocks. We define the key concepts, present the epochs analysis of
the stock time series and discuss the method used by T. A. Schmitt et al.
\cite{non_stationarity_fin_guhr} in Sect. \ref{sec:epochs}. In Sect.
\ref{sec:simulations} we simulate returns time series to test the rotation and
scaling method and check the influence of the epochs in the results. We define
the general considerations to use the exact multivariate distributions in Sect.
\ref{sec:exact_distributions}. In Sect. \ref{sec:comparison_returns} we compare
the theoretical distributions with the empirical aggregated distribution of
returns. Our conclusions follow in Sect. \ref{sec:conclusion}.