\subsection{PostProcessor}
\label{sec:models_postProcessor}
A Post-Processor (PP) can be considered as an action performed on a set of data
or other type of objects.
%
Most of the post-processors contained in RAVEN, employ a mathematical operation
on the data given as ``input''.
%
RAVEN supports several different types of PPs.

Currently, the following types are available in RAVEN:
\begin{itemize}
  \itemsep0em
  \item \textbf{BasicStatistics}
  \item \textbf{ComparisonStatistics}
  \item \textbf{ImportanceRank}
  \item \textbf{SafestPoint}
  \item \textbf{LimitSurface}
  \item \textbf{LimitSurfaceIntegral}
  \item \textbf{External}
  \item \textbf{TopologicalDecomposition}
  \item \textbf{RavenOutput}
  \item \textbf{DataMining}
  \item \textbf{Metric}
  \item \textbf{CrossValidation}
  \item \textbf{DataClassifier}
  \item \textbf{ValueDuration}
  \item \textbf{FastFourierTransform}
  \item \textbf{SampleSelector}
  %\item \textbf{PrintCSV}
  %\item \textbf{LoadCsvIntoInternalObject}
\end{itemize}

The specifications of these types must be defined within the XML block
\xmlNode{PostProcessor}.
%
This XML node needs to contain the attributes:
\vspace{-5mm}
\begin{itemize}
  \itemsep0em
  \item \xmlAttr{name}, \xmlDesc{required string attribute}, user-defined
  identifier of this post-processor.
  %
  \nb As with other objects, this is the name that can be used to refer to this
  specific entity from other input XML blocks.
  \item \xmlAttr{subType}, \xmlDesc{required string attribute}, defines which of
  the post-processors needs to be used, choosing among the previously reported
  types.
  %
  This choice conditions the subsequent required and/or optional
  \xmlNode{PostProcessor} sub nodes.
  %
\end{itemize}
\vspace{-5mm}

As already mentioned, all the types and meaning of the remaining sub-nodes
depend on the post-processor type specified in the attribute \xmlAttr{subType}.
%
In the following sections the specifications of each type are reported.

%%%%% PP BasicStatistics %%%%%%%
\subsubsection{BasicStatistics}
\label{BasicStatistics}
The \textbf{BasicStatistics} post-processor is the container of the algorithms
to compute many of the most important statistical quantities. It is important to notice that this
post-processor can accept as input both \textit{\textbf{PointSet}} and \textit{\textbf{HistorySet}}
data objects, depending on the type of statistics the user wants to compute:
\begin{itemize}
  \item \textit{\textbf{PointSet}}: Static Statistics;
  \item \textit{\textbf{HistorySet}}: Dynamic Statistics. Depending on a ``pivot parameter'' (e.g. time)
  the post-processor is going to compute the statistics for each value of it (e.g. for each time step).
  In case an \textbf{HistorySet} is provided as Input, the Histories needs to be synchronized (use
    \textit{\textbf{Interfaced}} post-processor of type  \textbf{HistorySetSync}).
\end{itemize}
%
\ppType{BasicStatistics post-processor}{BasicStatistics}
\begin{itemize}
  \item \xmlNode{"metric"}, \xmlDesc{comma separated string or node list, required field},
    specifications for the metric to be calculated.  The name of each node is the requested metric.  There are
    two forms for specifying the requested parameters of the metric.  For scalar values such as
    \xmlNode{expectedValue} and \xmlNode{variance}, the text of the node is a comma-separated list of the
    parameters for which the metric should be calculated.  For matrix values such as \xmlNode{sensitivty} and
    \xmlNode{covariance}, the matrix node requires two sub-nodes, \xmlNode{targets} and \xmlNode{features},
    each of which is a comma-separated list of the targets for which the metric should be calculated, and the
    features for which the metric should be calculated for that target.  See the example below.

    \nb When defining the metrics to use, it is possible to have multiple nodes with the same name.  For
    example, if a problem has inputs $W$, $X$, $Y$, and $Z$, and the responses are $A$, $B$, and $C$, it is possible that
    the desired metrics are the \xmlNode{sensitivity} of $A$ and $B$ to $X$ and $Y$, as well as the
    \xmlNode{sensitivity} of $C$ to $W$ and $Z$, but not the sensitivity of $A$ to $W$.   In this event, two
    copies of the \xmlNode{sensitivity} node are added to the input.  The first has targets $A,B$ and features
    $X,Y$, while the second node has target $C$ and features $W,Z$.  This could reduce some computation effort
    in problems with many responses or inputs.  An example of this is shown below.
  %
  \\ Currently the scalar quantities available for request are:
  \begin{itemize}
    \item \textbf{expectedValue}: expected value or mean
    \item \textbf{minimum}: The minimum value of the samples.
    \item \textbf{maximum}: The maximum value of the samples.
    \item \textbf{median}: median
    \item \textbf{variance}: variance
    \item \textbf{sigma}: standard deviation
    \item \textbf{percentile}: the percentile. If this quantity is inputted as \textit{percentile} the $5\%$ and $95\%$ percentile(s) are going to be computed.
                               Otherwise the user can specify this quantity with a parameter \textit{percent='X'}, where the \textit{X} represents the requested
                               percentile (a floating point value between 0.0 and 100.0)
    \item \textbf{variationCoefficient}: coefficient of variation, i.e. \textbf{sigma}/\textbf{expectedValue}. \nb If the \textbf{expectedValue} is zero,
    the \textbf{variationCoefficient} will be \textbf{INF}.
    \item \textbf{skewness}: skewness
    \item \textbf{kurtosis}: excess kurtosis (also known as Fisher's kurtosis)
    \item \textbf{samples}: the number of samples in the data set used to determine the statistics.
  \end{itemize}
  The matrix quantities available for request are:
  \begin{itemize}
    \item \textbf{sensitivity}: matrix of sensitivity coefficients, computed via linear regression method. (\nb The condition number is computed every time this quantity is requsted. If it results
    to be greater then $30$, a multicollinearity problem exists and the sensitivity coefficients
    might be incorrect and a Warning is spooned by the code)
    \item \textbf{covariance}: covariance matrix
    \item \textbf{pearson}: matrix of correlation coefficients
    \item \textbf{NormalizedSensitivity}: matrix of normalized sensitivity
    coefficients. \nb{It is the matrix of normalized VarianceDependentSensitivity}
    \item \textbf{VarianceDependentSensitivity}: matrix of sensitivity coefficients dependent on the variance of the variables
  \end{itemize}
  This XML node needs to contain the attribute:
  \begin{itemize}
    \itemsep0em
    \item \xmlAttr{prefix}, \xmlDesc{required string attribute}, user-defined prefix for the given \textbf{metric}.
      For scalar quantifies, RAVEN will define a variable with name defined as:  ``prefix'' + ``\_'' + ``parameter name''.
      For example, if we define ``mean'' as the prefix for \textbf{expectedValue}, and parameter ``x'', then variable
      ``mean\_x'' will be defined by RAVEN.
      For matrix quantities, RAVEN will define a variable with name defined as: ``prefix'' + ``\_'' + ``target parameter name'' + ``\_'' + ``feature parameter name''.
      For example, if we define ``sen'' as the prefix for \textbf{sensitivity}, target ``y'' and feature ``x'', then
      variable ``sen\_y\_x'' will be defined by RAVEN.
      \nb These variable will be used by RAVEN for the internal calculations. It is also accessible by the user through
      \textbf{DataObjects} and \textbf{OutStreams}.
  \end{itemize}
   %
  \nb If the weights are present in the system then weighted quantities are calculated automatically. In addition, if a matrix quantity is requested (e.g. Covariance matrix, etc.), only the weights in the output space are going to be used for both input and output space (the computation of the joint probability between input and output spaces is not implemented yet).
  \\
  \nb Certain ROMs provide their own statistical information (e.g., those using
  the sparse grid collocation sampler such as: \xmlString{GaussPolynomialRom}
  and \xmlString{HDMRRom}) which can be obtained by printing the ROM to file
  (xml). For these ROMs, computing the basic statistics on data generated from
  one of these sampler/ROM combinations may not provide the information that the
  user expects.
  \\
  In addition, RAVEN will automatically calculate the standard errors on the following scalar quantities:
  \begin{itemize}
    \item \textbf{expectedValue}
    \item \textbf{median}
    \item \textbf{variance}
    \item \textbf{sigma}
    \item \textbf{skewness}
    \item \textbf{kurtosis}
  \end{itemize}
  RAVEN will define a variable with name defined as: ``prefix for given \textbf{metric}'' + ``\_ste\_'' + ``parameter name'' to
  store standard error of given \textbf{metric} with respect to given parameter. This information will be stored in the DataObjects,
  i.e. \textbf{PointSet} and \textbf{HistorySet}, and by default will be printed out in the ``CSV'' output files by the
  \textbf{OutStreams}. Option node \xmlNode{what} can be used in the \textbf{OutStreams} to select the information that
  the users want to print.
  In the case when the users want to store all the calculations results in general \textbf{DataSets}, RAVEN will employ a variable
  with name defined as: ``\textbf{metric}'' + ``\_ste'' to store standard error with respect to all target parameters. An additional
  index ``target'' will added in the \textbf{DataSets} with respect to these variables. All these quantities will be automatically
  computed and stored in the given \textbf{DataSet}, and the users do not need to specify these quantities in their RAVEN input files.
  %
   \item \xmlNode{pivotParameter}, \xmlDesc{string, optional field}, name of the parameter that needs
   to be used for the computation of the Dynamic BasicStatistics (e.g. time). This node needs to
   be inputted just in case an \textbf{HistorySet} is used as Input. It represents the reference
   monotonic variable based on which the statistics is going to be computed (e.g. time-dependent
   statistical moments).
    \default{None}
  %
  \item \xmlNode{biased}, \xmlDesc{string (boolean), optional field}, if \textit{True} biased
  quantities are going to be calculated, if \textit{False} unbiased.
  \default{False}
  %
  \item \xmlNode{dataset}, \xmlDesc{boolean, optional field}, if \textit{True} \xmlString{DataSet}
    will be used to store the calculation results, if \textit{False} \xmlString{PointSet} or \xmlString{HistorySet}
    will be used to store the calculation results.
    \nb The optional \xmlString{DataSet} is added only to this PostProcessor, one can still use the \xmlString{OutStreams}
    to print the variables available in the \textit{DataSet}. The \xmlString{"metric"} names are used as the
    variable names, i.e. variable names listed in \xmlNode{Input} or \xmlNode{Output} in the defined \xmlString{DataSet}.
    In addition, the extra node \xmlNode{Index} is required, and the value for \xmlAttr{var} can be found in the following:
    \begin{itemize}
      \item scalar metrics, such as \xmlNode{expectedValue} and \xmlNode{variance},
        are requested, the index variable \xmlString{targets} will be required.
      \item vector metrics, such as \xmlNode{covariance} and \xmlNode{sensitivity}, are requested, the index variables
        \xmlString{targets} and \xmlString{features} will be required.
      \item If \xmlNode{percentile} is requested, an additional index variable \xmlString{percent} should be added.
      \item when dynamic BasicStatistics (e.g. time) is requested, the index variable \xmlString{time}  will be required.
    \end{itemize}
  \default{False}
  %
\item \xmlNode{multipleFeatures}, \xmlDesc(boolean, optional field), if \textbf{False}, this node can be used when
    the users want to compute sensitivities based on one target variable with respect to one feature variable,
    i.e. the sensitivity calculations are directly computed using the \textbf{Linear Regression} or
    \textbf{Best Linear Predictor} method with single feature. This method can be useful when the input features
    depend on each other. The default value is \textbf{True}, which means the sensitivity calculations are performed
    using \textbf{Linear Regression} or \textbf{Best Linear Predictor} method with multiple features. If the input
    features are not fully correlated, the default value for \xmlNode{multipleFeatures} is always recommanded.
    \nb this node only affects the calculations of metrics such as \xmlNode{sensitivity},
    \xmlNode{VarianceDependentSensitivity} and \xmlNode{NormalizedSensitivity}.
  \default{True}
\end{itemize}
\textbf{Example (Static Statistics):}  This example demonstrates how to request the expected value of
\xmlString{x01} and \xmlString{x02}, along with the sensitivity of both \xmlString{x01} and \xmlString{x02} to
\xmlString{a} and \xmlString{b}.
\begin{lstlisting}[style=XML,morekeywords={name,subType,debug}]
<Simulation>
  ...
  <Models>
    ...
    <PostProcessor name='aUserDefinedName' subType='BasicStatistics' verbosity='debug'>
      <expectedValue prefix='mean'>x01,x02</expectedValue>
      <sensitivity prefix='sen'>
        <targets>x01,x02</targets>
        <features>a,b</features>
      </sensitivity>
    </PostProcessor>
    ...
  </Models>
  ...
</Simulation>
\end{lstlisting}

In this case, the RAVEN variables ``mean\_x01, mean\_x02, sen\_x01\_a, sen\_x02\_a, sen\_x01\_b, sen\_x02\_b''
will be created and accessible for the RAVEN entities \textbf{DataObjects} and \textbf{OutStreams}.

\textbf{Example (Static, multiple matrix nodes):} This example shows how multiple nodes can specify
particular metrics multiple times to include different target/feature combinations.  This postprocessor
calculates the expected value of $A$, $B$, and $C$, as well as the sensitivity of both $A$ and $B$ to $X$ and
$Y$ as well as the sensitivity of $C$ to $W$ and $Z$.
\begin{lstlisting}[style=XML,morekeywords={name,subType,debug}]
<Simulation>
  ...
  <Models>
    ...
    <PostProcessor name='aUserDefinedName' subType='BasicStatistics' verbosity='debug'>
      <expectedValue prefix='mean'>A,B,C</expectedValue>
      <sensitivity prefix='sen1'>
        <targets>A,B</targets>
        <features>x,y</features>
      </sensitivity>
      <sensitivity prefix='sen2'>
        <targets>C</targets>
        <features>w,z</features>
      </sensitivity>
    </PostProcessor>
    ...
  </Models>
  ...
</Simulation>
\end{lstlisting}
\textbf{Example (Dynamic Statistics):}
\begin{lstlisting}[style=XML,morekeywords={name,subType,debug}]
<Simulation>
  ...
  <Models>
    ...
    <PostProcessor name='aUserDefinedNameForDynamicPP' subType='BasicStatistics' verbosity='debug'>
      <expectedValue prefix='mean'>x01,x02</expectedValue>
      <sensitivity prefix='sen'>
        <targets>x01,x02</targets>
        <features>a,b</features>
      </sensitivity>
      <pivotParameter>time</pivotParameter>
    </PostProcessor>
    ...
  </Models>
  ...
  <HistorySet name='basicStatHistorySet'>
    <Output>
      mean_x01,mean_x02,
      sen_x01_a, sen_x01_b,
      sen_x02_a, sen_x02_b
    </Output>
    <options>
      <pivotParameter>time</pivotParameter>
    </options>
  </HistorySet>
</Simulation>
\end{lstlisting}

\textbf{Example (Dumping the results into DataSet):}
\begin{lstlisting}[style=XML,morekeywords={name,subType,debug}]
<Simulation>
  ...
  <Models>
    ...
    <PostProcessor name='aUserDefinedNameForDynamicPP' subType='BasicStatistics' verbosity='debug'>
      <dataset>True</dataset>
      <expectedValue prefix='mean'>x01,x02</expectedValue>
      <sensitivity prefix='sen'>
        <targets>x01,x02</targets>
        <features>a,b</features>
      </sensitivity>
      <pivotParameter>time</pivotParameter>
    </PostProcessor>
    ...
  </Models>
  ...
  <DataObjects>
    <DataSet name='basicStatDataSet'>
      <Output>expectedValue,sensitivity</Output>
      <Index var='time'>expectedValue,sensitivity</Index>
      <Index var='targets'>expectedValue,sensitivity</Index>
      <Index var='features'>sensitivity</Index>
    </DataSet>
  </DataObjects>
</Simulation>
\end{lstlisting}
%%%%% PP ComparisonStatistics %%%%%%%
\subsubsection{ComparisonStatistics}
\label{ComparisonStatistics}
The \textbf{ComparisonStatistics} post-processor computes statistics
for comparing two different dataObjects.  This is an experimental
post-processor, and it will definitely change as it is further
developed.

There are four nodes that are used in the post-processor.

\begin{itemize}
\item \xmlNode{kind}: specifies information to use for comparing the
  data that is provided.  This takes either uniformBins which makes
  the bin width uniform or equalProbability which makes the number
  of counts in each bin equal.  It can take the following attributes:
  \begin{itemize}
  \item \xmlAttr{numBins} which takes a number that directly
    specifies the number of bins
  \item \xmlAttr{binMethod} which takes a string that specifies the
    method used to calculate the number of bins.  This can be either
    square-root or sturges.
  \end{itemize}
\item \xmlNode{compare}: specifies the data to use for comparison.
  This can either be a normal distribution or a dataObjects:
  \begin{itemize}
  \item \xmlNode{data}: This will specify the data that is used.  The
    different parts are separated by $|$'s.
  \item \xmlNode{reference}: This specifies a reference distribution
    to be used.  It takes distribution to use that is defined in the
    distributions block.  A name parameter is used to tell which
    distribution is used.
  \end{itemize}
\item \xmlNode{fz}: If the text is true, then extra comparison
  statistics for using the $f_z$ function are generated.  These take
  extra time, so are not on by default.
\item \xmlNode{interpolation}: This switches the interpolation used
  for the cdf and the pdf functions between the default of quadratic
  or linear.
\end{itemize}

The \textbf{ComparisonStatistics} post-processor generates a variety
of data.  First for each data provided, it calculates bin boundaries,
and counts the numbers of data points in each bin.  From the numbers
in each bin, it creates a cdf function numerically, and from the cdf
takes the derivative to generate a pdf.  It also calculates statistics
of the data such as mean and standard deviation. The post-processor
can generate a CSV file only.

The post-processor uses the generated pdf and cdf function to
calculate various statistics.  The first is the cdf area difference which is:
\begin{equation}
  cdf\_area\_difference = \int_{-\infty}^{\infty}{\|CDF_a(x)-CDF_b(x)\|dx}
\end{equation}
This given an idea about how far apart the two pieces of data are, and
it will have units of $x$.

The common area between the two pdfs is calculated.  If there is
perfect overlap, this will be 1.0, if there is no overlap, this will
be 0.0.  The formula used is:
\begin{equation}
  pdf\_common\_area = \int_{-\infty}^{\infty}{\min(PDF_a(x),PDF_b(x))}dx
\end{equation}

The difference pdf between the two pdfs is calculated.  This is calculated as:
\begin{equation}
  f_Z(z) = \int_{-\infty}^{\infty}f_X(x)f_Y(x-z)dx
\end{equation}
This produces a pdf that contains information about the difference
between the two pdfs.  The mean can be calculated as (and will be
calculated only if fz is true):
\begin{equation}
  \bar{z} = \int_{-\infty}^{\infty}{z f_Z(z)dz}
\end{equation}
The mean can be used to get an signed difference between the pdfs,
which shows how their means compare.

The variance of the difference pdf can be calculated as (and will be
calculated only if fz is true):
\begin{equation}
  var = \int_{-\infty}^{\infty}{(z-\bar{z})^2 f_Z(z)dz}
\end{equation}

The sum of the difference function is calculated if fz is true, and is:
\begin{equation}
  sum = \int_{-\infty}^{\infty}{f_z(z)dz}
\end{equation}
This should be 1.0, and if it is different that
points to approximations in the calculation.

\textbf{Example:}
\begin{lstlisting}[style=XML]
<Simulation>
   ...
   <Models>
      ...
      <PostProcessor name="stat_stuff" subType="ComparisonStatistics">
      <kind binMethod='sturges'>uniformBins</kind>
      <compare>
        <data>OriData|Output|tsin_TEMPERATURE</data>
        <reference name='normal_410_2' />
      </compare>
      <compare>
        <data>OriData|Output|tsin_TEMPERATURE</data>
        <data>OriData|Output|tsout_TEMPERATURE</data>
      </compare>
      </PostProcessor>
      <PostProcessor name="stat_stuff2" subType="ComparisonStatistics">
        <kind numBins="6">equalProbability</kind>
        <compare>
          <data>OriData|Output|tsin_TEMPERATURE</data>
        </compare>
        <Distribution class='Distributions' type='Normal'>normal_410_2</Distribution>
      </PostProcessor>
      ...
   </Models>
   ...
   <Distributions>
      <Normal name='normal_410_2'>
         <mean>410.0</mean>
         <sigma>2.0</sigma>
      </Normal>
   </Distributions>
</Simulation>
\end{lstlisting}

%%%%% PP ImportanceRank %%%%%%%
\subsubsection{ImportanceRank}
\label{ImportanceRank}
The \textbf{ImportanceRank} post-processor is specifically used
to compute sensitivity indices and importance indices with respect to input parameters
associated with multivariate normal distributions. In addition, the user can also request the transformation
matrix and the inverse transformation matrix when the PCA reduction is used.
%
\ppType{ImportanceRank}{ImportanceRank}
%
\begin{itemize}
  \item \xmlNode{what}, \xmlDesc{comma separated string, required field},
  %
  List of quantities to be computed.
  %
  Currently the quantities available are:
  \begin{itemize}
    \item \xmlString{SensitivityIndex}: used to measure the impact of sensitivities on the model.
    \item \xmlString{ImportanceIndex}: used to measure the impact of sensitivities and input uncertainties on the model.
    \item \xmlString{PCAIndex}: the indices of principal component directions, used to measure the impact
    of principal component directions on input covariance matrix.
    \nb \xmlString{PCAIndex} can be only requested when subnode \xmlNode{latent} is defined in \xmlNode{features}.
    \item \xmlString{transformation}: the transformation matrix used to map the latent variables to the manifest variables in the original input space.
    \item \xmlString{InverseTransformation}: the inverse transformation matrix used to map the manifest variables to the latent variables in the transformed space.
    \item \xmlString{ManifestSensitivity}: the sensitivity coefficients of \xmlNode{target} with respect to \xmlNode{manifest} variables defined in \xmlNode{features}.

    \nb In order to request \xmlString{transformation} matrix or \xmlString{InverseTransformation} matrix or \xmlString{ManifestSensitivity},
    the subnodes \xmlNode{latent} and \xmlNode{manifest} under \xmlNode{features} are required (more details can be found in the following).
    %
  \end{itemize}
  %
  \nb For each computed quantity, RAVEN will define a unique variable name so that the data can be accessible by the users
  through RAVEN entities \textbf{DataObjects} and \textbf{OutStreams}. These variable names are defined as follows:
  \begin{itemize}
    \item \xmlString{SensitivityIndex}: `sensitivityIndex' + `\_' + `targetVariableName' + `\_' + `latentFeatureVariableName'
    \item \xmlString{ImportanceIndex}: `importanceIndex' + `\_' + `targetVariableName' + `\_' + `latentFeatureVariableName'
    \item \xmlString{PCAIndex}: `pcaIndex' + `\_' + `latentFeatureVariableName'
    \item \xmlString{transformation}: `transformation' + `\_' + `manifestFeatureVariableName' + `\_' + `latentFeatureVariableName'
    \item \xmlString{InverseTransformation}: `inverseTransformation' + `\_' + `latentFeatureVariableName' + `\_' + `manifestFeatureVariableName'
    \item \xmlString{ManifestSensitivity}: `manifestSensitivity' + `\_' + `targetVariableName' + `\_' + `manifestFeatureVariableName'
  \end{itemize}
  %
  If all the quantities need to be computed, the user can input in the body of \xmlNode{what} the string \xmlString{all}.
  \nb \xmlString{all} equivalent to \xmlString {SensitivityIndex, ImportanceIndex, PCAIndex}.

  Since the transformation and InverseTransformation matrix can be very large, they are not printed with option \xmlString{all}.
  In order to request the transformation matrix (or inverse transformation matrix) from this post processor,
  the user need to specify \xmlString{transformation} or \xmlString{InverseTransformation} in \xmlNode{what}. In addition,
  both  \xmlNode{manifest} and \xmlNode{latent} subnodes are required and should be defined in node \xmlNode{features}. For example, let $\mathbf{L, P}$ represent
  the transformation and inverse transformation matrices, respectively. We will define vectors $\mathbf x$ as manifest variables and vectors $\mathbf y$
  as latent variables. If a absolute covariance matrix is used in given distribution, the following equation will be used:

  $
  \mathbf{\delta x} = \mathbf L * \mathbf y
  $

  $
  \mathbf y = \mathbf P * \mathbf \delta \mathbf x
  $

  If a relative covariance matrix is used in given distribution, the following equation will be used:

  $
  \frac{\mathbf \delta \mathbf x}{\mathbf \mu} = \mathbf L * \mathbf y
  $

  $
  \mathbf y = \mathbf P * {\frac{\mathbf \delta \mathbf x}{\mathbf \mu}}
  $

  where $\mathbf{\delta x}$ denotes the changes in the input vector $\mathbf x$, and $\mathbf \mu$ denotes the mean values of the input vector $\mathbf x$.

  %
  %
  \item \xmlNode{features}, \xmlDesc{XML node, required parameter}, used to specify the information for the input variables.
  In this xml-node, the following xml sub-nodes need to be specified:
    \begin{itemize}
      \item \xmlNode{manifest},\xmlDesc{XML node, optional parameter}, used to indicate the input variables belongs to the original input space.
      It can accept the following child node:
        \begin{itemize}
          \item \xmlNode{variables},\xmlDesc{comma separated string, required field}, lists manifest variables.
          \item \xmlNode{dimensions}, \xmlDesc{comma separated integer, optional field}, lists the dimensions corresponding to the manifest variables.
          If not provided, the dimensions are determined by the order indices of given manifest variables.
        \end{itemize}
      \item \xmlNode{latent},\xmlDesc{XML node, optional parameter}, used to indicate the input variables belongs to the transformed space.
      It can accept the following child node:
        \begin{itemize}
          \item \xmlNode{variables},\xmlDesc{comma separated string, required field}, lists latent variables.
          \item \xmlNode{dimensions}, \xmlDesc{comma separated integer, optional field}, lists the dimensions corresponding to the latent variables.
          If not provided, the dimensions are determined by the order indices of given latent variables.
        \end{itemize}
      \nb At least one of the subnodes, i.e. \xmlNode{manifest} and \xmlNode{latent} needs to be specified.
    \end{itemize}
  %
  \item \xmlNode{targets}, \xmlDesc{comma separated string, required field}, lists output responses.
  %
  \item \xmlNode{mvnDistribution}, \xmlDesc{string, required field}, specifies the
  multivariate normal distribution name. The \xmlNode{MultivariateNormal} node must be present. It requires two attributes:
    \begin{itemize}
      \item \xmlAttr{class}, \xmlDesc{required string attribute}, is the main
        ``class'' the listed object is from, the only acceptable class for
        this post-processor is \xmlString{Distributions};
      \item \xmlAttr{type}, \xmlDesc{required string attribute}, is the type of distributions,
        the only acceptable type is \xmlString{MultivariateNormal}
    \end{itemize}
\end{itemize}
  %
  %
  Here is an example to show the user how to request the transformation matrix, the inverse transformation matrix, the
  manifest sensitivities and other quantities.
  %

\textbf{Example:}
\begin{lstlisting}[style=XML,morekeywords={name,subType,debug}]
<Simulation>
  ...
  <Models>
    ...
    <PostProcessor name='aUserDefinedName' subType='ImportanceRank'>
      <what>SensitivityIndex,ImportanceIndex,Transformation, InverseTransformation,ManifestSensitivity</what>
      <features>
        <manifest>
          <variables>x1,x2</variables>
          <dimensions>1,2</dimensions>
        </manifest>
        <latent>
          <variables>latent1</variables>
          <dimensions>1</dimensions>
        </latent>
      </features>
      <targets>y</targets>
      <mvnDistribution>MVN</mvnDistribution>
    </PostProcessor>
    ...
  </Models>
  ...
</Simulation>
\end{lstlisting}

The calculation results can be accessible via variables ``sensitivityIndex\_y\_latent1, importanceIndex\_y\_latent1,
manifestSensitivity\_y\_x1, manifestSensitivity\_y\_x2, transformation\_x1\_latent1, transformation\_x2\_latent1,
inverseTransformation\_latnet1\_x1, inverseTransformation\_laent1\_x2'' through RAVEN entities \textbf{DataObjects}
and \textbf{OutStreams}.

%%%%% PP SafestPoint %%%%%%%
\subsubsection{SafestPoint}
\label{SafestPoint}
The \textbf{SafestPoint} post-processor provides the coordinates of the farthest
point from the limit surface that is given as an input.
%
The safest point coordinates are expected values of the coordinates of the
farthest points from the limit surface in the space of the ``controllable''
variables based on the probability distributions of the ``non-controllable''
variables.

The term ``controllable'' identifies those variables that are under control
during the system operation, while the ``non-controllable'' variables are
stochastic parameters affecting the system behaviour randomly.

The ``SafestPoint'' post-processor requires the set of points belonging to the
limit surface, which must be given as an input.
%
The probability distributions as ``Assembler Objects'' are required in the
``Distribution'' section for both ``controllable'' and ``non-controllable''
variables.

The sampling method used by the ``SafestPoint'' is a ``value'' or ``CDF'' grid.
%
At present only the ``equal'' grid type is available.

\ppType{Safest Point}{SafestPoint}

\begin{itemize}
  \item \xmlNode{Distribution}, \xmlDesc{Required}, represents the probability
  distributions of the ``controllable'' and ``non-controllable'' variables.
  %
  These are \textbf{Assembler Objects}, each of these nodes must contain 2
  attributes that are used to identify those within the simulation framework:
        \begin{itemize}
    \item \xmlAttr{class}, \xmlDesc{required string attribute}, is the main
    ``class'' the listed object is from.
                \item \xmlAttr{type}, \xmlDesc{required string attribute}, is the object
    identifier or sub-type.
        \end{itemize}
             \item  \xmlNode{outputName}, \xmlDesc{string, required field}, specifies the name of the output variable where the probability is going to be stored.
               \nb This variable name must be listed in the \xmlNode{Output} field of the Output DataObject
        \item \xmlNode{controllable}, \xmlDesc{XML node, required field},  lists the controllable variables.
  %
  Each variable is associated with its name and the two items below:
        \begin{itemize}
                \item \xmlNode{distribution} names the probability distribution associated
    with the controllable variable.
    %
                \item \xmlNode{grid} specifies the \xmlAttr{type}, \xmlAttr{steps}, and
    tolerance of the sampling grid.
    %
        \end{itemize}
        \item \xmlNode{non-controllable}, \xmlDesc{XML node, required field}, lists the non-controllable variables.
  %
  Each variable is associated with its name and the two items below:
        \begin{itemize}
                \item \xmlNode{distribution} names the probability distribution associated
    with the non-controllable variable.
    %
                \item \xmlNode{grid} specifies the \xmlAttr{type}, \xmlAttr{steps}, and
    tolerance of the sampling grid.
    %
                \end{itemize}
\end{itemize}

\textbf{Example:}
\begin{lstlisting}[style=XML,morekeywords={name,subType,class,type,steps}]
<Simulation>
  ...
    <Models>
    ...
    <PostProcessor name='SP' subType='SafestPoint'>
      <Distribution  class='Distributions'  type='Normal'>x1_dst</Distribution>
      <Distribution  class='Distributions'  type='Normal'>x2_dst</Distribution>
      <Distribution  class='Distributions'  type='Normal'>gammay_dst</Distribution>
      <controllable>
        <variable name='x1'>
          <distribution>x1_dst</distribution>
          <grid type='value' steps='20'>1</grid>
        </variable>
        <variable name='x2'>
          <distribution>x2_dst</distribution>
          <grid type='value' steps='20'>1</grid>
        </variable>
      </controllable>
      <non-controllable>
        <variable name='gammay'>
          <distribution>gammay_dst</distribution>
          <grid type='value' steps='20'>2</grid>
        </variable>
      </non-controllable>
    </PostProcessor>
    ...
  </Models>
  ...
</Simulation>
\end{lstlisting}
%%%%% PP LimitSurface %%%%%%%
\subsubsection{LimitSurface}
\label{LimitSurface}
The \textbf{LimitSurface} post-processor is aimed to identify the transition
zones that determine a change in the status of the system (Limit Surface).

\ppType{LimitSurface}{LimitSurface}

\begin{itemize}
  \item \xmlNode{parameters}, \xmlDesc{comma separated string, required field},
  lists the parameters that define the uncertain domain and from which the LS
  needs to be computed.
  \item \xmlNode{tolerance}, \xmlDesc{float, optional field}, sets the absolute
  value (in CDF) of the convergence tolerance.
 %
  This value defines the coarseness of the evaluation grid.
 %
 \default{1.0e-4}
  \item \xmlNode{side}, \xmlDesc{string, optional field}, in this node the user can specify
  which side of the limit surface needs to be computed. Three options are available:
  \\ \textit{negative},  Limit Surface corresponding to the goal function value of ``-1'';
  \\ \textit{positive}, Limit Surface corresponding to the goal function value of ``1'';
  \\ \textit{both}, either positive and negative Limit Surface is going to be computed.
  %
  %
\default{negative}
  % Assembler Objects
  \item \textbf{Assembler Objects} These objects are either required or optional
  depending on the functionality of the Adaptive Sampler.
  %
  The objects must be listed with a rigorous syntax that, except for the xml
  node tag, is common among all the objects.
  %
  Each of these nodes must contain 2 attributes that are used to map those
  within the simulation framework:
   \begin{itemize}
    \item \xmlAttr{class}, \xmlDesc{required string attribute}, is the main
    ``class'' of the listed object.
    %
    For example, it can be ``Models,'' ``Functions,'' etc.
    \item \xmlAttr{type}, \xmlDesc{required string attribute}, is the object
    identifier or sub-type.
    %
    For example, it can be ``ROM,'' ``External,'' etc.
    %
  \end{itemize}
  The \textbf{LimitSurface} post-processor requires or optionally accepts the
  following objects' types:
   \begin{itemize}
    \item \xmlNode{ROM}, \xmlDesc{string, optional field}, body of this xml
    node must contain the name of a ROM defined in the \xmlNode{Models} block
    (see section \ref{subsec:models_ROM}).
    \item \xmlNode{Function}, \xmlDesc{string, required field}, the body of
    this xml block needs to contain the name of an External Function defined
    within the \xmlNode{Functions} main block (see section \ref{sec:functions}).
    %
    This object represents the boolean function that defines the transition
    boundaries.
    %
    This function must implement a method called
    \textit{\_\_residuumSign(self)}, that returns either -1 or 1, depending on
    the system conditions (see section \ref{sec:functions}).
    %
    \end{itemize}
\end{itemize}

\textbf{Example:}
\begin{lstlisting}[style=XML,morekeywords={name,subType,debug,class,type}]
<Simulation>
 ...
 <Models>
  ...
    <PostProcessor name="computeLimitSurface" subType='LimitSurface' verbosity='debug'>
      <parameters>x0,y0</parameters>
      <ROM class='Models' type='ROM'>Acc</ROM>
      <!-- Here, you can add a ROM defined in Models block.
           If it is not Present, a nearest neighbor algorithm
           will be used.
       -->
      <Function class='Functions' type='External'>
        goalFunctionForLimitSurface
      </Function>
    </PostProcessor>
    ...
  </Models>
  ...
</Simulation>
\end{lstlisting}

%%%%% PP LimitSurfaceIntegral %%%%%%%

\subsubsection{LimitSurfaceIntegral}
\label{LimitSurfaceIntegral}
The \textbf{LimitSurfaceIntegral} post-processor is aimed to compute the likelihood (probability) of the event, whose boundaries are
represented by the Limit Surface (either from the LimitSurface post-processor or Adaptive sampling strategies).
The inputted Limit Surface needs to be, in the  \textbf{PostProcess} step, of type  \textbf{PointSet} and needs to contain
both boundary sides (-1.0, +1.0).
%\\ The \textbf{LimitSurfaceIntegral} post-processor accepts as outputs both files (CSV) and/or  \textbf{PointSet}s.
\\ The \textbf{LimitSurfaceIntegral} post-processor accepts as output  \textbf{PointSet}s only.

\ppType{LimitSurfaceIntegral}{LimitSurfaceIntegral}
\begin{itemize}
\item \variableDescription
 \variableChildIntro
 \begin{itemize}
     \item  \xmlNode{outputName}, \xmlDesc{string, required field}, specifies the name of the output variable where the probability is going to be stored.
               \nb This variable name must be listed in the \xmlNode{Output} field of the Output DataObject
    \item   \xmlNode{distribution}, \xmlDesc{string,
               optional field}, name of the distribution that is associated to this variable.
              Its name needs to be contained in the \xmlNode{Distributions} block explained
              in Section \ref{sec:distributions}. If this node is not present, the  \xmlNode{lowerBound}
              and  \xmlNode{upperBound} XML nodes must be inputted. It requires the following two attributes:
            \begin{itemize}
              \item \xmlAttr{class}, \xmlDesc{required string attribute}, is the main
              ``class'' the listed object is from, the only acceptable class for
              this post-processor is \xmlString{Distributions};
              \item \xmlAttr{type}, \xmlDesc{required string attribute}, is the type of distributions,
                i.e. Normal, Uniform.
            \end{itemize}
   \item   \xmlNode{lowerBound}, \xmlDesc{float,
               optional field}, lower limit of integration domain for this dimension (variable).
               If this node is not present, the  \xmlNode{distribution} XML node must be inputted.
   \item   \xmlNode{upperBound}, \xmlDesc{float,
               optional field}, upper limit of integration domain for this dimension (variable).
               If this node is not present, the  \xmlNode{distribution} XML node must be inputted.
  \end{itemize}

    \item  \xmlNode{tolerance}, \xmlDesc{float, optional field}, specifies the tolerance for
               numerical integration confidence.
                \default{1.0e-4}
     \item  \xmlNode{integralType}, \xmlDesc{string, optional field}, specifies the type of integrations that
                need to be used. Currently only MonteCarlo integration is available
                \default{MonteCarlo}
     \item  \xmlNode{seed}, \xmlDesc{integer, optional field}, specifies the random number generator seed.
                \default{20021986}
     \item  \xmlNode{target}, \xmlDesc{string, optional field}, specifies the target name that represents
                the $f\left ( \bar{x} \right )$ that needs to be integrated.
                \default{last output found in the inputted PointSet}
\end{itemize}

\textbf{Example:}
\begin{lstlisting}[style=XML,morekeywords={name,subType,debug,class,type}]
<Simulation>
 ...
 <Models>
  ...
    <PostProcessor name="LimitSurfaceIntegralDistributions" subType='LimitSurfaceIntegral'>
        <tolerance>0.0001</tolerance>
        <integralType>MonteCarlo</integralType>
        <seed>20021986</seed>
        <target>goalFunctionOutput</target>
        <outputName>EventProbability</outputName>
        <variable name='x0'>
          <distribution>x0_distrib</distribution>
        </variable>
        <variable name='y0'>
          <distribution>y0_distrib</distribution>
        </variable>
    </PostProcessor>
    <PostProcessor name="LimitSurfaceIntegralLowerUpperBounds" subType='LimitSurfaceIntegral'>
        <tolerance>0.0001</tolerance>
        <integralType>MonteCarlo</integralType>
        <seed>20021986</seed>
        <target>goalFunctionOutput</target>
        <outputName>EventProbability</outputName>
        <variable name='x0'>
          <lowerBound>-2.0</lowerBound>
          <upperBound>12.0</upperBound>
        </variable>
        <variable name='y0'>
            <lowerBound>-1.0</lowerBound>
            <upperBound>11.0</upperBound>
        </variable>
    </PostProcessor>
    ...
  </Models>
  ...
</Simulation>
\end{lstlisting}

%%%%% PP External %%%%%%%
\subsubsection{External}
\label{External}
The \textbf{External} post-processor will execute an arbitrary python function
defined externally using the \textit{Functions} interface (see
Section~\ref{sec:functions} for more details).
%

\ppType{External}{External}

\begin{itemize}
  \item \xmlNode{method}, \xmlDesc{comma separated string, required field},
  lists the method names of an external Function that will be computed (each
  returning a post-processing value). \nb New variable names will be defined as:
  ``Function Name in this post-processor'' + ``\_`` + ``variable name in XML
  node \xmlNode{method}''. These new varialbes will be used to store the computed
  values from the list of methods, and can be accessed by the users through RAVEN
  entities \textbf{DataObjects} and \textbf{OutStreams}.
  \item \xmlNode{Function}, \xmlDesc{xml node, required string field}, specifies
  the name of a Function where the \textit{methods} listed above are defined.
  %
  \nb This name should match one of the Functions defined in the
  \xmlNode{Functions} block of the input file.
  %
  The objects must be listed with a rigorous syntax that, except for the XML
  node tag, is common among all the objects.
  %
  Each of these sub-nodes must contain 2 attributes that are used to map them
  within the simulation framework:

   \begin{itemize}
     \item \xmlAttr{class}, \xmlDesc{required string attribute}, is the main
     ``class'' the listed object is from, the only acceptable class for
     this post-processor is \xmlString{Functions};
     \item \xmlAttr{type}, \xmlDesc{required string attribute}, is the object
     identifier or sub-type, the only acceptable type for this post-processor is
     \xmlString{External}.
  \end{itemize}
\end{itemize}

  This Post-Processor accepts as Input/Output both \xmlString{PointSet} and \xmlString{HistorySet}:
   \begin{itemize}
    \item If a \xmlString{PointSet}  is used as Input, the parameters are passed in the external  \xmlString{Function}
  as numpy arrays. The methods' return type must be either a new array or a scalar. In the following it is reported an example
  with two methods, one that returns a scalar and the other one that returns an array:
      \begin{lstlisting}[language=python]
import numpy as np
def sum(self):
  return np.sum(self.aParameterInPointSet)

def sumTwoArraysAndReturnAnotherone(self):
  return self.aParamInPointSet1+self.aParamInPointSet2
      \end{lstlisting}
    \item If a \xmlString{HistorySet}  is used as Input, the parameters are passed in the external  \xmlString{Function}
     as a list of numpy arrays. The methods' return type must be either a new list of arrays (if the Output is another
     \xmlString{HistorySet}), a scalar or a single array (if the  Output is  \xmlString{PointSet} . In the following it
     is reported an example
     with two methods, one that returns a new list of arrays (Output = HistorySet) and the other one that returns an array (Output =
     PointSet):
      \begin{lstlisting}[language=python]
import numpy as np
def newHistorySetParameter(self):
  x = []*len(self.time)
  for history in range(len(self.time)):
    for ts in range(len(self.time[history])):
      if self.time[history][ts] >= 0.001: break
    x[history] = self.x[history][ts:]
  return x

def aNewPointSetParameter(self):
  x = []*len(self.time)
  for history in range(len(self.time)):
    x[history] = self.x[history][-1]
  return x
      \end{lstlisting}
   \end{itemize}

\textbf{Example:}
\begin{lstlisting}[style=XML,morekeywords={subType,debug,name,class,type}]
<Simulation>
  ...
  <Models>
    ...
    <PostProcessor name="externalPP" subType='External' verbosity='debug'>
      <method>Delta,Sum</method>
      <Function class='Functions' type='External'>operators</Function>
        <!-- Here, you can add a Function defined in the
             Functions block. This should be present or
             else RAVEN will not know where to find the
             defined methods. -->
    </PostProcessor>
    ...
  </Models>
  ...
</Simulation>
\end{lstlisting}

\nb The calculation results from this post-processor are stored in the internal variables. These variables
are accessible by the users through RAVEN entities \textbf{DataObjects} and \textbf{OutStreams}. The names
of these variables are defined as: ``Function Name in this post-processor'' + ``\_`` + ``variable name in XML
node \xmlNode{method}''. For example, in previous case, variables ``operators\_Delta'' and ``operators\_Sum''
are defined by RAVEN to store the outputs of this post-processor.

%%%%% PP TopologicalDecomposition %%%%%%%
\subsubsection{TopologicalDecomposition}
\label{TopologicalDecomposition}
The \textbf{TopologicalDecomposition} post-processor will compute an
approximated hierarchical Morse-Smale complex which will add two columns to a
dataset, namely \texttt{minLabel} and \texttt{maxLabel} that can be used to
decompose a dataset.
%

The topological post-processor can also be run in `interactive' mode, that is
by passing the keyword \texttt{interactive} to the command line of RAVEN's
driver.
%
In this way, RAVEN will initiate an interactive UI that allows one to explore
the topological hierarchy in real-time and adjust the simplification setting
before adjusting a dataset. Use in interactive mode will replace the parameter
\xmlNode{simplification} described below with whatever setting is set in the UI
upon exiting it.

In order to use the \textbf{TopologicalDecomposition} post-processor, the user
needs to set the attribute \xmlAttr{subType}:
\xmlNode{PostProcessor \xmlAttr{subType}=\xmlString{TopologicalDecomposition}}.
The following is a list of acceptable sub-nodes:
\begin{itemize}
  \item \xmlNode{graph} \xmlDesc{, string, optional field}, specifies the type
  of neighborhood graph used in the algorithm, available options are:
  \begin{itemize}
    \item \texttt{beta skeleton}
    \item \texttt{relaxed beta skeleton}
    \item \texttt{approximate knn}
    %\item Delaunay \textit{(disabled)}
  \end{itemize}
  \default{\texttt{beta skeleton}}
  \item \xmlNode{gradient}, \xmlDesc{string, optional field}, specifies the
  method used for estimating the gradient, available options are:
  \begin{itemize}
    \item \texttt{steepest}
    %\item \xmlString{maxflow} \textit{(disabled)}
  \end{itemize}
  \default{\texttt{steepest}}
  \item \xmlNode{beta}, \xmlDesc{float in the range: (0,2], optional field}, is
  only used when the \xmlNode{graph} is set to \texttt{beta skeleton} or
  \texttt{relaxed beta skeleton}.
  \default{1.0}
  \item \xmlNode{knn}, \xmlDesc{integer, optional field}, is the number of
  neighbors when using the \xmlString{approximate knn} for the \xmlNode{graph}
  sub-node and used to speed up the computation of other graphs by using the
  approximate knn graph as a starting point for pruning. -1 means use a fully
  connected graph.
  \default{-1}
  \item \xmlNode{weighted}, \xmlDesc{boolean, optional}, a flag that specifies
  whether the regression models should be probability weighted.
  \default{False}
  \item \xmlNode{interactive}, if this node is present \emph{and} the user has
  specified the keyword \texttt{interactive} at the command line, then this will
  initiate a graphical interface for exploring the different simplification
  levels of the topological hierarchy. Upon exit of the graphical interface, the
  specified simplification level will be updated to use the last value of the
  graphical interface before writing any ``output'' results.
  \item \xmlNode{persistence}, \xmlDesc{string, optional field}, specifies how
  to define the hierarchical simplification by assigning a value to each local
  minimum and maximum according to the one of the strategy options below:
  \begin{itemize}
    \item \texttt{difference} - The function value difference between the
    extremum and its closest-valued neighboring saddle.
    \item \texttt{probability} - The probability integral computed as the
    sum of the probability of each point in a cluster divided by the count of
    the cluster.
    \item \texttt{count} - The count of points that flow to or from the
    extremum.
    % \item \xmlString{area} - The area enclosed by the manifold that flows to
    % or from the extremum.
  \end{itemize}
  \default{\texttt{difference}}
  \item \xmlNode{simplification}, \xmlDesc{float, optional field}, specifies the
  amount of noise reduction to apply before returning labels.
  \default{0}
  \item \xmlNode{parameters}, \xmlDesc{comma separated string, required field},
  lists the parameters defining the input space.
  \item \xmlNode{response}, \xmlDesc{string, required field}, is a single
  variable name defining the scalar output space.
\end{itemize}
\textbf{Example:}
\begin{lstlisting}[style=XML,morekeywords={subType}]
<Simulation>
  ...
  <Models>
    ...
    <PostProcessor name="***" subType='TopologicalDecomposition'>
      <graph>beta skeleton</graph>
      <gradient>steepest</gradient>
      <beta>1</beta>
      <knn>8</knn>
      <normalization>None</normalization>
      <parameters>X,Y</parameters>
      <response>Z</response>
      <weighted>true</weighted>
      <simplification>0.3</simplification>
      <persistence>difference</persistence>
    </PostProcessor>
    ...
  <Models>
  ...
<Simulation>
\end{lstlisting}

%%%%% PP DataMining %%%%%%%
\input{DataMining.tex}

%%%%% PP PrintCSV %%%%%%%
%\paragraph{PrintCSV}
%\label{PrintCSV}
%TO BE MOVED TO STEP ``IOSTEP''
%%%%% PP LoadCsvIntoInternalObject %%%%%%%
%\paragraph{LoadCsvIntoInternalObject}
%\label{LoadCsvIntoInternalObject}
%TO BE MOVED TO STEP ``IOSTEP''
%

%%%%% PP External %%%%%%%
\subsubsection{Interfaced}
\label{Interfaced}
The \textbf{Interfaced} post-processor is a Post-Processor that allows the user
to create its own Post-Processor. While the External Post-Processor (see
Section~\ref{External} allows the user to create case-dependent
Post-Processors, with this new class the user can create new general
purpose Post-Processors.
%

\ppType{Interfaced}{Interfaced}

\begin{itemize}
  \item \xmlNode{method}, \xmlDesc{comma separated string, required field},
  lists the method names of a method that will be computed (each
  returning a post-processing value). All available methods need to be included
  in the ``/raven/framework/PostProcessorFunctions/'' folder
\end{itemize}

\textbf{Example:}
\begin{lstlisting}[style=XML,morekeywords={subType,debug,name,class,type}]
<Simulation>
  ...
  <Models>
    ...
    <PostProcessor name="example" subType='InterfacedPostProcessor'verbosity='debug'>
       <method>testInterfacedPP</method>
       <!--Here, the xml nodes required by the chosen method have to be
       included.
        -->
    </PostProcessor>
    ...
  </Models>
  ...
</Simulation>
\end{lstlisting}

All the \textbf{Interfaced} post-processors need to be contained in the
``/raven/framework/PostProcessorFunctions/'' folder. In fact, once the
\textbf{Interfaced} post-processor is defined in the RAVEN input file, RAVEN
search that the method of the post-processor is located in such folder.

The class specified in the \textbf{Interfaced} post-processor has to inherit the
PostProcessorInterfaceBase class and the user must specify this set of
methods:
\begin{itemize}
  \item initialize: in this method, the internal parameters of the
  post-processor are initialized. Mandatory variables that needs to be
  specified are the following:
\begin{itemize}
  \item self.inputFormat: type of dataObject expected in input
  \item self.outputFormat: type of dataObject generated in output
\end{itemize}
  \item readMoreXML: this method is in charge of reading the PostProcessor xml
  node, parse it and fill the PostProcessor internal variables.
  \item run: this method performs the desired computation of the dataObject.
\end{itemize}

\begin{lstlisting}[language=python]
from PostProcessorInterfaceBaseClass import PostProcessorInterfaceBase
class testInterfacedPP(PostProcessorInterfaceBase):
  def initialize(self)
  def readMoreXML(self,xmlNode)
  def run(self,inputDic)
\end{lstlisting}

\paragraph{Data Format}
The user is not allowed to modify directly the DataObjects, however the
content of the DataObjects is available in the form of a python dictionary.
Both the dictionary give in input and the one generated in the output of the
PostProcessor are structured as follows:

\begin{lstlisting}[language=python]
inputDict = {'data':{}, 'metadata':{}}
\end{lstlisting}

where:

\begin{lstlisting}[language=python]
inputDict['data'] = {'input':{}, 'output':{}}
\end{lstlisting}

In the input dictonary, each input variable is listed as a dictionary that
contains a numpy array with its own values as shown below for a simplified
example

\begin{lstlisting}[language=python]
inputDict['data']['input'] = {'inputVar1': array([ 1.,2.,3.]),
                              'inputVar2': array([4.,5.,6.])}
\end{lstlisting}

Similarly, if the dataObject is a PointSet then the output dictionary is
structured as follows:

\begin{lstlisting}[language=python]
inputDict['data']['output'] = {'outputVar1': array([ .1,.2,.3]),
                               'outputVar2':array([.4,.5,.6])}
\end{lstlisting}

Howevers, if the dataObject is a HistorySet then the output dictionary is
structured as follows:

\begin{lstlisting}[language=python]
inputDict['data']['output'] = {'hist1': {}, 'hist2':{}}
\end{lstlisting}

where

\begin{lstlisting}[language=python]
inputDict['output']['data'][hist1] = {'time': array([ .1,.2,.3]),
                              'outputVar1':array([ .4,.5,.6])}
inputDict['output']['data'][hist2] = {'time': array([ .1,.2,.3]),
                              'outputVar1':array([ .14,.15,.16])}
\end{lstlisting}

\paragraph{Method: HStoPSOperator}

This Post-Processor performs the conversion from HistorySet to PointSet performing a projection of the output space.

In the \xmlNode{PostProcessor} input block, the following XML sub-nodes are available:

\begin{itemize}
   \item \xmlNode{pivotParameter}, \xmlDesc{string, optional field}, ID of the temporal variable. Default is ``time''.
   \nb Used just in case the  \xmlNode{pivotValue}-based operation  is requested
    \item \xmlNode{operator}, \xmlDesc{string, optional field}, the operation to perform on the output space:
      \begin{itemize}
        \item \textbf{min}, compute the minimum of each variable along each single history
         \item \textbf{max}, compute the maximum of each variable along each single history
         \item \textbf{average}, compute the average of each variable along each single history
       \end{itemize}
        \nb This node can be inputted only if \xmlNode{pivotValue} and \xmlNode{row} are not present
     \item \xmlNode{pivotValue}, \xmlDesc{float, optional field}, the value of the pivotParameter with respect to the other outputs need to be extracted.
       \nb This node can be inputted only if \xmlNode{operator} and \xmlNode{row} are not present
     \item \xmlNode{pivotStrategy}, \xmlDesc{string, optional field}, The strategy to use for the pivotValue:
       \begin{itemize}
        \item \textbf{nearest}, find the value that is the nearest with respect the \xmlNode{pivotValue}
        \item \textbf{floor}, find the value that is the nearest with respect to the \xmlNode{pivotValue} but less then the  \xmlNode{pivotValue}
        \item \textbf{celing}, find the value that is the nearest with respect to the \xmlNode{pivotValue} but greater then the  \xmlNode{pivotValue}
        \item \textbf{interpolate}, if the exact  \xmlNode{pivotValue}  can not be found, interpolate using a linear approach
       \end{itemize}

       \nb Valid just in case \xmlNode{pivotValue} is present
     \item \xmlNode{row}, \xmlDesc{int, optional field}, the row index at which the outputs need to be extracted.
       \nb This node can be inputted only if \xmlNode{operator} and \xmlNode{pivotValue} are not present
\end{itemize}

This example will show how the XML input block would look like:

\begin{lstlisting}[style=XML,morekeywords={subType,debug,name,class,type}]
<Simulation>
  ...
  <Models>
    ...
    <PostProcessor name="HStoPSperatorRows" subType="InterfacedPostProcessor">
      <method>HStoPSOperator</method>
      <row>-1</row>
    </PostProcessor>
    <PostProcessor name="HStoPSoperatorPivotValues" subType="InterfacedPostProcessor">
        <method>HStoPSOperator</method>
        <pivotParameter>time</pivotParameter>
        <pivotValue>0.3</pivotValue>
    </PostProcessor>
    <PostProcessor name="HStoPSoperatorOperatorMax" subType="InterfacedPostProcessor">
        <method>HStoPSOperator</method>
        <pivotParameter>time</pivotParameter>
        <operator>max</operator>
    </PostProcessor>
    <PostProcessor name="HStoPSoperatorOperatorMin" subType="InterfacedPostProcessor">
        <method>HStoPSOperator</method>
        <pivotParameter>time</pivotParameter>
        <operator>min</operator>
    </PostProcessor>
    <PostProcessor name="HStoPSoperatorOperatorAverage" subType="InterfacedPostProcessor">
        <method>HStoPSOperator</method>
        <pivotParameter>time</pivotParameter>
        <operator>average</operator>
    </PostProcessor>
    ...
  </Models>
  ...
</Simulation>
\end{lstlisting}

\paragraph{Method: HistorySetSampling}
This Post-Processor performs the conversion from HistorySet to HistorySet
The conversion is made so that each history H is re-sampled accordingly  to a
specific sampling strategy.
It can be used to reduce the amount of space required by the HistorySet.

In the \xmlNode{PostProcessor} input block, the following XML sub-nodes are required,
independent of the \xmlAttr{subType} specified:

\begin{itemize}
   \item \xmlNode{samplingType}, \xmlDesc{string, required field}, specifies the type of sampling method to be used (uniform, firstDerivative secondDerivative, filteredFirstDerivative or
   filteredSecondDerivative).
   \item \xmlNode{numberOfSamples}, \xmlDesc{integer, optional field}, number of samples (required only for the following sampling types: uniform, firstDerivative secondDerivative)
   \item \xmlNode{pivotParameter}, \xmlDesc{string, required field}, ID of the temporal variable
   \item \xmlNode{interpolation}, \xmlDesc{string, optional field}, type of interpolation to be employed for the history recostruction (required only for the following sampling types: uniform,
   firstDerivative secondDerivative). Valid types of interpolation to specified: linear, nearest, zero, slinear, quadratic, cubic, intervalAverage;
   \item \xmlNode{tolerance}, \xmlDesc{string, optional field}, tolerance level (required only for the following sampling types: filteredFirstDerivative or filteredSecondDerivative)
\end{itemize}

\paragraph{Method: HistorySetSync}
This Post-Processor performs the conversion from HistorySet to HistorySet
The conversion is made so that all histories are synchronized in time.
It can be used to allow the histories to be sampled at the same time instant.

There are two possible synchronization methods, specified through the \xmlNode{syncMethod} node.  If the
\xmlNode{syncMethod} is \xmlString{grid}, a \xmlNode{numberOfSamples} node is specified,
which yields an equally-spaced grid of time points. The output values for these points will be linearly derived
using nearest sampled time points, and the new HistorySet will contain only the new grid points.

The other methods are used by specifying \xmlNode{syncMethod} as \xmlString{all}, \xmlString{min}, or
\xmlString{max}.  For \xmlString{all}, the postprocessor will iterate through the
existing histories, collect all the time points used in any of them, and use these as the new grid on which to
establish histories, retaining all the exact original values and interpolating linearly where necessary.
In the event of \xmlString{min} or \xmlString{max}, the postprocessor will find the smallest or largest time
history, respectively, and use those time values as nodes to interpolate between.

In the \xmlNode{PostProcessor} input block, the following XML sub-nodes are required,
independent of the \xmlAttr{subType} specified:

\begin{itemize}
   \item \xmlNode{pivotParameter}, \xmlDesc{string, required field}, ID of the temporal variable
   \item \xmlNode{extension}, \xmlDesc{string, required field}, type of extension when the sync process goes outside the boundaries of the history (zeroed or extended)
   \item \xmlNode{syncMethod}, \xmlDesc{string, required field}, synchronization strategy to employ (see
     description above).  Options are \xmlString{grid}, \xmlString{all}, \xmlString{max}, \xmlString{min}.
   \item \xmlNode{numberOfSamples}, \xmlDesc{integer, optional field}, required if \xmlNode{syncMethod} is
     \xmlString{grid}, number of new time samples
\end{itemize}

\paragraph{Method: HistorySetSnapShot}
This Post-Processor performs the conversion from HistorySet to PointSet
The conversion is made so that each history H is converted to a single point P.
There are several methods that can be employed to choose the single point from the history:
\begin{itemize}
  \item min: Take a time slice when the \xmlNode{pivotVar} is at its smallest value,
  \item max: Take a time slice when the \xmlNode{pivotVar} is at its largest value,
  \item average: Take a time slice when the \xmlNode{pivotVar} is at its time-weighted average value,
  \item value: Take a time slice when the \xmlNode{pivotVar} \emph{first passes} its specified value,
  \item timeSlice: Take a time slice index from the sampled time instance space.
\end{itemize}
To demonstrate the timeSlice, assume that each history H is a dict of n output variables $x_1=[...],
x_n=[...]$, then the resulting point P is at time instant index t: $P=[x_1[t],...,x_n[t]]$.

Choosing one the these methods for the \xmlNode{type} node will take a time slice for all the variables in the
output space based on the provided parameters.  Alternatively, a \xmlString{mixed} type can be used, in which
each output variable can use a different time slice parameter.  In other words, you can take the max of one
variable while taking the minimum of another, etc.

In the \xmlNode{PostProcessor} input block, the following XML sub-nodes are required,
independent of the \xmlAttr{subType} specified:

\begin{itemize}
  \item \xmlNode{type}, \xmlDesc{string, required field}, type of operation: \xmlString{min}, \xmlString{max},
                        \xmlString{average}, \xmlString{value}, \xmlString{timeSlice}, or \xmlString{mixed}
   \item \xmlNode{extension}, \xmlDesc{string, required field}, type of extension when the sync process goes outside the boundaries of the history (zeroed or extended)
   \item \xmlNode{pivotParameter}, \xmlDesc{string, optional field}, name of the temporal variable.  Required for the
     \xmlString{average} and \xmlString{timeSlice} methods.
\end{itemize}

If a \xmlString{timeSlice} type is in use, the following nodes also are required:
\begin{itemize}
   \item \xmlNode{timeInstant}, \xmlDesc{integer, required field}, required and only used in the
     \xmlString{timeSlice} type.  Location of the time slice (integer index)
   \item \xmlNode{numberOfSamples}, \xmlDesc{integer, required field}, number of samples
\end{itemize}

If instead a \xmlString{min}, \xmlString{max}, \xmlString{average}, or \xmlString{value} is used, the following nodes
are also required:
\begin{itemize}
   \item \xmlNode{pivotVar}, \xmlDesc{string, required field},  Name of the chosen indexing variable (the
         variable whose min, max, average, or value is used to determine the time slice)
       \item \xmlNode{pivotVal}, \xmlDesc{float, optional field},  required for \xmlString{value} type, the value for the chosen variable
\end{itemize}

Lastly, if a \xmlString{mixed} approach is used, the following nodes apply:
\begin{itemize}
  \item \xmlNode{max}, \xmlDesc{string, optional field}, the names of variables whose output should be their
    own maximum value within the history.
  \item \xmlNode{min}, \xmlDesc{string, optional field}, the names of variables whose output should be their
    own minimum value within the history.
  \item \xmlNode{average}, \xmlDesc{string, optional field}, the names of variables whose output should be their
    own average value within the history. Note that a \xmlNode{pivotParameter} node is required to perform averages.
  \item \xmlNode{value}, \xmlDesc{string, optional field}, the names of variables whose output should be taken
    at a time slice determined by another variable.  As with the non-mixed \xmlString{value} type, the first
    time the \xmlAttr{pivotVar} crosses the specified \xmlAttr{pivotVal} will be the time slice taken.
    This node requires two attributes, if used:
    \begin{itemize}
      \item \xmlAttr{pivotVar}, \xmlDesc{string, required field}, the name of the variable on which the time
        slice will be performed.  That is, if we want the value of $y$ when $t=0.245$,
        this attribute would be \xmlString{t}.
      \item \xmlAttr{pivotVal}, \xmlDesc{float, required field}, the value of the \xmlAttr{pivotVar} on which the time
        slice will be performed.  That is, if we want the value of $y$ when $t=0.245$,
        this attribute would be \xmlString{0.245}.
    \end{itemize}
  Note that all the outputs of the \xmlNode{DataObject} output of this postprocessor must be listed under one
  of the \xmlString{mixed} node types in order for values to be returned.
\end{itemize}

\textbf{Example (mixed):}
This example will output the average value of $x$ for $x$, the value of $y$ at
time$=0.245$ for $y$, and the value of $z$ at $x=4.0$ for $z$.
\begin{lstlisting}[style=XML,morekeywords={subType,debug,name,class,type}]
<Simulation>
  ...
  <Models>
    ...
    <PostProcessor name="mampp2" subType="InterfacedPostProcessor">
      <method>HistorySetSnapShot</method>
      <type>mixed</type>
      <average>x</average>
      <value pivotVar="time" pivotVal="0.245">y</value>
      <value pivotVar="x" pivotVal="4.0">z</value>
      <pivotParameter>time</pivotParameter>
      <extension>zeroed</extension>
    </PostProcessor>
    ...
  </Models>
  ...
</Simulation>
\end{lstlisting}

\paragraph{Method: HSPS}

This Post-Processor performs the conversion from HistorySet to PointSet
The conversion is made so that each history H is converted to a single point P.
Assume that each history H is a dict of n output variables $x_1=[...],x_n=[...]$, then the resulting point P is as follows; $P=[x_1,...,x_n]$
Note: it is here assumed that all histories have been sync so that they have the same length, start point and end point. If you are not sure, do a pre-processing the the original history set.

In the \xmlNode{PostProcessor} input block, the following XML sub-nodes are required,
independent of the \xmlAttr{subType} specified (min, max, avg and value case):

\begin{itemize}
   \item \xmlNode{pivotParameter}, \xmlDesc{string, optional field}, ID of the temporal variable (only for avg)
\end{itemize}

\paragraph{Method: TypicalHistoryFromHistorySet}
This Post-Processor performs a simplified procedure of \cite{wilcox2008users} to form a ``typical'' time series from multiple time series. The input should be a HistorySet, with each history in the HistorySet synchronized. For HistorySet that is not synchronized, use Post-Processor method \textbf{HistorySetSync}  to synchronize the data before running this method.

Each history in input HistorySet is first converted to multiple histories each has maximum time specified in \xmlNode{outputLen} (see below). Each converted history $H_i$ is divided into a set of subsequences $\{H_i^j\}$, and the division is guided by the \xmlNode{subseqLen} node specified in the input XML. The value of \xmlNode{subseqLen} should be a list of positive numbers that specify the length of each subsequence. If the number of subsequence for each history is more than the number of values given in \xmlNode{subseqLen}, the values in \xmlNode{subseqLen} would be reused.

For each variable $x$, the method first computes the empirical CDF (cumulative density function) by using all the data values of $x$ in the HistorySet. This CDF is termed as long-term CDF for $x$. Then for each subsequence $H_i^j$, the method computes the empirical CDF by using all the data values of $x$ in $H_i^j$. This CDF is termed as subsequential CDF. For the first interval window (i.e., $j=1$), the method computes the Finkelstein-Schafer (FS) statistics \cite{finkelstein1971improved} between the long term CDF and the subsequential CDF of $H_i^1$ for each $i$. The FS statistics is defined as following.
\begin{align*}
FS & = \sum_x FS_x\\
FS_x &= \frac{1}{N}\sum_{n=1}^N\delta_n
\end{align*}
where $N$ is the number of value reading in the empirical CDF and $\delta_n$ is the absolute difference between the long term CDF and the subsequential CDF at value $x_n$. The subsequence $H_i^1$ with minimal FS statistics will be selected as the typical subsequence for the interval window $j=1$. Such process repeats for $j=2,3,\dots$ until all subsequences have been processed. Then all the typical subsequences will be concatenated to form a complete history.

In the \xmlNode{PostProcessor} input block, the following XML sub-nodes are required,
independent of the \xmlAttr{subType} specified:

\begin{itemize}
   \item \xmlNode{pivotParameter}, \xmlDesc{string, optional field}, ID of the temporal variable
   \default{Time}
   \item \xmlNode{subseqLen}, \xmlDesc{integers, required field}, length of the divided subsequence (see above)
   \item \xmlNode{outputLen}, \xmlDesc{integer, optional field}, maximum value of the temporal variable for the generated typical history
   \default{Maximum value of the variable with name of \xmlNode{pivotParameter}}
\end{itemize}

For example, consider history of data collected over three years in one-second increments,
where the user wants a single \emph{typical year} extracted from the data.
The user wants this data constructed by combining twelve equal \emph{typical month}
segments.  In this case, the parameter \xmlNode{outputLen} should be \texttt{31536000} (the number of seconds
in a year), while the parameter \xmlNode{subseqLen} should be \texttt{2592000} (the number of seconds in a
month).  Using a value for \xmlNode{subseqLen} that is either much, much smaller than \xmlNode{outputLen} or
of equal size to \xmlNode{outputLen} might have unexpected results.  In general, we recommend using a
\xmlNode{subseqLen} that is roughly an order of magnitude smaller than \xmlNode{outputLen}.

\paragraph{Method: dataObjectLabelFilter}
This Post-Processor allows to filter the portion of a dataObject, either PointSet or HistorySet, with a given clustering label.
A clustering algorithm associates a unique cluster label to each element of the dataObject (PointSet or HistorySet).
This cluster label is a natural number ranging from $0$ (or $1$ depending on the algorithm) to $N$ where $N$ is the number of obtained clusters.
Recall that some clustering algorithms (e.g., K-Means) receive $N$ as input while others (e.g., Mean-Shift) determine $N$ after clustering has been performed.
Thus, this Post-Processor is naturally employed after a data-mining clustering techniques has been performed on a dataObject so that each clusters
can be analyzed separately.

In the \xmlNode{PostProcessor} input block, the following XML sub-nodes are required,
independently of the \xmlAttr{subType} specified:

\begin{itemize}
   \item \xmlNode{label}, \xmlDesc{string, required field}, name of the clustering label
   \item \xmlNode{clusterIDs}, \xmlDesc{integers, required field}, ID of the selected clusters. Note that more than one ID can be provided as input
\end{itemize}

\paragraph{Method: HSPS}
The \xmlNode{HSPS} Post-Processor performs a filtering of the dataObject. This particular filtering is based on the labels generated by any clustering algorithm.
Given the selected label, this Post-Processor filters out all histories or points having a different label.
In the \xmlNode{PostProcessor} input block, the following XML sub-nodes are required:

\begin{itemize}
   \item \xmlNode{dataType}, \xmlDesc{string, required field}, type of dataObject (HistorySet or PointSet)
   \item \xmlNode{label}, \xmlDesc{string, required field}, varaiable which contains the cluster labels
   \item \xmlNode{clusterIDs}, \xmlDesc{int, required field}, cluster labels considered
\end{itemize}

\paragraph{Method: Discrete Risk Measures}
This Post-Processor calculates a series of risk importance measures from a PointSet. This calculation if performed for a set of input paramteres given an output target.

The user is required to provide the following information:
\begin{itemize}
   \item the set of input variables. For each variable the following need to be specified:
     \begin{itemize}
       \item the set of values that imply a reliability value equal to $1$ for the input variable
       \item the set of values that imply a reliability value equal to $0$ for the input variable
     \end{itemize}
   \item the output target variable. For this variable it is needed to specify the values of the output target variable that defines the desired outcome.
\end{itemize}

The following variables are first determined for each input variable $i$:
\begin{itemize}
   \item $R_0$ Probability of the outcome of the output target variable (nominal value)
   \item $R^{+}_i$ Probability of the outcome of the output target variable if reliability of the input variable is equal to $0$
   \item $R^{-}_i$ Probability of the outcome of the output target variable if reliability of the input variable is equal to $1$
\end{itemize}

Available measures are:
\begin{itemize}
   \item Risk Achievement Worth (RAW): $RAW = R^{+}_i / R_0 $
   \item Risk Achievement Worth (RRW): $RRW = R_0 / R^{-}_i$
   \item Fussell-Vesely (FV): $FV = (R_0 - R^{-}_i) / R_0$
   \item Birnbaum (B): $B = R^{+}_i - R^{-}_i$
\end{itemize}

In the \xmlNode{PostProcessor} input block, the following XML sub-nodes are required,
independent of the \xmlAttr{subType} specified:

\begin{itemize}
   \item \xmlNode{measures}, \xmlDesc{string, required field}, desired risk importance measures that have to be computed (RRW, RAW, FV, B)
   \item \xmlNode{variable}, \xmlDesc{string, required field}, ID of the input variable. This node is provided for each input variable. This nodes needs to contain also these attributes:
     \begin{itemize}
       \item \xmlAttr{R0values}, \xmlDesc{float, required field}, interval of values (comma separated values) that implies a reliability value equal to $0$ for the input variable
       \item \xmlAttr{R1values}, \xmlDesc{float, required field}, interval of values (comma separated values) that implies a reliability value equal to $1$ for the input variable
     \end{itemize}
   \item \xmlNode{target}, \xmlDesc{string, required field}, ID of the output variable. This nodes needs to contain also the attribute \xmlAttr{values}, \xmlDesc{string, required field}, interval of
                                                             values of the output target variable that defines the desired outcome
\end{itemize}

\textbf{Example:}
This example shows an example where it is desired to calculate all available risk importance measures for two input variables (i.e., pumpTime and valveTime)
given an output target variable (i.e., Tmax).
A value of the input variable pumpTime in the interval $[0,240]$ implies a reliability value of the input variable pumpTime equal to $0$.
A value of the input variable valveTime in the interval $[0,60]$ implies a reliability value of the input variable valveTime equal to $0$.
A value of the input variables valveTime and pumpTime in the interval $[1441,2880]$ implies a reliability value of the input variables equal to $1$.
The desired outcome of the output variable Tmax occurs in the interval $[2200,2500]$.
\begin{lstlisting}[style=XML,morekeywords={subType,debug,name,class,type}]
<Simulation>
  ...
  <Models>
    ...
    <PostProcessor name="riskMeasuresDiscrete" subType="InterfacedPostProcessor">
      <method>RiskMeasuresDiscrete</method>
      <measures>B,FV,RAW,RRW</measures>
      <variable R0values='0,240' R1values='1441,2880'>pumpTime</variable>
      <variable R0values='0,60'  R1values='1441,2880'>valveTime</variable>
      <target   values='2200,2500'                  >Tmax</target>
    </PostProcessor>
    ...
  </Models>
  ...
</Simulation>
\end{lstlisting}

This Post-Processor allows the user to consider also multiple datasets (a data set for each initiating event) and calculate the global risk importance measures.
This can be performed by:
\begin{itemize}
  \item Including all datasets in the step
\begin{lstlisting}[style=XML,morekeywords={subType,debug,name,class,type}]
<Simulation>
  ...
  </Steps>
    ...
    <PostProcess name="PP">
      <Input   class="DataObjects"  type="PointSet"        >outRun1</Input>
      <Input   class="DataObjects"  type="PointSet"        >outRun2</Input>
      <Model   class="Models"       type="PostProcessor"   >riskMeasuresDiscrete</Model>
      <Output  class="DataObjects"  type="PointSet"        >outPPS</Output>
      <Output  class="OutStreams"   type="Print"           >PrintPPS_dump</Output>
    </PostProcess>
  </Steps>
  ...
</Simulation>
\end{lstlisting}
  \item Adding in the Post-processor the frequency of the initiating event associated to each dataset
\begin{lstlisting}[style=XML,morekeywords={subType,debug,name,class,type}]
<Simulation>
  ...
  <Models>
    ...
    <PostProcessor name="riskMeasuresDiscrete" subType="InterfacedPostProcessor">
      <method>riskMeasuresDiscrete</method>
      <measures>FV,RAW</measures>
      <variable R1values='-0.1,0.1' R0values='0.9,1.1'>Astatus</variable>
      <variable R1values='-0.1,0.1' R0values='0.9,1.1'>Bstatus</variable>
      <variable R1values='-0.1,0.1' R0values='0.9,1.1'>Cstatus</variable>
      <variable R1values='-0.1,0.1' R0values='0.9,1.1'>Dstatus</variable>
      <target   values='0.9,1.1'>outcome</target>
      <data     freq='0.01'>outRun1</data>
      <data     freq='0.02'>outRun2</data>
    </PostProcessor>
    ...
  </Models>
  ...
</Simulation>
\end{lstlisting}

\end{itemize}

This post-processor can be made time dependendent if a single HistorySet is provided among the other data objects.
The HistorySet contains the temporal profiles of a subset of the input variables. This temporal profile can be only
boolean, i.e., 0 (component offline) or 1 (component online).
Note that the provided history set must contains a single History; multiple Histories are not allowed.
When this post-processor is in a dynamic configuration (i.e., time-dependent), the user is required to specify an xml
node \xmlNode{temporalID} that indicates the ID of the temporal variable.
For each time instant, this post-processor determines the temporal profiles of the desired risk importance measures.
Thus, in this case, an HistorySet must be chosen as an output data object.
An example is shown below:
\begin{lstlisting}[style=XML,morekeywords={subType,debug,name,class,type}]
<Simulation>
  ...
  <Models>
    ...
    <PostProcessor name="riskMeasuresDiscrete" subType="InterfacedPostProcessor">
      <method>riskMeasuresDiscrete</method>
      <measures>B,FV,RAW,RRW,R0</measures>
      <variable R1values='-0.1,0.1' R0values='0.9,1.1'>Astatus</variable>
      <variable R1values='-0.1,0.1' R0values='0.9,1.1'>Bstatus</variable>
      <variable R1values='-0.1,0.1' R0values='0.9,1.1'>Cstatus</variable>
      <target   values='0.9,1.1'>outcome</target>
      <data     freq='1.0'>outRun1</data>
      <temporalID>time</temporalID>
    </PostProcessor>
    ...
  </Models>
  ...
  <Steps>
    ...
    <PostProcess name="PP">
      <Input     class="DataObjects"  type="PointSet"        >outRun1</Input>
      <Input     class="DataObjects"  type="HistorySet"      >timeDepProfiles</Input>
      <Model     class="Models"       type="PostProcessor"   >riskMeasuresDiscrete</Model>
      <Output    class="DataObjects"  type="HistorySet"      >outHS</Output>
      <Output    class="OutStreams"   type="Print"           >PrintHS</Output>
    </PostProcess>
    ...
  </Steps>
  ...
</Simulation>
\end{lstlisting}

%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%% RavenOutput PP   %%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%

\subsubsection{RavenOutput}
\label{RavenOutput}
The \textbf{RavenOutput} post-processor is specifically used
to gather data from RAVEN output files and generate a PointSet suitable for plotting or other analysis.
It can do this in two modes: static and dynamic.  In static mode, the
PostProcessor reads from from several static XML output files produced by RAVEN.  In dynamic mode, the PostProcessor
reads from a single dynamic XML output file and builds a PointSet where the pivot parameter (e.g. time) is the
input and the requested values are returned for each of the pivot parameter values (e.g. points in time).  The
name for the pivot parameter will be taken directly from the XML structure.
%
Note: by default the PostProcessor operates in static mode; to read a dynamic file, the \xmlNode{dynamic} node must
be specified.
%
\ppType{RavenOutput}{RavenOutput}
%
\begin{itemize}
  \item \xmlNode{dynamic}, \xmlDesc{string, optional field}, if included will trigger reading a single dynamic
  file instead of multiple static files, unless the text of this field is \xmlString{false}, in which case it
  will return to the default (multiple static files).  \default(False)
  \item \xmlNode{File}, \xmlDesc{XML Node, required field}
  %
  For each file to be read by this postprocessor, an entry in the \xmlNode{Files} node must be added, and a
  \xmlNode{File} node must be added to the postprocessor input block.  The \xmlNode{File} requires two
  identifying attributes:
  \begin{itemize}
    \item \xmlAttr{name}, \xmlDesc{string, required field}, the RAVEN-assigned name of the file,
    \item \xmlAttr{ID}, \xmlDesc{float, optional field}, the floating point ID that will be unique to this
      file.  This will appear as an entry in the output \xmlNode{DataObject} and the corresponding column are
      the values extracted from this file.  If not specified, RAVEN will attempt to find a suitable integer ID
      to use, and a warning will be raised.

      When defining the \xmlNode{DataObject} that this postprocessor will write to, and when using the static
      (non-\xmlNode{dynamic}) form of the postprocessor, the \xmlNode{input} space should be given as
      \xmlString{ID}, and the output variables should be the outputs specified in the postprocessor. See the
      examples below.  In the data object, the variable values will be keyed on the \xmlString{ID} parameter.
  \end{itemize}
  Each value that needs to be extracted from the file needs to be specified by one of the following
  \xmlNode{output} nodes within the \xmlNode{File} node:
  \begin{itemize}
    \item \xmlNode{output}, \xmlDesc{|-separated string, required field},
           the specification of the output to extract from the file.
           RAVEN uses \texttt{xpath} as implemented in Python's \texttt{xml.etree} module to specify locations
           in XML.  For example, to search tags, use a path
           separated by forward slash characters (``/''), starting under the root; this means the root node should not
           be included in the path. See the example.  For more details on xpath options available, see
           \url{https://docs.python.org/2/library/xml.etree.elementtree.html#xpath-support}.
           %
           The \xmlNode{output} node requires the following attribute:
      \begin{itemize}
        \item \xmlAttr{name}, \xmlDesc{string, required field}, specifies the entry in the Data Object that
          this value should be stored under.
      \end{itemize}

  \end{itemize}
  %
\end{itemize}
\textbf{Example (Static):}
Using an example, let us have two input files, named \emph{in1.xml} and \emph{in2.xml}.  They appear as
follows.  Note that the name of the variables we want changes slightly between the XML; this is fine.

\textbf{\emph{in1.xml}}
\begin{lstlisting}[style=XML]
<BasicStatistics>
  <ans>
    <val1>6</val1>
    <val2>7</val2>
  </ans>
</BasicStatistics>
\end{lstlisting}
\textbf{\emph{in2.xml}}
\begin{lstlisting}[style=XML]
<ROM>
  <ans>
    <first>6.1</first>
    <second>7.1</second>
  </ans>
</BasicStatistics>
\end{lstlisting}

The RAVEN input to extract this information would appear as follows.
We include an example of defining the \xmlNode{DataObject} that this postprocessor will write out to, for
further clarity.

\begin{lstlisting}[style=XML]
<Simulation>
 ...
 <Files>
   <Input name='in1'>inp1.xml</Input>
   <Input name='in2'>inp2.xml</Input>
 </Files>
 ...
 <Models>
   ...
   <PostProcessor name='pp' subType='RavenOutput'>
     <File name='in1' ID='1'>
       <output name='first'>ans/val1</output>
       <output name='second'>ans/val2</output>
     </File>
     <File name='in2' ID='2'>
       <output name='first'>ans/first</output>
       <output name='second'>ans/second</output>
     </File>
   </PostProcessor>
   ...
 </Models>
 ...
 <DataObjects>
   ...
   <PointSet name='pointSetName'>
     <input>ID</input>
     <output>first,second</output>
   </PointSet>
   ...
 </DataObjects>
 ...
</Simulation>
\end{lstlisting}

\textbf{Example (Dynamic):}
For a dynamic example, consider this time-evolution of values example.  \emph{inFile.xml} is a RAVEN dynamic
XML output.

\textbf{\emph{in1.xml}}
\begin{lstlisting}[style=XML]
<BasicStatistics type='Dynamic'>
  <time value='0.0'>
    <ans>
      <val1>6</val1>
      <val2>7</val2>
    </ans>
  <\time>
  <time value='1.0'>
    <ans>
      <val1>9</val1>
      <val2>10</val2>
    </ans>
  <\time>
</BasicStatistics>
\end{lstlisting}
The RAVEN input to extract this information would appear as follows:
\begin{lstlisting}[style=XML]
<Simulation>
 ...
 <Files>
   <Input name='inFile'>inFile.xml</Input>
 </Files>
 ...
 <Models>
   ...
   <PostProcessor name='pp' subType='RavenOut'>
     <dynamic>true</dynamic>
     <File name='inFile'>
       <output name='first'>ans|val1</output>
     </File>
   </PostProcessor>
   ...
 </Models>
 ...
</Simulation>
\end{lstlisting}
The resulting PointSet has \emph{time} as an input and \emph{first} as an output.

%%%%%%%%%%%%%% ETImporter PP %%%%%%%%%%%%%%%%%%%

\subsubsection{ETImporter}
\label{ETImporterPP}
The \textbf{ETImporter} post-processor has been designed to import Event-Tree (ET) object into
RAVEN. Since several ET file formats are available, as of now only the OpenPSA format
(see https://open-psa.github.io/joomla1.5/index.php.html) is supported. As an example,
the OpenPSA format ET is shown below:

\begin{lstlisting}[style=XML,morekeywords={anAttribute},caption=ET in OpenPSA format., label=lst:ETModel]
<define-event-tree name="eventTree">
    <define-functional-event name="ACC"/>
    <define-functional-event name="LPI"/>
    <define-functional-event name="LPR"/>
    <define-sequence name="1"/>
    <define-sequence name="2"/>
    <define-sequence name="3"/>
    <define-sequence name="4"/>
    <initial-state>
        <fork functional-event="ACC">
            <path state="0">
                <fork functional-event="LPI">
                    <path state="0">
                        <fork functional-event="LPR">
                            <path state="0">
                                <sequence name="1"/>
                            </path>
                            <path state="+1">
                                <sequence name="2"/>
                            </path>
                        </fork>
                    </path>
                    <path state="+1">
                        <sequence name="3"/>
                    </path>
                </fork>
            </path>
            <path state="+1">
                <sequence name="4"/>
            </path>
        </fork>
    </initial-state>
</define-event-tree>
\end{lstlisting}

This is performed by saving the structure of the ET (from file) as a \textbf{PointSet}
(only \textbf{PointSet} are allowed), since an ET is a static Boolean logic structure. Each realization in the
\textbf{PointSet} represents a unique accident sequence of the ET, and the \textbf{PointSet} is structured as follows:
\begin{itemize}
  \item Input variables of the \textbf{PointSet} are the branching conditions of the ET. The value of each input variable can be:
  \begin{itemize}
    \item  0: event did occur (typically upper branch)
    \item  1: event did not occur (typically lower branch)
    \item -1: event is not queried (no branching occured)
  \end{itemize}
  \item Output variables of the \textbf{PointSet} are the ID of each branch of the ET (i.e., positive integers greater than 0)
\end{itemize}

\nb that the 0 or 1 values are specified in the \xmlNode{path state="0"} or \xmlNode{path state="1"} nodes in the ET OpenPSA file.

Provided this definition, the ET described in Listing~\ref{lst:ETModel} will be converted to \textbf{PointSet} that is characterized
by the following variables:
\begin{itemize}
	\item Input variables: statusACC, statusLPI, statusLPR
	\item Output variable: sequence
\end{itemize}
and the corresponding \textbf{PointSet} if the \xmlNode{expand} node is set to False is shown in Table~\ref{PointSetETExpandFalse}.
If \xmlNode{expand} set to True, the corresponding \textbf{PointSet} is shown in Table~\ref{PointSetETExpandTrue}.
\begin{table}[h]
    \centering
    \caption{PointSet generated by RAVEN by employing the ET Importer Post-Processor with \xmlNode{expand}
             set to False for the ET of Listing~\ref{lst:ETModel}.}
    \label{PointSetETExpandFalse}
	\begin{tabular}{c | c | c | c}
		\hline
		ACC & LPI & LPR & sequence \\
		\hline
		0.  &  0. &  0. & 1. \\
		0.  &  0. &  1. & 2. \\
		0.  &  1. & -1. & 3. \\
		1.  & -1. & -1. & 4. \\
		\hline
	\end{tabular}
\end{table}
\begin{table}[h]
    \centering
    \caption{PointSet generated by RAVEN by employing the ET Importer Post-Processor with \xmlNode{expand}
             set to True for the ET of Listing~\ref{lst:ETModel}.}
    \label{PointSetETExpandTrue}
	\begin{tabular}{c | c | c | c}
		\hline
		ACC & LPI & LPR & sequence \\
		\hline
		0.  &  0. &  0. & 1. \\
		0.  &  0. &  1. & 2. \\
		0.  &  1. &  0. & 3. \\
		0.  &  1. &  1. & 3. \\
		1.  &  0. &  0. & 4. \\
		1.  &  0. &  1. & 4. \\
		1.  &  1. &  0. & 4. \\
		1.  &  1. &  1. & 4. \\
		\hline
	\end{tabular}
\end{table}

The ETImporter PP supports also:
\begin{itemize}
  \item links to sub-trees
	      \nb If the ET is split in two or more ETs (and thus one file for each ET), then it is only required to list
	      all files in the Step. RAVEN automatically detect links among ETs and merge all of them into a single PointSet.
  \item by-pass branches
  \item symbolic definition of outcomes: typically outcomes are defined as either 0 (upper branch) or 1 (lower branch). If instead the ET uses the
    \textbf{success/failure} labels, then they are converted into 0/1 labels
    \nb If the branching condition is not binary or \textbf{success/failure}, then the ET Importer Post-Processor just follows
	   the numerical value of the \xmlNode{state} attribute of the \xmlNode{<path>} node in the ET OpenPSA file.
  \item symbolic/numerical definition of sequences: if the ET contains a symbolic sequence then a .xml file is generated.  This file contains
        the mapping between the sequences defined in the ET and the numerical IDs created by RAVEN. The file name is the concatenation of the ET name
        and "\_mapping". As an example the file "eventTree\_mapping.xml" generated by RAVEN:
        \begin{lstlisting}[style=XML]
            <map Tree="eventTree">
              <sequence ID="0">seq_1</sequence>
              <sequence ID="1">seq_2</sequence>
              <sequence ID="2">seq_3</sequence>
              <sequence ID="3">seq_4</sequence>
            </map>
        \end{lstlisting}
        contains the mapping of four sequences defined in the ET (seq\_1,seq\_2,seq\_3,seq\_4) with the IDs generated by RAVEN (0,1,2,3).
        Note that if the sequences defined in the ET are both numerical and symbolic then they are all mapped.
  \item The ET can contain a branch that is defined as a separate block in the \xmlNode{define-branch} node and it is
        replicated in the ET; in such case RAVEN automatically replicate such branch when generating the PointSet.
\end{itemize}
The \xmlNode{collect-formula} are not considered since this node is used to connect the Boolean formulae generated by the
Fault-Trees to the branch (i.e., fork) point.

%
\ppType{ETImporter}{ETImporter}
%
\begin{itemize}
  \item \xmlNode{fileFormat}, \xmlDesc{string, required field}, specifies the format of the file that contains the ET structure (supported format: OpenPSA).
  \item  \xmlNode{expand},\xmlDesc{bool, required parameter}, expand the ET branching conditions for all branches even if they are not queried
\end{itemize}

\textbf{Example:}
\begin{lstlisting}[style=XML,morekeywords={anAttribute},caption=ET Importer input example., label=lst:ET_PP_InputExample]
  <Files>
    <Input name="eventTreeTest" type="">eventTree.xml</Input>
  </Files>

  <Models>
    ...
    <PostProcessor name="ETimporter" subType="ETImporter">
      <fileFormat>OpenPSA</fileFormat>
      <expand>False</expand>
    </PostProcessor>
    ...
  </Models>

  <Steps>
    ...
    <PostProcess name="import">
      <Input   class="Files"        type=""                >eventTreeTest</Input>
      <Model   class="Models"       type="PostProcessor"   >ETimporter</Model>
      <Output  class="DataObjects"  type="PointSet"        >ET_PS</Output>
    </PostProcess>
    ...
  </Steps>

  <DataObjects>
    ...
    <PointSet name="ET_PS">
      <Input>ACC,LPI,LPR</Input>
      <Output>sequence</Output>
    </PointSet>
    ...
  </DataObjects>
\end{lstlisting}

%%%%%%%%%%%%%% FTImporter PP %%%%%%%%%%%%%%%%%%%

\subsubsection{FTImporter}
\label{FTImporterPP}
The \textbf{FTImporter} post-processor has been designed to import Fault-Tree (FT) object into
RAVEN. Since several FT file formats are available, as of now only the OpenPSA format
(see https://open-psa.github.io/joomla1.5/index.php.html) is supported. As an example,
the FT in OpenPSA format is shown in Listing~\ref{lst:FTModel}.

\begin{lstlisting}[style=XML,morekeywords={anAttribute},caption=FT in OpenPSA format., label=lst:FTModel]
<opsa-mef>
    <define-fault-tree name="FT">
        <define-gate name="TOP">
            <or>
                <gate name="G1"/>
                <gate name="G2"/>
                <gate name="G3"/>
            </or>
        </define-gate>
        <define-component name="A">
            <define-gate name="G1">
                <and>
                    <basic-event name="BE1"/>
                    <basic-event name="BE2"/>
                </and>
            </define-gate>
            <define-gate name="G2">
                <and>
                    <basic-event name="BE1"/>
                    <basic-event name="BE3"/>
                </and>
            </define-gate>
            <define-basic-event name="BE1">
                <float value="1.2e-3"/>
            </define-basic-event>
            <define-component name="B">
                <define-basic-event name="BE2">
                    <float value="2.4e-3"/>
                </define-basic-event>
                <define-basic-event name="BE3">
                    <float value="5.2e-3"/>
                </define-basic-event>
            </define-component>
        </define-component>
        <define-component name="C">
            <define-gate name="G3">
                <and>
                    <basic-event name="BE1"/>
                    <basic-event name="BE4"/>
                </and>
            </define-gate>
            <define-basic-event name="BE4">
                <float value="1.6e-3"/>
            </define-basic-event>
        </define-component>
    </define-fault-tree>
</opsa-mef>
\end{lstlisting}

This is performed by saving the structure of the FT (from file) as a \textbf{PointSet}
(only \textbf{PointSet} are allowed).

Each Point in the PointSet represents a unique combination of the basic events.
The PointSet is structured as follows: input variables are the basic events, output variable is the top event of the FT.
The value for each input and output variable can have the following values:
\begin{itemize}
  \item  0: False
  \item  1: True
\end{itemize}

Provided this definition, the FT model of Listing~\ref{lst:FTModel} can be converted to
\textbf{PointSet} that is characterized by these variables:
\begin{itemize}
	\item Input variables: BE1, BE2, BE3, BE4
	\item Output variable: out
\end{itemize}
and it is structured is shown in Table~\ref{PointSetFT}.

\begin{table}[h]
    \centering
    \caption{PointSet generated by RAVEN by employing the FT Importer Post-Processor for the FT of Listing~\ref{lst:FTModel}.}
    \label{PointSetFT}
	\begin{tabular}{c | c | c | c | c}
		\hline
		BE1 & BE2 & BE3 & BE4 & TOP \\
		\hline
		 0. &  0. &  0. &  0. &  0. \\
		 0. &  0. &  0. &  1. &  0. \\
		 0. &  0. &  1. &  0. &  0. \\
		 0. &  0. &  1. &  1. &  0. \\
		 0. &  1. &  0. &  0. &  0. \\
		 0. &  1. &  0. &  1. &  0. \\
		 0. &  1. &  1. &  0. &  0. \\
		 0. &  1. &  1. &  1. &  0. \\
		 1. &  0. &  0. &  0. &  0. \\
		 1. &  0. &  0. &  1. &  1. \\
		 1. &  0. &  1. &  0. &  1. \\
		 1. &  0. &  1. &  1. &  1. \\
		 1. &  1. &  0. &  0. &  1. \\
		 1. &  1. &  0. &  1. &  1. \\
		 1. &  1. &  1. &  0. &  1. \\
		 1. &  1. &  1. &  1. &  1. \\
		\hline
	\end{tabular}
\end{table}

%
\ppType{FTImporter}{FTImporter}
%
\begin{itemize}
  \item \xmlNode{fileFormat}, \xmlDesc{string, required field}, specifies the format of the file that contains the
    FT structure (supported format: OpenPSA).
  \item  \xmlNode{topEventID},\xmlDesc{string, required parameter}, the name of the top event of the FT
\end{itemize}

The example of FTImporter PostProcessor is shown in Listing~\ref{lst:FT_PP_InputExample}
\begin{lstlisting}[style=XML,morekeywords={anAttribute},caption=FT Importer input example., label=lst:FT_PP_InputExample]
  <Files>
    <Input name="faultTreeTest" type="">FTimporter_not.xml</Input>
  </Files>

  <Models>
    ...
    <PostProcessor name="FTimporter" subType="FTImporter">
      <fileFormat>OpenPSA</fileFormat>
      <topEventID>TOP</topEventID>
    </PostProcessor>
    ...
  </Models>

  <Steps>
    ...
    <PostProcess name="import">
      <Input   class="Files"        type=""                >faultTreeTest</Input>
      <Model   class="Models"       type="PostProcessor"   >FTimporter</Model>
      <Output  class="DataObjects"  type="PointSet"        >FT_PS</Output>
    </PostProcess>
    ...
  </Steps>

  <DataObjects>
    ...
    <PointSet name="FT_PS">
      <Input>BE1,BE2,BE3,BE4</Input>
      <Output>TOP</Output>
    </PointSet>
    ...
  </DataObjects>
\end{lstlisting}

Important notes and capabilities:
\begin{itemize}
	\item If the FT is split in two or more FTs (and thus one file for each FT), then it is only required to list
	      all files in the Step. RAVEN automatically detect links among FTs and merge all of them into a single PointSet.
	\item Allowed gates: AND, OR, NOT, ATLEAST, CARDINALITY, IFF, imply, NAND, NOR, XOR
	\item If an house-event is defined in the FT:
\begin{lstlisting}[style=XML,morekeywords={anAttribute},caption=FT Importer input example: house-event., label=lst:FT_house event]
<opsa-mef>
    <define-fault-tree name="FT">
        <define-gate name="TOP">
            <or>
                <basic-event name="BE1"/>
                <basic-event name="BE2"/>
                <house-event name="HE1"/>
            </or>
        </define-gate>
        <define-house-event name="HE1">
        	<constant value="true"/>
        </define-house-event>
    </define-fault-tree>
</opsa-mef>
\end{lstlisting}
           then the HE1 is not part of the PointSet (value is fixed)
\end{itemize}

%%%%%%%%%%%%%% Metric PP %%%%%%%%%%%%%%%%%%%

\subsubsection{Metric}
\label{MetricPP}
The \textbf{Metric} post-processor is specifically used to calculate the distance values among points from PointSets and histories from HistorySets,
while the \textbf{Metrics} block (See Chapter \ref{sec:Metrics}) allows the user to specify the similarity/dissimilarity metrics to be used in this
post-processor. Both \textbf{PointSet} and \textbf{HistorySet} can be accepted by this post-processor.
If the name of given variable is unique, it can be used directly, otherwise the variable can be specified
with $DataObjectName|InputOrOutput|VariableName$ like other places in RAVEN.
Some of the Metrics also accept distributions to calculate the distance against.
These are specified by using the name of the distribution.
%
\ppType{Metric}{Metric}
%
\begin{itemize}
  \item \xmlNode{Features}, \xmlDesc{comma separated string, required field}, specifies the names of the features.
    This xml-node accepts the following attribute:
    \begin{itemize}
      \item \xmlAttr{type}, \xmlDesc{required string attribute}, the type of provided features. Currently only
        accept `variable'.
    \end{itemize}
  \item \xmlNode{Targets}, \xmlDesc{comma separated string, required field}, contains a comma separated list of
    the targets. \nb Each target is paired with a feature listed in xml node \xmlNode{Features}. In this case, the
    number of targets should be equal to the number of features.
    This xml-node accepts the following attribute:
    \begin{itemize}
      \item \xmlAttr{type}, \xmlDesc{required string attribute}, the type of provided features. Currently only
        accept `variable'.
    \end{itemize}
  \item \xmlNode{multiOutput}, \xmlDesc{optional string attribute}, only used when \textbf{HistorySet} is used as
    input. Defines aggregating of time-dependent metrics' calculations. Available options include:
    \textbf{mean, max, min, raw\_values} over the time. For example, when `mean' is used, the metrics' calculations
    will be averaged over the time. When `raw\_values' is used, the full set of  metrics' calculations will be dumped.
    \default{raw\_values}
  \item \xmlNode{weight}, \xmlDesc{comma separated floats, optional field}, when `mean' is provided for \xmlNode{multiOutput},
    the user can specify the weights that can be used for the average calculation of all outputs.
  \item \xmlNode{pivotParameter}, \xmlDesc{optional string attribute}, only used when \textbf{HistorySet}
    is used as input. The pivotParameter for given metrics' calculations.
    \default{time}
  \item \xmlNode{Metric}, \xmlDesc{string, required field}, specifies the \textbf{Metric} name that is defined via
    \textbf{Metrics} entity. In this xml-node, the following xml attributes need to be specified:
    \begin{itemize}
      \item \xmlAttr{class}, \xmlDesc{required string attribute}, the class of this metric (e.g. Metrics)
      \item \xmlAttr{type}, \xmlDesc{required string attribute}, the sub-type of this Metric (e.g. SKL, Minkowski)
    \end{itemize}
\end{itemize}

\textbf{Example:}

\begin{lstlisting}[style=XML]
<Simulation>
 ...
  <Models>
    ...
    <PostProcessor name="pp1" subType="Metric">
      <Features type="variable">ans</Features>
      <Targets type="variable">ans2</Targets>
      <Metric class="Metrics" type="SKL">euclidean</Metric>
      <Metric class="Metrics" type="SKL">cosine</Metric>
      <Metric class="Metrics" type="SKL">manhattan</Metric>
      <Metric class="Metrics" type="ScipyMetric">braycurtis</Metric>
      <Metric class="Metrics" type="ScipyMetric">canberra</Metric>
      <Metric class="Metrics" type="ScipyMetric">correlation</Metric>
      <Metric class="Metrics" type="ScipyMetric">minkowski</Metric>
    </PostProcessor>
    ...
  </Models>
 ...
</Simulation>
\end{lstlisting}

In order to access the results from this post-processor, RAVEN will define the variables as ``MetricName'' +
``\_'' + ``TargetVariableName'' + ``\_'' + ``FeatureVariableName'' to store the calculation results, and these
variables are also accessible by the users through RAVEN entities \textbf{DataObjects} and \textbf{OutStreams}.
\nb We will replace ``|'' in ``TargetVariableName'' and ``FeatureVariableName'' with ``\_''.
In previous example, variables such as \textit{euclidean\_ans2\_ans, cosine\_ans2\_ans, poly\_ans2\_ans} are accessible
by the users.

%%%%%%%%%%%%%% Cross Validation PP %%%%%%%%%%%%%%%%%%%

\subsubsection{CrossValidation}
\label{CVPP}
The \textbf{CrossValidation} post-processor is specifically used to evaluate estimator (i.e. ROMs) performance.
Cross-validation is a statistical method of evaluating and comparing learning algorithms by dividing data into
two portions: one used to `train' a surrogate model and the other used to validate the model, based on specific
scoring metrics. In typical cross-validation, the training and validation sets must crossover in successive
rounds such that each data point has a chance of being validated against the various sets. The basic form of
cross-validation is k-fold cross-validation. Other forms of cross-validation are special cases of k-fold or involve
repeated rounds of k-fold cross-validation. \nb It is important to notice that this post-processor currently can
only accept \textbf{PointSet} data object.
%
\ppType{CrossValidation}{CrossValidation}
%
\begin{itemize}
  \item \xmlNode{SciKitLearn}, \xmlDesc{string, required field}, the subnodes specifies the necessary information
    for the algorithm to be used in the post-processor. `SciKitLearn' is based on algorithms in SciKit-Learn
    library, and currently it performs cross-validation over \textbf{PointSet} only.
  \item \xmlNode{Metric}, \xmlDesc{string, required field}, specifies the \textbf{Metric} name that is defined via
    \textbf{Metrics} entity. In this xml-node, the following xml attributes need to be specified:
    \begin{itemize}
      \item \xmlAttr{class}, \xmlDesc{required string attribute}, the class of this metric (e.g. Metrics)
      \item \xmlAttr{type}, \xmlDesc{required string attribute}, the sub-type of this Metric (e.g. SKL, Minkowski)
    \end{itemize}
    \nb Currently, cross-validation post-processor only accepts \xmlNode{SKL} metrics with \xmlNode{metricType}
    \xmlString{mean\_absolute\_error}, \xmlString{explained\_variance\_score}, \xmlString{r2\_score},
    \xmlString{mean\_squared\_error}, and \xmlString{median\_absolute\_error}.
\end{itemize}

\textbf{Example:}

\begin{lstlisting}[style=XML]
<Simulation>
 ...
  <Files>
    <Input name="output_cv" type="">output_cv.xml</Input>
    <Input name="output_cv.csv" type="">output_cv.csv</Input>
  </Files>
  <Models>
    ...
    <ROM name="surrogate" subType="SciKitLearn">
      <SKLtype>linear_model|LinearRegression</SKLtype>
      <Features>x1,x2</Features>
      <Target>ans</Target>
      <fit_intercept>True</fit_intercept>
      <normalize>True</normalize>
    </ROM>
    <PostProcessor name="pp1" subType="CrossValidation">
        <SciKitLearn>
            <SKLtype>KFold</SKLtype>
            <n_splits>3</n_splits>
            <shuffle>False</shuffle>
        </SciKitLearn>
        <Metric class="Metrics" type="SKL">m1</Metric>
    </PostProcessor>
    ...
  </Models>
  <Metrics>
    <SKL name="m1">
      <metricType>mean_absolute_error</metricType>
    </SKL>
  </Metrics>
  <Steps>
    <PostProcess name="PP1">
        <Input class="DataObjects" type="PointSet">outputDataMC</Input>
        <Input class="Models" type="ROM">surrogate</Input>
        <Model class="Models" type="PostProcessor">pp1</Model>
        <Output class="Files" type="">output_cv</Output>
        <Output class="Files" type="">output_cv.csv</Output>
    </PostProcess>
  </Steps>
 ...
</Simulation>
\end{lstlisting}

In order to access the results from this post-processor, RAVEN will define the variables as ``cv'' +
``\_'' + ``MetricName'' + ``\_'' + ``ROMTargetVariable'' to store the calculation results, and these
variables are also accessible by the users through RAVEN entities \textbf{DataObjects} and \textbf{OutStreams}.
In previous example, variable \textit{cv\_m1\_ans} are accessible by the users.

\paragraph{SciKitLearn}

The algorithm for cross-validation is chosen by the subnode \xmlNode{SKLtype} under the parent node \xmlNode{SciKitLearn}.
In addition, a special subnode \xmlNode{average} can be used to obtain the average cross validation results.

\begin{itemize}
  \item \xmlNode{SKLtype}, \xmlDesc{string, required field}, contains a string that
    represents the cross-validation algorithm to be used. As mentioned, its format is:

    \xmlNode{SKLtype}algorithm\xmlNode{/SKLtype}.
  \item \xmlNode{average}, \xmlDesc{boolean, optional field}, if `True`, dump the average cross validation results into the
    output files.
\end{itemize}

Based on the \xmlNode{SKLtype} several different algorithms are available. In the following paragraphs a brief
explanation and the input requirements are reported for each of them.

\paragraph{K-fold}
\textbf{KFold} divides all the samples in $k$ groups of samples, called folds (if $k=n$, this is equivalent to the
\textbf{Leave One Out} strategy), of equal sizes (if possible). The prediction function is learned using $k-1$ folds,
and fold left out is used for test.
In order to use this algorithm, the user needs to set the subnode:
\xmlNode{SKLtype}KFold\xmlNode{/SKLtype}.
In addition to this XML node, several others are available:
\begin{itemize}
  \item \xmlNode{n\_splits}, \xmlDesc{integer, optional field}, number of folds, must be at least 2. \default{3}
  \item \xmlNode{shuffle}, \xmlDesc{boolean, optional field}, whether to shuffle the data before splitting into
    batches.
  \item \xmlNode{random\_state}, \xmlDesc{integer, optional field}, when shuffle=True,
    pseudo-random number generator state used for shuffling. If not present, use default numpy RNG for shuffling.
\end{itemize}

\paragraph{Stratified k-fold}
\textbf{StratifiedKFold} is a variation of \textit{k-fold} which returns stratified folds: each set contains approximately
the same percentage of samples of each target class as the complete set.
In order to use this algorithm, the user needs to set the subnode:

\xmlNode{SKLtype}StratifiedKFold\xmlNode{/SKLtype}.

In addition to this XML node, several others are available:
\begin{itemize}
  \item \xmlNode{labels}, \xmlDesc{list of integers, (n\_samples), required field}, contains a label for each sample.
  \item \xmlNode{n\_splits}, \xmlDesc{integer, optional field}, number of folds, must be at least 2. \default{3}
  \item \xmlNode{shuffle}, \xmlDesc{boolean, optional field}, whether to shuffle the data before splitting into
    batches.
  \item \xmlNode{random\_state}, \xmlDesc{integer, optional field}, when shuffle=True,
    pseudo-random number generator state used for shuffling. If not present, use default numpy RNG for shuffling.
\end{itemize}

\paragraph{Label k-fold}
\textbf{LabelKFold} is a variation of \textit{k-fold} which ensures that the same label is not in both testing and
training sets. This is necessary for example if you obtained data from different subjects and you want to avoid
over-fitting (i.e., learning person specific features) by testing and training on different subjects.
In order to use this algorithm, the user needs to set the subnode:

\xmlNode{SKLtype}LabelKFold\xmlNode{/SKLtype}.

In addition to this XML node, several others are available:
\begin{itemize}
  \item \xmlNode{labels}, \xmlDesc{list of integers with length (n\_samples, ), required field}, contains a label for
    each sample. The folds are built so that the same label does not appear in two different folds.
  \item \xmlNode{n\_splits}, \xmlDesc{integer, optional field}, number of folds, must be at least 2. \default{3}
\end{itemize}

\paragraph{Leave-One-Out - LOO}
\textbf{LeaveOneOut} (or LOO) is a simple cross-validation. Each learning set is created by taking all the samples
except one, the test set being the sample left out. Thus, for $n$ samples, we have $n$ different training sets and
$n$ different tests set. This is cross-validation procedure does not waste much data as only one sample is removed from
the training set.
In order to use this algorithm, the user needs to set the subnode:

\xmlNode{SKLtype}LeaveOneOut\xmlNode{/SKLtype}.

\paragraph{Leave-P-Out - LPO}
\textbf{LeavePOut} is very similar to \textbf{LeaveOneOut} as it creates all the possible training/test sets by removing
$p$ samples from the complete set. For $n$ samples, this produces $(^n_p)$ train-test pairs. Unlike \textbf{LeaveOneOut}
and \textbf{KFold}, the test sets will overlap for $p > 1$.
In order to use this algorithm, the user needs to set the subnode:

\xmlNode{SKLtype}LeavePOut\xmlNode{/SKLtype}.

In addition to this XML node, several others are available:
\begin{itemize}
  \item \xmlNode{p}, \xmlDesc{integer, required field}, size of the test sets
\end{itemize}

\paragraph{Leave-One-Label-Out - LOLO}
\textbf{LeaveOneLabelOut} (LOLO) is a cross-validation scheme which holds out the samples according to a third-party
provided array of integer labels. This label information can be used to encode arbitrary domain specific pre-defined
cross-validation folds. Each training set is thus constituted by all samples except the ones related to a specific
label.
In order to use this algorithm, the user needs to set the subnode:

\xmlNode{SKLtype}LeaveOneLabelOut\xmlNode{/SKLtype}.

In addition to this XML node, several others are available:
\begin{itemize}
  \item \xmlNode{labels}, \xmlDesc{list of integers, (n\_samples,), required field}, arbitrary
    domain-specific stratificatioin of the data to be used to draw the splits.
\end{itemize}

\paragraph{Leave-P-Label-Out}
\textbf{LeavePLabelOut} is imilar as \textit{Leave-One-Label-Out}, but removes samples related to $P$ labels for
each training/test set.
In order to use this algorithm, the user needs to set the subnode:

\xmlNode{SKLtype}LeavePLabelOut\xmlNode{/SKLtype}.

In addition to this XML node, several others are available:
\begin{itemize}
  \item \xmlNode{labels}, \xmlDesc{list of integers, (n\_samples,), required field}, arbitrary
    domain-specific stratificatioin of the data to be used to draw the splits.
  \item \xmlNode{n\_groups}, \xmlDesc{integer, optional field}, number of samples to leave out in the test split.
\end{itemize}

\paragraph{ShuffleSplit}
\textbf{ShuffleSplit} iterator will generate a user defined number of independent train/test dataset splits. Samples
are first shuffled and then split into a pair of train and test sets. it is possible to control the randomness for
reproducibility of the results by explicitly seeding the \xmlNode{random\_state} pseudo random number generator.
In order to use this algorithm, the user needs to set the subnode:

\xmlNode{SKLtype}ShuffleSplit\xmlNode{/SKLtype}.

In addition to this XML node, several others are available:
\begin{itemize}
  \item \xmlNode{n\_splits}, \xmlDesc{integer, optional field}, number of re-shuffling and splitting iterations
    \default{10}.
  \item \xmlNode{test\_size}, \xmlDesc{float or integer, optional field}, if float, should be between 0.0 and 1.0 and
    represent the proportion of the dataset to include in the test split. \default{0.1}
    If integer, represents the absolute number of test samples. If not present, the value is automatically set to
    the complement of the train size.
  \item \xmlNode{train\_size}, \xmlDesc{float or integer, optional field}, if float, should be between 0.0 and 1.0 and represent
    the proportion of the dataset to include in the train split. If integer, represents the absolute number of train
    samples. If not present, the value is automatically set to the complement of the test size.
  \item \xmlNode{random\_state}, \xmlDesc{integer, optional field}, when shuffle=True,
    pseudo-random number generator state used for shuffling. If not present, use default numpy RNG for shuffling.
\end{itemize}

\paragraph{Label-Shuffle-Split}
\textbf{LabelShuffleSplit} iterator behaves as a combination of \textbf{ShuffleSplit} and \textbf{LeavePLabelOut},
and generates a sequence of randomized partitions in which a subset of labels are held out for each split.
In order to use this algorithm, the user needs to set the subnode:

\xmlNode{SKLtype}LabelShuffleSplit\xmlNode{/SKLtype}.

In addition to this XML node, several others are available:
\begin{itemize}
  \item \xmlNode{labels}, \xmlDesc{list of integers, (n\_samples)}, labels of samples.
  \item \xmlNode{n\_splits}, \xmlDesc{integer, optional field}, number of re-shuffling and splitting iterations
    \default{10}.
  \item \xmlNode{test\_size}, \xmlDesc{float or integer, optional field}, if float, should be between 0.0 and 1.0 and
    represent the proportion of the dataset to include in the test split. \default{0.1}
    If integer, represents the absolute number of test samples. If not present, the value is automatically set to
    the complement of the train size.
  \item \xmlNode{train\_size}, \xmlDesc{float or integer, optional field}, if float, should be between 0.0 and 1.0 and represent
    the proportion of the dataset to include in the train split. If integer, represents the absolute number of train
    samples. If not present, the value is automatically set to the complement of the test size.
  \item \xmlNode{random\_state}, \xmlDesc{integer, optional field}, when shuffle=True,
    pseudo-random number generator state used for shuffling. If not present, use default numpy RNG for shuffling.
\end{itemize}

%%%%%%%%%%%%%% Data Classifier PP %%%%%%%%%%%%%%%%%%%

\subsubsection{DataClassifier}
\label{DataClassifierPP}
The \textbf{DataClassifier} post-processor is specifically used to classify the data stored in the DataObjects. It
accepts two DataObjects, one is used as the classifier which must be a \textbf{PointSet}, the other one,
i.e. \textbf{PointSet} or \textbf{HistorySet}, is used as the input DataObject to be classified.
%
\ppType{DataClassifier}{DataClassifier}
%
\begin{itemize}
  \item \xmlNode{label}, \xmlDesc{string, required field}, the name of the label that are used for the classifier. This
    label must exist in the DataObject that is used as the classifer. This name will also be used as
    the label name for the DataObject that is classified.
  \item \xmlNode{variable}, \xmlDesc{required, xml node}. In this node, the following attribute should be specified:
    \begin{itemize}
      \item \xmlAttr{name}, \xmlDesc{required, string attribute}, the variable name, which should be exist in
        the DataObject that is used as classifier.
    \end{itemize}
    and the following sub-node should also be specified:
    \begin{itemize}
      \item \xmlNode{Function}, \xmlDesc{string, required field}, this function creates the mapping from input DataObject
        to the Classifier.
        \begin{itemize}
          \item \xmlAttr{class}, \xmlDesc{string, required field}, the class of this function (e.g. Functions)
          \item \xmlAttr{type}, \xmlDesc{string, required field}, the type of this function (e.g. external)
        \end{itemize}
    \end{itemize}
\end{itemize}
%
In order to use this post-processor, the users need to specify two different DataObjects, i.e.
\begin{lstlisting}[style=XML]
  <DataObjects>
    <PointSet name="ET_PS">
      <Input>ACC, LPI</Input>
      <Output>sequence</Output>
    </PointSet>
    <PointSet name="sim_PS">
      <Input>ACC_status, LPI_status</Input>
      <Output>out</Output>
    </PointSet>
  </DataObjects>
\end{lstlisting}
The first data object ``ET\_PS'' contains the event tree with input variables ``ACC, LPI'' and output label ``sequence''.
This data object will be used to classify the data in the second data object ``sim\_PS''. The results will be stored in
the output data object with the same label ``sequence''. Since these two data objects contain different inputs,
\xmlNode{Functions} will be used to create the maps between the inputs:
\begin{lstlisting}[style=XML]
  <Functions>
    <External file="func_ACC.py" name="func_ACC">
      <variable>ACC_status</variable>
    </External>
    <External file="func_LPI.py" name="func_LPI">
      <variable>LPI_status</variable>
    </External>
  </Functions>
\end{lstlisting}

The inputs to these functions are the inputs of the data object that will be classified, and the outputs of these functions
are the inputs of data object that is used as the classifier.

\textbf{Example Python Function for ``func\_ACC.py''}
\begin{lstlisting}[language=python]
def evaluate(self):
  return self.ACC_status
\end{lstlisting}

\textbf{Example Python Function for ``func\_LPI.py''}
\begin{lstlisting}[language=python]
def evaluate(self):
  return self.LPI_status
\end{lstlisting}

\nb All the functions that are used to create the maps should be include the ``evaluate'' method.

The \textbf{DataClassifier} post-processor is specifically used to classify the data stored in the DataObjects. It
accepts two DataObjects, one is used as the classifier which should be always \textbf{PointSet}, the other one, i.e.
either \textbf{PointSet} or \textbf{HistorySet} is used as the input DataObject to be classified.

The \textbf{DataClassifier} is provided below:
\begin{lstlisting}[style=XML]
    <PostProcessor name="ET_Classifier" subType="DataClassifier">
      <label>sequence</label>
      <variable name='ACC'>
        <Function class="Functions" type="External">func_ACC</Function>
      </variable>
      <variable name='LPI'>
        <Function class="Functions" type="External">func_LPI</Function>
      </variable>
    </PostProcessor>
\end{lstlisting}
The definitions for the XML nodes can be found in the RAVEN user manual. The label ``sequence''
and the variables ``ACC, LPI'' should be exist in the data object that is used as the classifier,
while the functions ``func\_ACC, func\_LPI'' are used to map relationships between the input data objects.

The classification can be achieved via the \xmlNode{Steps} as shown below:
\begin{lstlisting}[style=XML]
<Simulation>
 ...
  <Steps>
    <PostProcess name="classify">
      <Input   class="DataObjects"  type="PointSet"        >ET_PS</Input>
      <Input   class="DataObjects"  type="PointSet"        >sim_PS</Input>
      <Model   class="Models"       type="PostProcessor"   >ET_Classifier</Model>
      <Output  class="DataObjects"  type="PointSet"        >sim_PS</Output>
    </PostProcess>
  </Steps>
 ...
</Simulation>
\end{lstlisting}

%%%%%%%%%%%%%% ValueDuration %%%%%%%%%%%%%%%%%%%
\subsubsection{ValueDuration}
\label{ValueDurationPP}
The \xmlNode{ValueDuration} postprocessor is a tool to construct a particular kind of histogram, where the
independent variable is the number of times a variable exceeds a particular value, and the dependent variable
is the values themselves.  An example of this is the Load Duration Curve in energy modeling. This approach is
similar to that used in Lebesgue integration. Note that for each realization in the input
\xmlNode{HistorySet}, a seperate load duration curve will be created for each target.

The \xmlNode{ValueDuration} postprocessor can only act on \xmlNode{HistorySet} data objects, and generates a
\xmlNode{HistorySet} in return.  Two output variables are created for each \xmlAttr{target}:
\xmlString{counts\_x} and \xmlString{bins\_x}, where \xmlString{x} is replaced by the name of the target.
These must be specified in the output data object in order to be collected.

To plot a traditional Load Duration Curve, the x-axis should be the bins variable, and the y-axis should be
the counts variable.

\ppType{ValueDuration}{ValueDuration}
%
\begin{itemize}
  \item \xmlNode{target}, \xmlDesc{comma separated strings, required field}, specifies the names of the
    target(s) for which Value Duration histograms should be generated.
  \item \xmlNode{bins}, \xmlDesc{integer, required field}, specifies the number of bins that the values of the
    targets should be counted into.
\end{itemize}

\textbf{Example:}

\begin{lstlisting}[style=XML]
<Simulation>
 ...
  <Models>
    ...
    <PostProcessor name="pp" subType="ValueDuration">
      <target>x, y</target>
      <bins>100</bins>
    </PostProcessor>
    ...
  </Models>
 ...
</Simulation>
\end{lstlisting}

%%%%%%%%%%%%%% FastFourierTransform %%%%%%%%%%%%%%%%%%%
\subsubsection{FastFourierTransform}
\label{FastFourierTransformPP}
The \xmlNode{FastFourierTransform} postprocessor provides access to the Numpy fast fourier transform function
\texttt{numpy.fft.fft}
and provides the frequencies, periods, and amplitudes from performing the transform. The periods are simply
the inverse of the frequencies, and the frequency units are the deltas between pivot values in the provided
input. For example, if data is collected every 3600 seconds, the units of frequency are per-hour.  This
postrpocessor expects uniformly-spaced pivot values. Note that for each realization in the input data object,
a separate fft will be created for each target.

The \xmlNode{FastFourierTransform} postprocessor can act on any target in a DataObject that depends on a
single index, and generates three histories per sample per target: an independent variable
\xmlString{target\_fft\_frequency}, and two dependent values \xmlString{target\_fft\_period} and
\xmlString{target\_fft\_amplitude}, which both depend on the frequency by default. In all three outputs,
\emph{target} is replaced by the name of the target for which the fft was requested.

\ppType{FastFourierTransform}{FastFourierTransform}
%
\begin{itemize}
  \item \xmlNode{target}, \xmlDesc{comma separated strings, required field}, specifies the names of the
    target(s) for which the fast Fourier transform should be calculated.
 \end{itemize}
\textbf{Example:}

\begin{lstlisting}[style=XML]
<Simulation>
 ...
  <Models>
    ...
    <PostProcessor name="pp" subType="FastFourierTransform">
      <target>x, y</target>
    </PostProcessor>
    ...
  </Models>
 ...
</Simulation>
\end{lstlisting}
%%%%%%%%%%%%%% SampleSelector %%%%%%%%%%%%%%%%%%%
\subsubsection{SampleSelector}
\label{SampleSelectorPP}
The \xmlNode{SampleSelector} postprocessor is a tool to select a row from a dataset, depending on different
criteria. The different criteria that can be used are listed below.

The \xmlNode{SampleSelector} postprocessor can  act on any \xmlNode{DataObjects}, and generates a
\xmlNode{DataObject} with a single realization in return.

\ppType{SampleSelector}{SampleSelector}
%
\begin{itemize}
  \item \xmlNode{criterion}, \xmlDesc{string, required field}, specifies the criterion to select the
    realization from the input DataObject. Options are as follows:
    \begin{itemize}
      \item \xmlString{min}, choose the realization that has the lowest value of the \xmlNode{target}
        variable. The target must be scalar.
      \item \xmlString{max}, choose the realization that has the highest value of the \xmlNode{target}
        variable. The target must be scalar.
      \item \xmlString{index}, choose the realization that has the provided index. The index must be an
        integer and is zero-based, meaning the first entry is at index 0, the second entry is at index 1, etc.
        The realization order is taken from the order in which they were entered originally into the input
        DataObject. If this option is used, the \xmlNode{criterion} node must have an \xmlAttr{value}
        attribute that gives the index.
    \end{itemize}
  \item \xmlNode{target}, \xmlDesc{string, optional field}, required if the criterion targets a particular
    variable (such as the minimum and maximum criteria). Specifies the name of the
    target for which the criterion should be evaluated.
\end{itemize}

\textbf{Example:}

\begin{lstlisting}[style=XML]
<Simulation>
 ...
  <Models>
    ...
    <PostProcessor name="select_min" subType="SampleSelector">
      <target>x</target>
      <criterion>min</criterion>
    </PostProcessor>
    ...
    <PostProcessor name="select_index" subType="SampleSelector">
      <criterion value='3'>index</criterion>
    </PostProcessor>
    ...
  </Models>
 ...
</Simulation>
\end{lstlisting}