The proposed autoscaling system scales a web application in response to change in throughput at fixed intervals, which we denote by reconfiguration intervals of 10 minutes.  This system operates alongside the services deployed by a PaaS. Although it can be integrated into any other cloud platform easily. To feed this system, we rely on a monitoring engine provided by the cloud platform to track the application-workload and system resources. This architecture has the following key components:

%aThis system is able to provision the right amount and type of resources for minimizing the number of SLA violations while keeping the operational cost adapted to the customer requirements.

%Vertical and horizontal scaling actions can be applicable to satisfy the service demand.

\vspace{2mm}

\textbf{Predictor: } To prevent SLA violations in advance, workload prediction is needed to estimate the incoming traffic. However, this operation becomes more problematic by the fact that Web application workloads are often very unstable following an irregular pattern, with occasional short traffic spikes. Based on that and inspired by~\cite{wolski_network_1999}, our Predictor component takes the monitoring data as input and uses different time-series analysis techniques to predict the future service demand for the next time interval. To provide accurate forecasting measures, this component utilizes the technique that exhibited the lowest cumulative error measure during the previous monitoring window. By doing so, the Predictor is able to adapt the predictions to the current type of workload. This component supports five distinct statistical models that fits four types of workload: \emph{(1)} \emph{Linear Regression} for linear trends~\cite{muppala_regression-based_2012}, \emph{(2)} \emph{Auto Regression Moving Average} (ARMA) for linear with small oscillations~\cite{roy_efficient_2011}, \emph{(3)} \emph{Exponential Smoothing} for daily and seasonal~\cite{exponential_smoothing2010}, and \emph{(4)} \emph{Autoregression} and \emph{Vector Autoregression} for correlated trends~\cite{vector_autoregression_2006,chandra_dynamic_2003}.

\vspace{2mm}

\textbf{Profiler: } To choose a scaling plan, provisioning systems have to know the computing capacity of the different server configurations when running an application. To do that, the system creates profiles for each instance's type by analyzing its performance behavior (e.g. measuring the cpu usage, request rate and response times).  The definition of profiles per-instance's type improves the accuracy of the scaling actions, specifically in heterogeneous cloud infrastructures. Thus, the Profiler component calculates the optimized throughput of each type of provisioned configurations (VM instance in the following). By optimized throughput, we mean the performance pattern under which a resource assures the QoS requirements while avoiding its under/over-utilization.

% without any server instrumentation.

\vspace{2mm}

\textbf{Dynamic load balancer: } In order to adapt our system to the heterogeneity of requests and cloud resources, this component dynamically assigns weights to the backend servers to proportionally distribute the incoming traffic based on the performance characteristics of each backend. As an example, a server with four cores is able to process a higher number of requests so its weight has to a be greater than a server with only one core.

\vspace{2mm}

\textbf{Scaler:} This component governs our autoscaling system. As illustrated in Figure~\ref{autoScalingSys}, Scaler uses the Predictor and Profiler components to find the optimal provisioning strategy that fulfills a pre-established SLO (Service Level Objective). Furthermore, this component constantly analyzes the behavior of each provisioned VM triggering the Dynamic load balancer component when necessary.

\begin{figure}[htb]
  \begin{center}
    \includegraphics[width=.85\linewidth]{images/monitoringSchema}
  \end{center}
\vspace{-5mm}
  \caption{Autoscaling system.}
  \label{autoScalingSys}
\end{figure}

In the following, we focus on answering when to scale and which scaling plan to choose when running web applications.

\subsection{Provisioning on demand}
The Scaler component is the central component of this autoscaling system, and thereby is the responsible of triggering scaling actions. Our system supports horizontal scaling as a technique to add and remove resources to an application. Horizontal scaling enables to create a cluster of virtual machines associated to one application whose size is dynamically adapted to the workload fluctuations by adding or removing resources to the cluster. This scaling technique is commonly supported for the majority of resource provisioning systems. However, our system is able to use the heterogeneity of the cloud infrastructures to improve the efficiency of the scaling actions.

% On the other hand, in our system, vertical scaling enables to add or remove resources with high or less cpu and memory using \emph{service live migration}. Using this mechanism, the Scaler migrates the hosted service (web applications) from one virtual machine to another with a better hardware configuration. To reduce the degradations caused by this type of migration or by the booting time of the new machine, the Scaler checks the proper functioning of the new machine in order to release the older one.

In order to decide when to trigger a scaling action, our system uses the following three methods:
\begin{itemize}

\item \emph{Response time analysis:} A precise analysis of the monitoring data is crucial to minimize performance degradations that are sometimes omitted by traditional analytical methods such as the median, average or moving average. To obtain precise information (response times) that indicates the performance behavior of an application, we decided to extend a known smoothing technique called \emph{Weighted Moving Average} (WMA). Using our extension of WMA, it first associated weights in an increasing order giving more importance to the latest monitoring data; and second it doubles the original weight value of each data-point that exceeds the performance requirements (response time). By doing so, our system analyzes the monitoring data giving special importance to the data points on which SLO violations have occurred, and thereby is able to detect and avoid the maximum number of violations.

\item \emph{Definition of a reactive threshold:} Additionally to the fixed threshold pre-defined by the QoS requirements, our system also includes an extra threshold (based on these requirements) that will enable in advance to react against any traffic variation minimizing the number of SLO violations or under-utilization of the provisioned resources. This reactive threshold specifies one upper and lower boundaries creating two head-rooms between them and the performance requirements (response time) pre-defined by the user.

\item \emph{Workload forecasting: }  The \emph{Predictor} component is utilized to trigger short-term forecasts operations that will minimize at the maximum the number of SLA violations, as well as keep a high level of efficiency in the predictions.
\end{itemize}

Based on these techniques, the \emph{Scaler} evaluates whether the current performance behavior requires to trigger any of the following scaling actions:

\begin{itemize}

\item \textbf{Scale out:} Additional resources are provisioned if the loaded system state (response time or cpu utilization) exceeds the upper threshold, and the \emph{Predictor} confirms that such traffic changes will remain at least during the next monitoring window (in our experiments 5min).

\item \textbf{Scale back:} Resources are released if the loaded system state exceeds the lower threshold, and the \emph{Predictor} confirms that such traffic changes will remain at least during the next monitoring window.

\end{itemize}

%\subsubsection{Response time analysis - Time-series smoothing}

%When minimizing the SLA violations, a precise analysis of the monitoring data is crucial to improve the accuracy of the scaling decisions. It allows to %identify the resource requirements from the current workload type. This analyisis has even more importance when hosting web applications (e.g. %Wikipedia, Amazon) that need to provide high availability and performance to their clients. Moreover, the workload heterogeneity of this sites requires a %meticulous analysis of every monitoring data-point to reduce at the maximum the number of SLO violations.

%\begin{figure}[htb]
%	\begin{minipage}[b]{0.49\linewidth}
%		\includegraphics[width=4cm,height=3cm]{images/data2007/proxy_outputAvg.pdf}
%		\vspace{-4mm}
%	\end{minipage}
%	\hfill
%	\begin{minipage}[b]{0.49\linewidth}
%		\includegraphics[width=4cm,height=3cm]{images/data2007/proxy_outputWMA.pdf}
%		\vspace{-4mm}
%	\end{minipage}
%\caption{Response times for a trace: Average vs extended WMA.}
%\label{fig:data_analysis}
%\end{figure}

%To obtain precise information (response times) that reflects the performance behavior of an application, we decided to extend a known smoothing technique called \emph{Weighted Moving Average} (WMA). This technique is widely used in resource provisioning systems instead of others such as the median, average or moving average. Using our extension of WMA, it first associated weights in an increasing order giving more importance to the latest monitoring data; and second it doubles the original weight value of each data-point that exceeds the performance requirements (response time). By doing so, the Scaler analyzes the monitoring data giving special importance to the data points on which SLO violations have occurred, and thereby being able to detect and avoid the maximum number of violations. As an example, Figure~\ref{fig:data_analysis} shows how several SLO violantions are omitted when analyzing the monitoring data using an average analytical method (left) than when using our version of WMA (right).

%\subsubsection{Definition of a reactive threshold}

%\fixme{Perhaps, you may want to omit this subsection, as it is more related to When to provision than How to provision ?}

%Initially, the user defines several performance requirements that will be utilized by the Scaler to trigger scaling actions. As we mentioned, these QoS goals are previously specified in the Service Level Objective (SLO). In particual, when hosting web applications, these performance requirements are usually related to the maximum processing time needed to serve any request (response times). Regarding SLO fulfillment the majority of resource provisioning system aims to maintain the processing time as closer as possible to the performance requirements. However, this operation can become a problem, as provisioning systems are vulnerable to temporal traffic oscillations causing SLO violations.

%Therefore, and based on the requirements, the Scaler defines a reactive threshold that will enable in advance to react against any traffic oscillation minimizing the number of SLO violations or period of time under-utilization of the provisioned resources. As shown in Figure~\ref{threshold}, this reactive threshold specifies one upper and lower boundaries creating two head-rooms between them and the performance requirements pre-defined by the user (for the response time). In the future, these two boundaries could be adjusted depending on the hardware configuration of each provisioned vm, as introduced in~\cite{beloglazov_adaptive_2010}.

%\begin{figure}[htb]
 % \begin{center}
  %  \includegraphics[width=.85\linewidth]{images/thresholdGraphic.jpg}
 % \end{center}
%\vspace{-5mm}
%  \caption{Reactive threshold.}
%  \label{threshold}
%\end{figure}

%\subsection{Online profiling of allocated instances\label{profiling}}
\subsection{Measuring VM instance performance via online profiling\label{profiling}}

Regarding the resource heterogeneity of cloud infrastructures, a sensitivity analysis of the allocated resources is crucial to provide accurate scaling decisions. Accordingly, online profiling-based techniques~\cite{kaviani_profiling-as--service:_2011} have recently emerged as a solution to  estimate the resource's throughput under a certain workload. Traditionally, this technique replicates at runtime a server hosting an application, with a new server with profiling instrumentation that analyzes the performance behavior of a specific resource under a fixed percentage of the incoming traffic~\cite{jiangThesis,dejavu2012}. The use of fixed workloads to calculate the maximum throughput of a resource do not necessary imply the definition of more accurate profiles. A reason seems to be the lack of ideal performance isolation in cloud infrastructures which affects to the precision of the definitions, and consequently reduces the accuracy of the decisions. Furthermore, setting a profiling environment, in a heterogeneous cloud infrastructure, requires as many additional resources as number of available configurations.

Even though the use of online-profiling techniques is still under study, the configuration of a parallel environment and the heterogeneity of the cloud resources are the major drawbacks for its adoption. These techniques increase the operational cost and do not necessarily improve the accuracy of the scaling decisions.

%Even though the use of online-profiling techniques is still under study, the configuration of a parallel %environment and the limited accuracy of its scaling decisions are the major drawbacks for its %adoption.

As a consequence, we designed a novel online profiling technique that gives an estimation of the optimized throughput of each allocated instance's type without need for additional resources or parallel environments. To do that, the Profiler component use the provisioned resources for the analysis and estimation of the computing capacity of each available hardware configuration.

\begin{algorithm}
{\scriptsize
\SetAlgoLined
\SetInd{0mm}{2mm}
\KwData{ \\Service Level Objective, \emph{slo} \\ List of allocated instance types, \emph{inst\_types} \\ Compute units per \emph{inst\_type}, \emph{compute\_units$_{inst}$}}

\KwResult{VM instances performance classification, \emph{list\_perf}}

\While{ allocated instances to profile}{
\BlankLine
Collect profiling data of \emph{inst\_type}: \emph{req\_rate, cpu\_usage and resp\_time}\;

\While{ profiling data to smooth}{  \label{alg:smooth}
// Perform smoothing percentiles technique \;
\If{ \emph{resp\_time}$_i$ $>$ (slo * 0.25) and \emph{resp\_time}$_i$ $<=$ (slo * 0.75)}{
	\If{ \emph{req\_rate}$_i$ $>$ 0 and \emph{cpu\_usage}$_i$ $<$ 75}{
		\hspace{3mm} Add \emph{cpu\_usage$_i$} to cpu\_usage\_data\;
		\hspace{3mm} Add \emph{req\_rate$_i$} to req\_rate\_data\;
		\hspace{3mm} Add \emph{resp\_time$_i$} to resp\_time\_data\;
}
}
} \label{alg:endSmooth}
\BlankLine
Initialize instance capacity \emph{Ideal\_Throughput$_{inst}$} to 0 \;
\uIf{ \emph{resp\_time\_data}, \emph{cpu\_usage\_data}, \emph{req\_rate\_data}  \textbf{not empty}}{
Calculate average of \emph{cpu\_usage\_data}, \emph{CPU\_usage$_{inst}$} \; \label{alg:throughput_start}
Calculate average of \emph{req\_rate\_data}, \emph{Num\_requests$_{inst}$} \;
\BlankLine
\emph{Ideal\_Throughput$_{inst}$} = $\dfrac{\emph{CPU\_usage$_{inst}$} } {\emph{Num\_requests$_{inst}$} }$  \; \label{alg:throughput_end}
\BlankLine
Store instance computing capacity, \emph{Ideal\_Throughput$_{inst}$} \;
}
\Else{
	Use historic value of \emph{Ideal\_Throughput$_{inst}$} for \emph{inst\_type}\;
}
\BlankLine
// Classify \emph{inst\_type} based on its computing capacity \; \label{alg:clas}
\If{ Ideal\_Throughput$_{inst}$ == 0 }{
Use \emph{compute\_units$_{inst}$} of \emph{inst\_type} to rank \emph{inst\_type} in \emph{list\_perf}\;
}
\Else{
Use new value of \emph{Ideal\_Throughput$_{inst}$} to rank \emph{inst\_type} in \emph{list\_perf}\; \label{alg:end_clas}
}
}
}
\caption{VM instance profiling algorithm}
\label{profilingAlg}
\end{algorithm}

As detailed in Algorithm~\ref{profilingAlg}, once the Scaler component decides to trigger a scaling action, the Profiler component estimates the throughput of the allocated instances according the following steps:

\begin{enumerate}
\item Collect the latest hour of monitoring data from each VM instance's type. This profiling data contains information about monitoring metrics such as the request rate, cpu usage and response time, which provide enough feedback for the definition of instance profiles.

\item Perform a smoothing technique over the profiling data of each instance to remove the noise generated by traffic spikes. As pointed out in~\cite{gandhi_hybrid_2012}, when hosting web applications sudden changes in the workload, interference due to virtualization or OS activities may affect the precision of the profiling process. Hence, we decided to smooth the profiling data during the latest hour (or older if there is not enough data) to identify the performance capacity of one VM instance. In particular, the Profiler extracts the smoothed 75th and 25th percentiles from the response times below the SLO and 75th percentile from the cpu usage data-points. (See Lines \ref{alg:smooth}-\ref{alg:endSmooth}). This mechanism allows to identify the optimized throughput of each instance while enforcing the performance requirements and avoiding CPU saturation.

As an example in Figure~\ref{fig:vm_performance1} and Figure~\ref{fig:vm_performance2}, we show the profiling data and percentiles for one "m1.small" and one "c1.medium" EC2 instance types during one hour. The green areas represent the ranges of response times and cpu usage comprised by the percentiles. Red circles contain all data points that will be used to calculate the optimized throughput of one VM instance. As we mentioned above, some data points are excluded as they identify periods of time on which resources suffered from under-utilization or over-utilization (denoted by white areas). Furthermore, as shown in Figure~\ref{fig:vm_performance1}, a short number of data points are comprised between the percentiles for the "m1.small" instance. Its poor hardware configuration makes this type of instance more vulnerable to sudden changes in the workload.

%and three red circles highlight several data-points on which the performance of the instance was optimal while enforcing the %SLO.

%\begin{figure*}[htb]
%	\begin{minipage}[b]{0.5\linewidth}
%		\includegraphics[height=4.5cm]{images/vm_performance_resp_smallEC2Remark.pdf}		%\includegraphics[height=4.5cm]{images/vm_performance_cpu_smallEC2Remark.pdf}
%		\vspace{-4mm}
%	\end{minipage}
%	\hfill
%	\begin{minipage}[b]{0.5\linewidth}
%		\includegraphics[height=4.5cm]{images/vm_performance_resp_c1mediumRemark.pdf}		%\includegraphics[height=4.5cm]{images/vm_performance_cpu_c1mediumRemark.pdf}
%		\vspace{-4mm}
%	\end{minipage}
%\caption{Profiling data and percentiles of m1.small and c1.medium EC2 instances.}
%\label{fig:vm_performance}
%\end{figure*}

\begin{figure}
  \begin{center}
    \includegraphics[width=\linewidth]{images/idealSmallRemark.pdf}
  \end{center}
\vspace{-5mm}
  \caption{Profiling data and percentiles of m1.small.}
  \label{fig:vm_performance1}
\end{figure}

\begin{figure}
  \begin{center}
    \includegraphics[width=\linewidth]{images/idealc1MediumRemark.pdf}
  \end{center}
\vspace{-5mm}
  \caption{Profiling data and percentiles of c1.medium.}
  \label{fig:vm_performance2}
\end{figure}

%\begin{figure*}[htb]
%	\begin{minipage}[b]{0.45\linewidth}
%		\includegraphics[height=8.5cm]{images/idealSmallRemark.pdf}
%		\captionof*{figure}{ \textbf{m1.small}}
%		\vspace{-4mm}
%	\end{minipage}
%	\hfill
%	\begin{minipage}[b]{0.45\linewidth}
%		\includegraphics[height=8cm]{images/idealc1MediumRemark.pdf}
%		\captionof*{figure}{ \textbf{c1.medium}}
%		\vspace{-4mm}
%	\end{minipage}
%\caption{Profiling data and percentiles of m1.small and c1.medium EC2 instances.}
%\label{fig:vm_performance}
%\end{figure*}

\item Classify the different instance types depending on its computing capacity. Using the profiling smoothed data of each instance type (refer to Lines \ref{alg:throughput_start}-\ref{alg:throughput_end}), the Profiler computes a factor, named \emph{Ideal\_Throughput$_{inst}$}, as the average cpu utilization required to process the average of served requests (cycles/requests) for one instance, where \emph{CPU\_usage$_{inst}$} represents the average of  cpu usage (clocks/sec) and \emph{Num\_requests$_{inst}$} the average of request rate (requests/sec), as illustrated in the next Equation~\ref{cpu_speed}:

{\scriptsize
\begin{equation}\label{cpu_speed}
\begin{split}
CPU\_usage_{inst} = \dfrac{   \sum_{i=1}^N \big( cpu\_usage\_data_{i}  \big) } { N } \\
Num\_requests_{inst} = \dfrac{   \sum_{i=1}^N \big( req\_rate\_data_{i} \big) } { N } \\
Ideal\_Throughput_{inst} =\dfrac{   CPU\_usage_{inst}  } {  Num\_requests_{inst}  }
\end{split}
\end{equation}
}

The \emph{Ideal\_Throughput$_{inst}$} factor gives an estimation of the optimized performance capacity of one VM instance when processing the current workload. Based on the value of this factor per-instance, the \emph{Profiler} classifies the different instance-types based on its computing capacity when running an application. (See Lines \ref{alg:clas}-\ref{alg:end_clas}). The resulting classification gives an interesting feedback to the Scaler component, which is now able to identify the capacity of each instance type, and consequently to choose an optimal scaling plan. Initially, there are not instance profiles due to the lack of monitoring data, thereby the Scaler uses the number of compute units per instance as a priori classification of their performance capacities. Note that, this work only consider the compute units, but memory or bandwidth can be also taken into consideration to classify the VM instances.
\end{enumerate}

This profiling process allows to define a profile per instance type, thus facilitating the selection of an appropriate scaling plan that will satisfy the QoS requirements. As such, the profiling data can be used independently of factors such as performance isolation in clouds or request heterogeneity to improve the accuracy of the scaling decisions. Even though this profiling approach assumes that all the resources of the same type behave similarly, it can be easily extended to profile the allocated resources independently of its type.

\subsection{Scaling plan decision-making}

The definition of a scaling plan is the most important and challenging phase in a provisioning system. It is responsible of the selection of an appropriate combination of resources that fulfills the performance requirements with the lowest operational cost.

%This operation can become more laborious when running web applications, and therefore it has been barely addressed in the existing provisioning systems. Web applications are often a target of temporal traffic variations that may cause SLO violations having an impact in the user experience. Hence, the selection of an appropriate scaling plan becomes crucial to mitigate these penalties.

%On the other hand, to boost the volume of customers, the user experience appears as a mandatory requirement in web applications contrary to budget cuts that may reduce the advantage over competitors.

To overcome this task, the Scaler has to take into consideration two aspects: the workload requirements and the resource heterogeneity. An exhaustive analysis of the current workload allows to identify the future demand that will determine the new configuration to use. This operation can become more laborious when running web applications due to its workload heterogeneity. On the other hand, cloud infrastructures are highly heterogeneous, offering a wide range of different hardware configurations that can be combined to better satisfy the requirements. The selection of one configuration or another can expose the application to availability or performance issues, specifically when handling traffic anomalies. Hence, the selection of an appropriate scaling plan becomes crucial to mitigate these penalties.

%% Threshold fixed by the user

%\begin{figure*}[htb]
%	\begin{minipage}[b]{0.3\linewidth}
%		\includegraphics[height=4.5cm]{images/prediction_conpaas_6min.eps}
%		\vspace{-4mm}
%	\end{minipage}
%	\hfill
%	\begin{minipage}[b]{0.3\linewidth}
%		\includegraphics[height=4.5cm]{images/prediction_conpaas_10min}
%		\vspace{-4mm}
%	\end{minipage}
%	\hfill
%	\begin{minipage}[b]{0.3\linewidth}
%		\includegraphics[height=4.5cm]{images/prediction_conpaas_30min}
%		\vspace{-4mm}
%	\end{minipage}
%\caption{ConPaaS Predictions, response times for 5min, 10min and 30min ahead.}
%\label{fig:vm_performance}
%\end{figure*}

%\subsubsection{Scaling strategy decision-making}

\subsubsection{Analysis of the workload requirements}

Prior to any resource selection, the \emph{Scaler} has to measure the requirements of the current workload taken into account the traffic diversity of web applications. Traditional scaling systems gather monitoring data about the request volume or cpu usage consumed to identify the workload requirements. However, the workload of web applications is highly heterogeneous being defined by sudden changes in the traffic as well as constantly-changing request mixes. As a consequence, our system computes the workload complexity considering both the total cpu usage and request rate served by the current scaling plan. Both factors provide enough information to detect the causes of a performance degradation in web applications.

%In Equation~\ref{workload_complexity}, the \emph{$Workload_{complex}$} factor represents the complexity of the incoming traffic and includes degradations caused by operative system activities, low network performance, performance isolation or request heterogeneity. These degradations are omitted by individually analyzing the request rate or cpu usage consumed by all the allocated resources at any given time. Thus, the \emph{$Workload_{complex}$} is computed by using the sum of the cpu usage (denoted by \emph{$CPU\_usage_{i}$}) and the performance capacity of each allocated resource. The performance capacity is calculated as the average of the request rate served by each resource (denoted by $Num\_reqs_{i}$) and its ideal throughput (denoted by $Ideal\_Throughput_{inst_{i}}$). By using the ideal throughput instead of calculating the current throughput, the Scaler is able to measure how much the current workload is affecting the ideal performance pattern of a VM instance.

The workload complexity is a factor that identifies how much the current workload is affecting the ideal performance pattern of the allocated resources. It identifies the complexity of the incoming traffic and includes degradations caused by operative system activities, low network performance, performance isolation or request heterogeneity. These degradations affect the performance and are often omitted by individually analyzing the request rate or cpu usage consumed by the allocated resources.

To calculate the workload complexity (denoted by \emph{$Workload_{complex}$}), we first analyzes the monitoring data gathering the request rate (denoted by $Num\_reqs\_server_{i}$) and cpu utilization (denoted by \emph{$CPU\_usage\_server_{i}$}) consumed by the allocated resources. Secondly, unlike the \emph{CPU$_{inst}$},  we calculate the expected cpu usage of each resource (denoted by \emph{CPU\_usage\_expected$_{i}$} ) using its current request rate (denoted by $Num\_reqs\_server_{i}$) and its optimized throughput (denoted by $Ideal\_Throughput_{inst_{i}}$). Finally we compute the \emph{$Workload_{complex}$} factor using the sum of cpu usage consumed to handle the current workload and the sum of expected cpu usage of all the resources, as illustrated in Equation~\ref{workload_complexity}.

{\scriptsize
\begin{equation}\label{workload_complexity}
\begin{split}
CPU\_usage\_expected_{i} =  Num\_reqs\_server_{i}  * Ideal\_Throughput_{inst_{i}} \\
Workload_{complex}  = \dfrac{ \sum_{i=1}^N CPU\_usage\_server_{i}  }  {  \sum_{i=1}^N  CPU\_usage\_expected_{i}     } = \dfrac{ \ (cycles / sec) }  {  (cycles / sec ) }
\end{split}
\end{equation}
}

Additionally, the total cpu utilization ($\sum_{i=1}^N CPU\_usage\_server_{i} $) and request rate ($\sum_{i=1}^N  Num\_reqs\_server_{i}$) can be calculated by the Predictor component to estimate the future service demand of the application for the next monitoring windows (in our experiments 30min). This enables to select in advance a scaling strategy that will handle future fluctuations in the workload, and thereby saving cost. As an example in Figure~\ref{fig:forecast}, the Predictor shows the forecast values for the resource requirements during the next 30 minutes offering an acceptable level of accuracy. An aspect to notice is that the forecasting results never under-provision the system avoiding the generation of unnecessary SLO violations.

\begin{figure}[htb]
  \begin{center}
    \includegraphics[width=\linewidth,height=6cm]{images/prediction_conpaas_30min}
  \end{center}
\vspace{-5mm}
  \caption{Predictor component: forecasting results for 30min ahead.}
  \label{fig:forecast}
\end{figure}

\fixme{Calculate MAPE error for the forecasting results}

\subsubsection{Calculating the different scaling plans}

To decide which type and amount of VM instances to release or add, the \emph{Scaler} uses an optimal decision tree that calculates all the possible scaling strategies. By going through this tree, our system evaluates all the possible resource combinations using horizontal scaling. As shown by Figure~\ref{fig:scalingTree}, in the scaling decision tree, each node represents each type of hardware configuration offered by a cloud infrastructure. While the branches define all the possible resource combinations. When processing the current workload, to decide which resources to include in the scaling plan,  the \emph{Scaler} uses the following formula~\ref{resource_combination}.

%This formula determines the CPU usage consumed to process the current workload when using a specific resource combination (denoted by \emph{$CPU_{strategy}$}). Note that, CPU usage provides enough feedback about the performance capacity, as it is a function of the request complexity  and number of served requests.

%To calculate the different combinations of VM instances that enforce the performance requirements, we used the next

{\scriptsize
\begin{equation}\label{resource_combination}
\begin{split}
CPU_{strategy} = \dfrac{ \sum_{i=1}^n \bigg( \bigg( \dfrac{ Num\_reqs_{total} * Workload_{complex} }  {N}  \bigg) * Ideal\_Throughput_{inst_{i}} \bigg) }  {N} \\
%\\ {\small \textit{ If } CPU\_usage_{strategy} \leqslant CPU\_max_{SLO} }
\end{split}
\end{equation}
}

Equation~\ref{resource_combination} defines how the workload will be distributed across the different resources taken into account their performance capacities.  Thus, the \emph{Scaler} selects a strategy that uses \emph{N} resources to distribute the \emph{$Num\_reqs_{total}$} (total rate of served requests) with a workload complexity \emph{$Workload_{complex}$} across \emph{N} instances with an optimized throughput \emph{$Ideal\_Throughput_{inst}$}. \emph{$CPU_{strategy}$} measures the CPU usage consumed (cycles/sec) to process the current workload when using a specific resource combination of \emph{N} resources.

%As an example, Figure~\ref{fig:scalingTree} shows three scaling plans that propose three different resource combinations: (left) a strategy is selected by adding two new \emph{small} instances, (center) a strategy proposes to replace one \emph{small} instance by one \emph{large}, and (right) both operations are combined by replacing one \emph{small} instance by one \emph{medium} and then adding one \emph{small}.

Note that, the search space of possible strategies can be as large as the number of available hardware configurations, and allocated resources in the current scaling plan. It makes hard-to-find the best resource combination in a reasonable short period of time, so that the \emph{Scaler} filters the strategies based on the following criteria:
\begin{itemize}
\item Establish the maximum CPU usage consumed  by a strategy to process a particular workload (denoted by \emph{$CPU_{SLO}$}). It limits the number of possible plans where \emph{$CPU_{strategy} \leq CPU_{SLO}$}. \emph{$CPU_{SLO}$} specifies the cpu usage at which the application starts to experience SLO violations or over-utilization. As a result, the proposed scaling plans optimizes the current performance below values causing performance degradations.  Limit the number of possible scaling plans

%can be calculated either based on the Amazon EC2 recommendations ( CPU values always lower than 75\%), or based on the CPU values at which the application starts to experience SLO violations (or performance degradations).

\item Include a cost policy to avoid choosing wasteful scaling strategies. Considering the pricing model of cloud providers that charges users on a per-hour basis, the \emph{Scaler} includes a cost policy that rejects strategies releasing resources which have been recently started ( 5min < time to the end of its hour < 20). The \emph{Scaler} releases resources which are closed to the hour are free under the cloud pricing model, and there is no gain from terminating them before this hour price boundary.

\end{itemize}

These assumptions avoid to trigger new scaling actions in the next time interval, and consequently reduce the operational cost.

As an example, Figure~\ref{fig:scalingTree} shows a possible scaling plan obtained by using our decision tree and Equation~\ref{resource_combination}. This plan proposes to add two \emph{small} and one \emph{huge} instances to one initial \emph{small} instance. When the performance capacity of a current resource combination is not enough to handle the workload, we add new resources as long as the resulting \emph{$CPU_{strategy}$} (calculated by adding the new resource) is lower than \emph{$CPU_{SLO}$}. This searching process stop when all the possible combinations have been proved or there are not more combinations satisfying the aforementioned criteria.

\begin{figure}[htb]
  \begin{center}
    \includegraphics[width=\linewidth,height=4.5cm]{images/decisionTree2}
  \end{center}
\vspace{-5mm}
  \caption{Decision tree: example of a scaling plan using four VM instances.}
  \label{fig:scalingTree}
\end{figure}

In summary, using the Equation~\ref{resource_combination}, the Scaler is able to answer to the question \emph{"How many and which type of VM instances to provision?"}.

%is calculated using the Profiler component and will variate depending on the type of vm instance used in the strategy.

\subsubsection{Selecting the optimal scaling plan}

%To refine this search process according to the final goal and adapted to the customer preferences (user experience), the Scaler provides three classes of SLA agreements in function of the type of customer. These three classes of customers namely, gold, silver and bronze minimize the SLA violations with a different infrastructure cost. Accordingly, a gold customer pays more in order to get the best service at the cost of some extra over-provisioning. A silver customer gets good availability while a bronze customer obtains a reduced, but acceptable, SLA fulfillment but with very little over-provisioning.

%they are classified based on their infrastructure cost and degree of SLO fulfillmen

Once the \emph{Scaler} obtains a list of filtered scaling strategies, it computes the cost of each possible strategy how the cost incurred by its SLO fulfillment degree and the infrastructure cost. The infrastructure cost (denoted by \emph{infra\_cost}) specifies the cost required to provision additional resources for the next hour. While the SLO fulfillment degree (denoted by \emph{slo\_fulfillment}) indicates the vulnerability of a strategy to experience SLO violations.

As defined in the Equation~\ref{strategy_cost}, the \emph{slo\_fulfillment} cost is calculated given the percentage of \emph{$CPU_{strategy}$} and the \emph{$CPU_{SLO}$}. Obviously, higher values in the percentage of \emph{$CPU_{strategy}$} imply an increment in the probability of having SLO violations under traffic spikes, so the \emph{slo\_fulfillment} will increase as well.

%As defined in the Equation~\ref{strategy_cost}, the \emph{slo\_fulfillment} cost is calculated given the percentage of \emph{$CPU_{strategy}$} and the \emph{$CPU_{SLO}$}, multiply by a SLO penalty (in \$)  pre-established between the customer and provider. Obviously, higher values in the percentage of \emph{$CPU_{strategy}$} imply an increment in the probability of having SLO violations under traffic spikes, so the \emph{slo\_fulfillment} will increase as well.

%Therefore, according to the customer preferences, the Scaler can select one strategy with the lowest, highest or medium cost.

{\scriptsize
\begin{equation}\label{strategy_cost}
\begin{split}
infra\_cost = \sum_{i=1}^N \big( instance\_price_{i} \big) \\
%slo\_fulfillment =  \bigg( \dfrac{ CPU_{strategy} } {CPU_{SLO}} \bigg) * SLO penalty \\
slo\_fulfillment =  \bigg( \dfrac{ CPU_{strategy} } {CPU_{SLO}} \bigg)  \\
cost\_strategy = \dfrac{  slo\_fulfillment  } {infra\_cost}
\end{split}
\end{equation}
}

%\fixme{I could omit the use of the SLO penalty, as it is a constant in my experiments. Instead I could say that the system adapts to the user needs.}

The criteria for the selection of the optimal strategy is configurable, and thereby it can adapted to the customer preferences. For instance, large enterprises need to provide high availability and performance to their clients, disposing of a generous budget for such a mission.

%Moreover, \emph{$SLO_{penalty}$} can be also used as a criteria for the selection of the most appropriated strategy according to some requirements.

%To decide which type of instance to release, the Scaler uses a conservative algorithm that releases the resources with the lowest utilization by analyzing their monitoring data. Similarly, a cost policy was also included for avoiding to choose wasteful scaling decisions. Considering the pricing model of cloud providers that charges users on a per-hour basis, the Scaler includes a cost policy that rejects strategies releasing vm instances which have been recently started ( 5min < time to the end of its hour < 20). Scaler releases resources which are closed to the hour are free under the cloud pricing model, and there is no gain from terminating them before this hour price boundary.

%A complete point of view of the whole scaling flow is shown in Figure~\ref{autoScalingFlow}.

%\begin{figure}[htb]
%  \begin{center}
 %   \includegraphics[width=\linewidth]{images/NewAutoScalingFlow}
%  \end{center}
%\vspace{-5mm}
%  \caption{Auto scaling flow.}
%  \label{autoScalingFlow}
%\end{figure}

\subsection{Dynamic load balancing weights }

The problem we consider here is again the heterogeneity of cloud platforms.
Independently of its size, different VM'es from the same cloud might have different performance
characteristics, even when their specifications from the cloud vendor are
the same~\cite{ec2Performance}. This issue can be addressed through various
load balancing techniques, like assigning weights to the backend servers or
taking into account the current number of connections that each server
handles. Furthermore, the performance behavior of the virtual servers may
also fluctuate, either due to changes in the application's usage
patterns, or due to changes related to the hosting of the virtual servers
(e.g., VM migration).

\begin{figure}[htb]
  \begin{center}
    \includegraphics[height=5cm]{images/load_balancing}
  \end{center}
\vspace{-5mm}
  \caption{Weighted load balancing flow.}
  \label{fig:load_balancing}
\end{figure}

In order to address these issues, the Scaler implemented a weighted
load balancing system in which the weights of the servers are
periodically re-adjusted automatically, based on the monitoring data.
This method assigns the same weight to each backend server at the
beginning of the process. As illustrated in Figure~\ref{fig:load_balancing}, the weights are then periodically
adjusted (in our experiments, using a monitoring window of $\sim$ 15min) proportionally
with the difference among the average cpu usage and request rate of the servers
during this time interval. By adding this technique to our autoscaling system,
the workload can be dynamically and proportionally distributed across the provisioned VM instances depending on their performance capacities.

%we noticed a performance improvement when running the benchmarks, as discussed in the following.


\section{Conclusions \& Future Work}
\label{conclusion}

The WORG method provides a deployment schedule optimizer that converges
both closely enough and fast enough to be used inside
of a nuclear fuel cycle simulator. The algorithm can consistently obtain
tolerances of half-a-percent to a percent (1 GWe distances for over 200 GWe
deployable) for the once-through fuel cycle featured here within only five to
ten simulations. Such optimization problems are made
more challenging due to the integral nature of facility deployment and
that any demand curve may be requested.

WORG works by setting up a Gaussian process to model the production
as a function of time and the deployment schedule. This model may then
be evaluated orders of magnitude faster than running a full simulation, enabling
the search over many potential deployment schedules. The quality of these
possible schedules is evaluated based on the dynamic time warping distance
to the demand curve. The lowest distance curve is then evaluated in a
full fuel cycle simulation. The production curve that is computed by the
simulator in turn goes on to update the Gaussian process model and the
cycle repeats until the limiting conditions are met.

However, choosing the deployment schedules to estimate with the Gaussian
process may be performed in a number of ways. A blind approach would
simply be to choose such schedules randomly from a univariate. However,
the WORG method has more information available to it that helps drive
down the number of loop iterations. The first method discussed remains
stochastic but uses the inverse DTW distances of the GP model to
weight the deployment options, falling back to a Poisson distribution as
necessary. This second method minimizes the model distance for each point
in time from start to end, iteratively building up a solution. Finally,
another estimation strategy tries both previous options and chooses the
best result, forcing the stochastic method two of every four iterations
to avoid deterministic loops.  It is this last all-of-the-above method
that is seen to converge the fastest and to the lowest distance in most
cases.

It is important to note that the WORG algorithm is applicable to any
demand curves and fuel cycle facility types. It is not restricted to
reactors and power. Enrichment and separative work units, reprocessing
and separations capacity, and deep geologic repositories and their
space could be deployed via the WORG method for any applicable demand
curve. Furthermore, several of these demands and deployments can be
examined simultaneously. For instance, deployment schedules for light-water,
fast reactors, pebble-bed reactors, and separations facilities may be optimized
together on the joint
bases of minimizing cost, minimizing separated transuranics, and achieving
a specified power demand. The WORG algorithm fully supports such multivariate
problems. Reactors and power generation were chosen for study here as the
representative keystone example, though the algorithm is not tied to this
particular problem.

The speed of convergence of the WORG algorithm is effectively required for
high-fidelity simulation optimization. Single realizations may take hours or
longer to run. Executing hundred, thousands, or more realizations becomes
impractical. On the other hand, lower fidelity simulations that have short
execution times may be perfectly well optimized by an \emph{ex situ}
method. With fast run times, WORG would be implemented \emph{in situ} as
a mechanism for automation and avoiding external wrapper codes implicit
in an \emph{ex situ} optimizer. The speed of convergence for lower fidelity
simulations, while desirable conceptually, is not required in practice.

The next major step for this work is to actually employ the WORG method in
a fuel cycle simulator.  However, to the best knowledge of the
author, no existing simulator is capable of spawning forks of itself
during run time, rejoining the processes, and evaluating the results of the
child simulations in the parent simulation. Concisely, while many simulators
are `dynamic' in the fuel cycle sense, none are `dynamic' in the programming
language sense. This latter usage of the term is what is required to
take advantage of any sophisticated \emph{in situ} deployment optimizer.
The Cyclus fuel cycle simulator looks most promising as a platform
for such work to be undertaken. However, many technical roadblocks
on the software side remain, even for Cyclus.

Additionally, the WORG method does not utilize an expected decommissioning
schedule. If it is known prior to the simulation, this information
could help focus on a deployment schedule solution faster and more precisely.
Future work on WORG will consider optionally providing the decommissioning information.
This will likely affect the initial two optimizer cases and the
$M_p$ and $N_p$ deployment bounds.

Furthermore, adding \emph{in situ} capability also adds the additional
degree of freedom of how often to run the deployment schedule optimizer.
Running WORG each and every
time step seems excessive \emph{a priori}. Is every year, five years,
or ten years sufficient? How does this degree of freedom balance with the
time horizon $T$ specified in the optimizer? These questions remain unanswered, even
in a heuristic sense, and thus the frequency of optimization will be a key
parameter in a future \emph{in situ} study.


\section{Introduction}

A vital use of program analysis is correctness proving. Assuming we have a program and a set of criteria our program has to meet under any circumstance, proving that our program meets all of our criteria is an important task for critical software components. Additionally, if we expand our validity requirements to concurrent runs of one or more instances of the program, which itself might have several concurrent processes, our analysis threatens to become untractably complex. When we relax the constraints of sequential consistency to those defined by total store ordering (TSO) and partial store ordering (PSO) for programs employing weak memory models (WMM), we face a seemingly untractable problem when trying to perform program analysis.

\subsection{Proof of correctness}

Let $P$ be a multi-process program using a WMM instruction set, let $E$ be a list of predicates on the variables of $P$, and let $a$ be a boolean expression describing the correctness assertions for $P$ using the predicates provided in $E$ and the program counter. We consider $P$ to be correct as defined by $E$, iff for every possible run scenario of any number of concurrent instances of $P$, in which any number of processes of $P$ may run concurrently, the assertion $a$ holds true in every program point of $P$. Terms involving the program counter are used to limit the necessity for $a$ (or parts thereof) to hold true. By abstracting $P$ as a boolean program using $E$, such a proof becomes feasible using a model checker.

\subsection{Weak memory models}

In all of our programs, we will differentiate between local resources, declared as local variables, and their remote counterparts. The main feature of WMM languages is the differentiation between local assignments and remote reads and writes. We consider local variables as being local both in the semantic sense (i.e. a process constitutes the scope of a local variable) and the technical sense (i.e. a local variable is stored in fast-access memory, such as a register or local cache space). In contrast, remote variables are globally shared across all processes and are assumed to be stored in shared memory, that's more time-consuming to access. We will continue using the words ``local'' and ``remote'' in this manner throughout this work.\\

Thus, the main advantage of WMM programming becomes clear, as the programmer will now be able to avoid costly remote memory accesses for operations restricted to one process, along with any adverse side-effects depending on the caching policy. This does not, however, mean that a remote statement will always access remote memory. For practical reasons, remote writes are immediately stored in a buffer, and only committed to remote memory in the case of a flush statement, which, as apparent from its definition, is inherently non-deterministic. In the same vein, remote reads will read from the local buffer if possible, and only access remote memory if the local buffer is empty. In this work, we will abstract the buffer as a series of local variables representing a FIFO queue. The buffering method depends on the memory model. In the case of PSO, every global variable in every process will have a dedicated buffer, whereas in the case of TSO, every process will have one buffer shared by all global variables.

\subsection{Difficulties of WMM}

Our program behaves differently depending on the memory ordering chosen, without this difference being readily apparent from the code. More precisely, when using PSO, the available buffer size defines a set maximum buffer size for every global variable per process, whereas when using TSO, it only defines the sum of the buffer sizes of every global variable per process. To illustrate this with an example, if we have a program with 2 global variables $x$ and $y$, with 4 remote writes in $x$ and 2 remote writes in $y$ before flushing, a per-process buffer size of 6 wouldn't result in a buffer overflow, as the sum of the maximum buffer sizes of $x$ and $y$ does not exceed 6. However, the same code results in an overflow if we define a maximum per-process per-variable buffer size of 3, since the 4th write in $x$ would exceed the maximum buffer size allocated to $x$ in that process.\\

We tackle this problem by converting our program $P$ from one generally exploiting all the features available in WMM into another program $P'$ that lacks any ``true'' WMM statements. I.e., our resulting program will only contain local assignments, remote reads, and remote writes, with no fences or flushes (we will redefine flushes as statements later on). Though remote reads and writes will still be featured in $P'$, they will simply describe immediate and deterministic remote access statements, without any of the advantages or pitfalls of their WMM counterparts. Finally, we can convert $P'$ into a boolean program using predicate abstraction, which, as described in 1.1, can be used to prove the correctness of $P$ regarding $a$.\\

Flushing is a part of WMMs. It describes an operation triggered by memory management at times usually out of the programmer's reach. In essence, flushing writes the buffer content to memory, deleting it from the buffer. Since the behaviour of flushing is an important aspect of WMMs to consider when attempting to analyse programs running under them, we need to be able to simulate its behaviour in a non-WMM environment. To this end, we define \lstinline$flush;$ to be an explicit statement in our program, and we will place this statement once wherever a flush might occur.

\subsection{Main contributions}

This work describes the implementation of a software tool that takes a program written in SALPL as input, a language we define below, and outputs a boolean program parsable by the Fender \cite{fender} model checker for verification. The source code and software is available at \lstinline{https://github.com/hetmeter/awmm} . The components of this tool break down as follows:

\paragraph{Parser}

We have built a lightweight parser that takes a syntactically correct SALPL program and a language definition of SALPL as input, and outputs an abstract semantic tree representation into a file. This specific form of AST representation is then used to perform further operations on the program.

\paragraph{Buffer size analysis}

We traverse the control flow tree of the program, and record the number of store and fence statements we encounter on our path. For every store statement, we increment the buffer size needed to accomodate all stores so far. For PSO programs, this is stored separately for each process and each variable in a value to which we will refer as $s_{x, i}$ for the variable \lstinline$x$ and the process $i$. For TSO programs, this value is only kept by process, independent of variables. Analogously to its PSO counterpart, we will refer to this value as $s_i$. For buffer sizes exceeding $K$, as well as for values that may be arbitrarily high (which we will indicate by assigning the value ``TOP''), we will store the value $K$.

\paragraph{Program rewriting}

Converting the WMM program to one lacking buffer behaviour influence is done by visiting every program point of the type store, load, fence, or flush. For every program point, we will have already computed all parameters needed to generate the replacement code. In this phase, we carry out the replacements we define in 3.3.

\paragraph{Predicate abstraction}

We take our rewritten program, which is now devoid of any operations on the buffer. In this phase, our tool generates tables for every weakest liberal precondition and implicative cube disjunction we use for the predicate abstraction of our program. These components and techniques are expanded upon in section 4. We use Z3 \cite{z3} as a theorem prover to prove or disprove implications. As a general rule, we try to minimize the calls to the theorem prover, and we do this with a combination of storing all computed results for later use, omitting cubes that form supersets of already found implicative cubes, and renaming heuristics, with which we reduce the problem of finding cube disjunctions for predicates involving derived variables to the same problem involving the global variables from which they were derived.

\paragraph{Model checking}

While not strictly a part of this work, our output is tailored to be used with the model checker Fender \cite{fender}. Our output can be directly fed into the input of Fender for analysis.


% !TEX root = main.tex
% !TEX spellcheck = en-US
\section{Definitions and lemmas for multi-round SRS-based protocols}
\label{sec:se_definitions}

  \ourpar{Signatures of Knowledge from SE-SNARKs.}
The work of Groth and Maller  \cite{C:GroMal17} shows
how to construct a signature of knowledge for messages in $\{0, 1\}^*$
from  target collision-resistant hash-function and simulation-extractable SNARKs.
These schemes inherit the same properties as the underlying SNARK schemes,
they rely on a structured reference string, which can be updatable or universal and they are succinct.
In the following, our goal will be to show that recent efficient SNARKs are simulation extractable,  and therefore, they are a perfect candidate to build better signatures of knowledge.

  \ourpar{Simulation-Extractability for multi-round protocols.}
Most recent SNARK schemes are following the same blueprint: First, an interactive (multi-round) information-theoretic proof system is considered.
%However, this initial system is idealized and also inefficient.
Second,  this initial proof system is compiled into an efficient, non-interactive and computationally sound scheme using cryptographic tools such as polynomial commitments and the Fiat-Shamir transformation.

We remark that existing results on the Fiat-Shamir transform regarding existential unforgeability (for signatures) and simulation extraction (for proof systems and signatures of knowledge) work for $3$-round protocols without reference string that require two transcripts for standard model extraction, e.g., \cite{JC:PoiSte00,INDOCRYPT:FKMV12,C:RotSeg21}.
In this section we prepare our analysis for multi-round protocols with a universal or updatable SRS.
In order to prove simulation-extractability for such protocols,  we require more than
just two transcripts for extraction.  Moreover, in the updatable setting we consider protocols that rely on an SRS where the adversary gets to contribute to the SRS. This makes out analysis more challenging.
We first give a definition of simulation-extractability which we base on~\cite{INDOCRYPT:FKMV12} adapted to the updatable SRS setting. Then, to support multi-round SRS-based protocols compiled using the Fiat-Shamir transform we gneralize the unique response property, replace honest-verifier zero-knowledge property with trapdoor-less simulation and special soundness with a forking special soundness property that links up with our generalized forking lemma.
\subsection{Simulation extractability}
\begin{definition}[Simulation-extractable NIZK]
	\label{def:updsimext}
  \label{def:simext}
	Let $\ps = (\kgen, \prover, \verifier, \simulator)$ be a NIZK proof system. We say that $\ps$
  is \emph{updatable \COMMENT{forking }simulation-extractable} with \emph{extraction error} $\nu$ if for
  any $\ppt$ adversary $\adv$ that is given oracle access to an updatable SRS setup $\initU$, a
  simulation oracle $\simO$, cf.~\cref{fig:upd}, and a random oracle $\ro$, and that produces an
  accepting proof for $\ps$ with probability $\accProb$, where
	\[
	\accProb = \condprob{
	\begin{matrix}
	  \verifier(\srs, \inp_{\advse}, \zkproof_{\advse}) = 1  \\
	  \wedge
	(\inp_{\advse}, \zkproof_{\advse}) \not\in Q
	\end{matrix}
}{
	\begin{matrix}
	  r \sample \RND{\advse}\\
	(\inp_{\advse}, \zkproof_{\advse}) \gets \advse^{\initU, \simO,
		\ro} (1^\secpar; r)
	\end{matrix}
}
	\]
	there exists an extractor $\extse$ such that
	\[
	\extProb = \condprob{
	\begin{matrix}
  \verifier(\srs, \inp_{\advse}, \zkproof_{\advse}) = 1 \\
 \wedge  ~(\inp_{\advse}, \zkproof_{\advse}) \not\in Q   \\
	 \wedge  ~\REL(\inp_{\advse}, \wit_{\advse}) = 1
	\end{matrix}
}{
	\begin{aligned}
	& r \sample \RND{\advse},
	(\inp_{\advse}, \zkproof_{\advse}) \gets \advse^{\initU, \simO,
		\ro} (1^\secpar; r) \\
	& \wit_{\advse} \gets \ext_\se (\srs, \advse, r, \inp_{\advse}, \zkproof_{\advse},
	Q, Q_\ro, Q_\srs)
	\end{aligned}
}
	\]
	is at at least
	\[
	\extProb \geq \frac{1}{\poly} (\accProb - \nu)^d - \eps(\secpar)\,,
	\]
	for some polynomial $\poly$, constant $d$ and negligible $\eps(\secpar)$ whenever
	$\accProb \geq \nu$.
	Here, $\srs$ is the finalized SRS, list $Q$ contains all $(\inp, \zkproof)$ pairs where
	$\inp$ is an instance queried to $\simO$ by the adversary and
	$\zkproof$ is the simulator's answer. List $Q_\ro$ contains all $\advse$'s
	queries to $\ro$ and the random oracle's answers.
\end{definition}
\subsection{Unique-response protocols}
A technical hurdle identified by Faust et al.~\cite{INDOCRYPT:FKMV12} for
proving simulation extraction via the Fiat-Shamir transformation is that the
transformed prove system satisfies a unique response property. The original
Fischlin's formulation, although suitable for applications presented in
\cite{C:Fischlin05,INDOCRYPT:FKMV12}, does not suffice in our case. First, the
property assumes that the protocol has three messages, with the second being the
challenge from the verifier. That is not the case we consider here. Second, it
is not entirely clear how to generalize the property. Should one require that
after the first challenge from the verifier, the prover's responses are fixed?
That does not work since the prover needs to answer differently on different
verifier's challenges, as otherwise the protocol could have fewer rounds.
Another problem is that the protocol could have a round beyond the
first round where the message of the prover is randomized. Unique response cannot hold in
this case. Finally, the protocols we consider here are not in the standard
model, but use an SRS.

We work around these obstacles by providing a generalised notion of the unique
response property. More precisely, we say that a $(2\mu + 1)$-message protocol
has \emph{unique responses from $i$}, and call it a $\ur{i}$-protocol, if it
follows the definition below:

\begin{definition}[$\ur{i}$-protocol in the updatable setting]
	\label{def:wiuru}
	Let $\proofsystem$ be a $(2\mu + 1)$-message public coin proof system
	$\ps = (\kgen, \prover, \verifier, \simulator)$. Let $\proofsystem_\fs$ be
	$\proofsystem$ after the Fiat--Shamir transform and $\ro$ the random
	oracle. Denote by $a_1, \ldots, a_{\mu}, a_{\mu + 1}$ protocol messages output
	by the prover, We say that $\proofsystem$ has \emph{unique responses from $i$
		on} if for any $\ppt$ adversary $\adv$:
	\[
	\Pr\left[
	\begin{aligned}
	& \vec{a} \neq \vec{a'}, a_1, \ldots, a_{i} = a'_1,
	\ldots, a'_{i}, \\
	& \verifier^\ro_\fs (\srs, \inp, \vec{a}) =
	\verifier^\ro_\fs(\srs, \inp, \vec{a'}) = 1  \\
	\end{aligned}
	\,\left|\,
	\begin{aligned}
	& \inp, \vec{a},  \vec{a'}  \gets \adv^{\ro, \initU}(1^\secpar) \\
& \vec{a} = (a_1, \ldots, a_{\mu + 1}), \vec{a'} = (a'_1, \ldots,
	a'_{\mu + 1})
	\end{aligned}
	\right.\right]
	\]
	is upper-bounded by some negligible function $\negl$.
\end{definition}
Note that in the above definition, $\srs$ is the SRS that $\advse$ finalised using the update oracle $\initU$, defined in~\cref{fig:upd}.

Intuitively, a protocol is $\ur{i}$ if it is infeasible for a $\ppt$ adversary
to produce a pair of acceptable and different proofs $\zkproof$, $\zkproof'$
that are the same on the first $i$ prover messages.
We note that the definition above is also meaningful for protocols without an SRS. Intuitively in that case $\srs$ is the empty string.

 \subsection{Trapdoor-Less Simulation}
In security reductions in this paper it is sometimes needed to produce
simulated NIZK proofs without knowing the trapdoor, just by
programming the random oracle. We call protocols which allow for such a
simulation \emph{trapdoor-less simulatable} (TLS).

\begin{definition}[($k$-programmable) Trapdoor-Less Simulatable Proof System]
  Let $\ps$ be a $(2\mu + 1)$-message ZK proof system. Let $\ps_\fs= (\kgen_{\fs}, \prover_{\fs}, \verifier_{\fs}, \simulator_{\fs})$ be its
  Fiat--Shamir variant and $\ro$ the random oracle. $\simulator_{\fs}$ takes as input $\srs$
  and instance $\inp$, programs $\ro$, and outputs a proof $\zkproof_\simulator$.  We
  call $\ps$ \emph{trapdoor-less simulatable} if for any adversary
  $\adv$, $\eps_0 \approx \eps_1$, where
  \begin{align*}
    \eps_b = \Pr\left[
    \begin{aligned}
      \adv^{\initU,\ro,\oracleo_b} = 0
    \end{aligned}
    \right]
  \end{align*}
  On $\adv$'s query $\inp, \wit$, oracle $\oracleo_{b}$ responds
  with a real proof $\zkproof_\prover \gets
  \prover_{\fs}(\srs, \inp, \wit)$ if $b = 0$ or a simulated proof $\zkproof_\simulator \gets \simulator_{\fs} (\srs,
  \inp)$ if $b = 1$.

  Additionally, we say that $\ps_{\fs}$ is $k$-programmable, if $\simulator_\fs$ programs the random oracle \emph{only} for
    challenges from round $k$ to $\mu + 1$.
\end{definition}

\begin{remark}[TLS vs HVZK]
  We note that TLS notion is closely related to honest-verifier zero knowledge in the standard model. That is, if we consider an interactive proof system $\proofsystem$ that is HVZK in the standard model then its Fiat--Shamir compiled version $\proofsystem_\fs$ is TLS. This comes as the simulator $\simulator$ in $\proofsystem$ produces a valid simulated proof by picking verifier's challenges according to a predefined distribution and $\proofsystem_\fs$'s simulator $\simulator_\fs$ produces its proofs similarly by picking the challenges and additionally programming the random oracle to return the picked challenges. Importantly, in both $\proofsystem$ and $\proofsystem_\fs$ success of the simulator does not depend on access to an SRS trapdoor.
\end{remark}

% \begin{definition}[$k$-programmable ZK]
%   \label{def:kzk}
%   Let $\ps$ be a $(2\mu + 1)$-message ZK proof system and let $\ps_\fs$ be its
%   Fiat--Shamir variant. We say that $\ps_\fs$ is $k$-programmable ZK if there
%   exists a simulator $\simulator_\fs$ that
%   \begin{compactenum}
%   \item produces proofs indistinguishable from proofs output by an honest
%     prover;
%   \item $\simulator_\fs$ programs the random oracle \emph{only} for
%     challenges from round $k$ to $\mu + 1$.
%   \end{compactenum}
% \end{definition
%}
We note that $\plonk$ is $2$-programmable TLS, $\sonic$ is $1$-programmable TLS,
and $\marlin$ is $1$-programmable TLS. This follows directly from the proofs of
their standard model zero-knowledge property in
\cref{lem:plonk_hvzk,lem:sonic_hvzk,lem:marlin_hvzk}.

\subsection{Generalised forking lemma and forking special soundness}
%\label{sec:forking_lemma}
Although dubbed ``general'', the forking lemma of~\cref{lem:forking_lemma} is not
general enough for our purpose as it is useful only for protocols where a witness
can be extracted from just two transcripts. To be able to extract a witness
from, say, an execution of $\plonkprot$ we need at least
$(3 \numberofconstrains + 1)$ valid proofs, and $(\multconstr + \linconstr + 1)$ for $\sonicprot$. We
propose a generalisation of the general forking lemma that given a probability of
producing an accepting transcript, $\accProb$, lower-bounds the probability of
generating a \emph{tree of accepting transcripts} $\tree$, which allows to
extract a witness.

\begin{definition}[Tree of accepting transcripts, cf.~{\cite{EC:BCCGP16}}]
	\label{def:tree_of_accepting_transcripts}
	Consider a $(2\mu + 1)$-message proof system. A $(n_1,
  \ldots, n_\mu)$-tree of accepting transcripts is a tree where each node on
  depth $i$, for $i \in \range{1}{\mu + 1}$, is an $i$-th prover's message in an
  accepting transcript; edges between the nodes are labeled with verifier's
  challenges, such that no two edges on the same depth have the same
  label; and each node on depth $i$ has $n_{i} - 1$ siblings and $n_{i +
    1}$ children. The tree consists of $N = \prod_{i = 1}^\mu n_i$
  branches, where $N$ is the number of accepting transcripts. We require $N = \poly$.
\end{definition}

\begin{lemma}[General forking lemma II]
	\label{lem:generalised_forking_lemma}
	Fix $q \in \ZZ$ and set $H$ of size $h \geq m$. Let $\zdv$ be a $\ppt$
  algorithm that on input $y, h_1, \ldots, h_q$ returns $(i, s)$ where $i \in
  \range{0}{q}$ and $s$ is called a side output. Denote by $\ig$ a randomised
  instance generator. We denote by $\accProb$ the probability
	\[
		\condprob{i \neq 0}{ y \gets \ig;\ h_1, \ldots, h_q \sample H;\ (i, s)
		\gets \zdv(y, h_1, \ldots, h_q)}\,.
	\]
	Let $\genforking_{\zdv}^{m}$ denote the algorithm described in
  \cref{fig:genforking_lemma} then the probability $\frkProb := \condprob{b =
    1}{y \gets \ig;\ h_1, \ldots, h_{q} \sample H;\ (b, \vec{s}) \gets
    \genforking_{\zdv}^{m}(y, h_1, \ldots, h_q)}$ is at least
	\[
		\frac{\accProb^m}{q^{m - 1}} - \accProb \cdot \left(1 -
    \frac{h!}{(h - m)! \cdot h^{m}}\right).
	\]

	\begin{figure}[t]
		\centering
		\fbox{
		\procedure{$\genforking_{\zdv}^{m} (y,h_1^{1}, \ldots, h_{q}^{1})$}
		{
		\rho \sample \RND{\zdv}\\
		(i, s_1) \gets \zdv(y, h_1^{1}, \ldots, h_{q}^{1}; \rho)\\
    i_1 \gets i\\
		\pcif i = 0\ \pcreturn (0, \bot)\\
		\pcfor j \in \range{2}{m}\\
		\pcind h_{1}^{j}, \ldots, h_{i - 1}^{j} \gets h_{1}^{j - 1}, \ldots,
		h_{i - 1}^{j - 1}\\
		\pcind h_{i}^{j}, \ldots, h_{q}^{j} \sample H\\
		\pcind (i_j, s_j) \gets \zdv(y, h_1^{j}, \ldots, h_{i - 1}^{j}, h_{i}^{j},
		\ldots, h_{q}^{j}; \rho)\\
		\pcind \pcif i_j = 0 \lor i_j \neq i\ \pcreturn (0, \bot)\\
    \pcif \exists (j, j') \in \range{1}{m}^2, j \neq j' : (h_{i}^{j} = h_{i}^{j'})\
	\pcreturn (0, \bot)\\
		\pcelse \pcreturn (1, \vec{s})
	}}
	\caption{Generalised forking algorithm $\genforking_{\zdv}^{m}$}
	\label{fig:genforking_lemma}
\end{figure}
\end{lemma}
The proof is along similar lines as~\cite[Lemma 1]{CCS:BelNev06} with modifications required
by the fact that the protocol has more than $3$ rounds and the number of
transcripts required is larger. We defer the proof to
\cref{sec:forking_proof}.

\oursubsub{Forking special soundness}
Note that the special soundness property (as usually defined) holds for
all---even computationally unbounded---adversaries. Unfortunately, since a
simulation trapdoors for $\plonkprot$ and $\sonicprot$ exist, the protocols
cannot be special sound in that regard. This is because an unbounded adversary
can recover the trapdoor and build an arbitrary number of simulated proofs for a fake
statement. Hence, we provide a weaker, yet sufficient, definition of
\emph{forking special soundness}. More precisely, we state that an
adversary that is able to answer correctly multiple challenges either knows the
witness or can be used to break some computational assumption.
%\chaya{a notion of rewinding-based knowledge soundness has been used to mean exactly the above in prior works, like BBF19. we should clarify if forking soundness different from rewinding-based knowledge soundness? seems to me like the difference is just that here it is tailored for the NI version.}
%\chaya{I now see that the diff is the access to the simulator that the adversary gets. might be helpful to clarify this, either here or in a tech overview section. I am not sure why this needs to be defined this way by giving access to the simulator.}
However, differently from the standard definition of special soundness, we do
not require from the extractor to be able to extract the witness from \emph{any}
tree of acceptable transcripts. We require that the tree be produced honestly,
that is, all challenges are picked randomly --- exactly as an honest verifier would pick.
Intuitively, the tree is as it would be generated by a $\genforking$
algorithm from the generalized forking lemma.

% \begin{definition}[Forking soundness]
%   % Let $\proofsystem = (\kgen, \prover, \verifier, \simulator)$ be an
%   % $(2 \mu + 1)$-message proof system for a relation $\REL$. Let $\tree$ be an
%   % $(n_1, \ldots, n_\mu)$-tree of acceptable transcripts output by a $\ppt$ tree
%   % building algorithm $\tdv$ which plays the role of the verifier $\ps.\verifier$
%   % against $\ppt$ adversary $\adv$ which it interacts with.
% %
%   % We say that $\proofsystem$ is
%   % $(\epsss, (n_1, \ldots, n_\mu))$-\emph{computationally special sound} if there
%   % exists an extractor $\extt$ that given an $ (n_1, \ldots, n_\mu)$-tree of
%   % acceptable transcripts $\tree$ for an instance $\inp \in \LANG_\REL$ output by
%   % a $\ppt$ tree building adversary $\tdv$ some $\ppt$ adversary
%   % $\adv(\srs)$, for $\srs \sample \kgen(\REL)$, outputs $\wit$ such that
%   % $\REL(\inp, \wit) = 1$ with probability at least $1 - \epsss$.
%   Let $\proofsystem = (\kgen, \prover, \verifier, \simulator)$ be an
%   $(2 \mu + 1)$-message proof system for a relation $\REL$. Let $\tree$ be an
%   $(n_1, \ldots, n_\mu)$-tree of transcripts for an in valid instance
%   $\inp$. We say that $\ps$ is $(\epsss, (n_1, \ldots, n_\mu))$-forking sound if there is an extractor $\extt$ that given $\tree$ extracts $\wit$
%   such that $\REL(\inp, \wit) = 1$ with probability at least $1 - \epsss.$
% \end{definition}

% \begin{definition}[$k$-round forking soundness]
%   % Let $\proofsystem = (\kgen, \prover, \verifier, \simulator)$ be an
%   % $(2 \mu + 1)$-message proof system for a relation $\REL$. Let $\tree$ be an
%   % $(n_1, \ldots, n_\mu)$-tree of acceptable transcripts output by a $\ppt$ tree
%   % building algorithm $\tdv$ which plays the role of the verifier $\ps.\verifier$
%   % against $\ppt$ adversary $\adv$ which it interacts with.
% %
%   % We say that $\proofsystem$ is
%   % $(\epsss, (n_1, \ldots, n_\mu))$-\emph{computationally special sound} if there
%   % exists an extractor $\extt$ that given an $ (n_1, \ldots, n_\mu)$-tree of
%   % acceptable transcripts $\tree$ for an instance $\inp \in \LANG_\REL$ output by
%   % a $\ppt$ tree building adversary $\tdv$ some $\ppt$ adversary
%   % $\adv(\srs)$, for $\srs \sample \kgen(\REL)$, outputs $\wit$ such that
%   % $\REL(\inp, \wit) = 1$ with probability at least $1 - \epsss$.
%   Let $\proofsystem = (\kgen, \prover, \verifier, \simulator)$ be an
%   $(2 \mu + 1)$-message proof system for a relation $\REL$. Let $\tree$ be the
%   algorithm below that rewinds $\advse^{\simulator_\fs,
%           \ro} (\srs; r)$ to produce a $(1, n_k, 1)$-tree of
%   transcripts such that none of the challenges in round $k$ were used in
%   simulated proofs. We say that $\ps$ is $(\epsss, ((1, n_k, 1)))$-forking
%   special sound if there is an extractor $\extt$ that given a tree produced by
%   $\tree$ extracts $\wit$ such that $\REL(\inp, \wit) = 1$ with probability at
%   least $1 - \epsss.$
% \end{definition}

% Since we do not utilise the classical special soundness (that holds for all,
% even unbounded, adversaries) all references to that property should be
% understood as references to its computational version.

%\begin{definition}[$(\eps(\secpar), k,n)$-forking soundness]
%  Let $\proofsystem = (\kgen, \prover, \verifier, \simulator)$ be an
%  $(2 \mu + 1)$-message proof system for a relation $\REL$.
%
%For any $\ppt$ adversary $\advse^{\simulator_\fs,
%  \ro} (\srs; r)$ we consider the procedure $\zdv$ that provided the transcript $(\srs, \adv, r, Q, Q_{H})$ and $h_1, \ldots, h_q$ runs $\adv$ by providing it with random oracle queries
%and simulated proofs. While $Q_{H}$ is consistent with $h_1, \ldots, h_q$, it replays the proofs of $Q$.
%%
%$\zdv$ returns the index $i$ of the
%  random oracle query made for challenge $k$ and the proof $\adv$ returns.
%
%  Consider the algorithm $\genforking_{\zdv}^{n}$
%  that rewinds $\zdv$ to produce a $(1,\dots, n,\dots, 1)$-tree of
%  transcripts such that none of the $n$ challenges in round $k$ were used in
%  simulated proofs.
%
%  We say that $\ps$ is $(\eps(\secpar), k,n)$-forking sound if
%  for any PPT adversary the probability that
%  \begin{align*}
%    \Pr\left[
%    \REL(\inp, \wit) = 0
%    \,\Biggl|\,
%    \begin{aligned}
%       & \srs \sample \kgen(\REL),
%        r \sample \RND{\advse},
%         (\inp_{\advse}, \zkproof_{\advse}) \gets \advse^{\simulator_\fs,\ro} (\srs; r), \\
%       &    (1, \tree) \gets \genforking_{\zdv}^{m}((\srs,\adv,r,Q, Q_{H}),Q_{H}),
%           \wit \gets \extt(\tree)
%    \end{aligned}
%    \right] \leq \eps(\secpar).
%  \end{align*}
%   List $Q$ contains all $(\inp, \zkproof)$ pairs where
%  $\inp$ is an instance provided to the simulator by the adversary and
%  $\zkproof$ is the simulator's answer. List $Q_\ro$ contains all $\advse$'s
%  queries to $\ro$ and $\ro$'s answers.
%\end{definition}

%\hamid{The definition in the updatable setting should look like this:}

\begin{definition}[$(k,n)$-forking special soundness in the updatable setting]
	Let $\proofsystem = (\kgen, \prover, \verifier, \simulator)$ be an
	$(2 \mu + 1)$-message proof system for a relation $\REL$.  Let $\proofsystem_\fs$ be
	$\proofsystem$ after the Fiat--Shamir transform.

	For any $\ppt$ adversary $\advse^{\initU,
		\ro} (1^\secpar; r)$ with access to oracles $\initU$ defined in~\cref{fig:upd}, and random oracle $\ro$, we consider the procedure $\zdv$ that provided the transcript $(\srs, \adv, r)$ and $h_1, \ldots, h_q$ runs $\adv$ by providing it with random oracle queries
	and update oracle queries.
	%
	$\zdv$ returns the index $i$ of the
	random oracle query made for challenge $k$ and the proof $\adv$ returns.

	Consider the algorithm $\genforking_{\zdv}^{n}$
	that rewinds $\zdv$ to produce a $(1,\dots, n,\dots, 1)$-tree of
	accepting transcripts.

	We say that $\psfs$ is $(k,n)$-forking special sound with security loss $\eps(\secpar)$, if
	for any PPT adversary the probability that
	\begin{align*}
	\Pr\left[
	\REL(\inp, \wit) = 0
	\,\Biggl|\,
	\begin{aligned}
	%& \srs \sample \kgen(\REL),
	& r \sample \RND{\advse},
	(\inp_{\advse}, \zkproof_{\advse}) \gets \advse^{\initU, \ro} (1^\secpar; r), \\
	&    (1, \tree) \gets \genforking_{\zdv}^{m}((\srs,\adv,r),Q_{H}),
	\wit \gets \extt(\tree)
	\end{aligned}
	\right] \leq \eps(\secpar).
	\end{align*}
	Here, $\srs$ is the SRS that $\advse$ finalised using the update oracle $\initU$.
	%List $Q$ contains all $(\inp, \zkproof)$ pairs where $\inp$ is an instance provided to the simulator by the adversary and $\zkproof$ is the simulator's answer.
	List $Q_\ro$ contains all $\advse$'s
	queries to $\ro$ and the random oracle's answers.
	%List $Q_\srs$ contains the proofs of all honest SRS updates.
\end{definition}
% \chaya{why does $\genforking$ get $Q_H$ twice?}
% \markulf{20/09/2021}{This is a tricky one. If I remember correctly, we wanted $\zdv$ to be able to determin when $\adv$ starts to diverge from the first branch that results in $\pi_{\adv}$. $\adv$ needs to be fed recorded simulated proofs until that divergence. After that $\zdv$ can create fresh simulated proofs. I think its ok to use the same simulated proofs in multiple sub-branches as long as they are for the right instance. In any case simulation is now hidden within $\adv$.}

\paragraph{Importance of the general forking lemma.}
To highlight the importance of the generalised forking lemma, we outline
how it is used in our \COMMENT{forking} simulation-extractability proof.  Let $\psfs$ be a
forking special sound proof system where for an instance $\inp$ the
corresponding witness can be extracted from a
$(1, \ldots, 1, n_k, 1, \ldots, 1)$-tree of accepting transcripts.  Let $\advse$
be the simulation-extractability adversary that outputs an accepting proof with
probability at least $\accProb$. From $\adv$ we build an adversary $\adv$ that runs $\simO$ inside. Although we use the same $\accProb$ to denote
probability of $\zdv$ outputing a non-zero $i$ and the probability of $\advse$ (and $\adv$)
outputing an accepting proof, we claim that these probabilities are exactly the
same by the way we define $\zdv$ and $\bdv$. In the following we give the intuition of the proof without the additional indirection via $\bdv$ whose goal is primarily to simplify the forking special soundness definition.

Let $\adv$ output an accepting instance-proof pair
$(\inp_{\advse},\zkproof_{\advse})$; $r$ be $\adv$'s
randomness; \COMMENT{$Q$ the list of simulator queries made by $\advse$ \markulf{20/09/2021}{Q is now hidden within the forking special soundness adversary.} along with
$\simulator$'s answers; }%\hamid{$Q_\srs$ be the list of proofs for SRS honest updates;}
and $Q_\ro$ be the list of all random oracle
queries made by $\adv$.  All of these are given to the extractor $\ext$ that
internally runs the forking algorithm $\genforking_\zdv^{n_k}$.  Algorithm $\zdv$
takes $(\srs, \advse, \COMMENT{Q, }r)$ as input $y$ and $Q_\ro$ as input $h_1^1, \ldots,
h_q^1$.
(For the sake of completeness, we allow $\genforking_\zdv^{n_k}$ to
pick $h^1_{l + 1}, \ldots, h^1_q$ responses if $Q_\ro$ has only $l < q$
elements.)

Next, $\zdv$ internally runs $\adv(\srs; r)$ %\hamid{$\\advse(1^\secpar; r)$}
and responds to its random
oracle queries using $Q_\ro$\COMMENT{ and $Q$}. Note that $\adv$ makes
the same queries as it did before it output $(\inp_{\advse}, \zkproof_{\advse})$
as it is run on the same random tape and with the same answers from the
simulator and random oracle. Once $\adv$ outputs
$\zkproof_{\advse}$, algorithm $\zdv$ outputs $(i, \zkproof_{\advse})$, where
$i$ is the index of a random oracle query submitted by $\advse$ to receive the challenge after the
$k$-th message from the prover---a message where the tree of transcripts
branches.
Then, after the first run of $\advse$ is done, the extractor runs $\zdv$ again,
but this time it provides fresh random oracle responses $h^2_i, \ldots,
h^2_q$. Note that this is equivalent to rewinding $\advse$ to a point just
before $\advse$ is about to ask its $i$-th random oracle
query. The probability that the adversary produces an accepting transcript with the
fresh random oracle responses is at least $\accProb$. This continues until the
required number of transcripts is obtained.

We note that in the original forking lemma, the forking algorithm $\forking$,
cf.~\cref{fig:forking_lemma}, gets as input only $y$ and elements $h^1_1, \ldots,
h^1_q$ are randomly picked from $H$ internally by $\forking$. However, assuming
that $h^1_1, \ldots, h^1_q$ are random oracle responses, and thus random, makes
the change only notational.

We also note that the general forking lemma from
\cref{lem:generalised_forking_lemma} works for protocols with an extractor that can obtain the
witness from a $(1, \ldots, 1, n_k, 1, \ldots, 1)$-tree of accepting
transcripts. This limitation however does not affect the main result of this
paper, i.e.~showing that both $\plonk$, $\sonic$, and $\marlin$ are \COMMENT{forking }simulation extractable.

%\hamid{next definition should be commented!}
%\begin{definition}[$(\eps(\secpar), k,n)$-forking soundness]
%  Let $\proofsystem = (\kgen, \prover, \verifier, \simulator)$ be an
%  $(2 \mu + 1)$-message proof system for a relation $\REL$.
%
%  Let $\tdv$, called tree creator, be the
%  algorithm below that rewinds the $\ppt$ adversary $\advse^{\simulator_\fs,
%          \ro} (\srs; r)$ to produce a $(1,\dots, n,\dots, 1)$-tree of
%  transcripts such that none of the $n$ challenges in round $k$ were used in
%  simulated proofs.
%
%  $\tdv$ has oracle access to $\adv$ and provides it with (oracle) access to
%  random oracle $\ro$ and simulator $\simulator_\fs$ -- more precisely $\tdv$
%  has an internal procedure $\bdv$ that provided $\srs$ and random oracle
%  queries' responses $h_1, \ldots, h_q$ gives $\adv$ access to the random oracle
%  and simulates proof for it. In the end, $\bdv$ returns the index $i$ of the
%  random oracle query made for challenge $k$, the set $Q$ of simulator random
%  oracle indexes, the instance $\inp$, and the proof $\adv$ returns. Eventually,
%  $\tdv$ returns a $(1, \ldots, n, \dots, 1)$ tree of acceptable
%  transcripts~$\tree$.
%
%
%  \begin{figure}
%    \centering
% 		\fbox{ \procedure{$\tdv(\adv, \srs \sample \kgen(\REL))$}
%      {h_1^{1}, \ldots,
%      h_{q}^1 \sample H \\
%      (i, Q, \inp, \zkproof_1) \gets \bdv(\adv, \srs, h_1^{1}, \ldots, h_{q}^{1})\\
%        % i_1 \gets i\\
%        \pcif i\in Q \lor \verifier(\srs, \inp, \zkproof_1) = 0\ \pcreturn (0, \bot)\\
%        \pcfor j \in \range{2}{m}\\
%        \pcind h_{1}^{j}, \ldots, h_{i - 1}^{j} \gets h_{1}^{j - 1}, \ldots,
%        h_{i - 1}^{j - 1}\\
%        \pcind h_{i}^{j}, \ldots, h_{q}^{j} \sample H\\
%        \pcind (i_j, Q_j, \inp_j, \zkproof_j) \gets \bdv(\adv, \srs, h_1^{j}, \ldots, h_{i -
%          1}^{j}, h_{i}^{j},
%        \ldots, h_{q}^{j})\\
%        \pcind \pcif i \neq i_j \lor i_j \in Q_j \lor \inp \neq \inp_j \lor \verifier(\srs, \inp_j, \zkproof_i) = 0\ \pcreturn (0, \bot)\\
%        %\pcind \pcif i_j = 0 \lor i_j \neq i\ \pcreturn (0, \bot)\\
%        % \pcif \exists (j, j') \in \range{1}{m}^2, j \neq j' : (h_{i}^{j} =
%        % h_{i}^{j'})\
%        %
%        % pcreturn (0, \bot)\\
%        \pcelse \pcreturn (1, \tree = (\inp, \pmb{\pi}))}
%    }
%  \end{figure}
%%
%  We say that $\ps$ is $(\eps(\secpar), k,n)$-forking sound if
%  for any PPT adversary the probability that
%  \[
%    \Pr\left[
%    %  \begin{aligned}
%       % & \forall_{\zkproof \in \tree} \verifier(\srs, \inp, \zkproof) = 1, \\
%         \wit \gets \extt(\tree),
%         \REL(\inp, \wit) = 0
%    %  \end{aligned}
%      \,\left|\,
%        %\begin{aligned}
%           \srs \sample \kgen(\REL),
%           (1, \tree) \gets \tdv(\adv, \srs)
%     %   \end{aligned}
%      \right.\right] \leq \eps(\secpar).
%      \]
%\end{definition}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "main"
%%% End:


\section{Selecting Deployment Schedule Estimates}
\label{selecting}

There are three methods for choosing a new deployment schedule $\Theta$
to attempt to run in a simulator. The first is stochastic with weighted
probabilities for the $\theta_p$.  The second does a deterministic sweep
iteratively over all options, minimizing the dynamic time warping distance
at each point in time for each deployment parameter.  The last combines
these two and choose the one with the minimum distance to the demand curve.

All of these rely on a Gaussian process model of the production
curve. This is because constructing and evaluating GP model $g_*$ is
significantly faster than performing even a low-fidelity simulation.
As a demonstrative example, say each evaluation of $d(f, g_*)$ takes a tenth
of a second (which is excessively long) and $d(f, g_s)$ for a low fidelity
simulation takes ten seconds (which is reasonable), the model evaluation
is still one hundred times faster.  Furthermore, the cost of constructing
the GP model can is amortized over the number of guesses that are made.

However, the choice of which $\theta_p$ to pick is extremely important
as they drive the optimization. In a vanilla stochastic algorithm,
each $\theta_p$ would be selected as a univariate integer on the
range $[M_p, N_p]$.  However, this ignores the distance information $D$
that is known about the training set which is used to create the Gaussian
process.
More intelligent guesses for $\theta_p$ focus the model
evaluations to more promising regions of the option space.  This in turn
helps reduce the overall number of expensive simulations needed to find
a `good enough' deployment schedule.

The three WORG $\Theta$ selection methods are described in order in the
following subsections.

\subsection{Stochastic Estimation}
\label{stochastic}

The stochastic method works by randomly choosing $\Gamma$ deployment
schedules and evaluating $g_*(t, \Theta_\gamma)$ for each guess $\gamma$.
The $\Theta_\gamma$ which has the minimum distance $d_\gamma$ is
taken as the best-guess deployment schedule.  The number of guesses may
be as large or as small as desired.  However, a reasonable number to pick
spans the option space. This is simply is the $L_1$ norm of the difference
of the inclusive bounds. Namely, set $\Gamma$ as in Equation
\ref{Gamma-default} for a minimum number of for stochastic guesses.
\begin{equation}
\label{Gamma-default}
\Gamma = \sum_p^P (N_p - M_p + 1)
\end{equation}
Each $\theta_p$ has $N_p - M_p + 1$ options. Thus a reasonable
choice for $\Gamma$ is the sum of the number of independent options.

Still, each option for $\theta_p$ should not be equally likely.
For example, if the demand curve is relatively low, the number of deployed
facilities is unlikely to be relatively high. For this reason, the choice
of $\theta_p$ should be weighted.  Furthermore, note that each $\theta_p$
is potentially weighted differently as they are all independent parameters.
Denote $n \in [M_p, N_p]$ such that the n-th weight for the p-th parameter
is called $w_{n,p}$.

To choose weights, first observe that the distances $D$ can be said to be
inversely proportional to how likely each deployment schedule in
$\vec{\Theta}$ should be. A one-dimensional Gaussian process can thus be
constructed to model inverse distances given the values of the deployment
parameter for each schedule, namely $\vec{\theta_p}$.  Call this model
$d_*^{-1}$ as seen in Equation \ref{d-inv-model}.
\begin{equation}
\label{d-inv-model}
d_*^{-1}(\theta_p) = \GP\left(\mu(\vec{\theta_p}),
                              k(\vec{\theta_p}, \vec{\theta_p}^\prime)\right)
                   \equiv \GP\left[D^{-1}\right]
\end{equation}
The construction, regression of hyperparameters, and evaluation of this
model follows analogously to the production curve modeling presented in
\S\ref{gp}.

The weights for $\theta_p$ are then the normalized evaluation of the
inverse distance model for all $m$ and $n$ defined on the p-th range.
Symbolically,
\begin{equation}
\label{d-inv-w}
w_{n,p} = \frac{d_*^{-1}(n)}{\sum_{m=M_p}^{N_p} d_*^{-1}(m)}
\end{equation}
Equation \ref{d-inv-w} works very well as long as a valid model can
be established.  However, this is sometimes not the case when the $\theta_p$
are degenerate, the distances are too close together, the distances are
too close to zero, or other stability issues arise.

In cases where a valid model may not be formed for $d_*^{-1}(\theta_p)$,
a Poisson distribution may be used instead.  Take the mean of the Poisson
distribution $\lambda$ to be the value of $\theta_p$ where the distance
is minimized.
\begin{equation}
\label{lambda}
\lambda_p = \theta_p | \mathrm{argmin}(D)
\end{equation}
Hence, the Poisson probability distribution for the $n$-th weight of the
$p$-th deployment parameter is,
\begin{equation}
\label{poisson}
\mathrm{Poisson(n)} = \frac{(\lambda_p)^n}{n!} e^{-\lambda_p}
\end{equation}
Now, because $n$ is bounded, it is important to renormalize Equation
\ref{poisson} when constructing stochastic weights.
\begin{equation}
\label{poisson-w}
\begin{split}
w_{n,p} & = \frac{\frac{(\lambda_p)^n}{n!} e^{-\lambda_p}}
                 {\sum_{m=M_p}^{N_p} \frac{(\lambda_p)^m}{m!} e^{-\lambda_p}}\\
        & = \frac{(\lambda_p)^n}
                 {n!\sum_{m=M_p}^{N_p} \frac{(\lambda_p)^m}{m!}}\\
\end{split}
\end{equation}
Poisson-based weights could be used exclusively, foregoing the inverse
distance Gaussian process models completely. However, a Poisson-only
method takes into account less information about the demand-to-production
curve distances. It was therefore observed to converge more slowly
on an optimum than using Poisson weights as a backup.  Since the total
number of simulations is aiming to be minimized for \emph{in situ} use,
the WORG method uses Poisson weights as a fallback only.

After weights are computed for all $P$ deployment parameters, a set of
$\Gamma$ deployment schedules may be stochastically chosen. The Gaussian
process for each $g_*(\mathbf{t}, \Theta_\gamma)$ is then evaluated and the
dynamic time warping distance to the demand curve is computed. The
deployment schedule with the minimum distance is then selected and returned.

\subsection{Inner Product Estimation}
\label{inner-prod}

As an alternative to the stochastic method demonstrated in \S\ref{stochastic},
a best-guess for $\Theta$ can also be built up iteratively over all times.
The method here uses the same production curve Gaussian process $g_*$ to
predict production levels and measure the distance to the demand curve.
However, this method minimizes the distance at time
$t$ and then uses this to inform the minimization of $t+1$. Starting at $t=1$
and moving through the whole time grid to $t=T$, a complete deployment
schedule is generated.

The following description is for the simplified case when $P==T$. However,
this method is easily extended to the case where $P > T$, such as for
multiple reactor types.  When $P > T$, group $\theta_p$ that occur on the
same time step together and take the outer product of their options prior to
stepping through time.

For this method, define the time sub-grid $\mathbf{t}_p$ as the sequence of
all times less than or equal to the time when parameter $p$ occurs, $t(p)$.
\begin{equation}
\label{t-p}
\mathbf{t}_p = \left\{t | t \le t(p)\right\}
\end{equation}
Now define the deployment schedule $\Theta^t$ up to time $t$ through the
following recursion relations:
\begin{equation}
\label{Theta-t}
\begin{split}
\theta_1 & = n \, | \, \mathrm{min}\left[d(f, g_*(1, n))\right]
                       \forall n\in[M_1, N_1] \\
\Theta^1 & = \left\{\theta_1\right\}\\
\theta_p & = n \, | \, \mathrm{min}\left[d(f, g_*(\mathbf{t}_p,
                                                  \Theta^{t-1}, n))\right]
                       \forall n\in[M_p, N_p] \\
\Theta^t & = \left\{\Theta_1^{t-1}, \ldots, \Theta_{p-1}^{t-1}, \theta_p\right\}
\end{split}
\end{equation}
Equation \ref{Theta-t} has the effect of choosing the the number of
facilities to deploy at each time step that minimizes the distance function.
The current time step uses the previous deployment schedule and only searches
the option space of the its own deployment parameter $\theta_p$.
Once $\Theta^T$ is
reached, it is selected as the deployment schedule $\Theta$. The inner
product method here requires the same number of model evaluations of $g_*$ as
were selected for the default value of $\Gamma$ in Equation
\ref{Gamma-default} for stochastic estimation.

\subsection{All of the Above Estimation}
\label{all}

This method is simply to run both the stochastic method and the inner product
method and determine which has the lower $d(f, g_*)$ for the deployment
schedules they produce.  This method contains both the advantages and
disadvantages of its constituents.  Additionally, it has the disadvantage
of being more computationally expensive than the other methods individually.

The advantage from the stochastic method is that the entire space is
potentially searched. There are no forbidden regions.
This is important since there may be other
optima far away from the current $\vec{\Theta}$ that produce lower distances.
Searching globally prevents the stochastic method from becoming stuck locally.
However, the stochastic method may take many iterations to
make minor improvements on a $\Theta$ which is already close to a best-guess.
It is, after all, searching globally for something better.

On the other hand, the inner product method is designed to search around
the part of the Gaussian process model which already produces good results. It
is meant to make minor adjustments as it goes.  Unfortunately, this means
the inner product method can more easily get stuck in a cycle  where it
produces the same series of deployment schedules over and over again.
It has no mechanism on its own to break out of such cycles.

With the all-of-the-above option, the job of balancing the relative merits
of the stochastic and inner product methods is left to the optimization
loop itself.  This can be seen in \S\ref{algo}.  If the \allflag flag
is set as the estimation method, it is only executed as the \allflag flag
two of every four
iterations.  Other strategies for determining how and when each of the
three methods are used could be designed. However, any more complex strategy
should be able to show that it meaningfully reduces the number of
optimization loop iterations required.

At this point, the entire WORG method has been described. A
demonstration of how it performs for a representative fuel cycle is
presented in the next section.