%%% Intro.tex ---
%%
%% Filename: Intro.tex
%% Description:
%% Author: Ola Leifler
%% Maintainer:
%% Created: Thu Oct 14 12:54:47 2010 (CEST)
%% Version: $Id$
%% Version:
%% Last-Updated: Thu May 19 14:12:31 2016 (+0200)
%%           By: Ola Leifler
%%     Update #: 5
%% URL:
%% Keywords:
%% Compatibility:
%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%
%%% Commentary:
%%
%%
%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%
%%% Change log:
%% Completed Language Reading MBS
%%
%% RCS $Log$
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%
%%% Code:

\chapter{Integration of Optimization tool-chain into the Model-Based Development Process}
\label{cha:optimization}

This chapter is based on the following paper:

\begin{itemize}

	\item \textbf{Alachew Shitahun}, Vitalij Ruge, Mahder Gebremedhin, Bernhard Bachmann, Lars Eriksson, Joel Andersson, Moritz Diehl, and Peter Fritzson. \textbf{Model-Based Optimization with OpenModelica and CasADi.} In Proceedings of IFAC Conference in Tokyo, September 2013.

	\item  \textbf{Alachew Shitahun}, Vitalij Ruge, Mahder Gebremedhin, Bernhard Bachmann, Lars Eriksson, Joel Andersson, Moritz Diehl, Peter Fritzson. \textbf{Tool Demonstration Abstract: OpenModelica and CasADi for Model-Based Dynamic Optimization.} In Proceedings of the 5th International Workshop on Equation-Based Object-Oriented Modeling Languages and Tools, Nottingham, UK, April 19, 2013.

\end{itemize}

\section{Introduction}
\label{sec:optimizationintroduction}

During the last decade, nonlinear model predictive control (NMPC) and non-linear optimal control problems (NOCP)
based on Differential-Algebraic Equations (DAEs) have had a significant impact in the industrial community, particularly in
the control engineering area \cite{biegler, tamimi}. State-of-the-art methods use numerical
algorithms for dynamic optimization based on direct multiple shooting \cite{bock} or collocation algorithms
\cite{biegler}.

Use of equation-based, object-oriented modeling languages such as Modelica for
industrial applications has increased. These languages enable users to conveniently model large-scale physical systems described by
differential, algebraic, and discrete equations, primarily with the goal of performing virtual experiments (simulation)
on these systems, but recently also optimization.

Due to the influence of such equation-based, object-oriented modeling languages in the industrial community, there have
been several attempts to integrate tools for such languages with numerical algorithms for optimization. For example,
Dymola \cite{dymola} supports parameter and design optimization of models written in Modelica, whereas
JModelica.org \cite{akesson} and OpenModelica \cite{bernhard} have native support for optimal
control.

This chapter presents results of an effort in which OpenModelica and CasADi \cite{casadi} have been integrated to perform model-based dynamic optimization. The problem formulation and modeling is done in Modelica
including the optimization \cite{optimica} language extension. The integration is based
on standardized XML format presented in \cite{xml} for exchange of \acrshort{daes} models.
OpenModelica supports export of models written in Modelica and the optimization language extension using this XML
format, while CasADi supports importing of models represented in this format. This allows users to define optimal control problems (OCP) using
Modelica and optimization language specification, and solve the underlying model formulation using a
range of optimization methods, including direct collocation and direct multiple shooting. The proposed
solution has been tested on several industrially relevant optimal control problems, including a diesel-electric
power train.

\section{CasADi}
\label{sec:optcasadi}

CasADi \cite{casadi} is an open-source framework for C++ and Python that provides numerical optimization in general and optimal control in particular. The main idea of the tool is to provide users with the ability to easily and efficiently implement optimal control algorithms with a wide range of methods, including multiple shooting and collocation, rather than providing users with a “black-box” OCP solver.
The tool supports symbolic import of OCPs via an extended version of the functional mockup interface (FMI) format as explained in \cite{optandersson}. This OCP can then be transcribed into a nonlinear programming problem (NLP) and solved with one of CasADi’s interfaced NLP solvers.

\section{Modelica and the Optimization Language Extension}
\label{sec:optimizationoptimica}

Modelica is a mature and powerful language with regard to modeling of complex hybrid dynamical systems. However, it
lacks important features for describing or modeling optimization problems. This is not a surprise since Modelica
was not originally designed to help with dynamic optimization problems.

The optimization language extension \cite{optimica} complements Modelica by providing features that enable
formulation of dynamic optimization problems based on Modelica models. The optimization extension to Modelica
consists of the following elements:

\begin{itemize}

\item \textit{objective} and \textit{objectiveIntegerand}, which maps the Mayer and the Lagrange term in
            the objective function respectively.
\item \textit{startTime}, which defines the start of the optimization interval.
\item \textit{finalTime}, which defines the end of the optimization interval.
\item A new section: \textit{constraint}, which defines inequality constraints.
\end{itemize}

The requirement and motivation for introducing these specific features is covered in more detail in \cite{optimica}.

\section{Modeling NOCP and XML Export in OpenModelica Compiler}
\label{sec:optimizationopenmodelica}

The OpenModelica compiler front-end has been extended to support the optimization language extension described in
Section \ref{sec:optimizationoptimica}.

\begin{figure} [!h]
	\includegraphics[width=\linewidth]{opt_modeling_NOCP.png}
	\caption{Modeling NOCP using the OpenModelica Graphical and Textual Editor.}
	\label{fig:nocpmodel}
\end{figure}

This enables users to use the OpenModelica graphical editor (OMEdit) (see Figure \ref{fig:nocpmodel})to formulate and use model-based NOCP that can be solved by CasADi.
In addition, the OpenModelica compiler has recently been extended with XML export of models \cite{alachew} based on the XML
format defined in \cite{xml}. This schema is an extended version of the XML schema
defined by the Functional Mock-up Interface (FMI) \cite{fmi}, and is the most recent in a series of
Modelica-related XML schemas starting with ModelicaXML \cite{pop}. The XML export also includes the
optimization language extension, and OpenModelica is integrated with CasADi for the type of model-based dynamic optimization reported in this thesis.

\section{Complete Model-Based Dynamic Optimization tool-chain}
\label{sec:optimizationtoolchain}

Before exporting a Modelica and optimization language model to XML, the model should be symbolically
instantiated by the compiler in order to get a single flat system of equations. The model variables should also be
scalarized. The compiler front end performs this, including syntax checking, semantics and type checking, simplification
and constant evaluation, etc. Then the complete flattened model is exported to XML code. The exported
XML document can then be imported to CasADi for model-based dynamic optimization. The complete tool chain is
visualized in Figure \ref{fig:optimizationtoolchain}.

\begin{figure} [!h]
	\includegraphics[width=\linewidth]{opt_tool_chain.png}
	\caption{Model-Based Dynamic Optimization tool-chain for OpenModelica and CasADi.}
	\label{fig:optimizationtoolchain}
\end{figure}

The XML will be imported and symbolically pre-processed in CasADi. In particular, the fully-implicit \acrshort{dae} from
Modelica is reformulated in a semi-explicit form. With the NOCP now available in CasADi’s native data structures, the
NOCP can be reformulated to a NLP as outlined in \cite{alachewoptimization} (See Section 2.2).

At the time of our work, the efficient symbolic pre-processing model evaluation from OpenModelica is not yet completely
implemented to import into CasADi. So the symbolic preprocessing in CasADi can be used. The symbolic work in CasADi makes it easy to create the goal function and constraints.

The NLP is solved by one of the NLP solvers interfaced to CasADi, e.g., IPOPT \cite{wachter}. First and second order derivative information will be generated by CasADi using automatic differentiation and passed to the solver.

\section{Testing the Tool-chain Implementation}
\label{sec:optimizationtesting}

In this section, we describe the solution of an industrially-relevant optimal control problem for a diesel-electric powertrain. The
formulation of the underlying optimization problem and the corresponding optimization results are presented in the
following subsections.

\subsection{Fuel optimal control of a diesel electric powertrain}
\label{sec:optimizationdiesel}

The diesel-electric powertrain model presented in \cite{sivertsson,bernhard} is a nonlinear
mean value engine model (MVEM) containing four states and three control inputs, while the generator model is
simplified by considering constant efficiency and maximum power over the entire speed range, see Figure \ref{fig:dieselmodel} for the
schematic diagram of the model.

\begin{figure} [!h]
	\includegraphics[width=\linewidth]{opt_diesel_model.png}
	\caption{Diagram of the diesel-electric powertrain model.}
	\label{fig:dieselmodel}
\end{figure}

In a diesel-electric powertrain the operating point of the diesel engine can be freely chosen, which would potentially
decrease fuel consumption. Moreover, the electric machine has better torque characteristics. These are the main reasons
that the diesel-electric power-train concept is interesting for further studies.

To investigate the fuel-optimal transients of the powertrain from idling condition to a certain power level while the
accelerator pedal position is interpreted as a power level request, the following optimal control problem is solved:

\begin{equation*}
 \begin{aligned}
	\text{states}\quad x = 0, \quad \begin{pmatrix} w_{ice} \\ p_{im} \\p_{em} \\w_{tc}  \end{pmatrix}\quad = \quad \text{controls,}\quad u = \begin{pmatrix} u_{f} \\ u_{wg} \\p_{gen} \\w_{tc}  \end{pmatrix} \\
	\text{min}\;\int_{0}^{T}\dot{m}{_f} d_t
\end{aligned}
\end{equation*}

subject to

\begin{equation*}
	\begin{aligned}
		\dot{x}_1 = f_2 (x_2,x_3,u_1,u_3 ) \\
		\dot{x}_2= f_3 (x_1,x_2,x_4 ) \\
		\dot{x}_3=f_4 (x_1,x_2,x_3,u_1,u_2 ) \\
		\dot{x}_4=f_5 (x_2,x_3,x_4,u_2 ) \\
		0= f_6 (x_2,x_4 )- f_7 (x_1,x_2 ) \\
		0= f_7(x_1,x_2 ) + f_8 (x_1,u_1 )-f_9 (x_3 )-f_{10}(x_3,u_3 )  \\
		0= \frac{f_{11}(x_3)-f_{12}(x_1 )} {f_{13}(x_4 )}-f_{14} (x_4 )
\end{aligned}
\end{equation*}

\begin{equation*}
	\begin{aligned}
      54 rps \;\leq  \;  x_1  \;\leq \; 220 rps \\
      0.8 p_{amp} \;\leq \;  x_2 \; \leq \; 2P_{amb} \\
      P_{amb} \;  \leq\; x_3 \;\leq \;  3P_{amb} \\
      300rps  \; \leq \; x_4 \;\leq \;   10000 rps \\
      0 \;\leq \;u_1,u_2 \;\leq \; 1
	\end{aligned}
\end{equation*}

and boundary conditions are:

\begin{equation*}
 \begin{aligned}
	\text{at}\quad t = 0,  \begin{pmatrix} x_1 \\ x_2 \\x_3 \\x_4  \end{pmatrix} = \text{idle operating values,} \\
	\text{at}\quad t = T,  \begin{pmatrix} \dot{x}_1 \\ \dot{x}_2 \\\dot{x}_3\\\dot{x}_4 \end{pmatrix} = 0,  \begin{pmatrix} x1 \\ x2 \\x3 \\x4  \end{pmatrix} = \text{desired values} \\
	\text{and}\quad v_3 = P_{required}.
 \end{aligned}
\end{equation*}

The constraints are originated from components’ limitations and the functions $ f_i$ are described in \cite{sivertsson}.

\subsection{Model import into CasADi and NLP transcription}
\label{sec:optimizationxmlimport}

We used OpenModelica to translate Modelica/ Optimization
language extension code into an OCP in DAE and Lagrange
cost function. This OCP is then exported into an XML-based
symbolic expression format and imported into CasADi via
OpenModelica. The OCP can then be transcribed into a
nonlinear programming problem (NLP) using the approach
outlined in \cite{bernhard} of Section 5, and solved with one of CasADi’s interfaced NLP solvers.

\subsection{Solution of the NLP}
\label{sec:optimizationnlp}

The NLP was solved using IPOPT \cite{wachter} running by default with the MUMPS linear solver.
The right scaling is important for a solution without oscillations.
On the other hand, if the scaling does not work in all steps
then changing of the solver tolerance is helpful. By itself, the diesel model
scales well in the time interval [0.32, 0.5],
which is the critical interval here.

In order to cover the optimal solution of the diesel-electric
powertrain model, 140 NLP iterations for 128 sub-intervals
with the total collocation (Lobatto6 and Radau5) are required by
IPOPT.

A better initial guess will change the NLP iterations.
Table \ref{tab:table1} shows the total CPU time for the optimization. The calculations have been done on a Dell Latitude E6410 laptop
with an Intel Core i7 processor of 2.8 GHz, 8 GB of RAM, 4M Cache, running Windows.

\begin{table} [!h]
\begin{center}
	\caption{Execution times for the diesel-electric powertrain model.}
	\label{tab:table1}
	\begin{tabular}{ cc }
		\hline
		\bfseries Step & \bfseries Time  \\
		IPOPT (without function evaluation) & 2.140s \\
		NLP function evaluations & 1.158s \\
		\hline
	\end{tabular}
\end{center}
\end{table}

The control and state trajectories of the optimal solutions are shown in Figure \ref{fig:optimizationresultstatevariables} and Figure \ref{fig:optimizationresultcontrolvariables}, respectively.

%%% \begin{figure} [!h]
%%%	\includegraphics[width=\linewidth]{opt_initial_guess_state_variables.jpg}
%%%	\caption{Initial guess for diesel model - state variables.}
%%%	\label{fig:initialguessstatevariables}
%%% \end{figure}

%%% \begin{figure} [!h]
%%%	\includegraphics[width=\linewidth]{opt_initial_guess_control_variables.jpg}
%%%	\caption{Initial guess for diesel model - control variables.}
%%%	\label{fig:initialguesscontrolvariables}
%%% \end{figure}

The problem solved here is a minimum fuel problem for a transient from idle to 170 kW, for an end time of 0.5 s. For simplicity, only diesel operating condition is assumed which
means $(u_3=P_{gen} = 0)$. As expected, the fuel optimal results happen when the engine is accelerated only near the
end of the time interval $(t\approx 0.32 s)$, to meet the end constraints while minimizing the fuel consumption.

\begin{figure} [!h]
	\includegraphics[width=\linewidth]{opt_result_state_variables.jpg}
	\caption{Optimization result for diesel model - state variables.}
	\label{fig:optimizationresultstatevariables}
\end{figure}

\begin{figure} [!h]
	\includegraphics[width=\linewidth]{opt_result_control_variables.jpg}
	\caption{Optimization result for diesel model - control variables.}
	\label{fig:optimizationresultcontrolvariables}
\end{figure}

\clearpage
With amendments to the initial values, the process is robust. For this purpose, the initial values are changed slightly.

\begin{figure} [!h]
	\includegraphics[width=\linewidth]{opt_result_state_variables_changed.jpg}
	\caption{Optimization result for diesel model with changed initial values - state variables.}
	\label{fig:optimizationresultchangedstatevariables}
\end{figure}

\begin{figure} [!h]
	\includegraphics[width=\linewidth]{opt_result_control_variables_changed.jpg}
	\caption{Optimization result for diesel model with changed initial values - control variables.}
	\label{fig:optimizationresultchangedcontrolvariables}
\end{figure}

\clearpage
\section{Summary}
\label{sec:optimizationsummary}

This chapter demonstrates simulation-based optimization through the coupling of two open-source tools: OpenModelica, which is a Modelica-based modeling and simulation platform, and CasADi, a framework for numerical optimization. The coupling uses a standardized XML format for exchange of
differential-algebraic equations (DAE) models. OpenModelica supports export of models written in Modelica and the optimization language extension using this XML format, while CasADi supports import of models represented in this format. This allows users to define optimal control problems (OCP) using
Modelica and optimization language specification, and to solve the underlying model formulation using a range of optimization methods, including direct collocation and direct multiple shooting. The proposed solution has been tested on several industrially-relevant optimal control problems, including a diesel-electric power train.

\subsection*{Acknowledgements}
\label{sec:optimizationacknowledgements}

This work has been partially supported by Serc, by SSF in the EDOp project and by Vinnova as well as the German
Ministry BMBF (BMBF F\"{o}rderkennzeichen: 01IS09029C) in the ITEA2 OPENPROD project and in the ITEA2 MODRIO
project. The open-source Modelica Consortium supports the OpenModelica work.
JA and MD acknowledge support by PFV/10/002 OPTEC, GOA/10/09 and GOA/10/11, FWO G.0320.08, G.0377.09,
SBO LeCoPro; Belspo IUAP P7 DYSCO, FP7-EMBOCON (ICT-248940), SADCO (MC ITN-264735),
ERC ST HIGHWIND (259 166), Eurostars SMART, vicerp, ACCM.

%\nocite{scigen}
%We have included Paper \ref{art:scigen}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% Intro.tex ends here

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "demothesis"
%%% End:


\section{Performance indexes}\label{sec:indexes}

We have defined the following performance indexes:
\begin{description}
	\item[Total broadcast time] (\(t_B\)) The time required in order to
		deliver the message to all reachable users.
	\item[Percentage of covered users] (\(\mathit{\%U}\)) The number of
		users infected.
	\item[Energy efficiency] (\(\mathit{Eff}\)) We define the energy
		efficiency as an index that decreases with the increase of the
		broadcast radius \(R\) and the total number of transmissions
		\(M\). We will not develop a well-defined mathematical formula
		for the energy efficiency of wireless communications; we will
		just state that \(\mathit{Eff} \propto \frac{1}{R \cdot M}\). Of
		corse, this index is of great interest in wireless ad-hoc
		networks where each communicating device must optimize the
		energy used in order to avoid frequent battery recharges or
		replacements. Moreover, we note that from the point of view of a
		single user of our model, where the user must relay the message
		at most one time only, the only factor affecting this index is
		\(R\). So our network should try to minimize \(R\) in order to
		maximize the energy efficiency. We may consider the total number
		of transmissions only to evaluate the efficiency of the
		\emph{entire} network.
	\item[Total number of collisions] (\(C\)) The total number of
		collisions. This is of course highly related to the number of
		users in the network. We must consider that a network with an
		higher user density may have an higher number of collisions. So
		this index may not be user to compare the performance of
		networks with different number of users (at least, not without
		normalization).
\end{description}


%2multibyte Version: 5.50.0.2890 CodePage: 932

\documentclass[11pt]{article}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{geometry}
\usepackage[onehalfspacing]{setspace}
\usepackage{numinsec}
\usepackage{harvard}
\usepackage{hyperref}
\usepackage{rotating}
\usepackage[labelsep=none,format=default,labelfont={footnotesize},textfont={footnotesize},font={footnotesize},justification=justified,labelformat=empty]{caption}
\usepackage{bibmods}
\usepackage{appendix}

\setcounter{MaxMatrixCols}{10}
%TCIDATA{OutputFilter=LATEX.DLL}
%TCIDATA{Version=5.50.0.2890}
%TCIDATA{Codepage=932}
%TCIDATA{<META NAME="SaveForMode" CONTENT="1">}
%TCIDATA{BibliographyScheme=Manual}
%TCIDATA{Created=Saturday, July 19, 2008 23:46:52}
%TCIDATA{LastRevised=Tuesday, November 20, 2018 00:49:35}
%TCIDATA{<META NAME="GraphicsSave" CONTENT="32">}
%TCIDATA{<META NAME="DocumentShell" CONTENT="Standard LaTeX\Blank - Standard LaTeX Article">}
%TCIDATA{Language=American English}
%TCIDATA{CSTFile=40 LaTeX article.cst}

\setlength{\topmargin}{-0.8in}
\geometry{top=0.93in}
\setlength{\textheight}{9in}
\setlength{\evensidemargin}{0in}
\setlength{\oddsidemargin}{0in}
\setlength{\textwidth}{6.5in}
\renewcommand{\baselinestretch}{1.4}
\newtheorem{theorem}{Theorem}[section]
\newtheorem{acknowledgement}[theorem]{Acknowledgement}
\newtheorem{algorithm}[theorem]{Algorithm}
\newtheorem{axiom}[theorem]{Axiom}
\newtheorem{case}[theorem]{Case}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{conclusion}[theorem]{Conclusion}
\newtheorem{condition}[theorem]{Condition}
\newtheorem{conjecture}[theorem]{Conjecture}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{criterion}[theorem]{Criterion}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{example}[theorem]{Example}
\newtheorem{exercise}[theorem]{Exercise}
\newtheorem{lemma}{Lemma}[section]
\newtheorem{notation}[theorem]{Notation}
\newtheorem{problem}[theorem]{Problem}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{remark}[theorem]{Remark}
\newtheorem{solution}[theorem]{Solution}
\newtheorem{summary}[theorem]{Summary}
\newtheorem{assumption}{Assumption}[section]
\newenvironment{proof}[1][Proof]{\noindent \textbf{#1.} }{\  \rule{0.5em}{0.5em}}
\input{tcilatex}
\begin{document}

\noindent
%TCIMACRO{\TeXButton{Today}{\today}}%
%BeginExpansion
\today%
%EndExpansion

\bigskip

\section{Robust Inference for Risk Prices}

\subsection{Asymptotic Distribution of the Reduced-Form Parameter}

Write $\omega :=(\omega _{1},\omega _{2},\omega _{3})^{\prime },$ where $%
\omega _{1}=(\rho ,c,\delta )\in O_{1},$ $\omega _{2}=(\gamma ,\beta ,\psi
)\in O_{2},$ and $\omega _{3}=\zeta \in O_{3}.$ The parameter space for $%
\omega $ is $O=O_{1}\times O_{2}\times O_{3}\subset R^{d_{\omega }}.$ The
true value of $\omega $ is assumed to be in the interior of the parameter
space.

Below we describe the estimator $\widehat{\omega }:=(\widehat{\omega }_{1},%
\widehat{\omega }_{2},\widehat{\omega }_{3})^{\prime }$ and provide its
asymptotic distribution. We estimate these parameters separately because $%
\omega _{1}$ only shows up in the conditional mean and variance of $\sigma
_{t+1}^{2},$ $\omega _{2}$ only shows up in the conditional mean of $r_{t+1},
$ and $\omega _{3}$ only shows up in the conditional variance of $r_{t+1}.$

We first estimate $\omega _{1}=(\rho ,c)$ based on the conditional mean and
variance of $\sigma _{t+1}^{2}$, which can be equivalently written as
\begin{eqnarray}
E[\sigma _{t+1}^{2}|\sigma _{t}^{2}] &=&A\text{ and }E[\sigma
_{t+1}^{4}|\sigma _{t}^{2}]=B,\text{ where }  \notag \\
A &=&\rho \sigma _{t}^{2}+c\delta \text{ and }B=A^{2}+\left( 2c\rho \sigma
_{t}^{2}+c^{2}\delta \right) .
\end{eqnarray}%
Because the conditional mean of $\sigma _{t+1}^{2}$ and $\sigma _{t+1}^{4}$
are linear and quadratic functions, respectively, of the conditioning
variable $\sigma _{t}^{2},$ without loss of efficiency, they can be
transformed to the unconditional moments%
\begin{equation}
E[h_{t}(\omega _{10})]=0,\text{ where }h_{t}(\omega _{1})=[(1,\sigma
_{t}^{2})\otimes (\sigma _{t+1}^{2}-A),(1,\sigma _{t}^{2},\sigma
_{t}^{4})\otimes (\sigma _{t+1}^{4}-B)]^{\prime },
\end{equation}%
where $\omega _{10}$ represents the true value of $\omega _{1}.$ The
two-step GMM estimator of $\omega _{1}$ is%
\begin{equation}
\widehat{\omega }_{1}=\underset{\omega _{1}\in O_{1}}{\arg \min }\left(
T^{-1}\sum_{t=1}^{T}h_{t}(\omega _{1})\right) ^{\prime }\widehat{V}%
_{1}\left( T^{-1}\sum_{t=1}^{T}h_{t}(\omega _{1})\right) ,
\label{omega 1 est}
\end{equation}%
where $\widehat{V}_{1}$ is a consistent estimator of $V_{1}=\sum_{m=-\infty
}^{\infty }\mathbb{C}ov[h_{t}(\omega _{10}),h_{t+m}(\omega _{10})].$

We estimate $\omega _{2}$ by the generalized least squares (GLS) estimator
because the conditional mean of $r_{t+1}$ is a linear function of the
conditioning variable $\sigma _{t}^{2}$ and $\sigma _{t+1}^{2}$ and the
conditional variance is proportional to $\sigma _{t+1}^{2}.$ The GLS\
estimator of $\omega _{2}$ is%
\begin{eqnarray}
\widehat{\omega }_{2} &=&\left( \sum_{t=1}^{T}x_{t}x_{t}^{\prime }\right)
^{-1}\sum_{t=1}^{T}x_{t}y_{t},\text{ where }  \notag \\
x_{t} &=&\sigma _{t+1}^{-1}(1,\sigma _{t}^{2},\sigma _{t+1}^{2})^{\prime }%
\text{ and }y_{t}=\sigma _{t+1}^{-1}r_{t+1}.  \label{omega 2 est}
\end{eqnarray}%
We estimate $\omega _{3}$ by the sample variance estimator%
\begin{equation}
\widehat{\omega }_{3}=T^{-1}\sum_{t=1}^{T}\left( y_{t}-\widehat{y}%
_{t}\right) ^{2},\text{ where }\widehat{y}_{t}=x_{t}^{\prime }\widehat{%
\omega }_{2}.  \label{omega 3 est}
\end{equation}

Let
\begin{equation}
f_{t}(\omega )=\left(
\begin{array}{c}
h_{t}(\omega _{1}) \\
x_{t}(y_{t}-x_{t}^{\prime }\omega _{2}) \\
(y_{t}-x_{t}^{\prime }\omega _{2})^{2}%
\end{array}%
\right) \in R^{d_{f}}\text{ and }V=\sum_{m=-\infty }^{\infty }\mathbb{C}%
\mathbf{ov}[f_{t}(\omega _{0}),f_{t+m}(\omega _{0})].
\end{equation}%
The estimator $\widehat{\omega }$ defined above is based on the first moment
of $f_{t}(\omega ).$ Let $\widehat{V}$ denote a heteroskeasticity and
autocorrelation consistent (HAC) estimator of $V$. The estimator $\widehat{V}%
_{1}$ is a submatrix of $\widehat{V}$ associate with $V_{1}.$

Let $P$ denote the distribution of the data $\mathcal{W}=\{W_{t}=(r_{t+1},%
\sigma _{t+1}^{2}):t\geq 1\}$ and $\mathcal{P}$ denote the parameter space
of $P$. Note that the true values of the structural parameter and the
reduced-form parameters are all determined by $P.$ We allow $P$ to change
with $T.$ For notational simplicity, the dependence on $P$ and $T$ is
suppressed. Let $H_{t}(\omega _{1})=\partial h_{t}(\omega _{1})/\partial
\omega _{1}^{\prime }.$

\smallskip

\noindent \textbf{Assumption R}. The following conditions hold uniformly
over $P\in \mathcal{P}$.

\noindent (i) $V^{-1/2}\{T^{-1/2}(\sum_{t=1}^{T}f_{t}(\omega _{0})-\mathbb{E[%
}f_{t}(\omega _{0})\mathbb{]\} \rightarrow }_{d}N(0,I)$ and $\widehat{V}%
-V\rightarrow _{p}0.$

\noindent (ii) $T^{-1}\sum_{t=1}^{T}(h_{t}(\omega _{1})-\mathbb{E[}%
h_{t}(\omega _{1}))\rightarrow _{p}0$ and $T^{-1}\sum_{t=1}^{T}(H_{t}(\omega
_{1})-\mathbb{E[}H_{t}(\omega _{1})\mathbb{])}\rightarrow _{p}0,$ $\mathbb{E[%
}H_{t}(\omega _{1})\mathbb{]}$ is continuous in $\omega _{1},$ all uniformly
over the parameter space of $\omega _{1}.$

\QTP{Body Math}
\noindent (iii) $T^{-1}\sum_{t=1}^{T}(x_{t}x_{t}^{\prime }-\mathbb{E[}%
x_{t}x_{t}^{\prime }\mathbb{])\rightarrow }_{p}0.$

\noindent (iv) $C^{-1}\leq \lambda _{\min }(A)\leq \lambda _{\max }(A)\leq C$
for $A=V,\mathbb{E[}H_{t}\left( \omega _{1,0}\right) ^{\prime }H_{t}\left(
\omega _{1,0}\right) ]),\mathbb{E[}x_{t}x_{t}^{\prime }],\mathbb{E[}%
z_{t}z_{t}^{\prime }],$ where $z_{t}=(1,\sigma _{t}^{2},\sigma
_{t}^{4})^{\prime }.$

\smallskip

Assumptions R(i)-(iii) are the central limit theorem and the uniform law of
large numbers applied to weakly dependent time series data. Assumption
R(iv)-R(v) are typical regularity conditions for the identification and $%
T^{1/2}$ normality of the reduced-form parameter estimator.

Let $H(\omega _{1})=\mathbb{E[}H_{t}(\omega _{1})]$ and $\overline{H}(\omega
_{1})=T^{-1}\sum_{t=1}^{T}H_{t}(\omega _{1}).$ Define%
\begin{eqnarray}
F &=&diag\{[H(\omega _{10})V_{1}^{-1}H(\omega _{10})]^{-1}H(\omega
_{10})V_{1}^{-1},\mathbb{E[}x_{t}x_{t}^{\prime }]^{-1},1\},  \notag \\
\widehat{F} &=&diag\{[\overline{H}(\widehat{\omega }_{1})^{\prime }\widehat{V%
}_{1}^{-1}\overline{H}(\widehat{\omega }_{1})]^{-1}\overline{H}(\widehat{%
\omega }_{1})^{\prime }\widehat{V}_{1}^{-1},[T^{-1}%
\sum_{t=1}^{T}x_{t}x_{t}^{\prime }]^{-1},1\}.  \label{Fhat}
\end{eqnarray}%
The following Lemma provides the asymptotic distribution of the reduced-form
parameter and a consistent estimator of its asymptotic covariance. Note that
we put the asymptotic covariance on the left side of the convergence to
allow the distribution of the data to change with sample size $T.$

\begin{lemma}
\label{Lemma Reduce}Suppose Assumption R holds. The following results hold
uniformly over $P\in \mathcal{P}$.

\noindent \emph{(i)} $\xi _{T}:=\Omega ^{-1/2}T^{-1/2}(\widehat{\omega }%
-\omega _{0})\rightarrow _{d}\xi \sim N(0,I),$ where $\Omega =FVF^{\prime }.$

\noindent \emph{(ii)} $\widehat{\Omega }-\Omega \rightarrow _{p}0,$ where $%
\widehat{\Omega }=\widehat{F}\widehat{V}\widehat{F}^{\prime }.$
\end{lemma}

\subsection{Weak Identification}

The true value of the structural parameter $\lambda $ and the reduced-form
parameter $\omega $ satisfies the link function $g(\lambda _{0},\omega
_{0})=0.$In a standard problem without any identification issue, we can
estimate $\lambda _{0}$ by the minimum distance estimator $\widehat{\lambda }%
=(\widehat{\theta },\widehat{\pi },\widehat{\phi })$ that minimizes $%
Q_{T}(\lambda )=g(\lambda ,\widehat{\omega })^{\prime }W_{T}g(\lambda ,%
\widehat{\omega })$ for some weighting matrix $W_{T}$ and construct tests
and confidence sets for $\lambda _{0}$ based on the asymptotic normal
distribution of $T^{1/2}(\widehat{\lambda }-\lambda _{0})$. However, this
standard method does not work in the present problem when $\pi _{0}$ is only
weak identified. In this case, $g(\lambda ,\widehat{\omega })$ is almost
flat in $\pi $ and the minimum distance estimator of $\widehat{\pi }$ is not
even consistent. To make the problem even more complicated, the
inconsistency of $\widehat{\pi }$ has a spillover effect on $\widehat{\theta
}$ and $\widehat{\phi },$ making the distribution of $\widehat{\theta }$ and
$\widehat{\phi }$ non-normal even in large sample.

Before presenting the robust test, we first introduce some useful quantities
and provide some heuristic discussions of the identification problem and its
consequence. Let $G(\lambda ,\omega )$ denote the partial derivative of $%
g(\lambda ,\omega )$ wrt $\omega .$ Let $g_{0}(\lambda )=g(\lambda ,\omega
_{0})$ and $G_{0}(\lambda )=G(\lambda ,\omega _{0})$ be the link function
and its derivative evaluated at $\omega _{0}$ and $\widehat{g}(\lambda
)=g(\lambda ,\widehat{\omega })$ and $\widehat{G}(\lambda )=G(\lambda ,%
\widehat{\omega })$ be the same quantities evaluate at the estimator $%
\widehat{\omega }.$ The delta method gives
\begin{equation}
\eta _{n}(\lambda ):=T^{1/2}\left[ \widehat{g}(\lambda )-g_{0}(\lambda )%
\right] =G_{0}(\lambda )\Omega ^{1/2}\cdot \xi _{T}+o_{p}(1),
\label{emp pro}
\end{equation}%
where $\xi _{T}\rightarrow _{d}N(0,I)$ following Lemma \ref{Lemma Reduce}.
Thus, $\eta _{n}(\cdot )$ weakly converges to a Gaussian process $\eta
(\cdot )$ with covariance function $\Sigma (\lambda _{1},\lambda
_{2})=G_{0}(\lambda _{1})\Omega G_{0}(\lambda _{2})^{\prime }.$

Following (\ref{emp pro}), we can write $T^{1/2}\widehat{g}(\lambda )=\eta
_{n}(\lambda )+T^{1/2}g_{0}(\lambda ),$ where $\eta _{n}(\lambda )$ is the
noise from the reduced-form parameter estimation and $T^{1/2}g_{0}(\lambda )$
is the signal from the link function. Under weak identification, $%
g_{0}(\lambda )$ is almost flat in $\lambda ,$ modelled by the signal $%
T^{1/2}g_{0}(\lambda )$ being finite even for $\lambda \neq \lambda _{0}$
and $T\rightarrow \infty .$ Thus, the signal and the noise are of the same
order of magnitude, yielding an inconsistent minimum distance estimator $%
\widehat{\lambda }.$ This is in contrast with the strong identification
scenario, where $T^{1/2}g_{0}(\lambda )\rightarrow \infty $ for $\lambda
\neq \lambda _{0}$ as $T\rightarrow \infty $ and $g_{0}(\lambda _{0})=0.$ In
this case, the signal is so strong that the minimum distance estimator is
consistent.

The identification strength of $\lambda _{0}$ is determined by the function $%
T^{1/2}g_{0}(\lambda ).$ However, this function is unknown and cannot be
consistently estimated (due to $T^{1/2}$). Thus, we take the conditional
inference procedure as in Andrews and Mikusheva (2016)\ and view $%
T^{1/2}g_{0}(\lambda )$ as an infinite dimensionalnuisance parameter for the
inference for $\lambda _{0}$. The goal is to control robust confidence set
(CS) for $\lambda _{0}$ that has correct size asymptotically regardless of
this unknown nuisance parameter.

\subsection{Conditional QLR\ Test}

We construct a confidence set for $\lambda $ by inverting the test $%
H_{0}:\lambda =\lambda _{0}$ vs $H_{1}:\lambda \neq \lambda _{0}$. The test
statistic is a QLR\ statistic that takes the form%
\begin{equation}
QLR(\lambda _{0})=T\widehat{g}(\lambda _{0})^{\prime }\widehat{\Sigma }%
(\lambda _{0},\lambda _{0})^{-1}\widehat{g}(\lambda _{0})-\underset{\lambda
\in \Lambda }{\min }T\widehat{g}(\lambda )^{\prime }\widehat{\Sigma }%
(\lambda ,\lambda )^{-1}\widehat{g}(\lambda ),  \label{QLR stat}
\end{equation}%
where $\widehat{\Sigma }(\lambda _{1},\lambda _{2},)=\widehat{G}(\lambda
_{1})\widehat{\Omega }\widehat{G}(\lambda _{2})^{\prime }$ and $\widehat{%
\Omega }$ is the consistent estimator of $\Omega $ defined above$.$

Andrews and Mikusheva (2016) provide the conditional QLR\ test in a
nonlinear GMM problem, where $\widehat{g}(\lambda )$ is replaced by a sample
moment. The same method can be applied to the present nonlinear minimum
distance problem. Following AM, we first project $\widehat{g}(\lambda )$
onto $\widehat{g}(\lambda _{0})$ and construct a residual process%
\begin{equation}
\widehat{r}(\lambda )=\widehat{g}(\lambda )-\widehat{\Sigma }(\lambda
,\lambda _{0})\widehat{\Sigma }(\lambda _{0},\lambda _{0})^{-1}\widehat{g}%
(\lambda _{0}).  \label{red process}
\end{equation}%
The limiting distribution of $\widehat{r}(\lambda )$ and $\widehat{g}%
(\lambda _{0})$ are Gaussian and independent. Thus, conditional on $\widehat{%
r}(\lambda ),$ the asymptotic distribution of $\widehat{g}(\lambda )$ no
longer depends on the nuisance parameter $T^{1/2}g_{0}(\lambda ),$ making
the procedure robust to all identification strength.

Specifically, we obtain the $1-\alpha $ conditional quantile of the QLR
statistic, denoted by $c_{1-\alpha }(r,\lambda _{0}),$ as follows. For $%
b=1,..,B,$ we take independent draws $\eta _{b}^{\ast }\sim N(0,\widehat{%
\Sigma }(\lambda _{0},\lambda _{0}))$ and produce a simulated process
\begin{equation}
g_{b}^{\ast }(\lambda )=\widehat{r}(\lambda )+\widehat{\Sigma }(\lambda
,\lambda _{0})\widehat{\Sigma }(\lambda _{0},\lambda _{0})^{-1}\eta
_{b}^{\ast }
\end{equation}%
and a simulated statistic%
\begin{equation}
QLR_{b}^{\ast }(\lambda _{0})=T\widehat{g}(\lambda _{0})^{\prime }\widehat{%
\Sigma }(\lambda _{0},\lambda _{0})^{-1}\widehat{g}(\lambda _{0})-\underset{%
\lambda \in \Pi }{\min }Tg_{b}^{\ast }(\lambda )^{\prime }\widehat{\Sigma }%
(\lambda ,\lambda )^{-1}g_{b}^{\ast }(\lambda ).
\end{equation}%
Let $b_{0}=\lceil (1-\alpha )B\rceil ,$ the smallest integer no smaller than
$(1-\alpha )B$. Then the critical value $c_{1-\alpha }(r,\lambda _{0})$ is
the $b_{0}^{th}$ smallest value among $\{QLR_{b}^{\ast },b=1,...,B\}.\Omega $

\smallskip

To sum up, we execute the following steps for a robust CS for $\lambda .$

\noindent (i) Estimate the reduced-form parameter $\widehat{\omega }=(%
\widehat{\omega }_{1},\widehat{\omega }_{2},\widehat{\omega }_{3})^{\prime }$
following the estimators defined in (\ref{omega 1 est}) and (\ref{omega 2
est}). Obtain a consistent estimator of its asymptotic covariance $\widehat{%
\Omega }=\widehat{F}\widehat{V}\widehat{F}^{\prime },$ where $\widehat{F}$
is define in (\ref{Fhat}) and $\widehat{V}$ is a heteroskedastic and
autocorrelation consistent (HAC) estimator of $V.$

\noindent For $\lambda _{0}\in \Lambda ,$ execute steps (ii)-(iv) below.

\noindent (ii) Construct the QLR statistic $QLR(\lambda _{0})$ in (\ref{QLR
stat}) using $g(\lambda ,\omega ),$ $G(\lambda ,\omega ),$ $\widehat{\omega }%
,$ and $\widehat{\Omega }.$

\noindent (iii) Compute the residual process $\widehat{r}(\lambda )$ in (\ref%
{red process}).

\noindent (iv) Given $\widehat{r}(\lambda ),$ compute the critical value $%
c_{1-\alpha }(r,\lambda _{0})$ described above.

\noindent (v) Repeat steps (ii)-(iv) for different values of $\lambda _{0}$.
Construct a confidence set by collecting the null values that are not
rejected, i.e., nominal level $1-\alpha $ confidence set for $\lambda _{0}$
is%
\begin{equation}
CS_{T}=\{ \lambda _{0}:QLR_{T}(\lambda _{0})\leq c_{1-\alpha }(r,\lambda
_{0})\}.
\end{equation}

\smallskip

To obtain confidence intervals for each element of $\lambda _{0},$ one
simple solution is to project the confidence set constructed above to each
axis. The resulting confidence interval also has correct coverage. An
alternative solution is to first concentrate out the nuisance parameters
before apply the conditional inference approach above, see Section 5\ of AM.
However, this concentration approach only works when the nuisance parameter
is strongly identified. In the present set-up, this approach does not work
for $\theta $ and $\phi $ because the nuisance parameter $\pi $ is weakly
identified.

\smallskip

\noindent \textbf{Assumption S}. The following conditions hold over $P\in
\mathcal{P},$ for any $\lambda $ in its parameter space, and any $\omega $
in some fixed neighborhood around its true value.

\noindent (i) $g(\lambda ,\omega )$ is twice continuously differentiable in $%
\omega $ and the second order derivative $G_{\omega }(\lambda ,\omega )$
satisfies $||G_{\omega }(\lambda ,\omega )||\leq C.$

\noindent (ii) $C^{-1}\leq \lambda _{\min }(G(\lambda ,\omega )^{\prime
}G(\lambda ,\omega ))\leq \lambda _{\max }(G(\lambda ,\omega )^{\prime
}G(\lambda ,\omega ))\leq C$.

\smallskip

\begin{lemma}
\label{Lemma CS}Suppose Assumption R and S hold. Then,
\begin{equation*}
\underset{T\rightarrow \infty }{\lim \inf }\underset{P\in \mathcal{P}}{\inf }%
\Pr \left( \lambda _{0}\in CS_{T}\right) \geq 1-\alpha .
\end{equation*}
\end{lemma}

This Lemma states that the confidence set constructed by the conditional
QLR\ test has correct uniform asymptotic size. Uniformity is important for
this confidence set to cover the true parameter with a probability close to $%
1-\alpha $ in finite-sample. Most importantly, this uniform result is
established over a parameter $\mathcal{P}$ that is large enough to allow the
weak identification of the structural parameter $\lambda .$

\bigskip

\bigskip

\noindent Proof of Lemma \ref{Lemma Reduce}. Under the assumption that (i) $%
\mathbb{E(}z_{t}z_{t}^{\prime })$ has the smallest eigenvalue bounded away
from 0 and (ii) $c>\varepsilon $ and $\delta >\varepsilon $ for some $%
\varepsilon >0,$ we not only have $\omega _{10}$ as an uniquely minimizer of
$||\mathbb{E}[h_{t}(\omega _{1})]||$ but also have a uniform positive lower
bound for $||E[h_{t}(\omega _{1})]||$ for $||\omega _{1}-\omega _{10}||\geq
\varepsilon .$ Thus, consistency of $\widehat{\omega }_{1}$ follows from
standard arguments for the consistency of a GMM estimator under an uniform
convergence of the criterion under Assumption R(ii).

Let $\overline{h}(\omega _{1})=T^{-1}\sum_{t=1}^{T}h_{t}(\omega _{1})$ and $%
\overline{H}(\omega )=T^{-1}\sum_{t=1}^{T}H_{t}(\omega _{1}).$ By
construction, the estimator satisfies the first order condition%
\begin{eqnarray}
0 &=&\left(
\begin{array}{c}
\overline{H}(\widehat{\omega }_{1})^{\prime }\widehat{V}_{1}^{-1}\overline{h}%
(\widehat{\omega }_{1}) \\
T^{-1}\sum_{T=1}^{T}x_{t}(y_{t}-x_{t}^{\prime }\widehat{\omega }_{2}) \\
\widehat{\omega }_{3}-T^{-1}\sum_{t=1}^{T}\left( y_{t}-\widehat{y}%
_{t}\right) ^{2}%
\end{array}%
\right)   \notag \\
&=&\left(
\begin{array}{c}
\overline{H}(\widehat{\omega }_{1})^{\prime }\widehat{V}_{1}^{-1}\overline{h}%
(\omega _{10})+\overline{H}(\widehat{\omega }_{1})^{\prime }\widehat{V}%
_{1}^{-1}\overline{H}(\widetilde{\omega }_{1})(\widehat{\omega }_{1}-\omega
_{10}) \\
T^{-1}\sum_{t=1}^{T}x_{t}(y_{t}-x_{t}^{\prime }\omega
_{20})-T^{-1}\sum_{t=1}^{T}x_{t}x_{t}^{\prime }\left( \widehat{\omega }%
_{2}-\omega _{20}\right)  \\
\left( \widehat{\omega }_{3}-\omega _{3}\right) +\omega
_{3}-T^{-1}\sum_{t=1}^{T}\left( y_{t}-x_{t}\widehat{\omega }_{2}\right) ^{2}%
\end{array}%
\right) ,  \label{L-R-1}
\end{eqnarray}%
where the second equality follows from a mean value expansion of $\overline{h%
}(\widehat{\omega }_{1})$ around $\omega _{10},$ with $\widetilde{\omega }%
_{1}$ between $\omega _{10}$ and $\widehat{\omega }_{1}$. Let%
\begin{equation}
\widetilde{F}=diag\{[\overline{H}(\widehat{\omega }_{1})^{\prime }\widehat{V}%
_{1}^{-1}\overline{H}(\widetilde{\omega }_{1})]^{-1}\overline{H}(\widehat{%
\omega }_{1})^{\prime }\widehat{V}_{1}^{-1},[T^{-1}%
\sum_{t=1}^{T}x_{t}x_{t}^{\prime }]^{-1},1\}.
\end{equation}%
Then (\ref{L-R-1}) implies that
\begin{eqnarray}
T^{1/2}\left( \widehat{\omega }-\omega \right)  &=&\widetilde{F}\cdot
T^{-1/2}\sum_{t=1}^{T}\left(
\begin{array}{c}
-h_{t}(\omega _{10}) \\
x_{t}(y_{t}-x_{t}^{\prime }\omega _{20}) \\
\left( y_{t}-x_{t}\widehat{\omega }_{2}\right) ^{2}-\omega _{3}%
\end{array}%
\right)   \notag \\
&=&\widetilde{F}\cdot T^{-1/2}\sum_{t=1}^{T}\left(
\begin{array}{c}
-h_{t}(\omega _{10}) \\
x_{t}(y_{t}-x_{t}^{\prime }\omega _{20}) \\
\left( y_{t}-x_{t}^{\prime }\omega _{20}\right) ^{2}-\mathbb{E}[\left(
y_{t}-x_{t}^{\prime }\omega _{20}\right) ^{2}]%
\end{array}%
\right) +\left(
\begin{array}{c}
0 \\
0 \\
\varepsilon _{T}%
\end{array}%
\right) ,  \label{L-R-2}
\end{eqnarray}%
where the second equality uses $\omega _{3}=\mathbb{E}[\left(
y_{t}-x_{t}^{\prime }\omega _{20}\right) ^{2}]$ by definition and
\begin{eqnarray}
\varepsilon _{T} &=&T^{-1/2}\sum_{t=1}^{T}\left[ \left( y_{t}-x_{t}^{\prime }%
\widehat{\omega }_{2}\right) ^{2}-\left( y_{t}-x_{t}^{\prime }\omega
_{20}\right) ^{2}\right]   \notag \\
&=&2T^{-1}\sum_{t=1}^{T}\left( y_{t}-x_{t}^{\prime }\omega _{20}\right)
x_{t}^{\prime }\left[ T^{1/2}\left( \widehat{\omega }_{2}-\omega
_{20}\right) \right] +o_{p}(1)  \notag \\
&=&o_{p}(1)  \label{L-R-3}
\end{eqnarray}%
because $T^{-1}\sum_{t=1}^{T}\left( y_{t}-x_{t}^{\prime }\omega _{20}\right)
x_{t}^{\prime }\rightarrow _{p}0$ and $T^{1/2}(\widehat{\omega }_{2}-\omega
_{20})=O_{p}(1)$ following Assumption R. In addition,
\begin{equation}
\widetilde{F}\rightarrow _{p}F  \label{L-R-4}
\end{equation}%
following from the consistency of $\widehat{\omega }_{1}$ and Assumption R.
Finally, the desirable result follows from (\ref{L-R-2})-(\ref{L-R-4}) and
Assumption R. The consistency of $\widehat{\Omega }$ follows from the
consistency of $\widehat{F}$ and $\widehat{V}.$ $_{\square }$

\bigskip

\noindent Proof of Lemma \ref{Lemma CS}. Assumptions 1-3 of AM holds under
Assumption R, S, and Lemma \ref{Lemma Reduce}. This Lemma then follows from
Theorem 1 of AM. $_{\square }$

\end{document}

doc/early versions/Risk Price CLR Inference.tex


\documentclass{article}

\usepackage{fontspec}
\usepackage{unicode-math}
\usepackage{siunitx}
\sisetup{separate-uncertainty}
\usepackage{booktabs}
\usepackage{biblatex}
\addbibresource{main.bib}

\begin{document}

\thispagestyle{empty}
\subsection*{Highly unscientific calculation of expected $B\to D\mu\mu$ yields}

\begin{itemize}
  \item $\mathcal{L} = \SI{3.189}{\per\femto\barn}$
  \item $σ_{b\overline{b}} = \SI{295 \pm 29}{\micro\barn}$
  \item $f_d \approx \num{0.4}$
  \item $ε_\text{geom} \approx \num{0.15}$
  \item $ε_\text{trig} \approx \num{0.8}$
  \item $ε_\text{strip} \approx \num{0.2}$
  \item All BRs for $q^2 > \SI{1}{GeV}$
  \item $\mathrm{BR}(D^{0}\to K^-\pi^+) = \SI{3.87 \pm 0.05}{\percent}$
  \item $\mathrm{BR}(D^+\to K^-\pi^+\pi^+) = \SI{9.13 \pm 0.19}{\percent}$
  \item $\mathrm{BR}(D_s^+\to K^+K^-\pi^+) = \SI{5.49 \pm 0.27}{\percent}$
  \item $\mathrm{BR}(D^{*0} \to D^0\pi^0) = \SI{61.9 \pm 2.9}{\percent}$
  \item $\mathrm{BR}(D^{*+} \to D^0\pi^+) = \SI{67.7 \pm 0.5}{\percent}$
  \item $\mathrm{BR}(D_s^{+*}\to D_s^+\pi^0) = \SI{5.8 \pm 0.7}{\percent}$
\end{itemize}

\vspace{3em}

\begin{tabular}{l S S}
  \toprule
  Decay & {\text{Predicted BR}} \cite{evans1}\cite{evans2} & {\text{Expected yield}} \\
  \midrule
  $B^0\to D\mu\mu$ & 2.6e-9 & 1.8 \\
  $B^0\to D^*\mu\mu$ & 1.4e-8 & 6.1 \\
  $B^+\to D^+\mu\mu$ & 2.5e-12 & 0.41e-2 \\
  $B^+\to D^{+*}\mu\mu$ & 1.0e-11 & 0.48e-2 \\
  $B^+\to D_s^+\mu\mu$ & 4.3e-11 & 0.17e-2 \\
  $B^+\to D_s^{+*}\mu\mu$ & 2.0e-10 & 0.12e-1 \\
  $B_c\to D_s^+\mu\mu$ & ? & ? \\
  $B_c\to D_s^{*+}\mu\mu$ & ? & ? \\
  \bottomrule
\end{tabular}

\newpage

\printbibliography

\end{document}


\section{Method}
To evaluate the effect of method compression we will explore how the combination of said methods effects two Information Retrieval tasks: Question Answering and Passage Retrieval. For each of the aforementioned tasks, we train tasks specific models built on bert-base-uncased. Given this baseline, we proceed to train variants with layers removed, with distillation, and with various degrees of unstructured pruning.
\subsection{Question Answering}
Question answering is the goal of providing an answer to a query $q$ given some sort of context passage $p$. While there are abstractive methods because of difficulty in evaluation we will focus on extractive methods. Extractive systems predict two attributes for each query and context passage: $token_{start}$ and $token_{end}$ each of which denotes an token index in a context passage. As a result, the answer $a$ to a query $q$ is $a=\{t_{token_{start}},...,t_{token_{end}}\}$. \\
The \textbf{S}tanford \textbf{QU}estion \textbf{A}nswering \textbf{D}ataset 1.1 (SQUAD) dataset has become the standard for extractive question answering. Constructed out of 500 context passages derived from Wikipedia it contains 100,000+ question-answer where each question was written by a human annotator. While there have been updates to the dataset and additional question answering datasets the widespread use of SQUAD makes it a logical choice. \\
\subsection{Passage Re-Ranking}
Passage ranking is a task which seeks to provide relevant short context passages for a given user query. These systems are commonly used as first stage retrievers for con question answering models. Since passages tend to have short context windows (under 512 tokens) transformer \cite{Vaswani2017AttentionIA} have become efficient models. With the introduction of large scale ranking datasets like MSMARCO \cite{Campos2016MSMA} and ODQA \cite{Karpukhin2020DensePR} data hungry neural methods thrived. Passage Retrieval has three main formulations, sparse retrieval,  dense retrieval, and hybrid retrieval. Sparse retrieval leverages traditional retrieval mechanisms like BM25 to rank relevant candidates. Dense retrieval leverages encoders which map queries and passages to a shared vector space and ranks candidates using an Approximate Nearest Neighbor index. Hybrid retrieval focuses on some combination of dense retrieval and commonly includes a sparse 1st stage ranker followed by a dense re-ranker. Our work focuses on Hybrid Retrieval on the MSMARCO Passage Retrieval dataset.\\
The MSMARCO dataset was originally a 1,000,000+ sample question answering dataset where given 10 context passages $p$, a human judge selected the most relevant passage and wrote the answer. This dataset was re-purposed for passage ranking by creating a collection of all unique passages and evaluating candidates ranking ability. Since the corpus does not contain a definitive ranking to optimize models train using negative and positive examples of query-passage relevance. Since relevance labels are binary, we follow standard training procedures and treat the task as a binary classification task. Given a query $q$ and a passage $p$ a model predicts how likely this pair is to belong in the relevant class.
\subsection{Distillation}
Distillation can be considered an effective method for label smoothing \cite{Hinton2015DistillingTK}. Our focus is on self distillation where a teacher model is used to distill its learning into a student model. The teacher model is trained to convergence using cross entropy loss while the student is trained with a combination of the cross entropy loss and the Kullback–Leibler divergence between the student and teacher logits. Cross entropy is represented in equation \ref{cross}, and the distillation loss is shown in equation \ref{distill} where $sl$ are student logits, $tl$ are teacher logits, $\lambda$ is a distillation hardness,and $KL$ is the Kullback–Leibler divergence.
\begin{equation}
L_{crossentropy} = - \sum_{i} ({y_i' \log(y_i) + (1-y_i') \log (1-y_i)})
\label{cross}
\end{equation}
\begin{equation}
L_{distill} = (1- \lambda )(L_{crossentropy}) + \lambda (KL(sl, tl))
\label{distill}
\end{equation}
\subsection{Compression}
Our work explores compression via unstructured pruning and layer removal. Language models like BERT are created by stacking identical layers of transformers and as a result compression can be achieved by simply removing these layers. We assign a mapping of layers to keep given target network depth based on experiments on most efficient methodologies and we find that for 1 layer networks we keep the first layer, for 3 we keep 1,6,12, for 6 we keep odd layers(1,3,5,7,9,11), for 9 we keep 1,3,4,5,7,8,9,10,12.
Unstructured pruning is the removal of weights in a network until a desired amount of weights remain. This is commonly achieved by setting weights in a network to zero or by adding a mask on top of network weights. Our work focuses on the the gradual removal of weight which have the smallest magnitude (magnitude pruning). Network layers are pruned independently of each other meaning that all components in a network share sparsity levels. Pruning is implement using Neural Magic's \footnote{https://neuralmagic.com/} sparseML library which introduces configurable sparsity in models with minimal overhead. \\
Layer removal is perhaps the simplest method we study as it solely focus on removing layers from a language model. Language models like BERT are created by stacking identical layers of transformers and as a result compression can be achieved by simply removing these layers.


\subsection{Clustering Substitute Embeddings (${\bf S}$)}
\label{sec:clustering-s}

In the previous section we group word types rather than word tokens by
clustering the word embeddings.  In this section we remove this
one-tag-per-word restriction and group word tokens according to the
embeddings of their substitutes.  We sample 64 random substitutes for
each word token and input them to S-CODE as word (${\bf W}$) --
substitute (${\bf S}$) pairs.  The resulting embeddings of the
substitutes are clustered using the instance weighted k-means
algorithm.  The process yields 64 cluster-ids for each target word
token's context.  The predicted category for the target word token is
chosen to be the majority cluster-id among these 64 cluster-ids.  Ties
for the majority are broken randomly.  In effect, we are using random
substitutes as features of the context, and we are clustering contexts
of individual word tokens.  The many-to-one accuracy is \wsymto\ and
the V-measure is \wsyvm\ which is lower than word type clustering.
However, we show that the performance on highly ambiguous words
improve significantly.

In order to explain the merit of the token based POS induction, we
first define the gold-tag perplexity for word types as follows:
\begin{equation} \label{eq:tag-perp}
GP(w) = 2^{H(p_w)} = 2^{-\sum_{t} p_w(t)log_2 p_w(t)}
\end{equation}
\noindent where $w$ is a word, $t$ is a tag, $p_w$ is the gold POS tag
distribution of the word type $w$ and $H(p_w)$ is the entropy of the
$p_w$ distribution.  The gold-tag perplexity ($GP$) is used to determine
the POS ambiguity of a word type, relating how often a word type is
associated with different POS tags in the test corpus.  A $GP$ of 1
for a word type $w$ indicates $w$ is associated with same POS tag
throughout the test corpus, meaning the word type $w$'s POS is
unambiguous.  A word with $N$ equally probable tags would have a $GP$
of $N$.  As the $GP$ increases the ambiguity of a word type increases and
this poses a handicap for induction models that limits tag variety for
the word types.  To display the limitations, we split the test corpus
into two subsets: word types with $GP$ less than 1.75 and word types
with $GP$ equal to or greater than 1.75.  We performed \mto\ evaluation
on our induction output and obtained the induced-tag -- gold-tag
mappings. Using the mappings obtained over the test corpus, we
evaluated the accuracy in the two subsets.

\begin{table}[h]
  \small
  \centering
  \caption{The \mto\ accuracy of ${\bf W}$, ${\bf S}$ and ${\bf W}\oplus{\bf S}$
    based models on two subsets that consist of words with $GP$ smaller
    and larger than 1.75, respectively.  The
    percentage of each subset in the test data is reported in the title
    bar.  The average $GP$ of each
    clustering over the whole corpus is reported in the last column.  Each
    score is an average of 10 random starts of our algorithm and the
    standard error of each one is reported in parenthesis while
    statistically the best \mto\ score of each column is reported in bold.
  }
  \label{tab:bins}
  \begin{tabular}{|c|c|c|c||c|}
    \hline
    Model & \specialcell{$GP < 1.75$\\$89\%$} & \specialcell{$GP \ge 1.75$\\$11\%$} & \specialcell{$GP \ge 1$ \\ $100\%$} & Average $GP$ \\
    \hline
    \specialcell{Clustering ${\bf W}$ embeddings\\(Type based)} & {\bf .8054 (.0065)} & .4383 (.0104) & {\bf \wsmto} & 1.0 (.0)\\
    \hline
    \specialcell{Clustering ${\bf W} \oplus {\bf S}$ embeddings\\(Sparse-token based)}& .7322 (.0079) & {\bf .4671 (.0174)} & \wsxymto & 1.3406 (.0057)\\
    \hline
    \specialcell{Clustering ${\bf S}$ embeddings\\(Token based)} & .6620 (.0051) & .4309 (.0093) & \wsymto & 1.5318 (.0076)\\
    \hline
  \end{tabular}
\end{table}

The performance of our algorithm clustering the ${\bf S}$ embeddings
is summarized in Table~\ref{tab:bins}.  Due to the one-tag-per-word
nature of POS induction, the type based model outperforms the token
based one on the unambiguous words. The token based model achieves
statistically comparable results with the type based model on the
ambiguous words.  Type based model can not handle words with ambiguity
while the token based model can.  In order to take advantage of both
models we apply our algorithm on concatenation of ${\bf W}$ and ${\bf
  S}$ embeddings in the next section.

\subsection{Clustering Concatenation of Word and Context Embeddings (${\bf W}\oplus{\bf S}$)}
\label{sec:clustering-concatenation}

Two models presented in earlier sections perform POS induction either
by assuming (Section~\ref{sec:clustering-w}) or discarding
(Section~\ref{sec:clustering-c}) the one-tag-per-word assumption.  In
this section we define a sparse-token based model which clusters the
concatenation of ${\bf W}$ and ${\bf S}$ embeddings.  This model not
only tends to put instances of a word type into the same cluster but
also performs token based clustering by incorporating the word type
and context information together.

Similar to the previous models, we generate ${\bf W}$ -- ${\bf S}$
pairs as the input to S-CODE.  For each observed ${\bf W}$ -- ${\bf
  S}$ pair in the S-CODE input, corresponding 25-dimensional $\phi_w$
and $\psi_c$ embeddings are concatenated to create a 50-dimensional
representation.  We used the same experimental setting of the previous
section and predict the token clusters according to the majority
cluster-id of the corresponding pairs.  The many-to-one accuracy of
this model is \wsxymto\ and the V-measure is \wsxyvm\ .

Table~\ref{tab:bins} presents the performance of the ${\bf
  W}\oplus{\bf S}$ based model over the subsets and it achieves
statistically better \mto\ than both of the ${\bf W}$ and ${\bf S}$
based models on ambiguous words.  Due to the bias towards to the
sparse clustering, sparse-token based model statistically improves the
\mto\ accuracy on unambiguous words compared to the ${\bf S}$ based
model but it still can not achieve the performance of the ${\bf W}$
based model.  The ${\bf W}\oplus{\bf S}$ based model constructs token
based clusters that tend to assign instances of a word type into the
same cluster which leads to a smaller average $GP$ than the ${\bf S}$
based model as shown in Table~\ref{tab:bins}.

%% We don't really need this part
%% \subsubsection{Paradigmatic vs Syntagmatic Representations of Word Context}
%% \label{sec:bigram-token}
%% In order to compare the token clustering performance of the
%% paradigmatic and the syntagmatic context representations we use the
%% same 4 models defined in Section~\ref{sec:bigram-type}.  Following the
%% previous section we concatenate the 25-dimensional $\phi_x$ and
%% $\psi_y$ ($\psi_{y_{1}}$ and $\psi_{y_{2}}$ in the fourth model)
%% embeddings of the corresponding observed pairs (tuples in the fourth
%% model) and represent the first three models outputs with a
%% 50-dimensional vectors (75-dimensional vectors in the fourth model).
%% The resulting vectors are clustered using k-means algorithm with 128
%% restarts.
%% \begin{table}[ht]
%% \centering
%% \small
%% \caption{Accuracies of the token based S-CODE models on the gold-tag
%%   perplexity separated subsets.}
%% \begin{tabular}{|l|l|l|l|}
%% \hline
%% Model & \specialcell{$GP < 1.75$\\$89\%$} & \specialcell{$GP \ge 1.75$\\$11\%$} & \specialcell{$GP \ge 1.0$\\$100\%$}\\
%% \hline
%% $X$ (word) - $Y$ (left bigram) & .5950 (.0051) & .4783 (.0005) & .5821 (.0041)\\
%% $X$ (word) - $Y$ (right bigram) & .6239 (.0049) & .3075 (.0153) & .5891 (.0046)\\
%% $X$ (word) - $Y$ (left and right bigram concatenation) & .7523 (.0065) & .4492 (.0240) & .7190 (.0049)\\
%% $X$ (word) - $Y_1$, $Y_2$ (left and right bigrams) & .6697 (.0065) & .4579 (.0052) & .6464 (.0051)\\
%% $X$ (word) - $Y$ (random substitutes) & .7322 (.0079) & .4671 (.0174) & .7030 (.0073)\\
%% \hline
%% \end{tabular}
%% \label{tab:tokens}
%% \end{table}


\documentclass[journal]{IEEEtran}

\usepackage[pdfpagemode={UseOutlines},bookmarks=true,bookmarksopen=true,
   bookmarksopenlevel=0,bookmarksnumbered=true,hypertexnames=false,
   colorlinks,linkcolor={blue},citecolor={blue},urlcolor={blue},
   pdfstartview={FitV},unicode,breaklinks=true]{hyperref}

% *** MATH PACKAGES ***
\usepackage{amsmath,amssymb,bm}

% *** GRAPHICS RELATED PACKAGES ***
\ifCLASSINFOpdf
	\usepackage[pdftex]{graphicx}
	\graphicspath{{images/}}
	% \DeclareGraphicsExtensions{.pdf,.jpeg,.png}
\else
	\usepackage[dvips]{graphicx}
	\graphicspath{{images/}}
	% \DeclareGraphicsExtensions{.eps}
\fi

% *** SUBFIGURE PACKAGES ***
\ifCLASSOPTIONcompsoc
  \usepackage[caption=false,font=normalsize,labelfont=sf,textfont=sf]{subfig}
\else
  \usepackage[caption=false,font=footnotesize]{subfig}
\fi

\usepackage[pdftex,dvipsnames]{xcolor}
\usepackage{listings,xargs}
\usepackage[colorinlistoftodos,prependcaption,textsize=tiny]{todonotes}

\definecolor{dkgreen}{rgb}{0,0.6,0}
\definecolor{gray}{rgb}{0.5,0.5,0.5}
\definecolor{mauve}{rgb}{0.58,0,0.82}
\newcommand{\fsize}{\scriptsize}
%\newcommand{\fsize}{\tiny}
\newcommand{\tabsize}{4}

\lstdefinelanguage{diff}{
  morecomment=[f][\color{blue}]{@@},		% group identifier
  morecomment=[f][\color{red}]-,		% deleted lines
  morecomment=[f][\color{dkgreen}]+,		% added lines
  morecomment=[f][\color{magenta}]{---},	% Diff header lines (must appear after +,-)
  morecomment=[f][\color{magenta}]{+++},
}

\lstset{frame=tb,
  language=C++,
%  aboveskip=0mm,
  belowskip=0mm,
  showstringspaces=false,
  columns=flexible,
  basicstyle={\fsize\ttfamily},
  numberstyle=\fsize\color{gray},
%  numbers=left,
  keywordstyle=\color{blue},
  commentstyle=\color{dkgreen},
  stringstyle=\color{mauve},
  breaklines=true,
  breakatwhitespace=true,
  tabsize=\tabsize
}

% correct bad hyphenation here
\hyphenation{op-tical net-works semi-conduc-tor}

\newcommand{\fref}[1]{\figurename~\ref{#1}}
\newcommand{\eref}[1]{(\ref{#1})}
\newcommand{\tref}[1]{\tablename~\ref{#1}}
\newcommand{\lref}[1]{LISTING~\ref{#1}}

\newcommandx{\improv}[2][1=]{\todo[linecolor=Plum,backgroundcolor=Plum!25,bordercolor=Plum,#1]{#2}}
\newcommand{\improvi}[1]{\improv[inline]{#1}}

\begin{document}

% paper title
\title{Acceleration of An SVM Classifier}

% author names and IEEE memberships
\author{Group 6: Yubo~Zhi (yz4116), Jiayang~Sun (js11815)}

% The paper headers
\markboth{ADSD Design Coursework}%
{ADSD Design Coursework}

% make the title area
\maketitle

\begin{abstract}
This course work aims to design and optimise a hardware Support Vector Machine (SVM) classifier on an embedded system using High Level Synthesis (HLS) language. The hardware implementation was optimised by various directives in HLS. In addition, the performance of software and hardware implementations were also assessed and compared in this excise.
\end{abstract}

\section{Introduction}

Dedicated hardware accelerators can perform a particular task faster then generalised software design. However, there are a few trade-offs associated with using hardware accelerators. In this exercise, an SVM classifier IP core was developed and optimised using Vivado HLS toolkit. The hardware platform used in this exercise is Zedboard, which include a Xilinx Zynq SoC FPGA. This FPGA has an ARM Cortex-A9 core together with the FPGA fabric. Hardware acceleration modules developed on the FPGA fabric can communicate with the ARM core through AXI interconnect. The procedure of designing and optimising the accelerator will be explained in details. Finally, the performance differences between hardware and software implementations were evaluated and discussed.

\hfill \today

\section{IP core implementation}

\subsection{Overview}

\fref{fig:ip} shows the purposed SVM classifier accelerator architecture, designed according to \eref{eq:cls}.

\begin{align}
	f(\bm{x}) &= sgn \left( bias + \sum_{i=1}^{N_{SV}} \alpha_i k(\bm{{sv}_i}, \bm{x}) \right)
	\label{eq:cls}\\
	\text{where}~k(\bm{x}, \bm{y}) &= tanh(2 ( \bm{x} \cdot \bm{y} ) )
\end{align}

\begin{figure}[t]
	\centering
	\includegraphics[width=0.6\columnwidth]{IP}
	\caption{Flow chart of purposed SVM classifier IP core}
	\label{fig:ip}
\end{figure}

According to the provided sample data, vectors have a width of 16 elements and there are 1050 support vectors. These numbers were replaced by macro definitions in the design, which can be easily adjusted to meet the properties of potentially other sample data.

\subsection{Dot product calculation}

Dot product between 2 vectors was calculated using the module designed previously in Part I.

\subsection{\texttt{tanh} calculation}

COordinate Rotation DIgital Computer (CORDIC) \cite{volder1959cordic} was used to calculate the \texttt{tanh} function.

\subsubsection{CORDIC}

CORDIC calculates trigonometric functions through vector rotation in 2-dimension space. It can be generalised to calculate \texttt{sinh} and \texttt{cosh} functions. The computations involved with CORDIC are simple integer addition, subtraction and shifting. This makes it suitable to use on a resource constrained embedded platform.

\subsubsection{Range and precision}\label{subsubsec:range}

The precision of CORDIC can be improved by allowing more iterations, however, the input range of CORDIC is non-ideal. The CORDIC implementation used in this design has an input range of about $\pm 1.10$ radians, but the sample dataset can give an input as large as $83.6$ radians.

In order to improve the performance of the \texttt{tanh()}, the algorithm for optimising range detection for CORDIC based on the trigonometric rules were implemented.

\begin{align}
\sinh(\theta + \alpha) = \cosh(\theta)\sinh(\alpha) + \sinh(\theta)\cosh(\alpha)
\label{eq:tri}\\
\cosh(\theta + \alpha) = \cosh(\theta)\cosh(\alpha) + \sinh(\theta)\sinh(\alpha)
\end{align}

As equations \eref{eq:tri} describe, the $\theta$ can be used as a offset in order to increase the range of CORDIC.  In addition, declare a flag (\texttt{neg})to store whether the input is negative. If it is negative set the flag to one and invert the value to positive. Furthermore, it is easy to observe that $tanh()$ is infinitely approximate to 1 when the angle lager than 6 \texttt{rads}. As the result, when the input angle is larger than 6 \texttt{rads}, the result will be directly equal to \texttt{1}. Otherwise, the \texttt{trigo\_index} will be equal to the \texttt{round(theta)}. Also, \texttt{trigo\_index} is the index of look up table(LUT) of \texttt{lut\_sinh[]} and \texttt{lut\_cosh[]}. These value is used as the offset $\theta$ in the previous equation \eref{eq:tri}. After that the rest part of the result will be calculated inside \texttt{cordic()}. The results from \texttt{cordic()} can be used as $\alpha$ in equation \eref{eq:tri}. Moreover, do the operation from trigonometric rules inside \lref{lst:ext}, it will combine the results from CORDIC and offset in order to get the final output. Finally, if the \texttt{neg} is one, invert the output to negative. \cite{llamocca2007fixed}

\begin{lstlisting}[float,floatplacement=h,caption={Extending the range of CORDIC},captionpos=b,label=lst:ext]
	if (theta >= (mdata_t)6.0) {
		result = 1.0;
	} else {
		// Trigo index to extend range
		int trigo_index = theta;
		theta = theta - trigo_index;
		// Call Cordic function
		ldata_t outcosh, outsinh;
		cordic(theta, &outcosh, &outsinh);
		// Trigo rules
		result_sinh = (lut_sinh[trigo_index] * outcosh + lut_cosh[trigo_index] * outsinh);
		result_cosh = (lut_cosh[trigo_index] * outcosh + lut_sinh[trigo_index] * outsinh);
		result = result_sinh / result_cosh;
		*output = neg ? (ldata_t)-result : result;
	}
\end{lstlisting}

\subsection{SVM classifier}

The top-level SVM classifier was designed according to the flow chart \fref{fig:ip}, as shown in \lref{lst:clas}. It iterates through all support vectors, accumulates each result, finally output a boolean value according to the sign of accumulator after bias removal. The function \texttt{k()} is the same as the k inside of equation \eref{eq:cls}. It will calculate the dot product of two vectors, and apply \texttt{tanh()} to the dot product in order to get the output. Finally, output will be checked whether it is negative. If it is positive, then output 0, otherwise output 1.

\begin{lstlisting}[float,floatplacement=h,caption={Top-level SVM classifier},captionpos=b,label=lst:clas]
static void k(ldata_t u[N], ldata_t v[N], ldata_t *output)
{
	mdata_t res;
	dotp(u, v, &res);
	res = res << 1u;	// * 2
	fp_tanh(res, output);
}

void classifier(ldata_t x[N], int *output)
{
	hdata_t sum = 0;
	for (int i = 0; i != ASIZE(alpha); i++) {
		ldata_t res;
		k(SVs[i], x, &res);
		sum += res * alpha[i];
	}
	*output = sum + bias >= 0 ? 0 : 1;
}
\end{lstlisting}

\subsection{Hardware verification}

\subsubsection{Test bench}

The design of SVM classifier was verified using C simulation test benches. The test bench iterates through all \texttt{testData} vectors from the sample dataset and executes the classifier. The results from classifier were compared with \texttt{testDataLabel} ground truths from the same sample dataset. Any difference will be recorded, and an overall error rate will be calculated at the end of simulation.

\subsubsection{Reference generation}

A double precision example classifier implementation was available from the dataset. It produces 170 incorrect results from all 2000 ground truths values, corresponding to an error rate of $8.5 \%$. The results including errors from the double precision were considered the reference output, therefore it can reflect any precision mismatches between the hardware implementation and software double precision implementation.

\subsection{Design evaluation}

The original implementation without any optimisation directives gives a maximum latency of 164877 clock cycles, as shown by \tref{tbl:res_unopt}.

\begin{table}[ht]
	% increase table row spacing, adjust to taste
	\renewcommand{\arraystretch}{1.3}
	\caption{Performance and resource usage of unoptimised version}
	\label{tbl:res_unopt}
	\centering
	% Some packages, such as MDW tools, offer better commands for making tables
	% than the plain LaTeX2e tabular which is used here.
	\begin{tabular}{llll}
		\hline
		Item			& Unoptimised	\\
		\hline
		Estimated clock timing	& $8.77$	\\
		Latency			& $164877$	\\
		Interval		& $164878$	\\
		Co-simulation (average)	& $203$		\\
		\hline
		BRAM\_18K		& $36~(12\%)$	\\
		DSP48E			& $6~(2\%)$	\\
		FF			& $1436~(1\%)$	\\
		LUT			& $4086~(7\%)$	\\
		\hline
	\end{tabular}
\end{table}

Running RTL co-simulation using all 2000 test vectors with this order of delay interval between every test vector will be very time-consuming and not very useful. Therefore, only C simulation was used to ensure the correctness of the unoptimised design. RTL co-simulations were still used in later optimised design.

\section{Performance optimisation}

\subsection{Interface optimisation}

The interface for specified input vectors was optimised by reducing the interface width to a minimum and using the \texttt{ARRAY\_RESHAPE} directive. This directive can combine multiple elements of the input vector to a single 32-bit buffer, reduces the number of data transfers needed between the ARM core and the hardware accelerator.

\subsection{Resource usage optimisation}

In order to balanced the resources and accuracy, the bits of fix point number should be controlled by different bits of \texttt{ap\_fixed} type, as shown in \lref{lst:data_t}. The number of fractional bits for all types is 12. The only changed part is integer part, in order to have enough range to represent different data. For example, \texttt{hdata\_t} it is used for the accumulation, final result, and bias. Multiple times experimental tests are set in order to get exact number of bits for integer part of different arrays due to different range of data.

\begin{lstlisting}[float,floatplacement=h,caption={Optimised data type},captionpos=b,label=lst:data_t]
typedef ap_fixed<22, 10> hdata_t;
typedef ap_fixed<19, 7> mdata_t;
typedef ap_fixed<16, 4> data_t;
typedef ap_fixed<16, 4> ldata_t;
\end{lstlisting}

\subsection{Throughput optimisation}

\subsubsection{Pipelining}

By using the technique of pipelining, the throughput can be greatly improved without significant increases of hardware resource usages. This can be achieved by applying the \texttt{PIPELINE} directive. Vivado HLS will automatically insert pipeline stage registers in appropriate locations. An iteration interval of 1 clock cycles was achieved by pipelining. With the addition of a few pipeline stages filling the entire pipeline beforehand, the iteration of all 1050 support vectors now takes only over 1100 clock cycles.

\subsubsection{Loop unrolling}

There are also some small iteration loops in the design, such as the iteration inside dot product calculation which only iterates about 16 times. These loops can be unrolled to allow parallel computation of iterations, at the cost of more hardware resource usages. This can be achieved by applying the \texttt{UNROLL} directive. Vivado HLS is capable of duplicating the loop body multiple times, and applying map-reduce techniques to resolve dependencies automatically. Together with pipelining, this can reduce the number of pipeline stages.

\subsubsection{Parallel chains}

The latency can be further reduced by executing multiple iterations of the support vector loop inside the main classifier function in parallel. Instead of sequentially iterates through all 1050 support vectors, ideally this parallel execution can reduce the latency by a factor of parallel chains.
The disadvantage of this method is, the hardware resource usage will be increased by the same factor in order to implement multiple parallel chains. A trade-off between throughput and resource usage is implied.
This parallel execution was done by unrolling the main loop while specifying unroll factor. All constant values such as support vectors and alpha implemented by ROMs need to be partitioned by the same factor to prevent Vivado duplicating the entire ROM for each parallel chain. The array partition was done by specifying the \texttt{ARRAY\_PARTITION} directive.
\lref{lst:opt_cla} shows the classifier function with optimisations applied.

\begin{lstlisting}[float,floatplacement=h,caption={Optimised classifier top-level function},captionpos=b,label=lst:opt_cla]
void classifier(ldata_t x[N], int *output)
{
#pragma HLS ARRAY_RESHAPE variable=x cyclic factor=16 dim=1
#pragma HLS ARRAY_PARTITION variable=SVs cyclic factor=10 dim=1
#pragma HLS ARRAY_RESHAPE variable=SVs cyclic factor=16 dim=2
#pragma HLS ARRAY_PARTITION variable=alpha cyclic factor=10 dim=1
#pragma HLS INTERFACE s_axilite port=output
#pragma HLS INTERFACE s_axilite port=return
#pragma HLS INTERFACE s_axilite port=x
	hdata_t sum = 0, sums[12] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
#pragma HLS ARRAY_PARTITION variable=sums cyclic factor=10 dim=1
loop:	for (int i = 0; i != ASIZE(alpha); i++) {
#pragma HLS PIPELINE
#pragma HLS UNROLL factor=10
		ldata_t res;
		k(SVs[i], x, &res);
		sums[i % 10] += res * alpha[i];
	}
	for (int i = 0; i != 10; i++) {
#pragma HLS PIPELINE
#pragma HLS UNROLL
		sum += sums[i];
	}
	*output = sum + bias >= 0 ? 0 : 1;
}
\end{lstlisting}

The \texttt{ARRAY\_PARTITION} directive is used for cutting the array into small parts in order to parallel the program. \texttt{ARRAY\_RESHAPE} also can separate the array, however, it will combine the array together after the progress so it is better to use inside the dot product in order to save resources. The main \texttt{for} loop is optimised by \texttt{PIPELINE} and \texttt{UNTROLL} together. The factor of \texttt{UNROLL} means the number of threads paralleled is 10. The maximum number of threads is ten because of the limitation of hard ware resources.

\subsection{Clock timing issue}

\begin{figure*}[!t]
	\centering
	\includegraphics[width=0.9\textwidth]{bd}
	\caption{Block diagram design of the entire system}
	\label{fig:bd}
\end{figure*}

An unexpected issue occurred when applying unroll optimisations. The target clock period specified for this design is $10 ns$, but Vivado kept generating paths exceeding that clock constrain. By analysing the warning messages about the critical path, it is possible to resolve this issue by separating the arithmetic involved in the critical path, letting Vivado insert pipeline registers between them.

An example of resolving the timing issue is given by \lref{lst:tim}. The original multiplication and accumulation operation was broken into 10 separate \texttt{sums} variables as caches, partitioned to match the loop unroll factor. So that each parallel chain has a dedicated accumulator, summarised together in later stages. This removes the need for accumulate all 10 parallel chains in each iteration, hence reduces the length of the critical path.

\begin{lstlisting}[float,floatplacement=h,language=diff,caption={Code modification to resolve timing issue},captionpos=b,label=lst:tim]
--- old/classifier.cpp
+++ new/classifier.cpp
@@ -62,12 +62,19 @@ void classifier(ldata_t x[N],
 #pragma HLS INTERFACE s_axilite port=output
 #pragma HLS INTERFACE s_axilite port=return
     hdata_t sum = 0;
+    hdata_t sums[10] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
+#pragma HLS ARRAY_PARTITION variable=sums cyclic factor=10 dim=1
     for (int i = 0; i != ASIZE(alpha); i++) {
 #pragma HLS PIPELINE
 #pragma HLS UNROLL factor=10
        ldata_t res;
        k(SVs[i], x, &res);
-       sum += res * alpha[i];
+       sums[i % 10] += res * alpha[i];
+    }
+    for (int i = 0; i != 10; i++) {
+#pragma HLS PIPELINE
+#pragma HLS UNROLL
+       sum += sums[i];
     }
     *output = sum + bias >= 0 ? 0 : 1;
 }
\end{lstlisting}

\subsection{New \texttt{tanh()} reflection}

As the subsection \ref{subsubsec:range} mentioned, there is a new technique to enhance the range and accuracy of the \texttt{tanh()} fuction. However, it needs more resource and more instructions to complete, which means that it has higher latency than the previous CORDIC (133 cycles). However, the final latency of the design is shown in \fref{fig:latency}, which is 150 cycle. Although there are 17 more cycles to complete the calculation, it gets as accurate as the double precision answer generated by software test bench. Because if using the previous method to complete \texttt{tanh()} there are in total 169 errors, nonetheless it is different from the result come from the test bench which is 170 errors. As the result, the final implementation choose a more accuracy method in order to get enhanced classification.

\subsection{Design verification}

Correctness verifications by C simulations were done after every optimisations and code modifications, using the golden reference output generated from double precision implementation.

An unexpected discrepancy between Vivado HLS' C language and ISO C language standard (6.7.8.19 \cite{iso1999iec}) was discovered. All array elements must be explicitly initialised in Vivado HLS for a defined behaviour, whereas in ISO C standard, uninitialised elements in a partially initialised array should have the value 0, the same as static storage initialisation. Hence the 10 zeros written in the initialisation of \texttt{sums} in \lref{lst:tim}.

After applied all optimisations, the latency reduced significantly. RTL co-simulation was done at this stage before exporting as an IP core, to ensure the exported RTL implementation has the same functionality as the C implementation.

\subsection{Design evaluation}

\fref{fig:latency} shows the latency and varies aspects of hardware resources versus main loop unrolling factor.

\begin{figure}[ht]
	\centering
	\includegraphics[width=0.9\columnwidth]{latency}
	\caption{Latency and resource usage versus loop unrolling. unopt: Unoptimised version}
	\label{fig:latency}
\end{figure}

As loop unroll factor increases, hardware resource usages increase linearly. However, the effect on latency decreases exponentially. The latency can be estimated by \eref{eq:latency}.

\begin{equation}
	\text{latency} = \frac{N_{SV}}{\text{factor}} + \text{stages}
	\label{eq:latency}
\end{equation}

$N_{SV}$ equals the number of support vectors, which is 1050. Stages is the depth of pipeline, which is 44 in most cases, given by the loop latency value.

The equation shows, there is always some factors that can not be parallelised. Therefore the effect on performance will become more and more insignificant. Moreover, with a loop unroll factor of 10, DSP resource usage had already reached $95.45 \%$, impossible for further adjustments.

\section{System implementation}

After multiple optimisations, the improved implementation was packaged into Vivado as an IP core. The setup of Vivado and SDK were the same as in previous dot product exercise. \fref{fig:bd} shows the block diagram design of the system on FPGA. \fref{fig:sdk} describes an abstract structural overview of the testing program running on the ARM core, developed on the SDK.

\begin{figure}[t]
	\centering
	\includegraphics[width=0.3\columnwidth]{sdk}
	\caption{Overview of the testing program}
	\label{fig:sdk}
\end{figure}

\section{System evaluation}

Using the global timer inside ARM core, it is possible to get cycle accurate profiling information.

\lref{lst:test_int} shows one example of the tests. This test aims to record the time consumption of different procedures of the whole system. The whole progress can be divided into three parts which include data transmission from ARM core to IP core, starting, and calculation. After running this program, the total time and the time for each part will be print in the terminal.

\begin{lstlisting}[float,floatplacement=h,caption={Code for recoding execution time for different procedures},captionpos=b,label=lst:test_int]
unsigned int test_cls_hls_interrupt()
{
	XTime tick[4], perf[3] = {0, 0, 0};
	unsigned int err = 0;
	int *label = &testDataLabel[0];
	int16_t *x = &testDataI[0][0];
	for (size_t ix = ASIZE(testDataLabel); ix != 0; ix--) {
		XTime_GetTime(&tick[0]);
		XClassifier_Set_x_V(&cls, *(XClassifier_X_v *)x);
		XTime_GetTime(&tick[1]);
		XClassifier_Start(&cls);
		XTime_GetTime(&tick[2]);
		interrupt_wait();
		XTime_GetTime(&tick[3]);
		err += (!XClassifier_Get_output_r(&cls)) != (!*label++);
		perf[0] += tick[1] - tick[0];
		perf[1] += tick[2] - tick[1];
		perf[2] += tick[3] - tick[2];
		x += N;
	}
	printf("Data %llu, starting %llu, result %llu\r\n", perf[0], perf[1], perf[2]);
	return err;
}
\end{lstlisting}

An example output of the test program was given by \lref{lst:test_int_out}.

\begin{lstlisting}[float,floatplacement=h,language=,caption={Example output from testing program},captionpos=b,label=lst:test_int_out]
Build @ Mar 16 2017 22:15:28
HLS peripheral is ready
Converting data...
Conversion finished, 863684 ticks

Starting tests...
<double precision classifier> finished, 450697222 ticks, error count 170
<fixed point classifier> finished, 200704648 ticks, error count 170
Data 1097219, starting 266571, result 1100144
<HLS classifier by polling> finished, 2651697 ticks, error count 170
Data 1089646, starting 267273, result 1287449
<HLS classifier by interrupt> finished, 2826942 ticks, error count 170
SVM classifiers testing done.
\end{lstlisting}

\subsection{Performance and resource usage}

The support vector classifier was also implemented as software using double precision and fixed point CORDIC inside the ARM core, as a performance comparison to hardware implementation.

The performance data of software and notable hardware implementations are summarised in \tref{tbl:throughput}. \fref{fig:throughput} shows detailed hardware implementation throughputs and resource usage values. Both polling and interrupt methods for waiting results ready were implemented, however, only performance data from polling method was shown. The polling method gives slightly better overall performance values, probably due to some interfacing overheads of supporting interrupt.

\begin{table}[ht]
	% increase table row spacing, adjust to taste
	\renewcommand{\arraystretch}{1.3}
	\caption{Performance and resource usages of system implementations}
	\label{tbl:throughput}
	\centering
	% Some packages, such as MDW tools, offer better commands for making tables
	% than the plain LaTeX2e tabular which is used here.
	\begin{tabular}{llll}
		\hline
		Type			& Throughput (k vectors / second)	\\
		\hline
		Double precision	& $2.96$	\\
		Fixed precision CORDIC	& $6.64$	\\
		\hline
		Unoptimised hardware	& $1.47$	\\
		Pipelined hardware	& $152.17$	\\
		Unrolled with factor 10	& $541.18$	\\
		\hline
	\end{tabular}
\end{table}

It can be seen from the table, the final optimised hardware implementation gives a speed-up of 182 times over the double precision and 80.5 times over the fixed precision software implementation, quite substantial. However, as the figure suggests, the throughput almost saturates as loop unroll factor increases above 9.

\begin{figure}[ht]
	\centering
	\includegraphics[width=0.9\columnwidth]{throughput}
	\caption{System throughput and resource usages versus loop unroll factor}
	\label{fig:throughput}
\end{figure}

\fref{fig:time} shows a detailed breakdown of total time spent on each stage of interfacing the hardware SVM classifier. The time values shown on the figure were summarised for all 2000 test vectors.

\begin{figure}[ht]
	\centering
	\includegraphics[width=0.9\columnwidth]{time}
	\caption{Detailed time usage breakdown of interfacing hardware classifier}
	\label{fig:time}
\end{figure}

It can be seen from the figure that transferring the 16 elements input vectors (\texttt{tData}) and starting the classifier (\texttt{tStart}) are very inefficient and unavoidable, taking almost the same time as actual computation (\texttt{tResult}) in the final version where loop unroll factor is 10. These unavoidable overheads limit the maximum throughput achievable, result in the saturation.

\subsection{Reflection and Limitations}

Different from previous dot product exercise, the SVM classifier hardware accelerator designed in this exercise improves the system throughput by a large factor. The benefits of using a massively paralleled hardware accelerator greatly overcome the overheads of interfacing the accelerator, based on the number of computations offloaded onto hardware each time. The processor core is also freed to do some other sequential computations while waiting for the results to be available, further increases system performance and efficiency.

\section{Conclusion}

By evaluating the performance data of software and hardware implementation of the same algorithm, the advantages, use case and limiting factors of using hardware accelerators were investigated. Although the computation speed of dedicated hardware accelerator can be faster than software implementation, overheads such as data transfer performance may limit the actual efficiency.

% References section
\bibliographystyle{IEEEtran}
\bibliography{Reference}

% that's all folks
\end{document}


\documentclass{article}

\usepackage{amsmath,amssymb,amsthm,bm,graphicx,enumerate}
\usepackage{color}

\newcommand{\be}{\begin{equation}}
\newcommand{\ee}{\end{equation}}

\usepackage{geometry}
 \geometry{
 a4paper,
 total={210mm,297mm},
 left=35mm,
 right=35mm,
 top=30mm,
 bottom=30mm,
 }

 % make a note
\newcommand{\mynote}[1]{\noindent \textcolor{red} {{$\blacktriangleright$
   \small{\textsf{#1}} $\blacktriangleleft$}}}

\title{Covid-19 epidemic control}
\author{Lauri Neuvonen, Matthias Wildemeersch}
\date{}

\begin{document}

\maketitle
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Scenarios of interest}
\begin{enumerate}[-]
\item Limitations of the current model: (i) All asymptomatic become symptomatic (ii) recovered cannot turn susceptible again. These shortcomings can be resolved rather easily, but the corresponding transition rates are unknown and would require a sensitivity analysis. Possible extension in a later phase.
\item Distinction between lock-down and quarantine: Known infected people are put in quarantine with contact probability $\lambda^Q$; people in lockdown have a contact probability $\lambda^{LD}$; baseline contact probability is $\lambda$; $\lambda^Q < \lambda^{LD}<\lambda$.
\item Economic output
\be
Y = NQ + \frac{\lambda^{LD}}{\lambda} LD + \frac{\lambda^Q}{\lambda} Q \, ,
\ee
with NQ, Q, and LD representing the number of people not in quarantine, in quarantine, and in lockdown.
\mynote{MW: need to think if we need another scaling factor in this formula} Loss in output can be calculated then as $Y_\mathrm{loss} = Y_\mathrm{baseline} - Y$.
\item number of objectives: (i) output or loss of output/working hours, (ii) number of deaths / number of hospitalized people, (iii) cost of the strategy. The loss of output could potentially be combined with the cost of the strategy. Another alternative is to look at the loss of working hours over the cost of the strategy.
\item scenarios of interest
\begin{enumerate}[i.]
\item generalized lock-down: Control strength of the lock-down by means of the parameter $\lambda^{LD}$; testing only for people who start developing symptoms. Trade-off between economic cost vs health cost as a function of $\lambda^{LD}$. Testing cost small.
\item Paul Romer scenario: massive population-scale testing without a generalized lock-down. Only people who test positive are put in quarantine; tradeoff related to sensitivity of diagnostic testing, where false negatives can freely circulate. Cost of strategy is high. This approach requires implementing repeated testing (see below). Testing cost high.
\item test-and-trace: No generalized lockdown. Only people who develop symptoms are tested, and if positive track the (on average) 10 (?sensitivity?) people with whom infected person was in contact. These 10 people should be distributed over different compartments; requires some tweaks in the model.  \mynote{MW: to be worked out in the coming weeks} Test-and-trace cost moderately high.
\end{enumerate}
\item repeated testing: requires transition from NA* to NA, and transition from NA* to IA without star. The transition from NA* to NA happens on average in 7 days (sensitivity), so transition probability is $\sigma = 1/7$. Fixes are marked in excel file in green.
\end{enumerate}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Test and trace}
In this section, we present two methods that capture test-and-trace for compartmental models. Test-and-trace can easily be implemented in individual based models, which keep track of the history and individual contacts of each agent. In compartmental models, working with expected values and perfect mixing, test-and-trace is less straightforward. The aim of this section is to quantify how many undiagnosed people will get traced and quarantined. The test-and-trace can be implemented in two different ways, (i) people that are reached through contact tracing go into quarantine without diagnosis, or (ii) people that are traced get tested if the testing capacity is sufficient, and get quarantined based the test result.

\subsection{Method 1}
In the baseline model, the following compartments are affected by test-and-trace, NA-NQ (undiagnosed and diagnosed), IA-NQ undiagnosed, and FN-NQ. All said compartments transition to the quarantined state if they get traced. The transition probability from compartment $X$ to $Y$ can be written as
\be
\xi_{X \to Y} = \xi_\mathrm{TT} f^X \, ,
\ee
where $\xi_\mathrm{TT}$ is a generic test-and-trace efficiency parameter which is scaled by the fraction $f^X$ of the compartment with respect to the unquarantined population
\be
f^X = \frac{N_t^X}{N_t^\mathrm{U}} = \frac{N_t^X}{N_t^\mathrm{NA-NQ-U}+N_t^\mathrm{NA-NQ-D} + N_t^\mathrm{IA-NQ-U} + N_t^\mathrm{FN-NQ}} \, .
\ee
This method has as pro that it is easy and that we consider the test-and-trace strength over a large interval. (The scaling parameter could even be disregarded) As disadvantage, the method is coarse and it does not take into account changes of tracing efforts over time as a result of the number of newly identified cases. Delays between testing and sequential tracing can be included by introducing additional compartments for the different phases in the incubation and symptomatic period.

\subsection{Method 2}
We aim to describe the efficiency of test-and-trace as a function of time, in other words, as a function of the trajectory of the epidemic. Assuming approach (i) of contact tracing where contact-traced people go into quarantine, we can define the transition probability between the non-quarantined compartment $X$ and the quarantined compartment $Y$ as $\xi_{X \to Y}^\mathrm{CT}$, which can be expressed as
\begin{align}
\xi_{X \to Y}^\mathrm{CT}(t) &= P(\mathrm{CT} \, | \,C_\mathrm{T}) \cdot P(C_\mathrm{T}) \nonumber \\
& = \eta \cdot P(C_\mathrm{T}) \, ,
\end{align}
where $\eta$ represents the efficiency of the test-and-trace strategy, the probability that an individual get contact-traced conditioned on the event $C_\mathrm{T}$ that this person was in contact with a diagnosed/tested person. The probability $P(C_\mathrm{T})$ stands for the contact probability with a diagnozed person. We can further write
\begin{align}
\xi_{X \to Y}^\mathrm{CT}(t) &= \eta \cdot \Big( P(C_\mathrm{I}) \cdot P(\mathrm{IT}) + P(C_\mathrm{N}) \cdot P(\mathrm{FP}) \Big)\nonumber \\
&= \eta \cdot \Big( \lambda^X \, \pi_t^I \cdot (\pi_t^\mathrm{IST} + \pi_t^\mathrm{IAT}) + \lambda^X \pi_t^N \cdot \pi_t^\mathrm{FP} \Big) \, ,
\end{align}
where $\pi_t^I$ and $\pi_t^N$ are the probabilities to meet an infected and non-infected individual conditioned on the meeting an individual, $\lambda^X$ stands for the meeting rate and depends on the compartment $X$, $\pi_t^\mathrm{IST}$ and $\pi_t^{IAT}$ represent the probability at time $t$ that an infected person is tested in the case of  symptomatic and asymptomatic individuals, and $\pi_t^\mathrm{FP}$ is the probability at time $t$ that a non-infected person falsely tested positive.
\begin{align}
\pi_t^\mathrm{IST} & = \frac{ \delta (\lambda^\mathrm{Q} \, M^\mathrm{IA-Q}_{t-1} + \lambda \, M^\mathrm{IA-NQ}_{t-1})}{M_t^\mathrm{I}} \nonumber \\
\pi_t^\mathrm{IAT} &= \frac{\lambda \, M_{t-1}^\mathrm{IA-NQ}\cdot \tau \,q^{sens} + \lambda^\mathrm{Q} \, M_{t-1}^\mathrm{IA-Q}\cdot \tau \,q^{sens}}{M_t^\mathrm{I}} \nonumber \\
\pi_t^\mathrm{FP} &= \frac{\lambda \, M_{t-1}^\mathrm{NA-NQ}\cdot \tau \,(1-q^{spec}) + \lambda^\mathrm{Q} \, M_{t-1}^\mathrm{NA-Q}\cdot \tau \,(1- q^{spec})}{M_t^\mathrm{N}} \, .
\end{align}
In order to avoid double counting, only the new infected symptomatic individuals at time $t$ are accounted in the formulation of the diagnosis probability $\pi_t^\mathrm{IST}$, which depends on multiple compartments at time $t-1$. The same idea is used in the formulation of $\pi_t^\mathrm{IAT}$ and $\pi_t^\mathrm{FP}$. We assume here that a symptomatic person is correctly diagnosed with probability 1. Note that by including test-and-trace into the model, the difference equations can now be written as $X_{t+1} = f(X_t, X_{t-1})$.

The quarantine rate $\xi^{CT}$ would work for a situation, where the share of infected people  in those who have been contact traced would be the same that in total population. This is however not the case as both the risk of infection and chance of getting traced increases when you meet an infected person. Therefore, the transition rate $\xi^{CT}$ must be split into two different rates, $\xi^{CT, I}$ and $\xi^{CT, N}$ for infected and non-infected compartments respectively. This can be modeled by adjusting the rates as $\xi^{CT, I} = w^I \xi^{CT}$ and $\xi^{CT, N} = w^N \xi^{CT}$

\subsection{Lauri's new formulation}
One thing to note is that tracing probability and infection probability are affected by the same meeting. This means that the meeting rate should only be counted once for those who get traced, they don't neeed a separate meeting for getting traced after having met the person who gives (or doesn't) them the infection.

The following transition paths can now be defined from NA-NQ:

\textbf{Path 1: meeting a newly diagnosed infected person and getting infected and traced}: The chance of having met such a person on last time step, given a meeting, is
\begin{align}
	\pi_t^\mathrm{IST} &= \frac{\delta (\lambda^\mathrm{Q} \, M^\mathrm{IA-Q}_{t-1} + \lambda \, M^\mathrm{IA-NQ}_{t-1})}{ M_t^\mathrm{total}}\nonumber  \\
	\pi_t^\mathrm{IAT} &= \frac{\lambda \, M_{t-1}^\mathrm{IA-NQ}\cdot \tau \,q^{sens} + \lambda^\mathrm{Q} \, M_{t-1}^\mathrm{IA-Q}\cdot (\tau + \tau^{CT}) \,q^{sens}}{M_t^\mathrm{total}} \nonumber \\
	\pi_t^\mathrm{FP} &= \frac{\lambda \, M_{t-1}^\mathrm{NA-NQ}\cdot \tau \,(1-q^{spec}) + \lambda^\mathrm{Q} \, M_{t-1}^\mathrm{NA-Q}\cdot (\tau + \tau^{CT}) \,(1- q^{spec})}{M_t^\mathrm{total}}
\end{align}
Meeting such a person (out of all meetings) can lead to infections  at conditional rate $\rho^S$, giving an infection rate per random meeting:
\begin{equation}
  \alpha^{IS}_{t-1} = \pi^{IST}_{t-1}\rho^S + \pi^{IAT}_{t-1}\rho^A
\end{equation}
At this point these persons will move to IANQ compartment, at rate:
\begin{equation}
	\lambda\alpha^{IS}_{t-1} = \lambda \pi^{IST}_{t-1}\rho^S
\end{equation} and the mass calculated below. This mass is the one infected mass that can be traced on the next time step. It is a 'compartment in compartment' within IA-NQ.
\begin{equation}
	M_t^{TI} = \lambda\alpha^{IS}_{t-1} M_{t-1}^{NA-NQ} = \lambda (\pi^{IST}_{t-1}\rho^S + \pi^{IAT}_{t-1}\rho^A)M_{t-1}^{NA-NQ}
\end{equation}
In our model, this transition happens as part of the total transition from NA-NQ to IA-NQ, happening at rate
\begin{equation}
	\lambda \alpha_{t-1} = \lambda \pi_t^I[\pi_t^{IS} \rho^S + \pi_t^{IA} \rho^A]
\end{equation}
from where they may proceed to IA-Q, based on the tracing efficiency $\eta$.

This means that only part of the 'new arrivals' , and therefore also only part of the total mass in IA-NQ are subject to potential tracing efforts. Also, at this point the chance that they have already met an infected person is accounted for (because we know they have, that's how they got infected). Considering the share of these new members in the whole group of infected persons in IANQ leads to transition rate of:
\begin{equation}
\xi^{CT, I} = \eta   M^\mathrm{IT}_t  / M^\mathrm{IA-NQ}_t
\end{equation}
to IA-Q.
Fully extended, this becomes:
\begin{equation}
\xi^{CT, I} = \eta \cdot \lambda \rho^S   \frac{\lambda^{Q} (\delta  + (\tau + \tau^{CT}) \,q^{sens}) M^{IA-Q}_{t-2} + \lambda(\delta  + \tau \,q^{sens} ) M^{IA-NQ}_{t-2}) }{M_{t-1}^{total}} \frac{M_{t-1}^{NA-NQ}}{M^{IA-NQ}_t}
\end{equation}
\\
\\
\textbf{Path 1: meeting a newly diagnosed infected person and getting traced but not infected}:

This process moves on similarly to the one above. Instead of infection chance, one only needs to consider the complement, and instead of target compartment IA-NQ, the compartment NA-NQ. This gives:
\begin{equation}
	\xi^{CT, N} = \eta  M^\mathrm{NT}_t / M^\mathrm{NA-NQ}_t
\end{equation}

One way to think about this would be to build pass-through compartments for 'traceable' IA-NQ and NA-NQ that would be kept separate from the 'main' IN-NQ and NA-NQ compartments. The only transitions to quarantine through test and trace would then be from these 'traceable' compartments with rate $\eta$ (rest would go to main compartments because their 'traceability' would be lost). These both structures (with and without extra compartments) should lead to same transition \emph{massess} to quarantine compartments. This might clarify this process but is perhaps not necessary.

\end{document}