\subsection{Model parameters estimation}
\label{S:PARAMESTIMATION}

The default values for the parameters are either defined from heuristic knowledge or from statistics computed on the data itself (see~Table \ref{table:defaultparamreal}).
Each value is only an initial guess which has to be refined using an optimization algorithm (see Section~\ref{SS:THModelParameterEstimation}).
The algorithm will estimate the model parameter values from the data.
% that estimates the model parameters from the data.\\

\begin{table}[h!]
\caption{Default values for model parameters. $\hat{\sigma}_{y_{obs(1:\mathtt{T})}}$ corresponds to the empirical standard deviation calculated using observed data from the first data sample to the last data sample of index $\mathtt{T}$. Note that default value for synthetic data are different, see~Table \ref{table:defaultsynthetic} for details.}
\centering
\begin{tabular}{r|ll}
\toprule
$\bm{\theta}$ name  & $\bm{\theta}^{0}$ & $\bm{\theta}$ bounds \\\cmidrule(lr){1-3}
$\sigma_{w}^{\mathtt{LL}}$  &  $0$ & $[\text{NaN},\text{NaN}]$ \\
$\sigma_{w}^{\mathtt{LT}} $ &  $10^{-7}\times\hat{\sigma}_{y_{obs(1:\mathtt{T})}}$ & $[\text{NaN},\text{NaN}]$ \\
$\sigma_{w}^{\mathtt{LA}} $ &  $10^{-8}\times\hat{\sigma}_{y_{obs(1:\mathtt{T})}}$ & $[\text{NaN},\text{NaN}]$ \\
$\sigma_{w}^{\mathtt{LcT}} $ &  $10^{-7}\times\hat{\sigma}_{y_{obs(1:\mathtt{T})}}$ & $[\text{NaN},\text{NaN}]$ \\
$\sigma_{w}^{\mathtt{LcA}} $ &  $10^{-7}\times\hat{\sigma}_{y_{obs(1:\mathtt{T})}}$ & $[\text{NaN},\text{NaN}]$ \\
$\sigma_{w}^{\mathtt{TcA}} $ &  $10^{-8}\times\hat{\sigma}_{y_{obs(1:\mathtt{T})}}$ & $[\text{NaN},\text{NaN}]$ \\
$\sigma_{w}^{\mathtt{P}} $ &  $0$ & $[\text{NaN},\text{NaN}]$ \\
$\sigma_{w}^{\mathtt{AR}} $ &  $10^{-1}\times\hat{\sigma}_{y_{obs(1:\mathtt{T})}}$ & $[0,\text{Inf}]$ \\
$\sigma_{w,0}^{\mathtt{KR}}  $ &  $10^{-1}\times\hat{\sigma}_{y_{obs(1:\mathtt{T})}}$  & $[0,\text{Inf}]$ \\
$\sigma_{w,1}^{\mathtt{KR}}  $ &  $0$  &  $[\text{NaN},\text{NaN}]$ \\
$\sigma_{v}$ &  $0.05\times\hat{\sigma}_{y_{obs(1:\mathtt{T})}}$  &  $[0,\text{Inf}]$ \\
$\phi^{\mathtt{AR}} $ &  $0.75$ & $[0,1]$ \\
$p^{\mathtt{P}} $ &  $[365.24, 1, 182.62]$\footnote{One different default value for the period is given for each periodic component.} & $[\text{NaN},\text{NaN}]$ \\
$p^{\mathtt{KR}} $ &  $365.24$ & $[\text{NaN},\text{NaN}]$ \\
$\ell^{\mathtt{KR}} $ &  $2/\mathtt{L}^{\mathtt{KR}}$ (see \S\ref{SSS:KR})& $[0,\text{Inf}]$ \\
$\phi^{\cdot|\cdot} $ &  $0.01$  &  $[-\text{Inf}, \text{Inf}]$\\
$\mu_{b}^{\mathtt{LI}}$ &  $0$  &  $[-\text{Inf}, \text{Inf}]$\\
$\sigma_{b}^{\mathtt{LI}}$ &  $\hat{\sigma}_{y_{obs(1:\mathtt{T})}}$  &  $[0, \text{Inf}]$\\\bottomrule

\end{tabular}
\label{table:defaultparamreal}
\end{table}

In order to access the model parameter estimation menu from OpenBDLM, type  \colorbox{light-gray}{\lstinline[basicstyle = \mlttfamily \small, backgroundcolor = \color{light-gray}]!1!} (see Listing~\ref{LST:ModelParametersEstimationMenu}).
The optimization algorithm is either the Newton-Raphson (choice \colorbox{light-gray}{\lstinline[basicstyle = \mlttfamily \small, backgroundcolor = \color{light-gray}]!1!}) or the Stochastic Gradient Descent  (choice \colorbox{light-gray}{\lstinline[basicstyle = \mlttfamily \small, backgroundcolor = \color{light-gray}]!2!}) algorithms (see Section~\ref{SS:THModelParameterEstimation}). \begin{lstlisting}[ frame = single, basicstyle = \mlttfamily \scriptsize, caption = {OpenBDLM model parameter estimation menu}, label = LST:ModelParametersEstimationMenu ,  float =ht, linewidth=\linewidth, captionpos=b]
-------------------------------------------
/ Learn model parameters
-------------------------------------------

     1 ->  Newton-Raphson
     2 ->  Stochastic Gradient Ascent

     Type R to return to the previous menu

     choice >>
\end{lstlisting}
 By default, OpenBDLM uses all the data as the training set (see \lstinline[basicstyle = \mlttfamily \small ]!misc.options.Trainingperiod=[1 Inf]!).
OpenBDLM transform the model parameters to perform the optimization in an unbounded model parameter space (see Section~\ref{SS:THSpaceTransformation}).
Undefined bounds (\lstinline[basicstyle = \mlttfamily \small ]![NaN, NaN]!)  for a parameter means that the parameter is assumed to be known and thus, it will be excluded from the parameter estimation process.
By default, OpenBLDM assumes that the parameters for the period of the periodic component and the process noise standard deviation associated with the baseline and the periodic component have values fixed to 0 and are not optimized.\\

The model parameter can be learned by maximizing either the likelihood (Maximum Likelihood Estimation, MLE), or the posterior function (Maximum A Posteriori estimation, MAP) (see Section~\ref{SS:THModelParameterEstimation}).
Note that MAP requires a valid prior for each unknown model parameters.
In the case of of MAP, OpenBDLM supports gaussian prior only.
The default option is MLE (see \lstinline[basicstyle = \mlttfamily \small ]!misc.options.isMAP=false!).
There is the possibility to use the prediction capacity to drive the optimization process\footnote{The use of the prediction capacity is only available using the Stochastic Gradient Descent optimization.} (see  \lstinline[basicstyle = \mlttfamily \small ]!misc.options.isPredCap! and \lstinline[basicstyle = \mlttfamily \small ]!misc.options.SplitPercent!). The model parameter estimation framework computes point estimate of the model parameters.
However, it is also possible provide confidence intervals around the point estimate using the Laplace approximation\footnote{Laplace approximation is currently only available using the Newton-Raphson optimization.} (see \lstinline[basicstyle = \mlttfamily \small ]!misc.options.isLaplaceApproximation = true!).
Note that computing the Laplace approximation can significantly increase the computation time and it is not recommended when the number of model parameters is large.\\

Note also that Newton-Raphson and Stochastic Gradient Descent techniques are sensitive to the initial model parameters values.
%The algorithm can reach a local maximum, instead of the global maximum.
Therefore, it is advised to run the optimizations several times with different starting model parameters values in order to check if the proposed solution is the best attainable solution.
If \lstinline[basicstyle = \mlttfamily \small ]!misc.options.isMute = false!, outputs on the \MATLAB{} command line window allows to monitor the optimization process (see Listings~\ref{LST:OpenBLDMModelParameterLearning}-\ref{LST:OpenBLDMModelParameterLearningSGD}).
At each iteration, the quantity displayed are: the current value of the target function, the name as well as the current value of the unknown model parameter being optimized, the change in model parameters, the change in the target function, and the convergence status (\lstinline[basicstyle = \mlttfamily \small ]!1! if the model parameter converged according to the convergence criteria, \lstinline[basicstyle = \mlttfamily \small ]!0! otherwise).\\

The optimization stops when all the parameters converged, or if the maximum number of iterations (see \lstinline[basicstyle = \mlttfamily \small ]!misc.options.maxIterations!) (or epochs for Stochastic Gradient Descent algorithm, see \lstinline[basicstyle = \mlttfamily \small ]!misc.options.maxEpochs!) / the maximum time (see \lstinline[basicstyle = \mlttfamily \small ]!misc.options.maxTime!) is reached.
The values of the parameters are saved in the variable \lstinline[basicstyle = \mlttfamily \small ]!model!.

 \begin{lstlisting}[ frame = single, basicstyle = \mlttfamily \scriptsize, caption = {OpenBLDM output example when running Newton-Raphson algorithm.}, label = LST:OpenBLDMModelParameterLearning,  float =h!, linewidth=\linewidth, captionpos=b]
    \Start Newton-Raphson max. algorithm (finite difference method)

      Training period:                             1-Inf [days]
      Maximal number of iteration:                 100
      Total time limit for calibration :           60 [min]
      Convergence criterion:                       1e-07*LL
      Nb. of search levels for \lambda:            4*2

           Initial LL: 36626.8381
                       AR|M1|1         AR|M1|1         |M1|1
      parameter names: \phi            \sigma_w        \sigma_v
       initial values: +7.50e-01       +1.74e-02       +8.70e-03
--------------------------
    Loop #1 : |M1|1 | \sigma_v
       delta_param: -0.0040755
    log-likelihood : 36994.6374
    param change   : 0.0087002 -> 0.0046247

                    AR|M1|1         AR|M1|1         |M1|1
   parameter names: \phi            \sigma_w        \sigma_v
    current values: +7.50e-01       +1.74e-02       +4.62e-03
  current f.o. std: +0.00e+00       +0.00e+00       +1.93e-04
      previous dLL: +1.00e+06       +1.00e+06       +3.68e+02
         converged:         0               0               0
--------------------------
    Loop #2 : AR|M1|1 | \sigma_w
       delta_param: 0.0046034
    log-likelihood : 40998.3934
    param change   : 0.0174 -> 0.022003

                    AR|M1|1         AR|M1|1         |M1|1
   parameter names: \phi            \sigma_w        \sigma_v
    current values: +7.50e-01       +2.20e-02       +4.62e-03
  current f.o. std: +0.00e+00       +5.26e-05       +1.93e-04
      previous dLL: +1.00e+06       +4.00e+03       +3.68e+02
         converged:         0               0               0
--------------------------
\end{lstlisting}

 \begin{lstlisting}[ frame = single, basicstyle = \mlttfamily \scriptsize, caption = {OpenBLDM output example when running Stochastic Gradient Ascent algorithm.}, label = LST:OpenBLDMModelParameterLearningSGD,  float =h!, linewidth=\linewidth, captionpos=b]
    \Start SGD algorithm (finite difference method)

      Optimization mode                           MLE
      Optimizer                                   MMT
      Metric                                      logpdf
      Learning Rate mode                          hessian
      Training period:                            1 - Inf [days]
      Validation set portion:                     30 [%]
      Training set:                               13556 [data points]
      Validation set:                             5810 [data points]
      Mini batch:                                 3873 [data points]
      Number of max epoch:                        30+1 [epochs]
      Total time limit for calibration:           60 [min]

    Epoch #1
             Metric: 25972.5904
                    AR|M1|1         AR|M1|1         |M1|1
   parameter names: \phi            \sigma_w        \sigma_v
    initial values: +7.50e-01       +1.74e-02       +8.70e-03

--------------------------
    Epoch #2
            Metric: 33933.8856
   parameter names: AR|M1|1         AR|M1|1         |M1|1
    current values: +9.61e-01       +2.00e-02       +5.66e-03
      param change: +2.11e-01       +2.61e-03       -3.04e-03
  initialize param:         0               0               0

--------------------------
    Epoch #3
            Metric: 33933.8856
   parameter names: AR|M1|1         AR|M1|1         |M1|1
    current values: +9.61e-01       +2.00e-02       +5.66e-03
      param change: +0.00e+00       +0.00e+00       +0.00e+00
  initialize param:         1               0               0
\end{lstlisting}

\subsubsection{Model parameter estimation functions}

\begin{description}[style=unboxed]\setlength\itemsep{0em}
\item[Pilot function for optimization] \leavevmode
  \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[data,model,estimation,misc]=piloteOptimization(data,model,estimation,misc)
  \end{lstlisting}

\item[Estimates model parameter using Newton-Raphson algorithm] \leavevmode
  \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[optim,model]=NewtonRaphson(data,model,misc)
  \end{lstlisting}

 \item[Estimates model parameter using Stochastic Gradient Descent algorithm] \leavevmode
  \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[optim,model] = SGD(data,model,misc,varargin)
  \end{lstlisting}

 \item[Reads model parameter properties ] \leavevmode
 \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[arrayOut]=readParameterProperties(cellIn,Position)
  \end{lstlisting}

 \item[Writes model parameter properties ] \leavevmode
 \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[cellOut]=writeParameterProperties(cellIn,arrayIn,Position)
  \end{lstlisting}

 \item[Approximates the target function, as well as the first and second derivative of the logarithm of the target function with respect to parameter values ] \leavevmode
  \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[logpdf,Glogpdf,Hlogpdf, delta_grad] = logPosteriorPE(data,model,misc,varargin)
  \end{lstlisting}

 \item[Computes the gradient and hessian of the gaussian prior distribution of each model parameter ] \leavevmode
  \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[logprior,Glogprior,Hlogprior]= logPriorDistr(P,Mu,Sigma,varargin)
  \end{lstlisting}

 \item[Computes numerical hessian H of a function ] \leavevmode
  \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
H=numerical_hessian(x,fX,varargin)
  \end{lstlisting}

 \item[Defines transformation functions and their derivatives according to provided bounds for the model parameters ] \leavevmode
  \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[fct_TR,fct_inv_TR,grad_TR2OR,hessian_TR2OR]=parameter_transformation_fct(model,param_idx_loop)
  \end{lstlisting}

 \item[Performs Switching Kalman filter on time series ] \leavevmode
  \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[x,V,VV,S,loglik,U,D]=SwitchingKalmanFilter(data,model,misc)
  \end{lstlisting}

 \item[Computes the first and second derivative of the logarithm of the likelihood function with respect to parameter values] \leavevmode
 \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[grad,hessian,fail_gradHess,delta_grad] = gradHess(data, model,misc,pTR,pOR,log_lik_0,grad_TR2OR,delta_grad,param_idx_loop)
 \end{lstlisting}

 \item[Computes the optimal parameter step size for the approximation of the numerical derivative of the likelihood and compute the terms required to approximate the numerical derivative using finite-difference method] \leavevmode
 \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[delta_grad,fail_delta_grad,log_lik_1,log_lik_2] = StepSizeOptimization(model,data,misc,pOR,log_lik_0,delta_grad,param_idx_loop)
 \end{lstlisting}

 \item[Splits the full dataset into train and test dataset (for Stochastic Gradient Descent only) ] \leavevmode
  \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[data_train,data_valid] = dataSplit(data,idxTrain,alpha_split,varargin)
  \end{lstlisting}

 \item[Computes the metric function (for Stochastic Gradient Descent only) ] \leavevmode
  \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[metricVL,idxMaxM,logpdf_test,logpdf_train] = metricFct(data_train,data_test,model,misc,parameterSearch,parameterSearchTR)
  \end{lstlisting}

 \item[paramGrid (for Stochastic Gradient Descent only) ] \leavevmode
  \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[xM,momentumM,RMSpropM,gradM,learningRateM, mmtHessM, hessM]= paramGrid(x,momentum,RMSprop,grad,learningRate,mmtHess,hess)
  \end{lstlisting}

  \item[ADAM optimizer (for Stochastic Gradient Descent only)] \leavevmode
  \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[xsearch,xsearchTR,momentumTR,RMSpropTR] = ADAM(xsearchTRprev,momentumTRprev,RMSpropTRprev,grad,step,beta_1,beta_2,epsilon,Niter,fctInvTR)
  \end{lstlisting}

\item[MMT optimizer (for Stochastic Gradient Descent only)] \leavevmode
  \begin{lstlisting}[ basicstyle = \mlttfamily \small, breaklines=true]
[xsearch,xsearchTR,momentumTR] = MMT(xsearchTRprev,momentumTRprev,grad,step,beta,fctInvTR)
\end{lstlisting}

\end{description}

\begin{figure}[!h]
  \centering
  \captionsetup{justification=centering}
\scalebox{0.7}{
\begin{tikzpicture}

\node[parababyblueeyes](inputOptimization){ \begin{tabular}{c} \lstinline[ basicstyle = \mlttfamily \small]!data! \\ \phantom{}  \lstinline[ basicstyle = \mlttfamily \small]!model.param_properties! \phantom{} \end{tabular}};
\node[esbabyblueeyes](piloteOptimization)[below of = inputOptimization, yshift = -1cm]{\phantom{} piloteOptimization.m \phantom{}};
\node[testbabyblueeyes](testNRSGD)[below of = piloteOptimization, yshift = -1.5cm]{\begin{tabular}{c} NR or  \\ SGD ?  \end{tabular}};
\node[esbabyblueeyes](SeeNewtonRaphson)[below of = testNRSGD, yshift = -1cm, xshift = -3.5cm]{\phantom{} see Figure~\ref{FIG:NewtonRaphsonWorkflow} \phantom{}};
\node[esbabyblueeyes](SeeSGD)[below of = testNRSGD, yshift = -1cm, xshift = 3.5cm]{\phantom{} see Figure~\ref{FIG:SGDWorkflow} \phantom{}};
\node[parababyblueeyes](outputOptimization)[below of = inputOptimization, yshift = -8cm]{\phantom{}  \lstinline[ basicstyle = \mlttfamily \small]!model.param_properties! \phantom{}};

\path[->, draw, thick] (inputOptimization)edge(piloteOptimization);
\path[->, draw, thick] (piloteOptimization)edge(testNRSGD);
\path[->, draw, thick] (testNRSGD.east) -| (1.5cm,-4.5cm) -| node[pos=0.25, above]{ SGD} (SeeSGD.north);
\path[->, draw, thick] (testNRSGD.west) -| (-1.5cm,-4.5cm) -| node[pos=0.25, above]{NR} (SeeNewtonRaphson.north);
\path[->, draw, thick] (SeeNewtonRaphson.south) |- (0cm,-8cm) -|  (outputOptimization.north);
\path[->, draw, thick] (SeeSGD.south) |- (0cm,-8cm) -|  (outputOptimization.north);
\end{tikzpicture} }
\caption{Model parameter estimation workflow} \label{FIG:ModelParameterEstimationWorkflow}
\end{figure}

\begin{figure}[!h]
  \centering
  \captionsetup{justification=centering}
\scalebox{0.7}{
\begin{tikzpicture}

\node[esbabyblueeyes](NewtonRaphson){\phantom{} NewtonRapshon.m \phantom{}};
\node[esbabyblueeyes](readParameterEstimation)[below of = NewtonRaphson, yshift = -1cm]{\phantom{} readParameterEstimation.m \phantom{}};
\node[esbabyblueeyes](paramtransform)[below of = readParameterEstimation , yshift = -1cm]{\phantom{}  parameter\_transformation\_fct.m \phantom{}};
\node[esbabyblueeyes](logposterior)[below of = paramtransform  , yshift = -1cm]{\phantom{}  logPosteriorPE.m \phantom{}};
\node[esbabyblueeyes](logPrior)[right of = logposterior , xshift=4cm, yshift = 0.75cm]{\phantom{}  logPriorDistr.m \phantom{}};
\node[esbabyblueeyes](gradHess)[right of = logposterior , xshift=3.75cm, yshift = -0.75cm]{\phantom{}  gradHess.m \phantom{}};
\node[esbabyblueeyes](SSO)[right of = gradHess , xshift=3cm, yshift = 0cm]{\phantom{}  stepSizeOptimization.m \phantom{}};
\node[esbabyblueeyes](SKF)[below of = SSO , xshift=0cm, yshift = -1cm]{\phantom{}  SwitchingKalmanFilter.m \phantom{}};
\node[esbabyblueeyes](writeParameterEstimation)[below of = logposterior, yshift = -1.5cm]{\phantom{} writeParameterEstimation.m \phantom{}};

\path[->, draw, thick] (NewtonRaphson)edge(readParameterEstimation);
\path[->, draw, thick] (readParameterEstimation)edge(paramtransform);
\path[->, draw, thick] (paramtransform)edge(logposterior);
\path[->, draw, thick] (logposterior.east) -| (3cm,-5.25cm) --  (logPrior.west);
\path[->, draw, thick] (logposterior.east) -| (3cm,-6.75cm) --  (gradHess.west);
\path[->, draw, thick] (gradHess)edge(SSO);
\path[->, draw, thick] (SSO)edge(SKF);
\path[->, draw, thick] (logposterior)edge(writeParameterEstimation);

\end{tikzpicture} }
\caption{Model parameter estimation Newton-Raphson workflow} \label{FIG:NewtonRaphsonWorkflow}
\end{figure}

\begin{figure}[h]
  \centering
  \captionsetup{justification=centering}
\scalebox{0.7}{
\begin{tikzpicture}

\node[esbabyblueeyes](SGD){\phantom{} SGD.m \phantom{}};
\node[esbabyblueeyes](readParameterEstimation)[below of = SGD, yshift = -1cm]{\phantom{} readParamaterEstimation.m \phantom{}};
\node[esbabyblueeyes](paramtransform)[below of = readParameterEstimation , yshift = -1cm]{\phantom{}  parameter\_transformation\_fct.m \phantom{}};
\node[esbabyblueeyes](metric1)[below of = paramtransform , yshift = -1cm]{\phantom{}  metricFct.m \phantom{}};
\node[esbabyblueeyes](split)[below of =metric1 , yshift = -1cm]{\phantom{}  dataSplit.m \phantom{}};
\node[esbabyblueeyes](logposterior)[below of = split  , yshift = -1cm]{\phantom{}  logPosteriorPE.m \phantom{}};
\node[esbabyblueeyes](logPrior)[right of = logposterior , xshift=4cm, yshift = 0.75cm]{\phantom{}  logPriorDistr.m \phantom{}};
\node[esbabyblueeyes](gradHess)[right of = logposterior , xshift=3.75cm, yshift = -0.75cm]{\phantom{}  gradHess.m \phantom{}};
\node[esbabyblueeyes](SSO)[right of = gradHess , xshift=3cm, yshift = 0cm]{\phantom{}  stepSizeOptimization.m \phantom{}};
\node[esbabyblueeyes](SKF)[below of = SSO , xshift=0cm, yshift = -1cm]{\phantom{}  SwitchingKalmanFilter.m \phantom{}};
\node[esbabyblueeyes](grid)[below of = logposterior, yshift = -1cm]{\phantom{} paramGrid.m \phantom{}};
\node[esbabyblueeyes](adam)[below of = grid, yshift = -1cm]{\phantom{} \begin{tabular}{c} ADAM.m  \\ MMT.m  \end{tabular}};
\node[esbabyblueeyes](metric2)[below of = adam , yshift = -1cm]{\phantom{}  metricFct.m \phantom{}};
\node[esbabyblueeyes](writeParameterEstimation)[below of = metric2, yshift = -1cm]{\phantom{} writeParameterEstimation.m \phantom{}};

\path[->, draw, thick] (NewtonRaphson)edge(readParameterEstimation);
\path[->, draw, thick] (readParameterEstimation)edge(paramtransform);
\path[->, draw, thick] (paramtransform)edge(metric1);
\path[->, draw, thick] (metric1)edge(split);
\path[->, draw, thick] (split)edge(logposterior);
\path[->, draw, thick] (logposterior.east) -| (3cm,-9.25cm) --  (logPrior.west);
\path[->, draw, thick] (logposterior.east) -| (3cm,-10.75cm) --  (gradHess.west);
\path[->, draw, thick] (gradHess)edge(SSO);
\path[->, draw, thick] (SSO)edge(SKF);
\path[->, draw, thick] (logposterior)edge(grid);
\path[->, draw, thick] (grid)edge(adam);
\path[->, draw, thick] (adam)edge(metric2);
\path[->, draw, thick] (metric2)edge(writeParameterEstimation);

\end{tikzpicture} }
\caption{Model parameter estimation Stochastic gradient descent workflow} \label{FIG:SGDWorkflow}
\end{figure}


\documentclass[a4paper,11pt]{article}

\usepackage{physics}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{amsthm, mathtools}
%\usepackage{hyperref}
\usepackage{color}
\usepackage{jheppub}
\usepackage[T1]{fontenc} % if needed

% My Documents
\newcommand{\be}{\begin{equation}}
\newcommand{\ee}{\end{equation}}
\newcommand{\bes}{\begin{equation*}}
\newcommand{\ees}{\end{equation*}}
\newcommand{\bea}{\begin{flalign*}}
\newcommand{\eea}{\end{flalign*}}

\def\a{\alpha}
\def\b{\beta}
\def\g{\gamma}
\def\s{\sigma}
%\linespread{1.0}
%\setlength{\parindent}{0em}
%\setlength{\parskip}{0.8em}

\title{\textbf{Correlated Quantum Many-Body Systems}}
\author{Aditya Vijaykumar}
\affiliation{International Centre for Theoretical Sciences, Bengaluru, India.}
\emailAdd{aditya.vijaykumar@icts.res.in}

\begin{document}
\maketitle

\section{What is the problem?}
Want to understand and describe the propertiees of a quantum many body system with many moving parts.

Interested in describing the properties of a system: Mostly inclined towards {\bf linear response properties}. For example, the response of current through a resistor ($V = I R$). These can give us unperrturbed properties of a system.

\section{What are the moving parts: The constitutents of the system}
These can be ions, electons, stars, fluids, etc. We describe them using some effective theory, obviously not at the molecular level. This again, depends on the experiment we do, i.e, at what {\bf energy we probe the system}. These degrees of freedom are effective degrees of freedom. Once we have described the constituents, we need to understand the physics of them, i.e, the {\bf effective ``microscropic" description}.
\subsection{Physics of few constituents}
One question that one can ask: is classical mechanics enough to describe the system or does one need to use quantum mechanics. If for example is talking about sand particles one can do with CM. But for an elecron in a wire, one needs to essentially solve some QM.
\begin{itemize}
	\item {\bf CM}: Works by stating the position and momentum at any time and solve Newton's laws. For multiple particles, one can solve multiple equations. However in principle one can choose to follow one of the particle and understand it alone.

	\item {\bf QM}: In QM superposition and {\bf entanglement} doesn't allow us to understandpars of the system individually. This is a major departure from CM. Entangement doesn't have any classical analogue.\\
	{\bf Example:} Consider a state,
	\be
	\ket{\psi} = \frac{\ket{\uparrow \downarrow} + \ket{ \downarrow \uparrow}}{\sqrt{2}}
	\ee
	We define the density matrix as $\rho = \ket{\psi} \bra{\psi}$. The reduced density matrix is obtained by integrating out one spin,
	\be
	\rho_{red}^{(1)} = \braket{\uparrow|\rho|\uparrow_2} + \braket{\downarrow|\rho|\downarrow_2}
	\ee
	We define something called entanglement entropy $S_{ent}=-\mbox{Tr}[\rho_{red}^{(1)} \log\rho_{red}^{(1)}  ]$. If EE is zero then it's not an entangled state. Here it is zero because $\psi$ is  a tensor product state $\psi_1 \otimes \psi_2$.

	To understand the dynamics of a system we study the following related to that. Some of them include

	\subsubsection{Symmetries:} Tells aout the ``isotropies" in a system. For example,
	\be
	T_{\vec a}\ket{\vec x} = \ket{\vec x + \vec a}
	\ee
	This operator preserves the probabilities, and thus is unitary. It's the translation operator. Say a hamiltonian has a translational symmetry, then one can't really distinguish between $ \ket{x} $ and $ \ket{x + a} $. Thus,
	\begin{equation*}
	[H, T_{\vec a}] = 0 \implies T_a^{\dagger} H T_a = H
	\end{equation*}
	This brings a restriction n the dynamics of a system. In this particular case, it just means that the hamiltonian remains invariant under translations.

\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% 			Ising Model
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Ising Model}
The constituents in this system are spin $1/2$ particles. We have $N$ such spin half particles here. The $ i^{th} $ spin half is descrbed by two states $\ket{\sigma_i} \equiv \ket{1}, \ket{0} \equiv \ket{\uparrow}, \ket{ \downarrow } $. These are often called {\bf qbits}. The basis stats are spanned by objects like,
\begin{equation*}
\ket{\sigma_1 \sigma_2 \cdots \sigma_n}
\end{equation*}
Given this one can construct all operators in the system
\begin{equation*}
A_i = \sum_{\alpha \beta} A^{\alpha \beta}_i \ket{\alpha_i} \bra{\beta_i}
\end{equation*}
The operators above are local (depends on a particular site), and the claim is that these operators can be generated using the Pauli matrices $1, \sigma_x, \sigma_y, \sigma_z$, which satisfy,
\be
[\sigma_i^{\alpha}, \sigma_j^{\beta}] = 2 i \delta_{ij} \epsilon^{\alpha \beta \gamma}\sigma_i^{\gamma}
\ee
We now describe the symmetries of the system, (no matter what we choose we have to satisfy $S^{\dagger} H S = H$)
\begin{itemize}
	\item One dimensional Translation: We want translation symmetry in units of $n \in$ Integer. 	These can be realized in a lattice with lattice constant $a$.

	\item Given by the operator $X = \prod_i \s_i^X$. What does this do? It's essentially a rotation in spin space about the $X$ axis by $\frac{\pi}{2}$. We have the Pauli matrices as,
	\begin{equation*}
	\s_x = \left( \begin{array}{cc}
	1 & 0 \\
	0 & 1
	\end{array}
	 \right) \implies \s^x \ket{\uparrow} = \ket{\downarrow}
	\end{equation*}
	This also gives,
	\begin{equation*}
	\s^x \s^z \s^x = - \s^z
	\end{equation*}
	We can also compute for spins at different sites,
	\begin{equation*}
	X^{\dagger} \s_i^z \s_j^z X =
		X^{\dagger} \s_i^z X X^{\dagger} \s_j^z X  = \s_i^z \s_j^z
	\end{equation*}
	Thus to make it commute with the Hamiltonian we either choose an even number of $\s_z$ or work with $\s_x$.
\end{itemize}
One Hamiltonian that respects all the symmetries of the system,
\begin{equation*}
H = - J \sum_i \s_i^z \s_{i + 1}^z - \Gamma \sum_i \s_i^x
\end{equation*}
We are interested in determining the ground state and low energy properties of the system. The numbers $J$ and $ \Gamma $ are called coupling constants. For our case we take them to be positive. We vary the constant $\Gamma/J$  and ask how the ground state energy changes.\\ \\
 In the limit where $J \to 0$, we have a non-interacting Hamiltonian,
 \be
 H = \sum_i \s_i^x
 \ee
The two states having energy $+$ and $-$  in the $Z$ basis are repectively,
\begin{equation*}
\ket{\leftarrow}_x \equiv \frac{\ket{\uparrow} - \ket{\downarrow}}{\sqrt{2}}, \quad
\ket{\rightarrow}_x \equiv\frac{\ket{\uparrow} + \ket{\downarrow}}{\sqrt{2}}
\end{equation*}
The ground state is,
\be
\ket{GS} = \ket{\rightarrow \rightarrow \cdots \rightarrow}
\ee
The energy of the ground state is,
\be
E_{GS} = - \Gamma N
\ee
N being the number of spins. The first excited state is,
\begin{equation*}
\ket{1}_{ex} = \ket{\rightarrow \rightarrow \cdots \underbrace{\leftarrow}_{i} \cdots \rightarrow}
\end{equation*}
All the first excited states have the energy,
\be
E_1 = E_{GS} + 2 \Gamma
\ee

We now ask the action of $\s^z$ on $\ket{\leftarrow}$ and $\ket{\rightarrow}$,
\begin{equation*}
\s^z \ket{\rightarrow} = \ket{\leftarrow}, \quad
\s^z \ket{\leftarrow} = \ket{\rightarrow}
\end{equation*}

Now we consider the action of the operator,
\begin{equation*}
\s_i^z \s_{i+1}^z + \s_{i-1}^z \s_i^z
\end{equation*}
This takes the first excited state in the equation above ($\ket{1_{ex}} \equiv \ket{i}$) to $\ket{i-1}$ or $\ket{i+1}$ (keeps the energy invariant). Thus the hamiltonian can be written as,
\begin{equation*}
H = \sum_i \ket{i-1} \bra{i} + \ket{i+1}\bra{i}
\end{equation*}
In fourier space this becomes,
\begin{equation*}
\ket{k} = \frac{1}{\sqrt N} \sum_i e^{i k r_i} \ket{i}
\end{equation*}
These are the eigenstates of this Hamiltonian, an dit an be shown that,
\begin{equation*}
H \ket{k} = E_k \ket{k} \implies E_k = E_1 - J \cos ka
\end{equation*}
For soft $k$,
\begin{equation*}
E_k = (E_1 - J) + \frac{J a k^2}{2}
\end{equation*}
The first term is some shift, but the second one looks like an inverse of mass. Now using this we wish to compute the EE for a state broken into two halves. Thus for small $J$ it's almost a classical state with nearly zero EE since it's a state that can be broken into product state ($\ket{\rightarrow \cdots \rightarrow}$). This is the reason why one is able to construct a free particle state.

The question now is: The properties of ground state with lot of entanglement, and what kind of excited states they give rise to?

\end{document}


\subsection{Transfer}
\label{sec:721Transfer}

\sectlof
We continue with the notation and indices from the `\hyperref[sec:Mint]{Mint}' section.\\
\\
Suppose Alice wishes to transfer ownership of the ERC-721 token with tokenId `$\alpha$' to Bob, but under zero-knowledge.\\
\\
In the `Mint' section, we saw how Alice can create an `ERC-721 commitment' $Z_\alpha$ within the Shield contract which:
\begin{itemize}
  \item hides an underlying ERC-721 token with tokenId `$\alpha$'; and
  \item hides and binds Alice as the owner of $Z_\alpha$ (and hence of $\alpha$) through an ownership keypair $(sk_A^Z, pk_A^Z)$.
\end{itemize}
\ \\
Recall our privacy intentions:\\
Alice wishes to be able to transfer ownership of an ERC-721 token under zero-knowledge, so that the following become private:
	\begin{center}
		\begin{framed}
      \begin{enumerate}
        \item All details of the ERC-721 token (the `asset').
        \item The identity of the sender of the token (`Alice').
        \item The identity of the receiver of the token.
      \end{enumerate}
    \end{framed}
  \end{center}
Recall that minting a token commitment does not yet afford Alice any privacy (see the warning in Figure~\ref{fig:nfMintWarning}). Only with subsequent transfers will the whereabouts of $\alpha$ and the owner of $\alpha$ be hidden.

For Alice to transfer ownership of $\alpha$ within the Shield contract, under zero knowledge, she follows the steps in Figure~\ref{fig:nfTransferAlgorithm}.

\begin{figure}[htbp]
  \ContinuedFloat*
	\begin{center}
		\begin{framed}
      \begin{tabular}{p{16cm}}
       \textbf{ Non-fungible transfer algorithm} \\
        \\
        \midrule
        \textbf{Bob's steps:}\\
        \begin{enumerate}
          \item Before Alice can send him anything, Bob must register his public keys $pk_B^Z$ and $pk_B^W$ against both his public Ethereum address $pk_B^E$ and his unique name `Bob' within the \hyperref[sec:pkd]{PKD}.
          \setcounter{ongoingEnumCounter}{\value{enumi}}
        \end{enumerate}
        \ \\
        \midrule
        \textbf{Alice's steps:}\\
        \begin{enumerate}
          \setcounter{enumi}{\value{ongoingEnumCounter}}
          \item Generate a random salt $\sigma_{\vec{AB}}$.
          \item Lookup Bob's `zkp' public key $pk_B^Z$ from the PKD.
          \item Compute $Z_B := h(\;\alpha\;|\;pk^Z_B\;|\;\sigma_{\vec{AB}}\;)$, a token commitment which represents $\alpha$.
          \item Compute $N_A := h(\;\sigma_{A}\;|\;sk^Z_A\;)$, the nullifier of Alice's commitment $Z_A$.
          \item Get $\psi_{Z_A}$ -- the sister-path of $Z_A$ -- from the Shield contract (see Details below).
          \item Get the latest Merkle root from the Shield contract: $\roott_{n+m-1}$ (see Details below).
          \item Set public inputs $x = (\;N_A,\;\roott_{n+m-1},\;Z_B)$
          \item Set private inputs $\omega = (\alpha,\;\psi_{Z_A},\;sk_A,\;\sigma_{A},\;pk_B,\;\sigma_{\vec{AB}})$
          \item Select $C_{nft-transfer}(\;\omega,\;x\;)$ -- the set of constraints which are satisfied if and only if:
          \begin{enumerate}
            \item $pk_A$ equals $h(\;sk_A\;)$; (Proof of knowledge of the secret key to $pk_A$) (see Details for why $pk_A$ isn't an input to $C$)
            \item $Z_A$ equals $h(\;\alpha\;|\;pk_A\;|\;\sigma_A\;)$ (Proof of the constituent values of $Z_A$) (see Details for why $Z_A$ isn't an input to $C$)
            \item $\roott_{n+m-1}$ equals $h\br*{\psi_{1}\;|...|\;h\br*{\psi_{d-2}\;|\;h\br*{\psi_{d-1}\;|\;Z_A}\;}...}$ (Proof that $Z_A$ belongs to the on-chain Merkle Tree)
            \item $N_A$ equals $h(\;\sigma_{A}\;|\;sk^Z_A\;)$ (Proof $N_A$ is indeed the nullifier of $Z_A$)
            \item $Z_B$ equals $h(\;\alpha\;|\;pk^Z_B\;|\;\sigma_{\vec{AB}}\;)$ (Proof that $Z_B$ contains the same asset as $Z_A$)
          \end{enumerate}
          \item Generate $\pi := P(\;p_C\;,\;x,\;\omega\;)$; a proof of knowledge of satisfying arguments $(\omega, x)\;s.t.\;C(\omega, x) = 1$. Recall: $p_C$ -- the proving key for $C$ -- will be stored on Alice's computer.

          The pair $(\pi, x)$ is the zk-SNARK which attests to knowledge of private inputs $\omega$ without revealing them.
          \item Send $(\pi, x)$ to the Shield contract for verification.

          Using web3: \texttt{nfTokenShield.transfer(proof, inputs, vkId)}
          %remember where the count (enumi) is up to and store it in ongoingEnumCounter:
          \setcounter{ongoingEnumCounter}{\value{enumi}}
        \end{enumerate}
        \ \\
        \midrule
        \textbf{Shield contract's steps:}\\
        \begin{enumerate}
          %resume counter
          \setcounter{enumi}{\value{ongoingEnumCounter}}
          \item Verify the proof as correct: call a Verifier contract to verify the \texttt{(proof, inputs)} pair against the verification key represented by \texttt{vkId}.
          \setcounter{ongoingEnumCounter}{\value{enumi}}
        \end{enumerate}
        \ \\
        \hline
        ...
			\end{tabular}
		\end{framed}
	\end{center}
\caption{Non-Fungible Transfer Algorithm}
\label{fig:nfTransferAlgorithm}
\end{figure}

%continue on next page
\begin{figure}[htbp]
  \ContinuedFloat %to continue
	\begin{center}
		\begin{framed}
      \begin{tabular}{p{16cm}}
       \textbf{ Verifier contract's steps:}\\
        \begin{enumerate}
          \setcounter{enumi}{\value{ongoingEnumCounter}}
          \item Compute \texttt{result = verify(proof, inputs, vkId)}.

          I.e. Verify the \texttt{(proof, inputs)} pair against the verification key.
          \item Return \texttt{result}$\in$\texttt{\{false, true\}} to the Shield contract.
          \setcounter{ongoingEnumCounter}{\value{enumi}}
        \end{enumerate}
        \ \\
        \midrule
        \textbf{Shield contract's steps:}\\
        \begin{enumerate}
          \setcounter{enumi}{\value{ongoingEnumCounter}}
          \item If \texttt{result = false}, revert.
          \item Else:
          \begin{enumerate}
            \item Check $\roott_{n+m-1}$ is in $\rootsList$. (Revert if not).
            \item Check $N_A$ is not already in its list of `spent' nullifiers. (Revert if not).
            \item Add $Z_B$ to the next empty leaf of the Merkle Tree.
            \item Recalculate the path to the root of the Merkle Tree from $Z_B$ for future users.
            \item Append the newly calculated root $\roott_{n+m}$ to the ever-increasing array $\rootsList$
            \item Similarly append the nullifier $N_{A}$ to the ever-increasing array $\bm N$.
          \end{enumerate}
          \setcounter{ongoingEnumCounter}{\value{enumi}}
        \end{enumerate}
        \ \\
        \midrule
        \textbf{Alice's steps:}\\
        \begin{enumerate}
          \setcounter{enumi}{\value{ongoingEnumCounter}}
          \item Store relevant data in her local database, including the leafindex of $Z_B$.
          \item Send Bob important data privately via Whisper (using his public key $pk^W_B$):
          \begin{enumerate}
            \item The salt $\sigma_{\vec{AB}}$ of $Z_B$.
            \item The public key of Bob, $pk^Z_B$, used by Alice in the preimage of $Z_B$ (for completeness, so Bob can check the correctness of $Z_B$ himself).
            \item The tokenId $\alpha$.
            \item $Z_B$.
            \item The leafIndex of $Z_B$ within the on-chain Merkle Tree $M$ (so Bob can locate it).
          \end{enumerate}
          \setcounter{ongoingEnumCounter}{\value{enumi}}
        \end{enumerate}
        \ \\
        \midrule
        \textbf{Bob's steps:}\\
        \begin{enumerate}
          \setcounter{enumi}{\value{ongoingEnumCounter}}
          \item Check the correctness of the information provided by Alice:
          \begin{enumerate}
            \item Check $Z_B$ equals $h(\;\alpha\;|\;pk^Z_B\;|\;\sigma_{\vec{AB}}\;)$
            \item Check that $Z_B$ is stored at the leafIndex of $M$ which Alice claimed.
          \end{enumerate}
          \item Store relevant data in his local database, including whether or not his `correctness checks' passed.
          \setcounter{ongoingEnumCounter}{0} %reset for next figure
        \end{enumerate}
			\end{tabular}
		\end{framed}
	\end{center}
\caption{Non-Fungible Transfer Algorithm} %same caption as the first part of this figure
%\label{fig:nfTransferAlgorithm} - no label in this second part of the figure
\end{figure}

\newpage
\subsubsection{Details}
\label{sec:721TransferDetails}

We refer to the numbered steps of Figure~\ref{fig:nfTransferAlgorithm}.

\textbf{Step $1$}
\ \\
This is handled at the time Bob creates an account through the \hyperref[sec:ui]{UI}.\\
\\
\textbf{Step $2$}
\ \\
This is handled within the \hyperref[sec:ui]{UI} microservice (or within the api-gateway).\\
\\
\textbf{Step $3$}
\ \\
This is handled within the api-gateway when a call is made by Alice to transfer to Bob.\\
\\
\textbf{Steps $4 - 5$}
\ \\
These steps are handled within \hyperref[sec:nf-token-controller]{\texttt{nf-token-controller.js}}.\\
\\
\textbf{Steps $6 - 7$}
\ \\
These calls to the Shield contract are handled within \hyperref[sec:nf-token-zkp]{\texttt{nf-token-zkp.js}}.\\
\\
It is important at this stage to note that there are an unknown number of other parties utilising the Shield contract.
Hence, the dynamic array of tokens $\bm{Z}$ might have grown since Alice appended her $Z_A$ as the $n^{th}$ leaf of $M$ (during the \hyperref[sec:721Mint]{Mint} explanation).\\
Suppose there have been $m-1$ additional tokens added to $M$ since Alice added $Z_A$.
That is,\\
\begin{align*}
  \bm{Z}_{n+m-1} = (Z_0, Z_1,...,Z_{n-1}, Z_A, Z_{n+1},..., Z_{n+m-1})
\end{align*}
We denote the corresponding Merkle Tree which holds tokens $\bm{Z}_{n+m-1}$ by $M_{n+m-1}$.
We denote its root by $\roott_{n+m-1}$; an element of $\rootsList = (\roott_0, \roott_1,...,\roott_{n+m-1})$.\\
\\
\begin{align*}
  \begin{forest}
    [{$\roott_{n+m-1}:= h\br*{
                        h\br*{
                          h\br*{
                            h\br*{
                              Z_0,Z_1
                            },
                            ...
                          },
                          h\br*{
                            h\br*{
                              Z_{n-1},Z_A
                            },
                            h\br*{
                              Z_{n+1},...
                            }
                          }
                        },
                        h\br*{
                          h\br*{
                            h\br*{
                              Z_{n+m-1},0
                            },
                            0
                          },
                          0
                        }
                      }
                    $}
      [{$ h\br*{
            h\br*{
              h\br*{
                Z_0,Z_1
              },
              ...
            },
            h\br*{
              h\br*{
                Z_{n-1},Z_A
              },
              h\br*{
                Z_{n+1},...
              }
            }
          }
        $}
        [{$ h\br*{
              h\br*{
                Z_0,Z_1
              },
              ...
            }
          $}
          [{$ h\br*{
                Z_0,Z_1
              }
            $}
            [{$Z_0$}][{$Z_1$}]
          ]
          [...
            [...][...]
          ]
        ]
        [{$ h\br*{
              h\br*{
                Z_{n-1},Z_A
              },
              h\br*{
                Z_{n+1},...
              }
            }
          $}
          [{$ h\br*{
                Z_{n-1},Z_A
              }
            $}
            [{$Z_{n-1}$}][{$Z_A$}]
          ]
          [{$ h\br*{
                Z_{n+1},...
              }
            $}
            [$Z_{n+1}$][...]
          ]
        ]
      ]
      [{$ h\br*{
            h\br*{
              h\br*{
                Z_{n+m-1},0
              },
              0
            },
            0
          }
        $}
        [{$ h\br*{
              h\br*{
                Z_{n+m-1},0
              },
              0
            }
          $}
          [{$ h\br*{
                Z_{n+m-1},
                0
              }
            $}
            [{$Z_{n+m-1}$}][0]
          ]
          [0
            [0][0]
          ]
        ]
        [0
          [0
            [0][0]
          ]
          [0
            [0][0]
          ]
        ]
      ]
    ]
  \end{forest}
\end{align*}

\noindent
Alice retrieves the value of the current Merkle root, $\roott_{n+m-1}$, from the Shield contract.\\
\\
Since Alice knows that $Z_A$ is at leaf-index $n$ of $M_{n+m-1}$, Alice can also retrieve the path from the leaf $Z_{n}=Z_A$ to the root $\roott_{n+m-1}$. Path computations are done in \texttt{zkp/src/compute-vectors.js}.\\
\\
We denote this path:
\begin{align*}
  \phi_{Z_A} = [\phi_{d-1}, \phi_{d-2},..., \phi_{1}, \phi_0]
\end{align*}
Note that $\phi_0 = \roott_{n+m-1}$.\\
\\
Alice also retrieve's the `sister-path' of this path:
\begin{align*}
  \psi_{Z_A} = [\psi_{d-1}, \psi_{d-2},..., \psi_{1}, \psi_0]
\end{align*}
where $\psi_0 = \phi_0 = \roott_{n+m-1}$\\
\\
For ease of reading, let's focus only on the nodes of $M_{n+m-1}$ which Alice cares about for the purposes of transferring to Bob:

\begin{align*}
  \begin{forest}
    [{$\roott_{n+m-1}:=\phi_0=\psi_0$}
      [{$\phi_1$}
        [{$\psi_2$}
          [...
            [...][...]
          ]
          [...
            [...][...]
          ]
        ]
        [{$\phi_2$}
          [{$\phi_3$}
            [{$\psi_4$}][{$Z_A$}]
          ]
          [{$\psi_3$}
            [...][...]
          ]
        ]
      ]
      [{$\psi_1$}
        [...
          [...
            [...][0]
          ]
          [0
            [0][0]
          ]
        ]
        [0
          [0
            [0][0]
          ]
          [0
            [0][0]
          ]
        ]
      ]
    ]
  \end{forest}
\end{align*}

\noindent
Equipped with $\psi_{Z_A}$, Alice can prove that she owns a token commitment at one of the leaves of $M_{n+m-1}$, without revealing that it is "$Z_n$ located at leaf-index $n$".\\
\\
\textbf{Steps $8 - 9$}
\ \\
These steps are handled within \hyperref[sec:nf-token-controller]{\texttt{nf-token-controller.js}}.\\
\\
As a reminder, we let:
\begin{center}
  \begin{tabular}{l l}
    $x = (N_{A},\
          \roott_{n+m-1},\
          Z_B)$ & Public Inputs used to generate the Proof\\
    $\omega = (\alpha,\
              \psi_{Z_A},\
              sk_A,\
              \sigma_{A},\
              pk_B,\
              \sigma_{\vec{AB}})$ & Private Inputs used to generate the Proof\\
  \end{tabular}
\end{center}
\ \\
\textbf{Steps $10 - 11$}
\ \\
These steps are handled within a \hyperref[sec:zokrates]{ZoKrates} container.\\
\\
Alice uses the $C_{nft-transfer}$ (or $C$) -- the set of constraints for a non-fungible transfer, located in \texttt{zkp/code/gm17/nft-transfer} (see \hyperref[sec:trustedSetup]{Trusted Setup}). $C_{nft-transfer}(\;\omega,\;x\;)$ returns a value of $true$ if Alice provides a set of valid `satisfying' arguments $(\omega, x)$ to $C$.\\
\\
Let's elaborate on each of the checks and calculations constraining the inputs to $C$ (we highlight public inputs in \textbf{bold} below):
\begin{enumerate}
  \item Calculate $h(sk_A) =: pk_A'$.\\
    Note that this newly calculated $pk_A'$ should equal $pk_A$ (Alice's public key), but we don't need to pass $pk_A$ as a private input and explicitly check that $pk_A'=pk_A$; a check on the correctness of $sk_A$ (and hence $pk_A'$) is implicitly achieved in the next two steps:
  \item Calculate $h(\alpha\;|\;pk_A'\;|\;\sigma_A) =: Z_A'$.\\
    Note again that this newly calculated $Z_A'$ should equal $Z_A$ (Alice's token commitment), but we don't need to pass $Z_A$ as a private input and explicitly check that $Z_A'=Z_A$; a check on the correctness of $Z_A$ (and hence $Z_A'$) is implicitly achieved in the next step:
  \item Check inputs $\psi_{Z_A}=[\psi_{d-1}, \psi_{d-2},..., \psi_{1}, \bm{\psi_{0}=\roott_{n+m-1}}]$ and the newly calculated $Z_A'$ satisfy:\\
    $h\br*{\psi_{1}\;|...|\;h\br*{\psi_{d-2}\;|\;h\br*{\psi_{d-1}\;|\;Z_A'}\;}...} = \roott_{n+m-1} ( =: \bm{\psi_{0}})$\\
    Given the one-way nature of our hashing function $h$, the only feasible way we could have arrived at the correct value of $\roott_{n+m-1}$ is if the sister-path $\psi_{Z_A}$ is correct, and if $Z_A'$ is correct, which (working backwards) must mean that $sk_A$ is correct.

    How does the circuit know the value of $\roott_{n+m-1}$ is correct? It doesn't; but it is a `public input', and we can rely upon the Shield smart contract to check the correctness of all public inputs.\\
  \\
  We've therefore shown in the steps so far, that:
  \begin{itemize}
    \item[--] Alice is the owner of a token commitment (because she knows its secret key)
    \item[--] Said token commitment is indeed a leaf of the on-chain Merkle Tree $M_{n+m-1}$.
  \end{itemize}

  Alice commits to spending her token $Z_A$ in the next step:
  \item Check inputs $\sigma_{A}, sk_A, \bm{N_A}$ satisfy:
    $h(\sigma_{A}\;|\;sk_A) = \bm{N_{A}}$\\
    $N_A$ is referred to as a `nullifier' because it is understood by all participants to be an indisputable commitment to spend (`nullify') a token commitment. Remember that the token commitment being spent isn't revealed; the earlier steps have allowed Alice to demonstrate hidden knowledge of the secret key $sk_A$ of a token commitment which does indeed exist. By including $sk_A$ in the nullifier's preimage, Alice is binding herself as the executor of this transfer. By including $\sigma_A$, Alice is specifying a serial number which is unique to the token $Z_A$ (thereby distinguishing this nullifier from those which would nullify any other token commitments she may own).\\
  \item Check inputs $\alpha, pk_B, \sigma_{\vec{AB}}, \bm{Z_B}$ satisfy:
    $h(\alpha\;|\;pk_B\;|\;\sigma_{\vec{AB}}) = \bm{Z_B}$\\
    This final step constrains the same asset $\alpha$ to be included in $Z_B$ as was included in $Z_A$.\\
    You might notice that the circuit doesn't actually constrain Alice to use the correct values for Bob's public key $pk_B$, nor the serial number $\sigma_{\vec{AB}}$ as inputs to the circuit. Alice is free to transfer ownership of the token commitment to anyone.
\end{enumerate}
Notice how each stage is linked to the last, and that at each of the `Check' stages, private inputs are being reconciled against at least one public input (highlighted in \textbf{bold} to help you notice). By structuring the circuit $C$ in this way, we are able to share only the public inputs with the Shield contract (along with a `proof' $\pi_{C,x,\omega}$). We'll see shortly that the Shield contract checks the correctness of each of the public inputs against its current states.\\
\\

\noindent
If all of the above constraints are satisfied by the public and private inputs, ZoKrates will generate the proof $\pi_{C,x,\omega}$; a proof of knowledge of satisfying arguments $(\omega, x) \ s.t. \ C(\omega, x) = 1$.\\
\\
\textbf{Step $12$}
\ \\
This transaction is handled within \hyperref[sec:nf-token-zkp]{\texttt{nf-token-zkp.js}}.\\
\\
Having generated $\pi_{C,x,\omega}$, Alice then sends the following to the Shield contract from her anonymous Ethereum address $\Xi_{A,1}$:
\begin{align*}
  &\pi_{C,x,\omega}\\
  &x = (N_{A}, \roott_{n+m-1}, Z_B)
\end{align*}
\\
Recall that everyone knows the checks and calculations which have been performed in the circuit $C_{nft-transfer}$, because it is a public file in the Nightfall repository. Further, everyone knows the verification key $vk_C$ which uniquely represents this circuit, because it has been publicly stored in the Verifier Registry contract. Therefore, when this anonymous caller (Alice) shares the pair $(x, \pi_{C,x,\omega})$, and the `unique id' of the relevant verification key $vk_C$; everyone will interpret this information as the caller's intention to transfer, and everyone will be convinced that the caller knows the secret key which permits them to transfer ownership of a token commitment.\\
\\

\textbf{Steps $13 - 15$}
\ \\
The Verifier Registry contract already has stored within it the verification key $vk_C$.
It runs a verification function $V(vk_C, \pi_{C,x,\omega}, x)$.
\begin{align*}
  V: (vk_C, \pi_{C,x,\omega}, x) \to \{0,1\}
\end{align*}
where:
\[
    V=
\begin{cases}
    1,& \text{if } \pi_{C,x,\omega} \text{ and } x \text{ satisfy } vk_C\\
    0,& \text{otherwise}
\end{cases}
\]
\ \\

\textbf{Steps $16 - 17$}
\ \\
If the Verifier contract returns $1$ ($true$) (verified) to the Shield contract, then the Shield contract will be satisfied that Alice's proof and public inputs represent her commitment to relinquish ownership of a token commitment, and to transfer ownership of the underlying asset to someone via the newly proposed token commitment $Z_B$. If the Verifier contract returns $0$, then the transaction will revert.\\
\\
Let's suppose Alice's $(x, \pi_{C,x,\omega})$ pair is verified.\\
\\
Following verification of the proof, the Shield contract will do the following:
\begin{enumerate}
  \item Check $\roott_{n+m-1}$ is in $\rootsList$.\\
    (If not, the transfer will fail)
  \item Check $N_A$ is not already in the list of nullifiers, which we denote $\bm{N}$.\\
    (If $N_A$ is already in $\bm{N}$, the transfer will fail)
  \item Append the commitment $Z_B$ to the ever-increasing array of tokens, $\bm{Z}_{n+m}$, so that $\bm{Z}_{n+m}=(Z_0, Z_1, ... Z_{n-1}, Z_A, Z_{n+1}, ... Z_{n+m-1}, Z_B)$\\
  \item Recalculate a Merkle Root $\roott_{n+m}$ of $M_{n+m}$\\
    \\
    \begin{align*}
      \begin{forest}
        [{$\roott_{n+m}:= h\br*{
                            h\br*{
                              h\br*{
                                h\br*{
                                  Z_0,Z_1
                                },
                                ...
                              },
                              h\br*{
                                h\br*{
                                  Z_{n-1},Z_A
                                },
                                h\br*{
                                  Z_{n+1},...
                                }
                              }
                            },
                            h\br*{
                              h\br*{
                                h\br*{
                                  Z_{n+m-1}, Z_B
                                },
                                0
                              },
                              0
                            }
                          }
                        $}
          [{$ h\br*{
                h\br*{
                  h\br*{
                    Z_0,Z_1
                  },
                  ...
                },
                h\br*{
                  h\br*{
                    Z_{n-1},Z_A
                  },
                  h\br*{
                    Z_{n+1},...
                  }
                }
              }
            $}
            [{$ h\br*{
                  h\br*{
                    Z_0,Z_1
                  },
                  ...
                }
              $}
              [{$ h\br*{
                    Z_0,Z_1
                  }
                $}
                [{$Z_0$}][{$Z_1$}]
              ]
              [...
                [...][...]
              ]
            ]
            [{$ h\br*{
                  h\br*{
                    Z_{n-1},Z_A
                  },
                  h\br*{
                    Z_{n+1},...
                  }
                }
              $}
              [{$ h\br*{
                    Z_{n-1},Z_A
                  }
                $}
                [{$Z_{n-1}$}][{$Z_A$}]
              ]
              [{$ h\br*{
                    Z_{n+1},...
                  }
                $}
                [$Z_{n+1}$][...]
              ]
            ]
          ]
          [{$ h\br*{
                h\br*{
                  h\br*{
                    Z_{n+m-1}, Z_B
                  },
                  0
                },
                0
              }
            $}
            [{$ h\br*{
                  h\br*{
                    Z_{n+m-1}, Z_B
                  },
                  0
                }
              $}
              [{$ h\br*{
                    Z_{n+m-1}, Z_B
                  }
                $}
                [{$Z_{n+m-1}$}][{$Z_B$}]
              ]
              [0
                [0][0]
              ]
            ]
            [0
              [0
                [0][0]
              ]
              [0
                [0][0]
              ]
            ]
          ]
        ]
      \end{forest}
    \end{align*}

    Note that the Shield contract only needs to calculate the hashes on the path from $Z_B$ to the root.

  \item Append $\roott_{n+m}$ to the ever-increasing array $\rootsList$
  \item Similarly append the nullifier $N_{A}$ to the ever-increasing array $\bm N$.
\end{enumerate}
\ \\

\textbf{Steps $18 - 19$}
\ \\
The api-gateway routes the data resulting from a transfer to her local database.\\
\\
Similarly, the api-gateway ensures any sensitive data (data which is private to Alice alone) is filtered before Alice sends data to Bob.\\
\\
Data which is crucial to Bob verifying his ownership of the new $Z_B$ is encrypted with Bob's public whisper key $pk^W_B$ and broadcast to the Whisper network.\\
\\

\textbf{Steps $20 - 21$}
\ \\
Nightfall uses web3.shh to use Whisper. Bob's logged-in application will listen for all Whisper messages, and will try to decrypt all messages with his private whisper key $sk^W_B$. If decryption is successful, the data will be stored in the relevant database on Bob's local machine.\\
\\
\hyperref[sec:nf-token-zkp]{\texttt{nft-token-zkp.js}} includes functions to cross-reference the data Bob has received from Alice against the data stored in the Shield contract.\\
\\
Bob will store all important information in his private database.


%-------------------------------------------------------------------------------
%	NAME:	report.tex
%	AUTHOR: Connor Beardsmore - 15504319
%	LAST MOD:	01/05/17
%	PURPOSE:	FCC Assignment Report
%	REQUIRES:	NONE
%-------------------------------------------------------------------------------

\documentclass[]{article}
\usepackage[ margin=3cm ]{geometry}
\usepackage{graphicx}
\usepackage{fancyhdr}
\usepackage{float}
\usepackage{hyperref}
\usepackage{transparent}
\usepackage{multicol}
\usepackage{amsmath}
\usepackage[final]{pdfpages}
\usepackage{listings}
\usepackage{color}
\usepackage{algorithmicx}
\usepackage{algpseudocode}
\usepackage{amssymb}
\usepackage[style=chicago-authordate,backend=biber]{biblatex}

\pagestyle{fancy}
\fancyhf{}
\lhead{Connor Beardsmore - 15504319}
\rhead{FCC200}
\lfoot{May 2017}
\rfoot{\thepage}

\pagenumbering{arabic}
\graphicspath{{./images/}}

\addbibresource{bib/references.bib}
\nocite{*}

%-------------------------------------------------------------------------------
% CODE HIGHLIGHTING FOR LISTINGS
\definecolor{codegreen}{rgb}{0,0.6,0}
\definecolor{codegray}{rgb}{0.5,0.5,0.5}
\definecolor{codepurple}{rgb}{0.58,0,0.82}
\definecolor{backcolour}{rgb}{0.99,0.99,0.99}

\lstdefinestyle{mystyle}{
	backgroundcolor=\color{backcolour},
	commentstyle=\color{codegreen},
	keywordstyle=\color{magenta},
	numberstyle=\tiny\color{codegray},
	stringstyle=\color{codepurple},
	basicstyle=\footnotesize,
	breakatwhitespace=false,
	breaklines=true,
	captionpos=b,
	keepspaces=true,
	numbers=left,
	numbersep=5pt,
	showspaces=false,
	showstringspaces=false,
	showtabs=false,
	tabsize=2
}

\lstset{style=mystyle}

%-------------------------------------------------------------------------------
\begin{document}
%-------------------------------------------------------------------------------
% TITLE PAGE

\includepdf[]{./images/cover_page.pdf}

\begin{titlepage}
	\begin{center}
		\vspace*{1cm}
		\LARGE\textbf{FCC200 Report}
		\break
		RSA Cryptosystem Implementation
		\vspace{1cm}
		\break
		\Large\textbf{Connor Beardsmore - 15504319}
		\vspace{15cm}

		\normalsize
		Curtin University \\
		Science and Engineering \\
		Perth, Australia \\
	    May 2017

	\end{center}
\end{titlepage}

%-------------------------------------------------------------------------------
% BINARY MODULAR EXPONENTIATION
\vspace*{-0.8cm}
\section*{\hfil RSA Implementation\hfil}

\subsection*{Modular Exponentiation}
\noindent
Modular exponentiation is used to calculate the remainder when a base \textit{b} is raised to an exponent \textit{e} and reduced by some modulus \textit{m}. The simple right-to-left method provided by \cite{alttext} utilizes exponentiation by squaring. The full Java code for the implementation of this method is illustrated below. The running time of this algorithm is $O(log\;e)$ which provides a significant improvement over more simplistic methods of time complexity $O(e)$ (\cite{maintext}). This calculation is a core component of RSA and thus its efficiency is crucial to the speed of the RSA implementation.

\vspace{0.2cm}
\lstinputlisting[language=C,linerange={59-92} ]{../../rsa/numberTheory.c}{}

\noindent
The code above was utilized to calculate the following example:

$$236^{239721}\;mod\;2491=236$$

\vspace{0.2cm}
\noindent
The running of this code provided the following output:

\begin{figure}[H]
		\centering
	\includegraphics[height=\textheight/10,width=\textwidth/3]{exponentiation.png}
	\caption{Modular Exponentiation Example}
\end{figure}

\pagebreak

%-------------------------------------------------------------------------------
% RSA IMPLEMENTATION
\vspace*{-0.8cm}

\subsection*{RSA Testing}

The implemented RSA cipher works correctly with any key generated by the system. The recovered plaintext in Figure 4 is identical to the original plaintext of Figure 2, as confirmed by the Linux \textit{diff} command. No major problems were encountered during the implementation phase. The bytes visible in the ciphertext are not readable as plaintext and as a result, standard text editors cannot appropriately display the information.

\vspace{0.5cm}
\begin{figure}[H]
	\includegraphics[height=\textheight/6,width=\textwidth]{rsa_plain1.png}
	\includegraphics[height=\textheight/6,width=\textwidth]{rsa_plain2.png}
	\caption{RSA Plaintext}
	\centering
\end{figure}

\begin{figure}[H]
	\includegraphics[width=\textwidth]{rsa_cipher1.png}
	\includegraphics[width=\textwidth]{rsa_cipher2.png}
	\caption{RSA Ciphertext}
	\centering
\end{figure}

\begin{figure}[H]
	\includegraphics[height=\textheight/6,width=\textwidth]{rsa_plain1.png}
	\includegraphics[height=\textheight/6,width=\textwidth]{rsa_plain2.png}
	\caption{RSA Recovered Plaintext}
	\centering
\end{figure}

\pagebreak

%-------------------------------------------------------------------------------
% ADDITIONAL QUESTIONS

\vspace*{-0.8cm}
\section*{\hfil Additional Questions\hfil}

\subsection*{Signature Forgery}

RSA can be utilized as a message signature scheme to provide authentication to messages. If Alice wants to send a signed message to Bob, she first produces a \textit{hash value} of the message $H(m)$, then raises this to the power $d(modulo\;n)$ and attaches it to the message as a signature. When Bob receives the message, he utilizes the same hashing function, raises the result to the power of $e(modulo\;n)$ and compares to the message signature ($H(m)=H(m')$). If they are the same, Bob can be assured that the message was signed by Alice or someone with knowledge of Alice's private key.\\

\begin{figure}[H]
	\centering
	\includegraphics[height=\textheight/7,width=12cm]{rsasignature.png}
	\caption{RSA Signature Scheme (\cite{alttext})}
\end{figure}

\noindent
In this scheme, it is not possible to completely ensure the message was sent by Alice. If Bob has some alternate message $m''$ with a hash value $H(m'')$ matching that of Alice's $H(m)$ as discussed in \cite{RSA}, Bob can pretend to be Alice. He can simply resend Alice's signature he received onwards and the receiver of this message will believe that the message was sent by Alice. He can then perform malicious actions such as replay attacks depending on his intent. This can only occur if Bob finds a hash collision where $H(m)=H(m')$. It is however unlikely for Bob to create a \textit{meaningful} message with a hash value matching Alice's (\cite{alttext}). This is analogous to the idea of Birthday attacks mentioned in \cite{lecture3}.\\

\noindent
To prevent this situation from occurring, the hash function utilized for the digital signature must be strongly collision resistant (\cite{RSA}). Strong collision resistance asserts that there exists no $m$ and $m'$ where $m!=m'$ so that $H(m)=H(m')$. Thus if the hash function adheres to this property, the above situation cannot occur.

\break
\subsection*{Birthday Attack}

\vspace{0.5cm}
\begin{center}
\textit{In a group of 23 randomly selected people, the probability that two\\ of them share the same birthday is larger than 50\%}
\end{center}
\vspace{0.5cm}

\noindent
Firstly, the probability that two people have different birthdays is found:

$$1-\frac{1}{365}=\frac{364}{365}=0.99726$$

\vspace{0.5cm}
\noindent
This can be extended to determine if three people have different birthdays:

$$1-\frac{2}{365}=\frac{363}{365}=0.99452$$

\vspace{0.5cm}
\noindent
Utilizing conditional probability (\cite{lecture2}) we can construct the probability that all 23 people have different birthdays. This is simply represented as a series of fractions with their product producing the resultant probability:

$$1\times(1-\frac{1}{365})(1-\frac{2}{365})...(1-\frac{22}{365})=0.493$$

\vspace{0.5cm}
\noindent
To find the probability that two of the people have the same birthday, we inverse this number by subtracting from the total probability (1):

$$1-0.493=0.507=50.7\%$$

\vspace{0.5cm}
\noindent
It is thus evident that the probability of two people in a set of 23 random selected sharing the same birthday is greater than 50\%.

\pagebreak

%-------------------------------------------------------------------------------
% RSA CODE

\vspace*{-0.8cm}
\begin{center}
	\section*{RSA Source Code}
\end{center}

\subsection*{makefile}
\lstinputlisting[language=make,linerange={} ]{../../rsa/Makefile}\pagebreak{}
\subsection*{numberTheory.h}
\lstinputlisting[language=C,linerange={} ]{../../rsa/numberTheory.h}\pagebreak{}
\subsection*{numberTheory.c}
\lstinputlisting[language=C,linerange={} ]{../../rsa/numberTheory.c}\pagebreak{}
\subsection*{main.h}
\lstinputlisting[language=C,linerange={} ]{../../rsa/main.h}\pagebreak{}
\subsection*{main.c}
\lstinputlisting[language=C,linerange={} ]{../../rsa/main.c}\pagebreak{}

%-------------------------------------------------------------------------------
% REFERENCES

\break
\setlength\bibitemsep{4\itemsep}
\printbibliography[title={References}]

%-------------------------------------------------------------------------------
\end{document}
%-------------------------------------------------------------------------------:


\section{Nested hierarchy as the random variability structure}
\label{sec:variabilityModel}
\label{math:variability}

This section describes the variability structure of the random effects and the related naming convention. It is largely based on the discussions and conclusions from the Copenhagen focus meeting \cite{Copenhagen:2013}. Accordingly, in the following we will distinguish:
\begin{itemize}
\item
(related to the observations) -- \textit{residual variability}, also known as \textit{intra-individual variability} and
\item
(related to the parameters) -- \textit{inter-individual} and \textit{inter-occasion variabilities}
\end{itemize}
The former is described in the section \ref{sec:residualErrorModel}, while the latter is described in this section.

\begin{figure}[htb!]
\centering
  \includegraphics[width=70mm]{Subject-level}
 \caption{Inter-individual variability typically occurring in an experiment, here $\log(V_{i=1\cdots5})$ i.e. values for five subjects, varying around a typical value $\log(V_{pop})$, are shown.}
 \label{fig:subjectLevelVariability}
\end{figure}

\begin{figure}[htb!]
\centering
  \includegraphics[width=120mm]{Subject-occasion-level}
 \caption{Subject variability level, $(0)$, and within-subject (or occasion) variability level, $(-1)$, typically occurring in an experiment, with index $i$ for subjects and $k$ for occasion. Here two subjects only are visualised, each of them having four or three occasions, respectively.}
 \label{fig:subjectOccasionLevelVariability}
\end{figure}

\subsection{Motivation}
One way to look at variability is to consider the following simple experiment: in this experiment, we estimate the volume of distribution in five subjects. Following a drug administration we collect blood samples over a time interval and estimate each subject's PK parameters. The result will be a set of five individual estimates such as those in Figure \ref{fig:subjectLevelVariability}. The values vary around a certain typical/population value. It is apparent that the only variability source is the fact that these are different persons, i.e. we have a rough estimate of the so called \textit{inter-individual} variability.\\
As an extension of this setup, we can now consider different number of occasions, when the PK parameters are estimated for each subject. If we restrict the discussion to two subjects only, each of them having three or four occasions, respectively, we can illustrate the results such those in Figure \ref{fig:subjectOccasionLevelVariability}. Repeatedly performing the same experiment for each subject is equivalent to create an additional level of variability, the \textit{inter-occasion within individual} variability.\\
Similarly, one can add e.g. 'country' as a new variability level. If a clinical trial has been conducted in various countries, it is reasonable to ask if the geographic location influences the outcome of the study.

\subsection{General case}
As a generalisation of the examples described above, one can derive the \textit{nested hierarchy} (also known as \textit{inclusion hierarchy}) of the variability structure of random effects. It can be visualised as a tree or alternatively using a Venn diagram, see Figures \ref{IOVgeneral_tree}, \ref{IOVgeneral_venn}. \\
The tree representation consists of \textit{nodes} and \textit{links} or \textit{edges}. It has the advantage that it visualises the whole structure explicitly from the top level, the \textit{root} node, down to lowest level of the variability. It provides immediate insights needed to understand or to verify the setup of a trial design. However, in case of a very complex structure, with high number of levels and/or subjects, it can become very large, making the tree difficult to represent in a typical document. It this case showing only partial branches will be more helpful, e.g. Figure \ref{IOVgeneral_tree}. On the contrary, the Venn diagram visualises the levels only, and it might be more suited for the complex cases. Usually, the variability structure consists of only one or two levels, e.g. \textit{individual} or  \{\textit{individual}, \textit{occasion}\}, see examples below.

The \textit{root}, i.e. the top node in the tree structure, stands for the population/typical value of a parameter. Following the current nomenclature, every subsequent variability level is either 'positive' or 'negative' dependent on its position relative to the 'subject level', denoted as 0 -- the level 'zero'. Each level has a covariance matrix associated with it, i.e.
\begin{itemize}
\item
$\Omega^{+n}$ -- for levels above the 'zero' level -- their names will vary according to the nature of the levels. For example the variability on country level is called 'between-country variability'.
\item
$\Omega^0$ -- also called BSV (between subject variability) or IIV (inter-individual variability).
\item
$\Omega^{-n}$ -- for levels below the 'zero' level -- called WSV (within-subject variability) or IOV (inter-occasion variability).
\end{itemize}
The number of levels will vary dependent on the nature of the study. Cases without or with only positive/negative levels are possible. Please note that \pharmml doesn't require or use numbers to be assigned to the various levels of variability. Instead the user can define meaningful identifiers.

\begin{figure}[htb!]
\centering
  \includegraphics[width=120mm]{IOV-general-TREE}
 \caption{General nested hierarchy of the variability structure -- as tree. Note that \pharmml doesn't require or use numbers to be assigned to the various levels of variability. Instead the user can define meaningful identifiers.}
 \label{IOVgeneral_tree}
\end{figure}

\begin{figure}[htb!]
\centering
  \includegraphics[width=100mm]{IOV-general-VENN}
 \caption{General nested hierarchy of the variability structure -- as Venn diagram.}
 \label{IOVgeneral_venn}
\end{figure}

\paragraph{Example 1}
This example handles the simplest scenario, with only one level of variability: \textit{subject}--level, see Figure \ref{tree_IOV0}. The following symbols are used
\begin{itemize}
\item
$i$ -- subject index, $1\le i \le N$
\end{itemize}
with $N_l$ -- number of subjects.\\
The typical parameter model, without covariate, reads as follows:
\begin{align*}
& \log(V_i) = \log(V_{pop}) + \eta_i^{(0)}
\end{align*}
or alternatively:
\begin{align*}
& V_i = V_{pop} \,e^{\eta_i^{(0)}}
\end{align*}
with $\eta_i^{(0)} \sim \mathcal{N}\big(0,\Omega^{(0)}\big)$.

\begin{figure}[htb!]
\centering
  \includegraphics[width=120mm]{tree_IOV0}
 \caption{Example 1 -- single level of variability: \textit{subject}-- level}
 \label{tree_IOV0}
\end{figure}

\paragraph{Example 2}
In this example there are three levels of variability: \{\textit{centre, subject, occasion}\}, see Figure \ref{tree_IOV1}. Following symbols are used:
\begin{itemize}
\item
$l$ -- centre index, $1\le l \le L$
\item
$i$ -- subject index, $1\le i \le N_l$
\item
$k$ -- occasion index, $1\le k \le N_{li}$
\end{itemize}
with
\begin{itemize}
\item
$L$ -- number of centres
\item
$N_l$ -- number of subjects in centre \textit{l}
\item
$N_{li}$ -- number of occasions in subject \textit{i} in centre \textit{l}
\end{itemize}
The parameter model, without covariate, reads as follows:
\begin{align*}
& \log(V_{lik}) = \log(V_{pop}) + \eta_l^{(1)} + \eta_{li}^{(0)} + \eta_{lik}^{(-1)}
\end{align*}
or alternatively:
\begin{align*}
& V_{lik} = V_{pop} \, e^{\eta_l^{(1)}} e^{\eta_{li}^{(0)}} e^{\eta_{lik}^{(-1)}}
\end{align*}
with
\begin{align*}
 & \eta_l^{(1)} \sim \mathcal{N}\big(0,\Omega^{(1)}\big), \quad \eta_{li}^{(0)} \sim \mathcal{N}\big(0,\Omega^{(0)}\big),
\quad \eta_{lik}^{(-1)} \sim \mathcal{N}\big(0,\Omega^{(-1)}\big)
\end{align*}

\begin{figure}[htb!]
\centering
  \includegraphics[width=120mm]{tree_IOV1}
 \caption{Example 1 -- three levels of variability: \{\textit{centre, subject, occasion}\}}
 \label{tree_IOV1}
\end{figure}

\paragraph{Example 3}
In this example there are four levels of variability: \{\textit{country, centre, subject, occasion}\}, see Figure \ref{tree_IOV2}. The symbol list is extended by one for 'country' as follows:
\begin{itemize}
\item
$m$ -- country index, $1\le m \le M$
\item
$l$ -- centre index, $1\le l \le N_m$
\item
$i$ -- subject index, $1\le i \le N_{ml}$
\item
$k$ -- occasion index, $1\le k \le N_{mli}$
\end{itemize}
with
\begin{itemize}
\item
$M$ -- number of countries
\item
$N_m$ -- number of centres in country \textit{m}
\item
$N_{ml}$ -- number of subjects in centre \textit{l} in country \textit{m}
\item
$N_{mli}$ -- number of occasions in subject \textit{i} in centre \textit{l} in country \textit{m}
\end{itemize}
The parameter model reads as follows:
\begin{align*}
& \log(V_{mlik}) = \log(V_{pop}) + \eta_m^{(2)} + \eta_{ml}^{(1)} + \eta_{mli}^{(0)} + \eta_{mlik}^{(-1)}
\end{align*}
or alternatively:
\begin{align*}
& V_{mlik} = V_{pop} \, e^{\eta_m^{(2)}} e^{\eta_{ml}^{(1)}} \; e^{\eta_{mli}^{(0)}} \; e^{\eta_{mlik}^{(-1)}}
\end{align*}
with
\begin{align*}
 & \eta_m^{(2)} \sim \mathcal{N}\big(0,\Omega^{(2)}\big), \quad \eta_{ml}^{(1)} \sim \mathcal{N}\big(0,\Omega^{(1)}\big), \quad
 \eta_{mli}^{(0)} \sim \mathcal{N}\big(0,\Omega^{(0)}\big), \quad \eta_{mlik}^{(-1)} \sim \mathcal{N}\big(0,\Omega^{(-1)}\big)
\end{align*}

\begin{figure}[htb!]
\centering
  \includegraphics[width=120mm]{tree_IOV2}
 \caption{Example 2 -- four levels of variability:  \{\textit{country, centre, subject, occasion}\}}
 \label{tree_IOV2}
 \end{figure}


\documentclass[ignorenonframetext,notheorems,aspectratio=1610]{beamer}
\usetheme[compress]{Madrid}
\useoutertheme{infolines}
\usecolortheme{iwr}
\usepackage{../mathsim}
\input{./fig/tikzsettings}
\mathtoolsset{showonlyrefs}
\excludecomment{solution}
\externaldocument{main}

\makeatletter
\def\blocktheorem#1#2(#3){\begin{block}{#3 \capitalizewords{#1}%
      \def\tmp{#2}\ifx\tmp\empty{}\else (#2)\fi}}
\def\endblocktheorem{\end{block}}
\makeatother
\def\mylabel#1{}
\let\label\mylabel
%\renewcommand{\label}[1]{}
\renewcommand{\eqref}[1]{(\ref{#1})}
\renewcommand{\define}[1]{\textbf{#1}}
\begin{document}
\frame{\tableofcontents[hideallsubsections]}
\section{From elliptic to mixed problems}
\frame{\tableofcontents[currentsection,hideothersubsections]}
\frame {\input {blocks/Notation-vector-diff-operators.tex}}
\frame {\input {blocks/Definition-strain-tensor.tex}}
\frame {\input {blocks/Definition-hooke.tex}}
\frame {\input {blocks/Definition-weak-lame-navier.tex}}
\frame {\input {blocks/Problem-frobenius.tex}}
\frame {\input {blocks/Assumption-korn-inequality.tex}
  \input {blocks/Lemma-korn.tex}}
\frame {\small\input {blocks/Problem-elasticity-standard.tex}}

\begin{frame}
  \frametitle{Example: hanging sheet}
  \centering
  \begin{tabular}{ccc}
    \includegraphics[width=.25\textwidth]{./graph/elasticity/stalactite-0}
    &\includegraphics[width=.25\textwidth]{./graph/elasticity/stalactite-1}
    &\includegraphics[width=.25\textwidth]{./graph/elasticity/stalactite-2}
    \\
    $\lambda = 1$&$\lambda = 10$&$\lambda = 100$
    \\\\
    \includegraphics[width=.25\textwidth]{./graph/elasticity/stalactite-3}
    &\includegraphics[width=.25\textwidth]{./graph/elasticity/stalactite-4}
    &\includegraphics[width=.25\textwidth]{./graph/elasticity/stalactite-5}
    \\
    $\lambda = 1000$&$\lambda = 10000$&$\lambda = 100000$
  \end{tabular}
\end{frame}
\frame {\input {blocks/Definition-displacement-pressure.tex}}
\frame {\input {blocks/Definition-lame-navier-strong.tex}}
\frame {\input {blocks/Definition-saddle-point-operators.tex}}
\frame {\input {blocks/Definition-saddle-point-abstract.tex}}
\frame {\input {blocks/Notation-saddle-point-form.tex}}
\frame {\input {blocks/Definition-schur-complement.tex}
  \input {blocks/Lemma-schur-complement1.tex}}
\frame {\input {blocks/Lemma-schur-definiteness.tex}}
\frame {\input {blocks/Definition-stokes-eq1.tex}}
\frame {\input {blocks/Definition-solenoidal.tex}}
\frame {\input {blocks/Lemma-stokes-equivalence.tex}}
\frame {\input {blocks/Definition-stokes-eq2.tex}}
\frame {\input {blocks/Definition-stokes-boundary2.tex}}
\frame {\input {blocks/Lemma-divergence-compatibility.tex}
\input {blocks/Notation-pressure-constant.tex}}
\frame {\input {blocks/Theorem-minimization.tex}}
\frame {\input {blocks/Definition-reduced-problem.tex}
\input {blocks/Lemma-reduced-wellposedness.tex}}
\frame {\input {blocks/Theorem-lagrange-multiplier.tex}}
\frame {\input {blocks/Problem-lagrange-multiplier.tex}}
\frame {\input {blocks/Corollary-stokes-lagrange.tex}}

\section{Conditions for well-posedness}
\frame{\tableofcontents[currentsection,hideothersubsections]}
\frame {\input {blocks/Problem-unbounded-inverse.tex}}
\frame {\input {blocks/Problem-lax-milgram-not-applicable.tex}}
\frame {\input {blocks/Theorem-la-invertible.tex}}
\frame {\input {blocks/Theorem-svd.tex}}
\frame {\input {blocks/Corollary-svd-order.tex}}
\frame {\input {blocks/Definition-ker-range-rn.tex}
\input {blocks/Definition-orthogonal1.tex}}
\frame {\input {blocks/Lemma-ker-coker-rn.tex}
\input {blocks/Corollary-ker-coker-iso.tex}
\input {blocks/Corollary-svd-infsup.tex}}
\frame {\input {blocks/Definition-infsup1.tex}
\input {blocks/Lemma-infsup2.tex}
\input {blocks/Problem-inf-sup-equivalence.tex}}
\frame {\input {blocks/Definition-polar-orthogonal.tex}
\input {blocks/Lemma-orthogonal-closed.tex}
\input {blocks/Theorem-orthogonal-complement.tex}}
\frame {\input {blocks/Definition-orthogonal-projection.tex}}
\frame {\input {blocks/Lemma-polar-orthogonal-hilbert.tex}}
\frame {\input {blocks/Theorem-closed-range.tex}}
\frame {\input {blocks/Theorem-open-mapping.tex}
\input {blocks/Lemma-closed-infsup.tex}}
\frame {\input {blocks/Theorem-infsup-well-equivalence.tex}}
\frame {\input {blocks/Corollary-infsup-well-posedness1.tex}}
\frame {\input {blocks/Theorem-infsup-well-posedness2.tex}}
\frame {\input {blocks/Theorem-infsup-mixed1.tex}}
\frame {\input {blocks/Theorem-infsup-mixed2.tex}}
\frame {\input {blocks/Problem-inhomogeneous-continuity.tex}}
\frame {\input {blocks/Assumption-mixed-elliptic.tex}}

\subsection{Galerkin approximation of mixed problems}

\frame {\input {blocks/Definition-kerbh.tex}}
\frame {\input {blocks/Definition-mixed-galerkin.tex}}
\frame {\input {blocks/Theorem-galerkin-mixed-u-kerbh.tex}}
\frame {\input {blocks/Corollary-galerkin-mixed-u-kerb.tex}}
\frame {\input {blocks/Theorem-galerkin-mixed-existence-p.tex}}
\frame {\input {blocks/Problem-infsup-uniform.tex}}
\frame {\input {blocks/Lemma-fortin.tex}}
\frame {\input {blocks/Assumption-mixed-elliptic-stabilized.tex}}
\frame {\input {blocks/Theorem-mixed-stabilized-well-posed.tex}}
\frame {\input {blocks/Definition-mixed-residual.tex}}
\frame {\input {blocks/Corollary-mixed-residual-bounded.tex}}
\frame {\input {blocks/Lemma-stabilized-mixed-approximation.tex}}
\frame {\input {blocks/Corollary-stabilized-mixed-convergence.tex}}

\section{Stokes equations}
\frame{\tableofcontents[currentsection,hideothersubsections]}

\frame {\input {blocks/Lemma-stokes-a-elliptic.tex}}
\frame {\input {blocks/Lemma-stokes-helmholtz.tex}
  \input {blocks/Lemma-stokes-grad.tex}}
\frame {\input {blocks/Corollary-stokes-iso.tex}
  \input {blocks/Theorem-stokes-infsup.tex}}
\frame {\input {blocks/Theorem-stokes-convergence.tex}}
\frame {\input {blocks/Corollary-stokes-convergence2.tex}}
\frame {\input {blocks/Problem-checker-board.tex}}

\begin{frame}
  \frametitle{Example: one-dimensional $P_1-P_1$ elements}
  \begin{center}
      \includegraphics[width=.7\textwidth]{./fig/p1-p1-1d}
  \end{center}
\end{frame}

\begin{frame}
  \frametitle{Small patches with Dirichlet boundary}
  \begin{center}
    \hfill
    \includegraphics[width=.3\textwidth]{./fig/patch1.tikz}
    \hfill
    \includegraphics[width=.3\textwidth]{./fig/patch2.tikz}
    \hfill\mbox{}
  \end{center}
\end{frame}

\begin{frame}
  \frametitle{Checkerboard modes}

\end{frame}

\subsection{MINI element}
\frame {\input {blocks/Definition-barycentric-coordinates.tex}}

\begin{frame}
  \frametitle{The $P_1$ element in barycentric coordinates}
  \begin{columns}
    \begin{column}{.5\textwidth}
      \begin{center}
        \includegraphics[width=.6\textwidth]{./fig/p1-p.tikz}
      \end{center}
    \end{column}
    \begin{column}{.5\textwidth}
      \begin{gather*}
        \phi_i = \lambda_i,
        \quad i=0,1,2
      \end{gather*}
    \end{column}
  \end{columns}
\end{frame}

\begin{frame}
  \frametitle{The $P_2$ element in barycentric coordinates}
  \begin{columns}
    \begin{column}{.5\textwidth}
      \begin{center}
        \includegraphics[width=.6\textwidth]{./fig/p2-p.tikz}
      \end{center}
    \end{column}
    \begin{column}{.5\textwidth}
      \begin{xalignat*}2
        \phi_{ii} &= 2\lambda_i^2 - \lambda_i,
        &i&=0,1,2\\
        \phi_{ij} &= 4\lambda_i\lambda_j
        &j&\neq i
      \end{xalignat*}
    \end{column}
  \end{columns}
\end{frame}

\begin{frame}
  \frametitle{The $P_3$ element in barycentric coordinates}
  \begin{columns}
    \begin{column}{.5\textwidth}
      \begin{center}
        \includegraphics[width=.6\textwidth]{./fig/p3-p.tikz}
      \end{center}
    \end{column}
    \begin{column}{.5\textwidth}
      \begin{xalignat*}2
        \phi_{iii} &= \tfrac12 \lambda_i(3\lambda_i-1)(3\lambda_i-2)
        &i&=0,1,2\\
        \phi_{ij} &= \tfrac92\lambda_i\lambda_j(3\lambda_j-1)
        &j&\neq i\\
        \phi_0 &= 27\lambda_0\lambda_1\lambda_2
      \end{xalignat*}
    \end{column}
  \end{columns}
\end{frame}

\frame {\input {blocks/Notation-piecewise-polynomial-spaces.tex}}
\frame {\input {blocks/Definition-h1-bubble-space.tex}}
\frame {\input {blocks/Definition-mini-element-p.tex}}
\frame {\input {blocks/Lemma-fortin-construction-1.tex}}
\frame {\input {blocks/Assumption-h1-stable-interpolation.tex}}
\frame {\input {blocks/Definition-locally-quasi-uniform.tex}
  \input {blocks/Assumption-locally-quasi-uniform.tex}}
\frame {\input {blocks/Theorem-mini-stability.tex}}
\frame {\input {blocks/Notation-broken-bilinear-form.tex}}
\frame {\input {blocks/Lemma-mini-stabilized.tex}}
\frame {\input {blocks/Problem-quadrilateral-mini.tex}}
\frame {\input {blocks/Problem-mini-3d.tex}}
\frame {\input {blocks/Definition-higher-order-bubble.tex}}
\frame {\input {blocks/Theorem-higher-order-bubble.tex}
  \input {blocks/Corollary-pk-bubble.tex}}

\subsection{P2-P0}

\frame {\input {blocks/Definition-p2-p0-element.tex}
  \input {blocks/Lemma-p2-p0-stability.tex}}
\frame {\input {blocks/Theorem-p2-p0-convergence.tex}}
\frame {\input {blocks/Problem-q2-q0.tex}}
\frame {\input {blocks/Lemma-bubble-discontinuous.tex}}
\frame {\input {blocks/Corollary-bubble-p2.tex}
  \input {blocks/Corollary-pk-pk2.tex}}

\frame {\input {blocks/Theorem-discontinuous-pressure-normal-velocity.tex}}
\frame {\input {blocks/Corollary-qk-pk1.tex}}
\frame {\input {blocks/Definition-moment-dofs-qk-2d.tex}}
\frame {\input {blocks/Definition-moment-dofs-qk-3d.tex}}
\frame {\input {blocks/Example-qk-pk1.tex}}

\subsection{The Hood-Taylor elements}

\frame {\input {blocks/Definition-hood-taylor.tex}}
\frame {\input {blocks/Example-hood-taylor-triangle.tex}}
\frame {\input {blocks/Example-hood-taylor-quad.tex}}
\frame {\input {blocks/Definition-macro-equivalence.tex}}
\frame {\input {blocks/Problem-reference-macros.tex}}
\frame {\input {blocks/Definition-macro-spaces.tex}}
\frame {\input {blocks/Definition-verfuerth-norm.tex}}
\frame {\input {blocks/Definition-macro-seminorm.tex}}
\frame {\input {blocks/Lemma-macro1.tex}}
\frame {\input {blocks/Lemma-macro-local.tex}}
\frame {\input {blocks/Problem-macro-local.tex}}
\frame {\input {blocks/Lemma-verfuerth1.tex}}
\frame {\input {blocks/Lemma-verfuerth2.tex}}
\frame {\input {blocks/Theorem-hood-taylor-stability.tex}}

\frame {\input {blocks/Lemma-patch-test-triangle.tex}}
\frame {\input {blocks/Lemma-patch-test-quad.tex}}

\subsection{Almost incompressible elasticity}

\frame {\input {blocks/Lemma-reduced integration.tex}}
\frame {\input {blocks/Theorem-mixed-stabilized-well-posed.tex}}
\frame {\input {blocks/Corollary-stabilized-mixed-convergence.tex}}

\section{Mixed formulation of elliptic problems}
\frame{\tableofcontents[currentsection,hideothersubsections]}

\frame {\input {blocks/Definition-primal-mixed.tex}}
\frame {\input {blocks/Definition-hdiv.tex}}
\frame {\input {blocks/Definition-dual-mixed.tex}}
\frame {\input {blocks/Theorem-Hdiv-separable.tex}
  \input {blocks/Theorem-Hdiv-trace.tex}}
\frame {\input {blocks/Problem-trace-dnu.tex}}
\frame {\input {blocks/Theorem-Hdiv-trace-surjective.tex}
  \input {blocks/Theorem-Hdiv-trace-kernel.tex}}
\frame {\input {blocks/Theorem-Hdiv-helmholtz.tex}}
\frame {\input {blocks/Problem-mixed-inhomogeneous-bc.tex}}
\frame {\input {blocks/Lemma-darcy-reduced-wellposed.tex}}
\frame {\input {blocks/Lemma-darcy-infsup.tex}
  \input {blocks/Theorem-darcy-well-posed.tex}}
\frame {\small\input {blocks/Theorem-infsup-mixed2.tex}}

\subsection{Discretization of dual mixed problems}

\frame {\input {blocks/Lemma-normal-continuity.tex}}
\frame {\input {blocks/Definition-rt-simplex.tex}}
\frame {\input {blocks/Example-rt-simplex.tex}}
\frame {\input {blocks/Lemma-rt-simplex-1.tex}}
\frame {\input {blocks/Lemma-rt-simplex-dimension.tex}}
\frame {\input {blocks/Lemma-rt-simplex-unisolvence.tex}}

\frame {\input {blocks/Definition-bdm-simplex.tex}}
\frame {\input {blocks/Example-bdm-simplex.tex}}
\frame {\input {blocks/Lemma-bdm-simplex-unisolvence.tex}}
\frame {\input {blocks/Definition-canonical-interpolation.tex}}
\frame {\input {blocks/Lemma-commuting-diagram-hdiv.tex}}

\subsection{Quarilaterals and hexahedra}

\frame {\input {blocks/Notation-recap-reference-transform.tex}}
\frame {\input {blocks/Definition-Piola-transform.tex}}
\frame {\input {blocks/Lemma-Piola-transform-integrals.tex}
  \input {blocks/Problem-Piola-transform-integrals.tex}}
\frame {\input {blocks/Notation-tensor-product-polynomials.tex}}
\frame {\input {blocks/Definition-rt-quad.tex}}
\frame {\input {blocks/Example-rt-quad.tex}}
\frame {\input {blocks/Lemma-rt-quad-1.tex}}
\frame {\input {blocks/Definition-bdm-quad.tex}}
\frame {\input {blocks/Example-bdm-quad.tex}}
\frame {\input {blocks/Lemma-bdm-quad.tex}
  \input {blocks/Problem-bdm-quad.tex}}
\frame {\input {blocks/Corollary-darcy-convergence-affine.tex}}
\frame {\input {blocks/Definition-abf-quad.tex}}

\section{Divergence conforming DG}
\frame{\tableofcontents[currentsection,hideothersubsections]}

\subsection{The interior penalty method}
\frame {\input {blocks/Definition-dg-faces.tex}
  \input {blocks/Definition-dg-spaces.tex}}
\frame {\input {blocks/Definition-broken-integrals.tex}}
\frame {\input {blocks/Notation-dg-operators.tex}}
\frame {\input {blocks/Definition-ip.tex}}
\frame {\input {blocks/Definition-ip-norm.tex}
  \input {blocks/Problem-ip-norm.tex}}
\frame {\input {blocks/Lemma-ip-stability.tex}
  \input {blocks/Problem-ip-stability.tex}}
\frame {\input {blocks/Lemma-ip-consistence.tex}}
%\frame {\input {blocks/Theorem-ip-convergence.tex}}

\subsection{Formulation with lifting operators}

\frame {\input {blocks/Definition-dg-lifting.tex}
  \input {blocks/Lemma-ip-lifting-bounded.tex}}
\frame {\input {blocks/Definition-ip-lifting.tex}
  \input {blocks/Lemma-ip-equivalence.tex}}
\frame {\input {blocks/Definition-ip-residual.tex}}
\frame {\input {blocks/Lemma-ip-lifting-strang.tex}}
\frame {\input {blocks/Lemma-ip-lifting-residual-2.tex}}
\frame {\input {blocks/Lemma-ip-lifting-residual-1.tex}}
\frame {\input {blocks/Theorem-ip-lifting-h1.tex}}
\frame {\input {blocks/Theorem-ip-lifting-l2.tex}}

\subsection{Divergence conforming IP}

\frame {\input {blocks/Definition-hdiv-ip.tex}}
\frame {\input {blocks/Lemma-dg-fortin.tex}}
\frame {\input {blocks/Corollary-hdivdg-infsup.tex}}
\frame {\input {blocks/Theorem-hdivdg-convergence.tex}}

\subsection{Duality}

\frame {\input {blocks/Definition-dual-stokes.tex}
  \input {blocks/Assumption-stokes-regularity.tex}}
\frame {\input {blocks/Definition-hdivdg-residual-operators.tex}}
\frame {\input {blocks/Lemma-hdivdg-residual-1.tex}
  \input {blocks/Corollary-hdivdg-residual-2.tex}}
\frame {\input {blocks/Theorem-hdivdg-l2.tex}
  \input {blocks/Problem-hdivdg-l2.tex}}

\section{Maxwell}
\frame {\input {blocks/Notation-curl.tex}}
\frame {\input {blocks/Lemma-curl-green.tex}}
\frame {\input {blocks/Definition-Maxwell-boundary.tex}}
\frame {\input {blocks/Definition-curl-traces.tex}}
\frame {\input {blocks/Theorem-curl-traces.tex}}
\frame {\input {blocks/Definition-Maxwell-mixed-0.tex}}

\section{The de Rham complex}
\frame{\tableofcontents[currentsection,hideothersubsections]}

\frame {\input {blocks/Notation-hlambda.tex}}
\frame {\input {blocks/Notation-hlambda-2d.tex}}
\frame {\input {blocks/Notation-hlambda-norm.tex}}
\frame {\input {blocks/Theorem-de-rham.tex}}
\frame {\input {blocks/Lemma-hlambda-0.tex}}

\frame {\input {blocks/Theorem-div-curl-well-posed.tex}
  \input {blocks/Problem-darcy-derham.tex}}

\subsection{Polynomial complexes for simplicial meshes}

\frame {\input {blocks/Notation-pk-complex.tex}}
\frame {\input {blocks/Definition-Koszul-complex.tex}}
\frame {\input {blocks/Lemma-kd-plus-dk.tex}
  \input {blocks/Lemma-d-kappa-injective.tex}}
\frame {\input {blocks/Theorem-polynomial-exact.tex}
  \input {blocks/Corollary-pk-complexes.tex}}
\frame {\input {blocks/Definition-pk-plus.tex}}
\frame {\input {blocks/Lemma-pk-plus-d.tex}}
\frame {\input {blocks/Lemma-pr-pr-plus.tex}}
\frame {\input {blocks/Theorem-dimension-pr-lambda.tex}}

\subsection{Degrees of freedom and bases for simplicial meshes}

\frame {\input {blocks/Definition-mesh-pk.tex}
  \input {blocks/Definition-spanned-simplex.tex}
  \input {blocks/Definition-subsimplices.tex}}
\frame {\input {blocks/Definition-pr-f.tex}}
\frame {\input {blocks/Lemma-subsimplex-polynomials.tex}
  \input {blocks/Problem-subsimplex-polynomials.tex}}
\frame {\input {blocks/Example-h1-moment-dofs.tex}}
\frame {\input {blocks/Definition-v-of-f.tex}
  \input {blocks/Definition-w-of-f.tex}}
\frame {\input {blocks/Lemma-pr-geometric.tex}}
\frame {\input {blocks/Theorem-decomp-pr-plus.tex}}
\frame {\input {blocks/Example-rt-complex-decomp.tex}}
\frame {\input {blocks/Theorem-decomp-pr.tex}}
\frame {\input {blocks/Example-bdm-complex-decomp.tex}}
\frame {\input {blocks/Lemma-qr-complex.tex}}
\frame {\input {blocks/Definition-pr-complex-1d.tex}}
\frame {\input {blocks/Definition-tensor-dofs.tex}}
\frame {\input {blocks/Lemma-2d-tensor-rt-nedelec.tex}}

\end{document}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End: