\documentclass{article}

\usepackage{lipsum}
\usepackage[margin=1.5in]{geometry}
\usepackage{titlesec}
\usepackage{graphicx}
\usepackage{amsmath}

\usepackage{mathtools, amssymb, nccmath}
\usepackage{bigstrut, changepage, lipsum}

\newcommand{\code}{\texttt}
\newcommand{\norm}[1]{\left\lVert#1\right\rVert}

\usepackage{siunitx} % Required for alignment

% Specify images directory
\graphicspath{ {./report-images/} }

% Header and Footer stuff
\usepackage{fancyhdr}
\pagestyle{fancy}
\fancyhead{}
\fancyfoot{}
\fancyfoot[R]{ \thepage\ }
\renewcommand{\headrulewidth}{0pt}
\renewcommand{\footrulewidth}{0pt}
\newcommand{\sectionbreak}{\clearpage}
\setlength{\parindent}{0pt}

%

\begin{document}

%----------------------------------------------------------------------------------------
%	TITLE PAGE
%----------------------------------------------------------------------------------------

\begin{titlepage} % Suppresses displaying the page number on the title page and the subsequent page counts as page 1
	\newcommand{\HRule}{\rule{\linewidth}{0.5mm}}% Defines a new command for horizontal lines, change thickness here

	\center % Centre everything on the page

	%------------------------------------------------
	%	Headings
	%------------------------------------------------

	\textsc{\Large Data Smoothing}\\[0.5cm] % Major heading such as course name

	\textsc{\large Exercise 3}\\[0.5cm] % Minor heading such as course title

	%------------------------------------------------
	%	Title
	%------------------------------------------------

	\HRule\\[0.6cm]

	{\huge\bfseries Data Smoothing Report}\\[0.25cm] % Title of your document

	\HRule\\[1.5cm]

	%------------------------------------------------
	%	Author(s)
	%------------------------------------------------

	\begin{minipage}{0.4\textwidth}
		\begin{flushleft}
			\large
			\textit{Author}\\
			\textsc{Cesare De Cal} % Your name
		\end{flushleft}
	\end{minipage}
	~
	\begin{minipage}{0.4\textwidth}
		\begin{flushright}
			\large
			\textit{Professor}\\
			\textsc{Annie Cuyt}\\ % Supervisor's name
			[0.25cm]
			\textit{Assistant Professor}\\
			\textsc{Ferre Knaepkens} % Supervisor's name

		\end{flushright}
	\end{minipage}

	\vfill\vfill\vfill

	{\large\today}

	\vfill

\end{titlepage}

%----------------------------------------------- Introduction ------------------------------------------------------
\section{Introduction}\label{sec:intro}
This exercise asks to use the linearly independent basis function:
$$\phi_{3,i}(x)=\binom{3}{i}x^{i}(1-x)^{3-i}\quad\quad i = 0, 1, 2, 3$$
to find the optimal combination
$$\phi(x)=\lambda_0\phi_{3,0}(x)+\lambda_1\phi_{3,1}(x)+\lambda_2\phi_{3,2}(x)+\lambda_3\phi_{3,3}(x)$$
that minimizes the Euclidean norm
$$\sqrt{\sum_{j=0}^{19}{(\phi(x_j)-y_j)^2}}$$
for the 20 data points $(x_j,y_j)$ given in

\begin{table}[!ht]
\large        %% not "\fontsize{12}{12}\selectfont"
\centering    %% not "\center{...}"
\begin{tabular}{|c|c|c|}
\hline
$j$ & $x_j$ & $y_j$ \\     %% no "&" at start of row
\hline
0 & 0.0 & -0.80\\
1 & 0.6 & -0.34\\
2 & 1.5 & 0.59\\
3 & 1.7 & 0.59\\
4 & 1.9 & 0.23\\
5 & 2.1 & 0.10\\
6 & 2.3 & 0.28\\
7 & 2.6 & 1.03\\
8 & 2.8 & 1.50\\
9 & 3.0 & 1.44\\
10 & 3.6 & 0.74\\
11 & 4.7 & -0.82\\
12 & 5.2 & -1.27\\
13 & 5.7 & -0.92\\
14 & 5.8 & -0.92\\
15 & 6.0 & -1.04\\
16 & 6.4 & -0.79\\
17 & 6.9 & -0.06\\
18 & 7.6 & 1.00\\
19 & 8.0 & 0.00\\
\hline
\end{tabular}
\end{table}

This is an overdetermined system. I'll first find the coefficient matrix $A$ and then use $QR$ decomposition to solve the system. I'll also calculate the $l_2$-norm of the residual, and the condition number of the coefficient matrix. Finally, I'll plot the data points with the graph of the function $\phi(x)$.

%---------------------------------- Introduction ------------------------------------------------------
\section{Tools}
The following programming language and libraries have been used in this exercise:
\begin{itemize}
  \item C
  \item GSL (GNU Scientific Library)
  \item C Math Library
  \item Python (for plotting)
  \item SciPy (for plotting)
\end{itemize}
The following double-precision GSL data types have been used in the exercise:
\begin{itemize}
  \item \code{gsl\_matrix}
  \item \code{gsl\_vector}
\end{itemize}
The following GSL methods have been used in the exercise:
\begin{itemize}
  \item \code{gsl\_matrix\_alloc(size1, size2)}
  \item \code{gsl\_matrix\_set(matrix, row, column, value)}
  \item \code{gsl\_matrix\_get(matrix, row, column)}
  \item \code{gsl\_vector\_alloc(size)}
  \item \code{gsl\_vector\_set(vector, index, value)}
  \item \code{gsl\_vector\_set\_zero(vector)}
  \item \code{gsl\_vector\_get(vector, index)}
  \item \code{gsl\_matrix\_memcpy(destinationMatrix, matrixToCopyFrom)}
  \item \code{gsl\_linalg\_SV\_decomp(A, V, S, workspaceVector)}
  \item \code{gsl\_vector\_minmax(vector, minInVector, maxInVector)}
  \item \code{gsl\_sf\_fact(number)} to calculate the factorial
  \item \code{gsl\_sf\_pow\_int(base, exponent)}
  \item \code{gsl\_blas\_dnrm2(vector)} to calculate the Euclidean norm
  \item \code{gsl\_matrix\_free(matrixToDeallocate)}
  \item \code{gsl\_vector\_free(vectorToDeallocate)}
\end{itemize}
In order to decompose the coefficient matrix into its QR decomposition, and then solve the square system $A\vec{x}=\vec{b}$, I've used the following methods:
\begin{itemize}
  \item \code{gsl\_linalg\_QR\_decomp(matrixQR, vectorTau)}
  \item \code{gsl\_linalg\_QR\_lssolve(matrixQR, vectorTau, vectorB, vectorX, vectorResidual)}
  \item \code{gsl\_permutation\_alloc(size)}
\end{itemize}
The following method from the C Math library was used in this exercise to calculate the absolute value of a number:
\begin{itemize}
  \item \code{fabs(x)}
\end{itemize}

\section{Computation}
First of all, I compute the coefficients A of the linear system by using the linearly independent basis function. For the $(i, j)$ cell in the matrix, I call the basis function by passing the $i$-th element of the $x$ data points, and by using the column $j$ for the $i$ parameter of the basis function. This results in a $20\times4$ matrix. The following is a representation of A:
\begin{adjustwidth}{-2em}{-2em}
    \[ \medmath{\begin{bmatrix*}[r]
1.000000000000000\,e+00 & 0.000000000000000\,e+00 & 0.000000000000000\,e+00 & 0.000000000000000\,e+00 \bigstrut[t]\\
6.400000000000002\,e-02 & 2.880000000000000\,e-01 & 4.320000000000001\,e-01 & 2.160000000000000\,e-01 \\
-1.250000000000000\,e-01 & 1.125000000000000\,e+00 & -3.375000000000000\,e+00 & 3.375000000000000\,e+00 \\
-3.429999999999999\,e-01 & 2.499000000000000\,e+00 & -6.068999999999998\,e+00 & 4.912999999999999\,e+00 \\
-7.289999999999998\,e-01 & 4.616999999999998\,e+00 & -9.747000000000000\,e+00 & 6.858999999999999\,e+00 \\
-1.331000000000000\,e+00 & 7.623000000000002\,e+00 & -1.455300000000000\,e+01 & 9.261000000000001\,e+00 \\
-2.196999999999999\,e+00 & 1.166100000000000\,e+01 & -2.063099999999999\,e+01 & 1.216700000000000\,e+01 \\
-4.096000000000001\,e+00 & 1.996800000000001\,e+01 & -3.244800000000000\,e+01 & 1.757600000000000\,e+01 \\
-5.831999999999998\,e+00 & 2.721599999999999\,e+01 & -4.233599999999999\,e+01 & 2.195199999999999\,e+01 \\
-8.000000000000000\,e+00 & 3.600000000000000\,e+01 & -5.400000000000000\,e+01 & 2.700000000000000\,e+01 \\
-1.757600000000000\,e+01 & 7.300800000000001\,e+01 & -1.010880000000000\,e+02 & 4.665600000000001\,e+01 \\
-5.065300000000001\,e+01 & 1.930290000000000\,e+02 & -2.451990000000000\,e+02 & 1.038230000000000\,e+02 \\
-7.408800000000001\,e+01 & 2.751840000000000\,e+02 & -3.407040000000000\,e+02 & 1.406080000000000\,e+02 \\
-1.038230000000000\,e+02 & 3.777390000000001\,e+02 & -4.581090000000000\,e+02 & 1.851930000000000\,e+02 \\
-1.105920000000000\,e+02 & 4.008960000000000\,e+02 & -4.844160000000000\,e+02 & 1.951120000000000\,e+02 \\
-1.250000000000000\,e+02 & 4.500000000000000\,e+02 & -5.400000000000000\,e+02 & 2.160000000000000\,e+02 \\
-1.574640000000000\,e+02 & 5.598720000000002\,e+02 & -6.635520000000001\,e+02 & 2.621440000000001\,e+02 \\
-2.053790000000000\,e+02 & 7.205670000000001\,e+02 & -8.426970000000001\,e+02 & 3.285090000000001\,e+02 \\
-2.874960000000000\,e+02 & 9.931679999999998\,e+02 & -1.143648000000000\,e+03 & 4.389759999999999\,e+02 \\
-3.430000000000000\,e+02 & 1.176000000000000\,e+03 & -1.344000000000000\,e+03 & 5.120000000000000\,e+02 \bigstrut[b]
\end{bmatrix*}} \]%
\end{adjustwidth}

% --------- CONDITION NUMBER ---------
Then, I calculate the condition number of the coefficient matrix $A$. In GSL there is no direct function that calculates the condition number, but it's possible to use the ratio of the largest singular value of matrix A, $\sigma_n (A)$, to the smallest $\sigma_1 (A)$:

$$\kappa(A) := \frac{\sigma_n (A)}{\sigma_1 (A)}= \frac{\norm{A}}{\norm{A^{-1}}^{-1}}$$

I proceed to factorize $A$ into its singular value decomposition $SVD$ using the \code{gsl\_linalg\_SV\_decomp} method, and then use $\code{gsl\_vector\_minmax}$ to extract the minimum and maximum singular values out of the vector $S$ that contains the diagonal elements of the singular value matrix. The condition number of the matrix $A$ is equal to $3.741019262503867e+03$.\\

% --------- CONDITION NUMBER ---------

The column vector $\vec{b}$ is formed by the input $y_j$ values:
$$
\begin{bmatrix}
-8.000000000000000e-01\\
-3.400000000000000e-01\\
5.900000000000000e-01\\
5.900000000000000e-01\\
2.300000000000000e-01\\
1.000000000000000e-01\\
2.800000000000000e-01\\
1.030000000000000e+00\\
1.500000000000000e+00\\
1.440000000000000e+00\\
7.400000000000000e-01\\
-8.200000000000000e-01\\
-1.270000000000000e+00\\
-9.200000000000000e-01\\
-9.200000000000000e-01\\
-1.040000000000000e+00\\
-7.900000000000000e-01\\
-6.000000000000000e-02\\
1.000000000000000e+00\\
0.000000000000000e+00\\
\end{bmatrix}
$$

I proceed in solving the system (finding the $\lambda$ values that approximate to a solution of the system). I use QR decomposition to solve the system ($\code{gsl\_linalg\_QR\_decomp}$ and $\code{gsl\_linalg\_QR\_lssolve}$ GSL methods). The following is the QR decomposition of A:

\begin{adjustwidth}{-2em}{-2em}
    \[ \medmath{\begin{bmatrix*}[r]
-5.607137694902810e+02 & 1.956202843752372e+03 & -2.276212763756861e+03 & 8.833928944999552e+02 \\
1.139370324819986e-04 & -4.671135706796601e+01 & 1.122030535262658e+02 & -6.782418228480103e+01 \\
-2.225332665664034e-04 & 1.460497799957120e-02 & 7.581872706318327e+00 & -8.735870723873337e+00 \\
-6.106312834582108e-04 & 2.762429988027977e-02 & 1.797466673315370e-01 & -2.470454024049311e+00 \\
-1.297814010615264e-03 & 4.400918925292650e-02 & 2.113118885654055e-01 & 1.354991299702775e-01 \\
-2.369534222399064e-03 & 6.326913656703972e-02 & 2.350053228690535e-01 & 9.686987975376644e-02 \\
-3.911244693171105e-03 & 8.491363227214763e-02 & 2.513310790520081e-01 & 6.050043190845661e-02 \\
-7.291970078847909e-03 & 1.207784144212678e-01 & 2.631080548709738e-01 & 1.049772432830278e-02 \\
-1.038251208492211e-02 & 1.462383018205487e-01 & 2.632200917938682e-01 & -1.959354810540480e-02 \\
-1.424212906024982e-02 & 1.723659541841733e-01 & 2.577288314294048e-01 & -4.692208258102275e-02 \\
-3.128995754536886e-02 & 2.498504055563917e-01 & 2.126763547071380e-01 & -1.108942994397425e-01 \\
-9.017582041110428e-02 & 3.520968788593226e-01 & 5.605686650054093e-02 & -1.473630562460556e-01 \\
-1.318963572269736e-01 & 3.635568863958127e-01 & -2.831753402090372e-02 & -1.240982343374392e-01 \\
-1.848325706777897e-01 & 3.424022086740725e-01 & -1.099043818739413e-01 & -7.278910904340619e-02 \\
-1.968831921288935e-01 & 3.335830602668363e-01 & -1.251940955112117e-01 & -5.896438574071358e-02 \\
-2.225332665664034e-01 & 3.108660410390103e-01 & -1.542417580429807e-01 & -2.760834500928011e-02 \\
-2.803278262928973e-01 & 2.434003295032724e-01 & -2.044456433018340e-01 & 5.043304854412397e-02 \\
-3.656292780331310e-01 & 1.127004233192019e-01 & -2.472054281433762e-01 & 1.782091144383195e-01 \\
-5.118193920381977e-01 & -1.705668773785795e-01 & -2.556299463550832e-01 & 4.175378294298211e-01 \\
-6.106312834582110e-01 & -3.921595584433181e-01 & -2.261060282111244e-01 & 5.880648466528077e-01 \\
\end{bmatrix*}} \]%
\end{adjustwidth}

The solutions vector is directly given by the $\code{gsl\_linalg\_QR\_lssolve}$ method.

$$
\begin{bmatrix}
-1.118262027055321e+00\\
-4.515724133992401e-01\\
2.972563550105292e-03\\
2.972882574075207e-01\\
\end{bmatrix}
$$

The residual vector $\vec{r}$ is also directly given by the $\code{gsl\_linalg\_QR\_lssolve}$ method.
$$
\begin{bmatrix}
3.182620270553753e-01\\
-2.038767862631474e-01\\
-3.507925507654632e-02\\
-1.076211346528342e-01\\
-5.104297656943454e-01\\
-6.559970851745927e-01\\
-4.668150300670836e-01\\
3.379120178166642e-01\\
8.680652851344475e-01\\
8.842461476326978e-01\\
4.840349364751467e-01\\
-4.332442092900336e-01\\
-6.426370573997744e-01\\
-1.387527086448619e-01\\
-1.218089886087358e-01\\
-2.242466352247385e-01\\
-1.314405461740510e-02\\
5.037445845512266e-01\\
8.885672158267567e-01\\
-7.311795037780225e-01\\
\end{bmatrix}
$$

The Euclidean norm of the residual vector, which can be easily calculated with $\code{gsl\_blas\_dnrm2(vector)}$, is equal to $2.282876480420795e+00$.
\section{Plot}

\includegraphics[width=\textwidth,height=\textheight,keepaspectratio]{graph.png}

\section{Observations}
In this problem I have worked with an overdetermined system of equations which has more equations that unknowns ($m >> n \rightarrow20 >> 4$). It is not possible to find the exact $\vec{x}$ vector, but I found a set of $\lambda$ that satisfies the system as close as possible through $QR$ factorization. \\

At the end of the exercise, I plotted the computed function together with the data points. In order to plot the graph of the $\phi(x)$ function, I used the $\lambda$ values contained in $\vec{x}$ and plugged them into the linear combination. Plus, I created an array of equidistant $x$ values and calculated the function value for each of the values. We can observe that the function gives a general idea of the data points, but overall I don't think it is very accurate.

\end{document}


\documentclass[jou]{apa6}

\usepackage[american]{babel}

\usepackage{csquotes}
\usepackage[style=apa,sortcites=true,sorting=nyt,backend=biber]{biblatex}
\DeclareLanguageMapping{american}{american-apa}
\addbibresource{bibliography.bib}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Discrete Structures
%% The start of RBS stuff
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Working internal and external links in PDF
\usepackage{hyperref}
% Extra math symbols in LaTeX
\usepackage{amsmath}
\usepackage{gensymb}
\usepackage{amssymb}
% Enumerations with (a), (b), etc.
\usepackage{enumerate}

\let\OLDitemize\itemize
\renewcommand\itemize{\OLDitemize\addtolength{\itemsep}{-6pt}}

\usepackage{etoolbox}
\makeatletter
\preto{\@verbatim}{\topsep=3pt \partopsep=3pt }
\makeatother

% These sizes redefine APA for A4 paper size
\oddsidemargin 0.0in
\evensidemargin 0.0in
\textwidth 6.27in
\headheight 1.0in
\topmargin -24pt
\headheight 12pt
\headsep 12pt
\textheight 9.19in

\title{Sample Quiz 4}
\author{Discrete Structures, Fall 2020}
\affiliation{RBS}

\leftheader{Discrete Sample Quiz 4}

\abstract{%
}

%\keywords{}

\begin{document}

%\thispagestyle{empty}

\twocolumn
\section{Worksheet 4: Sets}

\vspace{10pt}
{\bf Question 1.} We define three sets in the universe $U$ of integer numbers
between $1$ and $70$ (inclusive):
$$\left\{ \begin{array}{rcl}
K_2 & = & \{ x \in U \,\mid\, 2 \mid x \},\\
K_5 & = & \{ x \in U \,\mid\, 5 \mid x \},\\
K_7 & = & \{ x \in U \,\mid\, 7 \mid x \},
\end{array} \right.$$

Find the size of the following sets.
Here $|X|$ denotes the number of elements in a finite set
(BTW, $|X|$ is also the notation for the cardinality of an infinite set).\\
{\em Note.} It was not intentional, but vertical bar: $\mid$
in this exercise happens to be used in three different ways:
It is a separator when defining sets $K_2$; it is used to denote
divisibility; it also denotes set cardinality.

\begin{tabular}{ll}
$\left| K_2 \cup K_5 \right|$ & $\ldots$ \\
$\left| K_2 \cap K_7 \right|$ & $\ldots$ \\
$\left| \overline{K_7} \right|$ & $\ldots$ \\
$\left| \overline{K_2 \cap K_5} \right|$ & $\ldots$ \\
$\left| K_2 \cap K_5 \cap K_7 \right|$ & $\ldots$ \\
$\left| K_2 \cap K_5 \cap \overline{K_7} \right|$ & $\ldots$ \\
$\left| K_2 \cup K_5 \cup K_7 \right|$ & $\ldots$ \\
$\left| \overline{K_2 \cup K_5 \cup K_7} \right|$ & $\ldots$ \\
\end{tabular}

\vspace{6pt}
{\bf Question 2.}
Find a counterexample to refute the following predicate expression:\\
$(\exists x \in U,\;P(x)) \wedge (\exists x\in U,\;Q(x)) \rightarrow$\\
$\rightarrow \exists x\in U,\;(P(x) \wedge Q(x))$.\\
Here $P(x)$ is true iff $P$ is a full square (a square of some integer number),
$Q(x)$ is true iff $x$ is divisible by $5$, and $U$ is the set of all integers
from the interval $[120;130]$.\\
{\em Note.} The three $x$'s in this formula refer to
three unrelated (local) variables.
If it looks confusing, you can rewrite it like this:\\
$(\exists x_1 \in U,\;P(x_1)) \wedge (\exists x_2\in U,\;Q(x_2)) \rightarrow$\\
$\rightarrow \exists x_3\in U,\;(P(x_3) \wedge Q(x_3))$.

\noindent
{\bf (A)} Identify the variables which you need to pick for your counter-example.\\
{\bf (B)} Pick the values for these variables to make the above statement false.

\vspace{6pt}
{\bf Question 3.} Determine the cardinality of the following sets (some finite number?
equal to $|\mathbb{N}|$? equal to $|\mathcal{P}(\mathbb{N})| = |\mathbb{R}|$? equal to $|\mathcal{P}(\mathbb{R})|$?)
\begin{enumerate}[(A)]
\item The set of positive real numbers from $(0;1)$ with decimal representation
containing only digits $0$ and $1$?
\item The set of positive real numbers from $(0;1)$ with decimal representation
containing only digits $0$ and $1$ (and it is known that the number of $0$s is finite)?
\item The set of positive real numbers from $(0;1)$ that are fully periodic decimal fractions with
a period of $2020$ digits?
\item The set of positive real numbers from $(0;1)$ that have decimal representation
without any digits "9"?
\item The set of all irrational $x \in (0;1)$ such that $x^3$ is rational?
\item Ordered pairs of real numbers $x_1,x_2$ such that $x_1,x_2 \in (0;1)$.
\item Finite sequences of real numbers: $x_1,\ldots,x_n$, and all $x_i \in (0;1)$? (Here $n$ can
be any positive integer)?
\item Infinite sequences of real numbers from $(0;1)$: $\{ x_n \}$: $x_1,x_2,x_3,\ldots$.
\end{enumerate}

\vspace{6pt}
{\bf Question 4.}
Let ${\displaystyle f(x) = \left\lfloor \frac{x^3}{3} \right\rfloor}$. Find $f(S)$ if $S$ is:
\begin{enumerate}[(A)]
\item $S = \{ −2, −1, 0, 1, 2, 3 \}$.
\item $S = \{0, 1, 2, 3, 4, 5 \}$.
\item $S = \{1, 5, 7, 11 \}$.
\end{enumerate}
Is function $f: \mathbb{Z} \rightarrow \mathbb{Z}$ injective? Is it surjective?
If it is not, mention counterexamples to show this.

\vspace{6pt}
{\bf Question 5.}
Determine, if the given set is a powerset of some other set. If yes, which one?
\begin{enumerate}[(A)]
\item $\{\emptyset,\; \{\emptyset\},\; \{a\},\; \{\{a\}\},\; \{\{\{a\}\}\},\; \{\emptyset, a\},\; \{\emptyset, \{a\}\},\; \{\emptyset, \{\{a\}\}\},$\\
\mbox{}$\;\{a, \{a\}\},\; \{a, \{\{a\}\}\},\; \{\{a\}, \{\{a\}\}\},\; \{\emptyset, a, \{a\}\},\; \{\emptyset, a, \{\{a\}\}\},$\\
\mbox{}$\;\{\emptyset, \{a\}, \{\{a\}\}\},\; \{a, \{a\}, \{\{a\}\}\},\; \{\emptyset, a, \{a\}, \{\{a\}\}\}\}$.
\item $\{\emptyset, \{a\}\}$.
\item $\{\emptyset, \{a\}, \{\emptyset, a\}\}$.
\item $\{\emptyset, \{a\}, \{\emptyset\}, \{a, \emptyset\}\}$.
\item $\{\emptyset, \{a, \emptyset\}\}$.
\end{enumerate}

\vspace{6pt}
{\bf Question 6.}
Given two sets $A = \{ x, y \}$ and $B = \{x, \{x \}\}$, check, if statements are true or false:
\begin{enumerate}[(A)]
\item $x \subseteq B$.
\item $\emptyset \in \mathcal{P}(B)$.
\item $\{x\} \subseteq A - B$.
\item $|\mathcal{P}(A)| = 4$.
\end{enumerate}

\vspace{6pt}
{\bf Question 7.}
We define functions $g\,:\, A \rightarrow A$ and $f : A \rightarrow A$, where $A \{1, 2, 3, 4\}$ by
listing all argument-value pairs:
$$g = \{(1, 4), (2, 1), (3, 1), (4, 2)\},\;\;f = \{(1, 3), (2, 2), (3, 4), (4, 1)\}.$$
Find these functions by listing their argument/value pairs (or establish that they do not exist).
\begin{enumerate}[(A)]
\item Find $f \circ g$.
\item Find $g \circ f$.
\item Find $g \circ g$.
\item Find $g \circ (g \circ g)$.
\item Find $f^{-1}$.
\item Find $g^{-1}$.
\end{enumerate}

\vspace{6pt}
{\bf Question 8.} Find these sums:
\begin{enumerate}[(A)]
\item $1/4 + 1/8 + 1/16 + 1/32 + \ldots$.
\item $2 + 4 + 8 + 16 + 32 + \ldots + 2^{28}$.
\item $2 - 4 + 8 - 16 + 32 - \ldots - 2^{28}$.
\item $1 - 1/2 + 1/4 - 1/8 + 1/16 - \ldots$.
\end{enumerate}

\vspace{6pt}
{\bf Question 9.} Find an appropriate $O(g(n))$ for each function $f(n)$ defined below
(pick your $g(n)$ to be the slowest growing among the functions such that $f(n)$ is in $O(g(n))$).
\begin{enumerate}[(A)]
\item $f(n) = 1^2 + 2^2 + \ldots + n^2$.
\item ${\displaystyle f(n) = \frac{3n - 8 - 4n^3}{2n - 1}}$.
\item ${\displaystyle f(n) = \sum\limits_{k=1}^{n} k^3}$.
\item ${\displaystyle f(n) = \frac{6n + 4n^5 - 4}{7n^2 - 3}}$.
\item ${\displaystyle f(n) = \sum\limits_{k=2}^{n} k\cdot(k-1)}$.
\item ${\displaystyle f(n) = 3n^2 + 8n + 7}$
\end{enumerate}

\vspace{6pt}
{\bf Question 10.}
For the given functions, find an optimal $O(g(n))$; find
$C$ and $n_0$ (from the definition $|f(n)| < C\cdot |g(n)|$ as long as $n > n_0$).
\begin{enumerate}[(A)]
\item ${\displaystyle f(n) = 3n^4 + \log_2 n^8}$.
\item ${\displaystyle f(n) = \sum\limits_{k=1}^{n} (k^3 + k)}$.
\item ${\displaystyle f(n) = (n + 2)\log_2 (n^2 + 1) + \log_2 (n^3 + 1)}$.
\item ${\displaystyle f(n) = n^3 + \sin n^7}$.
\end{enumerate}

\vspace{6pt}
{\bf Question 11.} This is a Python fragment; variable {\tt n} can become very large; {\tt t} is some
fixed parameter. Denote by $f(n)$ the number of operations depending on the variable {\tt n},
where an operation is an addition or a multiplication, or raising to the power 2.
Find the slowest growing $g(n)$ so that $f(n)$ is in $O(g(n))$.
\begin{verbatim}
sum = 0
for i in range(1,n+1):
    for j in range(1,n+1):
        sum += (i*t + j*t + 1)**2
\end{verbatim}

\vspace{6pt}
{\bf Question 12.} There are two functions $f,g: \mathbb{R} \rightarrow \mathbb{R}$ defined
for all real numbers and taking real values.
Find, which predicate logic expressions describe a statement that is logically
equivalent to the English sentence ``The function $f(n)$ is in $O(g(n))$''.\\
{\em Note.} There may be multiple correct answers.

\begin{enumerate}[(A)]
\item $\forall n \in \mathbb{R}\;\exists n_0 \in \mathbb{R}\;\exists C \in \mathbb{R},$\\
$\left(n > n_0 \rightarrow |f(n)| \leq C\cdot{}|g(n)|\right)$.
\item $\exists n_0 \in \mathbb{R}\;\forall n \in \mathbb{R}\;\exists C \in \mathbb{R},$\\
$(n > n_0 \rightarrow |f(n)| \leq C\cdot{}|g(n)|)$.
\item $\exists n_0 \in \mathbb{R}\;\exists C \in \mathbb{R}\;\forall n \in \mathbb{R},$\\
$(n > n_0 \rightarrow |f(n)| \leq C\cdot{}|g(n)|)$.
\item $\exists n_0 \in \mathbb{R}\;\exists C \in \mathbb{R}\;\forall n \in \mathbb{R},$\\
$(n > n_0 \rightarrow f(n) \leq C\cdot{}|g(n)|)$.
\item $\exists n_0 \in \mathbb{R}\;\exists C \in \mathbb{R}\;\forall n \in \mathbb{R},$\\
$(n > n_0 \rightarrow |f(n)| \leq C\cdot{}g(n))$.
\item $\exists n_0 \in \mathbb{R}\;\exists C \in \mathbb{R}\;\forall n \in \mathbb{R},$\\
$(n \geq n_0 \rightarrow |f(n)| < C\cdot{}|g(n)|)$.
\item $\exists n_0 \in \mathbb{Z}^{+}\;\exists C \in \mathbb{Z}^{+}\;\forall n \in \mathbb{R},$\\
$(n > n_0 \rightarrow |f(n)| \leq C\cdot{}|g(n)|)$.
\end{enumerate}

\newpage
\subsection{Answers}

\vspace{6pt}
{\bf Question 1.} Answer:\\

\begin{tabular}{ll}
$\left| K_2 \cup K_5 \right|$ & $42$ \\
$\left| K_2 \cap K_7 \right|$ & $5$ \\
$\left| \overline{K_7} \right|$ & $60$ \\
$\left| \overline{K_2 \cap K_5} \right|$ & $63$ \\
$\left| K_2 \cap K_5 \cap K_7 \right|$ & $1$ \\
$\left| K_2 \cap K_5 \cap \overline{K_7} \right|$ & $6$ \\
$\left| K_2 \cup K_5 \cup K_7 \right|$ & $46$ \\
$\left| \overline{K_2 \cup K_5 \cup K_7} \right|$ & $24$ \\
\end{tabular}

To find $\left| K_2 \cup K_5 \cup K_7 \right|$ we might use inclusion-exclusion principle:
$$\left| K_2 \cup K_5 \cup K_7 \right| = (35 + 14 + 10) - (7 + 5 + 2) + 1 = 46.$$

\vspace{6pt}
{\bf Question 2.} Answer: TBD\\

\vspace{6pt}
{\bf Question 3.} Answer: TBD\\

\vspace{6pt}
{\bf Question 4.} Answer: TBD\\

\vspace{6pt}
{\bf Question 5.} Answer: TBD\\

\vspace{6pt}
{\bf Question 6.} Answer: TBD\\

\vspace{6pt}
{\bf Question 7.} Answer: TBD\\

\vspace{6pt}
{\bf Question 8.} Answer: TBD\\

\vspace{6pt}
{\bf Question 9.} Answer: TBD\\

\vspace{6pt}
{\bf Question 10.} Answer: TBD\\

\vspace{6pt}
{\bf Question 11.} Answer: TBD\\

\vspace{6pt}
{\bf Question 12.} Answer: TBD\\

\end{document}


%!TEX root = ../OGUSAdoc.tex

We start the \ogindia section on modeling the household with a description of the demographics of the model. \citet{Nishiyama:2015} and \citet{DeBackerEtAl:2017} have recently shown that demographic dynamics are likely the biggest influence on macroeconomic time series, exhibiting more influence than fiscal variables or household preference parameters.

In this chapter, we characterize the equations and parameters that govern the transition dynamics of the population distribution by age. In \ogindia, we take the approach of taking mortality rates and fertility rates from outside estimates. But we estimate our immigration rates as residuals using the mortality rates, fertility rates, and at least two consecutive periods of population distribution data. This approach makes sense if one modeling a country in which in one is not confident in the immigration rate data. If the country has good immigration data, then the immigration residual approach we describe below can be skipped.

We define $\omega_{s,t}$ as the number of households of age $s$ alive at time $t$. A measure $\omega_{1,t}$ of households is born in each period $t$ and live for up to $E+S$ periods, with $S\geq 4$.\footnote{Theoretically, the model works without loss of generality for $S\geq 3$. However, because we are calibrating the ages outside of the economy to be one-fourth of $S$ (e.g., ages 21 to 100 in the economy, and ages 1 to 20 outside of the economy), it is convenient for $S$ to be at least 4.} Households are termed ``youth'', and do not participate in market activity during ages $1\leq s\leq E$. The households enter the workforce and economy in period $E+1$ and remain in the workforce until they unexpectedly die or live until age $s=E+S$. We model the population with households age $s\leq E$ outside of the workforce and economy in order most closely match the empirical population dynamics.

The population of agents of each age in each period $\omega_{s,t}$ evolves according to the following function,
\begin{equation}\label{EqPopLawofmotion}
  \begin{split}
    \omega_{1,t+1} &= (1 - \rho_0)\sum_{s=1}^{E+S} f_s\omega_{s,t} + i_1\omega_{1,t}\quad\forall t \\
    \omega_{s+1,t+1} &= (1 - \rho_s)\omega_{s,t} + i_{s+1}\omega_{s+1,t}\quad\forall t\quad\text{and}\quad 1\leq s \leq E+S-1
  \end{split}
\end{equation}
where $f_s\geq 0$ is an age-specific fertility rate, $i_s$ is an age-specific net immigration rate, $\rho_s$ is an age-specific mortality hazard rate, and $\rho_0$ is an infant mortality rate.\footnote{The parameter $\rho_s$ is the probability that a household of age $s$ dies before age $s+1$.} The total population in the economy $N_t$ at any period is simply the sum of households in the economy, the population growth rate in any period $t$ from the previous period $t-1$ is $g_{n,t}$, $\tilde{N}_t$ is the working age population, and $\tilde{g}_{n,t}$ is the working age population growth rate in any period $t$ from the previous period $t-1$.
\begin{equation}\label{EqPopN}
  N_t\equiv\sum_{s=1}^{E+S} \omega_{s,t} \quad\forall t
\end{equation}
\begin{equation}\label{EqPopGrowth}
  g_{n,t+1} \equiv \frac{N_{t+1}}{N_t} - 1 \quad\forall t
\end{equation}
\begin{equation}\label{EqPopNtil}
  \tilde{N}_t\equiv\sum_{s=E+1}^{E+S} \omega_{s,t} \quad\forall t
\end{equation}
\begin{equation}\label{EqPopGrowthTil}
  \tilde{g}_{n,t+1} \equiv \frac{\tilde{N}_{t+1}}{\tilde{N}_t} - 1 \quad\forall t
\end{equation}
We discuss the approach to estimating fertility rates $f_s$, mortality rates $\rho_s$, and immigration rates $i_s$ in Sections \ref{SecDemogFert}, \ref{SecDemogMort}, and \ref{SecDemogImm}.

\section{Fertility rates}\label{SecDemogFert}

  In \ogindia, we assume that the fertility rates for each age cohort $f_s$ are constant across time. However, this assumption is conceptually straightforward to relax. Our data for U.S. fertility rates by age come from \citet[Table 3, p. 18]{MartinEtAl:2015} National Vital Statistics Report, which is final fertility rate data for 2013. Figure \ref{FigFertRates} shows the fertility-rate data and the estimated average fertility rates for $E+S=100$.

  \begin{figure}[htbp]\centering \captionsetup{width=4.0in}
    \caption{\label{FigFertRates}\textbf{Fertility rates by age ($f_s$) for $E+S=100$}}
    \fbox{\resizebox{4.0in}{3.0in}{\includegraphics{./images/fert_rates.png}}}
  \end{figure}

  The large blue circles are the 2013 U.S. fertility rate data from \citet{MartinEtAl:2015}. These are 9 fertility rates $[0.3, 12.3, 47.1, 80.7, 105.5, 98.0, 49.3, 10.4, 0.8]$ that correspond to the midpoint ages of the following age (in years) bins $[10-14, 15-17, 18-19, 20-24, 25-29, 30-34, 35-39, 40-44, 45-49]$. In order to get our cubic spline interpolating function to fit better at the endpoints we added to fertility rates of zero to ages 9 and 10, and we added two fertility rates of zero to ages 55 and 56. The blue line in Figure \ref{FigFertRates} shows the cubic spline interpolated function of the data.

  The red diamonds in Figure \ref{FigFertRates} are the average fertility rate in age bins spanning households born at the beginning of period 1 (time = 0) and dying at the end of their 100th year. Let the total number of model years that a household lives be $E+S\leq 100$. Then the span from the beginning of period 1 (the beginning of year 0) to the end of period 100 (the end of year 99) is divided up into $E+S$ bins of equal length. We calculate the average fertility rate in each of the $E+S$ model-period bins as the average population-weighted fertility rate in that span. The red diamonds in Figure \ref{FigFertRates} are the average fertility rates displayed at the midpoint in each of the $E+S$ model-period bins.

\section{Mortality rates}\label{SecDemogMort}

  The mortality rates in our model $\rho_s$ are a one-period hazard rate and represent the probability of dying within one year, given that an household is alive at the beginning of period $s$. We assume that the mortality rates for each age cohort $\rho_s$ are constant across time. The infant mortality rate of $\rho_0=0.00587$ comes from the 2015 U.S. CIA World Factbook. Our data for U.S. mortality rates by age come from the Actuarial Life Tables of the U.S. Social Security Administration \citep[see][]{SocSec:2015}, from which the most recent mortality rate data is for 2011. Figure \ref{FigMortRates} shows the mortality rate data and the corresponding model-period mortality rates for $E+S=100$.

  \begin{figure}[htbp]\centering \captionsetup{width=4.0in}
    \caption{\label{FigMortRates}\textbf{Mortality rates by age ($\rho_s$) for $E+S=100$}}
    \fbox{\resizebox{4.0in}{3.0in}{\includegraphics{./images/mort_rates.png}}}
  \end{figure}

  The mortality rates in Figure \ref{FigMortRates} are a population-weighted average of the male and female mortality rates reported in \citet{SocSec:2015}. Figure \ref{FigMortRates} also shows that the data provide mortality rates for ages up to 111-years-old. We truncate the maximum age in years in our model to 100-years old. In addition, we constrain the mortality rate to be 1.0 or 100 percent at the maximum age of 100.

\section{Immigration rates}\label{SecDemogImm}

  Because of the difficulty in getting accurate immigration rate data by age, we estimate the immigration rates by age in our model $i_s$ as the average residual that reconciles the current-period population distribution with next period's population distribution given fertility rates $f_s$ and mortality rates $\rho_s$. Solving equations \eqref{EqPopLawofmotion} for the immigration rate $i_s$ gives the following characterization of the immigration rates in given population levels in any two consecutive periods $\omega_{s,t}$ and $\omega_{s,t+1}$ and the fertility rates $f_s$ and mortality rates $\rho_s$.

  \begin{equation}\label{EqPopImmRates}
    \begin{split}
      i_1 &= \frac{\omega_{1,t+1} - (1 - \rho_0)\sum_{s=1}^{E+S}f_s\omega_{s,t}}{\omega_{1,t}}\quad\forall t \\
      i_{s+1} &= \frac{\omega_{s+1,t+1} - (1 - \rho_s)\omega_{s,t}}{\omega_{s+1,t}}\qquad\qquad\forall t\quad\text{and}\quad 1\leq s \leq E+S-1
    \end{split}
  \end{equation}

  \begin{figure}[htbp]\centering \captionsetup{width=4.0in}
    \caption{\label{FigImmRates}\textbf{Immigration rates by age ($i_s$), residual, $E+S=100$}}
    \fbox{\resizebox{4.0in}{3.0in}{\includegraphics{./images/imm_rates_orig.png}}}
  \end{figure}

  We calculate our immigration rates for three different consecutive-year-periods of population distribution data (2010 through 2013). Our four years of population distribution by age data come from \citet{Census:2015}. The immigration rates $i_s$ that we use in our model are the the residuals described in \eqref{EqPopImmRates} averaged across the three periods. Figure \ref{FigImmRates} shows the estimated immigration rates for $E+S=100$ and given the fertility rates from Section \ref{SecDemogFert} and the mortality rates from Section \ref{SecDemogMort}.

  At the end of Section \ref{SecDemogPopSSTP}, we describe a small adjustment that we make to the immigration rates after a certain number of periods in order to make computation of the transition path equilibrium of the model compute more robustly.

\section{Population steady-state and transition path}\label{SecDemogPopSSTP}

  This model requires information about mortality rates $\rho_s$ in order to solve for the household's problem each period. It also requires the steady-state stationary population distribution $\bar{\omega}_{s}$ and population growth rate $\bar{g}_n$ as well as the full transition path of the stationary population distribution $\hat{\omega}_{s,t}$ and population grow rate $\tilde{g}_{n,t}$ from the current state to the steady-state. To solve for the steady-state and the transition path of the stationary population distribution, we write the stationary population dynamic equations \eqref{EqPopLawofmotionStat} and their matrix representation \eqref{EqPopLOMstatmat}.
  \begin{equation}\label{EqPopLawofmotionStat}
    \begin{split}
      \hat{\omega}_{1,t+1} &= \frac{(1-\rho_0)\sum_{s=1}^{E+S} f_s\hat{\omega}_{s,t} + i_1\hat{\omega}_{1,t}}{1+\tilde{g}_{n,t+1}}\quad\forall t \\
      \hat{\omega}_{s+1,t+1} &= \frac{(1 - \rho_s)\hat{\omega}_{s,t} + i_{s+1}\hat{\omega}_{s+1,t}}{1+\tilde{g}_{n,t+1}}\qquad\quad\:\forall t\quad\text{and}\quad 1\leq s \leq E+S-1
    \end{split}
  \end{equation}
  \begin{equation}\label{EqPopLOMstatmat}
    \begin{split}
      & \begin{bmatrix}
        \hat{\omega}_{1,t+1} \\ \hat{\omega}_{2,t+1} \\ \hat{\omega}_{2,t+1} \\ \vdots \\ \hat{\omega}_{E+S-1,t+1} \\ \hat{\omega}_{E+S,t+1}
      \end{bmatrix}= \frac{1}{1 + g_{n,t+1}} \times ... \\
      & \begin{bmatrix}
        (1-\rho_0)f_1+i_1 & (1-\rho_0)f_2 & (1-\rho_0)f_3 & \hdots & (1-\rho_0)f_{E+S-1} & (1-\rho_0)f_{E+S} \\
        1-\rho_1 & i_2 & 0 & \hdots & 0 & 0 \\
        0 & 1-\rho_2 & i_3 & \hdots & 0 & 0 \\
        \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\
        0 & 0 & 0 & \hdots & i_{E+S-1} & 0 \\
        0 & 0 & 0 & \hdots & 1-\rho_{E+S-1} & i_{E+S}
      \end{bmatrix}
      \begin{bmatrix}
        \hat{\omega}_{1,t} \\ \hat{\omega}_{2,t} \\ \hat{\omega}_{2,t} \\ \vdots \\ \hat{\omega}_{E+S-1,t} \\ \hat{\omega}_{E+S,t}
      \end{bmatrix}
    \end{split}
  \end{equation}
  We can write system \eqref{EqPopLOMstatmat} more simply in the following way.
  \begin{equation}\label{EqPopLOMstatmat2}
    \bm{\hat{\omega}}_{t+1} = \frac{1}{1+g_{n,t+1}}\bm{\Omega}\bm{\hat{\omega}}_t \quad\forall t
  \end{equation}
  The stationary steady-state population distribution $\bm{\bar{\omega}}$ is the eigenvector $\bm{\omega}$ with eigenvalue $(1+\bar{g}_n)$ of the matrix $\bm{\Omega}$ that satisfies the following version of \eqref{EqPopLOMstatmat2}.
  \begin{equation}\label{EqPopLOMss}
    (1+\bar{g}_n)\bm{\bar{\omega}} = \bm{\Omega}\bm{\bar{\omega}}
  \end{equation}

  \begin{proposition}
    If the age $s=1$ immigration rate is $i_1>-(1-\rho_0)f_1$ and the other immigration rates are strictly positive $i_s>0$ for all $s\geq 2$ such that all elements of $\bm{\Omega}$ are nonnegative, then there exists a unique positive real eigenvector $\bm{\bar{\omega}}$ of the matrix $\bm{\Omega}$, and it is a stable equilibrium.
  \end{proposition}

  \begin{proof}
    First, note that the matrix $\bm{\Omega}$ is square and non-negative.  This is enough for a general version of the Perron-Frobenius Theorem to state that a positive real eigenvector exists with a positive real eigenvalue. This is not yet enough for uniqueness. For it to be unique by a version of the Perron-Fobenius Theorem, we need to know that the matrix is irreducible. This can be easily shown. The matrix is of the form
    $$\bm{\Omega} =
    \begin{bmatrix}
      * & *  & * & \hdots & * & * & *\\
      * & * & 0 & \hdots & 0 & 0 & 0 \\
      0 & * & * & \hdots & 0 & 0 & 0 \\
      \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\
      0 & 0 & 0 & \hdots & *  & * & 0 \\
      0 & 0 & 0 & \hdots & 0 & * & *
    \end{bmatrix}
    $$
    Where each * is strictly positive. It is clear to see that taking powers of the matrix causes the sub-diagonal positive elements to be moved down a row and another row of positive entries is added at the top. None of these go to zero since the elements were all non-negative to begin with.
    $$\bm{\Omega}^2 =
    \begin{bmatrix}
      * & *  & * & \hdots & * & * & *\\
      * & * & * & \hdots & * & * & * \\
      0 & * & * & \hdots & 0 & 0 & 0 \\
      \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\
      0 & 0 & 0 & \hdots & *  & * & 0 \\
      0 & 0 & 0 & \hdots & 0 & * & *
    \end{bmatrix}; ~~~
    \bm{\Omega}^{S+E-1} =
    \begin{bmatrix}
      * & *  & * & \hdots & * & * & *\\
      * & * & * & \hdots & * & * & * \\
      * & * & * & \hdots & * & * & * \\
      \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\
      * & * & * & \hdots & *  & * & * \\
      0 & 0 & 0 & \hdots & 0 & * & *
    \end{bmatrix}
    $$
    $$\bm{\Omega}^{S+E} =
    \begin{bmatrix}
      * & *  & * & \hdots & * & * & *\\
      * & * & * & \hdots & * & * & * \\
      * & * & * & \hdots & * & * & * \\
      \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\
      * & * & * & \hdots & * & * & * \\
      * & * & * & \hdots & * & * & *
    \end{bmatrix}
    $$
    Existence of an $m \in \mathbb N $ such that $\left(\bf\Omega^m\right)_{ij} \neq 0 ~~ ( > 0)$ is one of the definitions of an irreducible (primitive) matrix. It is equivalent to saying that the directed graph associated with the matrix is strongly connected. Now the Perron-Frobenius Theorem for irreducible matrices gives us that the equilibrium vector is unique.

    We also know from that theorem that the eigenvalue associated with the positive real eigenvector will be real and positive. This eigenvalue, $p$, is the Perron eigenvalue and it is the steady state population growth rate of the model. By the PF Theorem for irreducible matrices, $| \lambda_i | \leq p$ for all eigenvalues $\lambda_i$ and there will be exactly $h$ eigenvalues that are equal, where $h$ is the period of the matrix. Since our matrix $\bf\Omega$ is aperiodic, the steady state growth rate is the unique largest eigenvalue in magnitude. This implies that almost all initial vectors will converge to this eigenvector under iteration.
  \end{proof}

  For a full treatment and proof of the Perron-Frobenius Theorem, see \citet{Suzumura:1983}. Because the population growth process is exogenous to the model, we calibrate it to annual age data for age years $s=1$ to $s=100$.

  Figure \ref{FigOrigVsFixSSpop} shows the steady-state population distribution $\bm{\bar{\omega}}$ and the population distribution after 120 periods $\bm{\hat{\omega}}_{120}$. Although the two distributions look very close to each other, they are not exactly the same.

  \begin{figure}[htbp]\centering \captionsetup{width=4.0in}
    \caption{\label{FigOrigVsFixSSpop}\textbf{Theoretical steady-state population distribution vs. population distribution at period $t=120$}}
    \fbox{\resizebox{4.0in}{3.0in}{\includegraphics{./images/OrigVsFixSSpop.png}}}
  \end{figure}

  Further, we find that the maximum absolute difference between the population levels $\hat{\omega}_{s,t}$ and $\hat{\omega}_{s,t+1}$ was $1.3852\times 10^{-5}$ after 160 periods. That is to say, that after 160 periods, given the estimated mortality, fertility, and immigration rates, the population has not achieved its steady state. For convergence in our solution method over a reasonable time horizon, we want the population to reach a stationary distribution after $T$ periods. To do this, we artificially impose that the population distribution in period $t=120$ is the steady-state. As can be seen from Figure \ref{FigOrigVsFixSSpop}, this assumption is not very restrictive. Figure \ref{FigImmRateChg} shows the change in immigration rates that would make the period $t=120$ population distribution equal be the steady-state. The maximum absolute difference between any two corresponding immigration rates in Figure \ref{FigImmRateChg} is 0.0028.

  \begin{figure}[htbp]\centering \captionsetup{width=4.0in}
    \caption{\label{FigImmRateChg}\textbf{Original immigration rates vs. adjusted immigration rates to make fixed steady-state population distribution}}
    \fbox{\resizebox{4.0in}{3.0in}{\includegraphics{./images/OrigVsAdjImm.png}}}
  \end{figure}

  The most recent year of population data come from \citet{Census:2015} population estimates for both sexes for 2013. We those data and use the population transition matrix \eqref{EqPopLOMstatmat2} to age it to the current model year of 2015. We then use \eqref{EqPopLOMstatmat2} to generate the transition path of the population distribution over the time period of the model. Figure \ref{FigPopDistPath} shows the progression from the 2013 population data to the fixed steady-state at period $t=120$. The time path of the growth rate of the economically active population $\tilde{g}_{n,t}$ is shown in Figure \ref{FigGrowthPath}.

  \begin{figure}[htbp]\centering \captionsetup{width=4.0in}
    \caption{\label{FigPopDistPath}\textbf{Stationary population distribution at periods along transition path}}
    \fbox{\resizebox{4.0in}{3.0in}{\includegraphics{./images/PopDistPath.png}}}
  \end{figure}

  \begin{figure}[htbp]\centering \captionsetup{width=4.0in}
    \caption{\label{FigGrowthPath}\textbf{Time path of the population growth rate $\tilde{g}_{n,t}$}}
    \fbox{\resizebox{4.0in}{3.0in}{\includegraphics{./images/GrowthPath.png}}}
  \end{figure}


\documentclass[12pt]{article}

\setlength{\parindent}{0pt}
\setlength{\parskip}{1ex}

\begin{document}

\section{Notation}
\begin{itemize}
    \item $t_n$: date step, $n = 1, \cdots, N$
    \item $m$: item (e.g.\ contract number), $m = 1, \cdots, M$
    \item $s(m,t_n)$: a series of marks.
    \item If $t_{n}$ is a roll date,
    \begin{itemize}
        \item $s(m,t_{n-1})$ becomes $s(m-1, t_n)$
        \item $s(1,t_{n-1})$ rolls off, i.e.\ disappears
        \item $s(M, t_{n})$ appears as a new item
    \end{itemize}
    \item $q(m,t_n)$: adjusted marks
    \item $\alpha(m, t)$: drift slope calculated for the $m$-th item as of $t$
    \item $T(m,t)$: the roll-off (i.e.\ maturity) date for the $m$-th item as of $t$
    \item $\phi(m,j)$: last drift slope tracked for the $m$-th item over the $j$-th rolling period $[r_j, r_{j+1})$
    \item $t' - t$: year-fraction from $t$ to $t'$ calculated using the Act 365 convention.

\end{itemize}

\section{Drift Adjustment}

\subsection{Remarking Criteria}

$t_n$ is a remarked date for the $m$-th item if
\begin{itemize}
    \item if $t_n$ is not a roll date:
    \begin{equation}
        \left|\frac{s(m, t_n) - s(m, t_{n-1})}{s(m, t_{n-1})} \right| > \epsilon
    \end{equation}
    \item if $t_n$ is a roll date:
    \begin{equation}
        \left|\frac{s(m, t_n) - s(m + 1, t_{n-1})}{s(m + 1, t_{n-1})} \right| > \epsilon
    \end{equation}
\end{itemize}
for some $\epsilon > 0$.

\subsection{Slope Function}

Given $\{s(m,t)\}_{m}$, the $m$-th slope $\alpha(m, t)$ is calculated by
\begin{eqnarray}
    \alpha(1, t) & = & \frac{s(1,t)}{T(1, t) - t} \\
    \alpha(m, t) & = & \frac{s(m,t) - s(m-1,t)}{T(m,t) - T(m-1,t)} \textrm{ for } m = 2, \cdots, M
\end{eqnarray}

{\bf Note}: For standardised products like Futures, we may assume $T(m, t) - T(m-1, t) = 90/365$ for simplicity.

{\bf Motivation}: Consider two marks: $s(m,t)$ and $s(m-1,t)$.
After time is elapsed by $\Delta := (T(m,t) - T(m-1,t))$, the time-to-roll-off of the first item becomes $(T(m-1,t)-t)$, which is the current time-to-roll-off of the second item.

Therefore, the natural adjustment slope is the one makes
$$q(m, t + \Delta) = s(m-1, t)$$
if no remarks occurs for all future time steps.

\subsection{Algorithm}

Consider a rolling period $I_j = \{t_n| t_n \in [r_j, r_{j+1}]\}$ where $r_j$ and $r_{j+1}$ are two adjacent rolling dates. Assume that $t_0 \in I_0$ and $t_N \in I_J$.

For each rolling period $j$ starting from $0$ to $J$:
\begin{enumerate}
    \item Initialisation: Let $t^j$ be the first date in the period, i.e.\ $t^j := \max\{t_0, r_j\}$
    \begin{itemize}
        \item If $j = 0$ or $t^j$ is a remark date, simply set
        \begin{itemize}
            \item $q(m,t^j) = s(m, t^j)$
            \item $\phi(m, j) = \alpha(m, t^j)$
        \end{itemize}
        for $m = 1, \cdots, M$.
        \item Otherwise, $j > 0$ and $t^j$ is not a remark date.

        For $m = 1, \cdots M-1$, set
        \begin{itemize}
            \item $\phi(m, j) = \phi(m+1, j-1)$, i.e.\ carry over the slope from the previous period
            \item $q(m,t^j) = q(m + 1, p(t^j)) - \phi(m,j) (t^j - p(t^j))$
        \end{itemize}
        where $p(t)$ is the previous time step in $\{t_n\}$.

        For $m = M$, borrowing from $m = M-1$, set
        \begin{itemize}
            \item $\phi(M, j) = \phi(M-1, j)$, i.e. the slope is carried over.
            \item $q(M,t^j) = s(M, t^j) + [q(M-1,t^j) - s(M-1,t^j)]$
        \end{itemize}

    \end{itemize}

    \item For each $t_n$ with $t^j < t_n < r_{j+1}$:
    \begin{itemize}
        \item If $t_n$ is a remarked date, set
        \begin{itemize}
            \item $q(m,t_n) = s(m, t_n)$
            \item $\phi(m, j) = \alpha(m, t_n)$
        \end{itemize}
        for $m = 1, \cdots, M$
        \item Otherwise i.e. not remarked

        \begin{itemize}
            \item $q(m,t_n) = q(m, t_{n-1}) - \phi(m,j) (t_{n} - t_{n-1})$

            \item $\phi(m, j)$: no update
        \end{itemize}
        for $m = 1, \cdots, M$

    \end{itemize}
\end{enumerate}

\end{document}$


\SecDef{expo}{Decomposition based on Finite Field Logarithm}

\subsection{BelT Block Cipher and its S-Box}

BelT is a block cipher from the Belarusian cryptographic standard~\cite{BELT}. It uses an 8-bit S-Box $H\colon \field{8} \to \field{8}$ which is given as a look-up table in the standard. The rationale behind this S-Box is not given in the standard, but instead in a separate rationale document by Agievich~\etal{}~\cite{BELTratio}.

\begin{proposition}[The BelT S-Box Construction, \cite{BELTratio} (translated)]
The look-up tables of the S-Box coordinate functions were chosen as different segments of length 255 of different linear recurrences defined by the irreducible polynomial $p(\lambda)$:
$$
p(\lambda) = \lambda^8 + \lambda^6 + \lambda^5 + \lambda^2 + 1.
$$
Additionally, a zero element was inserted in a fixed position of each segment.
\end{proposition}

Agievich also explains in~\cite{AA2004,AA2004rus} that such a construction is equivalent to an \emph{exponential} function in the finite field.

\newcommand\fexp[2]{#1^{(#2)}}
\newcommand\pexp[1]{\mathsf{exp}_{#1}}
\newcommand\plog[1]{\mathsf{log}_{#1}}

\begin{definition}
For a primitive element $w \in \fielde{n}$ let $x \mapsto \fexp{w}{x}$ be the map from $\field{n}$ to itself, obtained by raising $w$ to the power given by the integer represented by $x \in \field{n}$, and representing the result as an element of $\field{n}$, where the polynomial defining the field should be clear from context.

The exponential mapping can be turned into a permutation of $\field{n}$ by letting it map 0 to 0. Let $w \in \fielde{n}$ be a primitive element. Let $\pexp{w}$ be a permutation of $\field{n}$ given by:
$$
\pexp{w}(x) \eqdef \mapsto \begin{cases}
0, &~\text{if}~x = 0,\\
\fexp{w}{x}, &~\text{otherwise}.
\end{cases}
$$
Let $\plog{w}\colon \field{n} \to \field{n}$ denote the functional inverse of $\pexp{w}$:
$$
\plog{w} \eqdef \pexp{w}^{-1}.
$$
\end{definition}

\begin{remark}
In the S-Box $H$ used in BelT, the zero was inserted at $x = \hex{0a} \in \field{8}$ instead of 0.
\end{remark}

\FigRef{h_lat} shows the Jackson Pollock representations of the column and row frequency tables of the LAT of $H$. In the row frequency table several rows stick out, similarly to the special columns in the column frequency table of the LAT of $\pi$. This similarity suggests that there might be a relation between $H$ and the inverse of $\pi$.

Since this chapter is devoted to $\pi$, for a closer analysis of the S-Box used in BelT, I refer to our paper~\cite{OurKuz2}.

\FigTex{h_lat.tex}

\subsection{Exponential Behaviour of $\pi$}

An exponential function $x \mapsto w^x$ has the following property: for all $x,c \in \field{8}$
$$
\fexp{w}{x + c} = \fexp{w}{x} \fmult \fexp{w}{c}.
$$
This property can be used to distinguish exponential permutations or functions close to them. However, the integer addition can be partially hidden by a whitening affine layer. Still, a strong property can be observed if the addition is approximated by \txor{}. Indeed, for a unit vector $e_i$ of $\field{n}$ and all $x \in \field{8}$,
$$
x \oplus e_i =
\begin{cases}
x \boxplus e_i, & ~\text{if}~ \inprod{x, e_i} = 0,\\
x \boxminus e_i, & ~\text{if}~ \inprod{x, e_i} = 1.
\end{cases}
$$

An advantage of this approximation is that the \txor{} with $e_i$ after a whitening input linear map $L \in \linbij{n}$ maps back to the \txor{} with $L^{-1}(e_i)$ before the application of $L$. And indeed such behaviour can be observed in $\pi$! By an exhaustive search over the parameters, the following relations were found in $\pi$.

\begin{observation}
Let $c \in \pround{\hex{12}, \hex{26}, \hex{24}, \hex{30}}$. For any $i \in \seg{1}{4}$
\begin{equation}
\Label{eq:exprel}
\Pr_{x \in \field{8}} \psquare{
    \begin{aligned}
    \pi^{-1}(x \oplus c_i) = \pi^{-1}(x) \fmult X^{2^{i-1}} && ~\text{or}\\
    \pi^{-1}(x \oplus c_i) = \pi^{-1}(x) \fdiv X^{2^{i-1}}. &&
    \end{aligned}
} = 240/256,
\end{equation}
where multiplication $\fmult$ and division $\fdiv$ are performed in the finite field $\fielde{8} \simeq \field{}[X]/(X^8 + X^4 + X^3 + X^2 + 1)$ and $X$ defines a primitive element.
\end{observation}

\newcommand\taupi{\plog{X} \circ \pi^{-1}}

This strong property suggests that the output side of $\pi^{-1}$ is not masked by a random linear layer. Otherwise, the multiplication and the division by $X^{2^i}$ would be masked and not triggered by the constant \txor{} in the input. Therefore, we assume that the output of $\pi^{-1}$ is the output of an exponential function composed with some simple layer. The simple layer then can be analyzed separately as $\taupi$.

\subsection{Decomposing the Arithmetic Layer}

Our hypothesis was that there  is a linear whitening layer mapping all $c_i$ to unit vectors. Equation~\Ref{eq:exprel} suggests that the unit vectors are consecutive powers of 2. Let $\alpha \in \linbij{8}$ be given by
$$
\alpha \eqdef \matb{
    1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
    0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \\
    0 & 0 & 0 & 0 & 1 & 1 & 1 & 0 \\
    0 & 0 & 0 & 0 & 1 & 0 & 0 & 1 \\
    0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 \\
    0 & 0 & 0 & 0 & 0 & 1 & 1 & 0 \\
    0 & 0 & 0 & 0 & 0 & 0 & 1 & 1 \\
    0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 \\
}^{-1}.
$$
It is such that for all $i \in \pset{1,2,3,4}$, $\alpha(c_i)$ is the unit vector corresponding to $2^{i-1}$. Preimages of the other four unit vectors were chosen randomly to complete the map. The look-up table of $\taupi \circ \alpha^{-1}$ is given in \TabRef{taupialpha}.

\FigTex{taupialpha.tex}

The rows of \TabRef{taupialpha} are clearly structured. We observe that each row can be sorted by modifying the linear mapping $\alpha$ (except the zero value). Indeed, let $\beta \in \linbij{8}$ be given by
$$
\beta \eqdef \matb{
    1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
    0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \\
    0 & 0 & 0 & 0 & 1 & 1 & 1 & 0 \\
    1 & 0 & 0 & 0 & 1 & 0 & 0 & 1 \\
    0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 \\
    0 & 1 & 0 & 0 & 0 & 1 & 1 & 0 \\
    1 & 0 & 1 & 0 & 0 & 0 & 1 & 1 \\
    0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 \\
}.
$$
Then the look-up table of $\taupi \circ \beta^{-1}$ is the same as the look-up table of $\taupi \circ \alpha^{-1}$ with sorted rows.
It is shown in \TabRef{taupibeta}.

\FigTex{taupibeta.tex}

Furthermore, the rows can be reordered by applying a 4-bit nonlinear mapping to the left branch. Let $q$ be a permutation of $\field{4}$ given by its lookup table
$$
\lookup{q} \eqdef (12, 2, 9, 10, 13, 6, 3, 5, 11, 4, 8, 15, 14, 7, 0, 1).
$$
Let $q_L$ be a permutation of $\field{8}$ made by applying $q$ to the left half of the input:
$$
q_L(x, y) \eqdef (q(x), y)).
$$
Then the look-up table of $\taupi \circ \beta^{-1} \circ q_L^{-1}$ has a very simple structure. It is shown in \TabRef{taupibetaq}. This structure has a simple arithmetic expression. As a result, an algorithmic decomposition of $\pi^{-1}$ can be deduced. It is given in Algorithm~\Ref{alg:pinv}.

\begin{remark}
It is also possible to define $q$ such that it moves the row $(17, 34, \ldots, 255, 0)$ to the end of the table. This change results in similar expressions.
\end{remark}

\FigTex{taupibetaq.tex}

\begin{algorithm}
    \caption{\Label{alg:pinv}
        Computing the inverse of $\pi$: $y = \pi^{-1}(x)$.
    }
    \begin{algorithmic}[1]
        \State{$(l, r) \gets \beta(x)$}
        \State{$l \gets q(l)$}
        \If{$l = 0$}
            \State{$z \gets 17 \times ((r + 1) \mod 16)$} \Comment{integer arithmetic}
        \Else
            \State{$z \gets 17 \times l + r - 16$} \Comment{integer arithmetic}
        \EndIf
        \State{$y \gets \plog{X}(z)$}
        \State{\Return{$y$}}
    \end{algorithmic}
\end{algorithm}

\subsection{Obtaining a Decomposition of $\pi$}

\newcommand\arith{\widehat{\pi}}
Let $\arith$ be the permutation of $\field{8}$ given by
$$
    \arith \eqdef \plog{X} \circ \pi^{-1} \circ \beta^{-1} \circ q_L^{-1}.
$$
It corresponds to the arithmetic part of the decomposition. Observe that it has a TU-decomposition as $\pi$ has itself (see Section~\Ref{sec:tu}).

\begin{observation}
There exist permutations of $\field{4}$ $T_{\hex{0}},\ldots,T_{\hex{f}}$ and $U_{\hex{0}}, \ldots, U_{\hex{f}}$ such that for all $l,r \in \field{4}$
$$
\arith(l,r) = U_{T_l(r)}(l), T_l(r).
$$

\FigTex{arith-tu.tex}

Such $T,U$ are given in \TabRef{arith-tu}. They can also be expressed arithmetically:

\begin{equation*}
    T_k(x) = \begin{cases}
    x + k, \mbox{ if } k \ne 0, \\
    x + k + 1, \mbox{ otherwise},
    \end{cases}
    U_k(x) = \begin{cases}
    ((x - k - 1) \mod{15}) + k + 1, \mbox{ if } x \ne 0, \\
    k, \mbox{ otherwise}.
    \end{cases}
\end{equation*}
\end{observation}

$T$ and $U$ can be inverted separately. By further using the finite field logarithm and inverses of $\beta$ and $q_L$, the logarithmic decomposition of $\pi$ is obtained. The corresponding algorithm is given in Algorithm~\Ref{alg:forward} and graphical representation is given in \FigRef{logarithmic}.

\begin{algorithm}
    \caption{Computing the S-Box $y = \pi(x)$ using the logarithmic decomposition.}
    \Label{alg:forward}
    \begin{algorithmic}
        \State{$(l, r) \gets \plog{X}(x)$}
        \State{$l \gets l - r$}
        \If{$l = 0$}
            \State{$r \gets r - 1$}
        \Else
            \State{$l \gets (l + r - 1) \mod 15 + 1$}
        \EndIf
        \State{$r \gets r - l$}
        \State{$l \gets q^{-1}(l)$}
        \State{$y \gets \beta^{-1}(l||r)$}
        \State{\Return{$y$}}
    \end{algorithmic}
\end{algorithm}

\FigTex{logarithmic}


\section{Hierarchization on Full Grids (Unidirectional Principle)}
\label{sec:42fullGrids}

If $\sgset$ is a full grid $\fgset{\*l}$
(see \cref{sec:21nodalSpaces}),
the well-known \up
can be used to apply $\linop$ to input data $\vlinin$.
As shown in \cref{fig:unidirectionalPrinciple} for a sparse grid,
the idea of the \up
is to apply the corresponding one-dimensional operators on the
one-dimensional subgrids (the \term{poles}) of $\sgset$,
which is repeated for all dimensions.
In this section, we first formulate the \up for
general linear operators $\linop$ and then prove its correctness for
the case $\linop = \intpmatinv$ of hierarchization.
The correctness for arbitrary tensor product operators
will follow from \cref{sec:45spatAdaptiveUP}.

\begin{figure}
  \includegraphics{unidirectionalPrinciple_1}%
  \caption[%
    Unidirectional principle%
  ]{%
    Application of a linear operator $\linop$
    on two-dimensional sparse grid data with the unidirectional principle.
    \emph{Left:}
    The univariate operator $\upopuv{1}{\lisetpole}$ is applied on
    the input data $\vlinin$
    along poles $\lisetpole$ of the first dimension $x_1$.
    \vspace{-0.2em}%
    \emph{Center:}
    The univariate operator $\upopuv{2}{\lisetpole}$ is applied on the
    resulting intermediate data $\vlinout[(1)]$
    along poles $\lisetpole$ of the second dimension $x_2$.
    \emph{Right:}
    Final values $\vlinout = \linop[\vlinin]$ after the application
    on both dimensions.
    All grid points of the same color are part of the same pole $\lisetpole$
    (equivalence classes of $\samepole{t}$ in
    \cref{alg:unidirectionalPrinciple}).%
  }%
  \label{fig:unidirectionalPrinciple}%
\end{figure}

\paragraph{Unidirectional principle and its correctness}

We state the \up in \cref{alg:unidirectionalPrinciple}.
The algorithm is given a permutation $(t_1, \dotsc, t_d)$ of $(1, \dotsc, d)$
that specifies the order of dimensions in which the \up should be applied.
We denote with $\upopuv{t_j}{\lisetpole}$ the one-dimensional version of $\linop$
applied in dimension $t_j$ ($j = 1, \dotsc, d$) on the pole $\lisetpole$.
Formally, a pole is an equivalence class of the
\term{pole equivalence relation} $\samepole{t_j}$ on $\liset$:
\begin{equation}
  \label{eq:poleEquivalenceRelation}
  \*k' \samepole{t_j} \*k'' \iff \*k'_{-t_j} = \*k''_{-t_j},\quad
  \*k', \*k'' \in \liset.
\end{equation}
We prove the correctness of the \up with the following invariant:

\begin{algorithm}
  \begin{algorithmic}[1]
    \Function{$\vlinout = \texttt{unidirectionalPrinciple}$}{%
      $\vlinin$, $\liset$, $(t_1, \dotsc, t_d)$%
    }
      \State{$\vlinout[(0)] \gets \vlinin$}
      \For{$j = 1, \dotsc, d$}
        \For{$\lisetpole \in \eqclasses{\liset}{\samepole{t_j}}$}
          \State{%
            $(\linout[(j)]{\*k})_{\*k \in \lisetpole} \gets
            \upopuv{t_j}{\lisetpole}
            \bracket*{(\linout[(j-1)]{\*k})_{\*k \in \lisetpole}}$%
          }
          \Comment{apply univariate operator on pole}%
          \label{line:algUnidirectionalPrinciple1}
        \EndFor{}
      \EndFor{}
      \State{$\vlinout \gets \vlinout[(d)]$}
    \EndFunction{}
  \end{algorithmic}
  \caption[%
    Unidirectional principle%
  ]{%
    Application of a tensor product operator $\linop$ with
    the unidirectional principle.
    Inputs are
    the vector $\vlinin = (\linin{\*k})_{\*k \in \liset}$ of input data,
    the set $\liset$ of grid indices, and
    the permutation $(t_1, \dotsc, t_d)$ specifying the order in which
    the one-dimensional operators $\upopuv{t_j}{\lisetpole}$ should be applied.
    The output is the vector $\vlinout = (\linout{\*k})_{\*k \in \liset}$
    of output data.%
  }%
  \label{alg:unidirectionalPrinciple}%
\end{algorithm}

\begin{proposition}[invariant of unidirectional principle for hierarchization]
  \label{prop:invariantUnidirectionalPrinciple}
  Let $\linop$ be the hierarchization operator on a full grid,
  i.e.,
  $\linop = \intpmatinv$,
  $\vlinin = (\fcnval{\*k})_{\*k \in \liset}$,
  $\vlinout = (\surplus{\*k})_{\*k \in \liset}$,
  $\upopuv{t_j}{\lisetpole}$ is the univariate interpolation operator
  $\intpmatuvinv{t_j}$, and
  $\liset = \{\*0, \dotsc, \*2^\*l\}$
  corresponds to a full grid $\fgset{\*l}$ of level $\*l$.
  After iteration $j$ of \cref{alg:unidirectionalPrinciple}
  ($j = 1, \dotsc, d$), it holds for $T \ceq (t_1, \dotsc, t_j)$
  \begin{equation}
    \sum_{\*k_T=\*0}^{\*2^{\*l_T}}
    \linout[(j)]{(\*k_T,\*k'_{-T})} \basis{\*k_T}(\gp{\*k'_T})
    = \fcnval{\*k'},\quad
    \*k' = \*0, \dotsc, \*2^\*l,
  \end{equation}
  where $(\*k_T,\*k'_{-T})$ is shorthand for $\*k''$
  with $k''_t \ceq k_t$ if $t \in T$ and $k''_t \ceq k'_t$ if $t \notin T$.
\end{proposition}

\begin{proof}
  We prove the assertion by induction over $j = 1, \dotsc, d$.
  We set $T' \ceq (t_1, \dotsc, t_{j-1})$,
  $T \ceq (t_1, \dotsc, t_{j-1}, t_j)$,
  and we exploit the tensor product structure of the basis
  to write the \lhs of the assertion for $j$
  and arbitrary $\*k' = \*0, \dotsc, \*2^\*l$ as
  \begin{equation}
    \label{eq:proofPropInvariantUnidirectionalPrinciple1}
    \sum_{\*k_T=\*0}^{\*2^{\*l_T}}
    \linout[(j)]{(\*k_T,\*k'_{-T})} \basis{\*k_T}(\gp{\*k'_T})
    = \sum_{\*k_{T'}=\*0}^{\*2^{\*l_{T'}}}
    \basis{\*k_{T'}}(\gp{\*k'_{T'}})
    \sum_{k_{t_j}=0}^{2^{l_{t_j}}}
    \linout[(j)]{(\*k_T,\*k'_{-T})} \basis{k_{t_j}}(\gp{k'_{t_j}}).
  \end{equation}
  If we choose the equivalence class
  $\lisetpole \ceq \eqclass{(\*k_T,\*k'_{-T})}{\samepole{t_j}}$
  ($\*k_T$ arbitrary),
  then the inner sum over $k_{t_j}$ equals
  \begin{equation}
    \label{eq:proofPropInvariantUnidirectionalPrinciple2}
    \sum_{\*k'' \in \lisetpole}
    \linout[(j)]{\*k''} \basis{k''_{t_j}}(\gp{k'_{t_j}})
    = \paren*{
      (\upopuv{t_j}{\lisetpole})^{-1}
      \bracket*{(\linout[(j)]{\*k''})_{\*k'' \in \lisetpole}}
    }_{k'_{t_j}}
    = \linout[(j-1)]{(\*k_{T'},\*k'_{-T'})}
  \end{equation}
  by \cref{line:algUnidirectionalPrinciple1} of
  \cref{alg:unidirectionalPrinciple}.
  We can conclude that the \lhs
  \cref{eq:proofPropInvariantUnidirectionalPrinciple1} equals
  \begin{equation}
    \sum_{\*k_{T'}=\*0}^{\*2^{\*l_{T'}}}
    \linout[(j-1)]{(\*k_{T'},\*k'_{-T'})}
    \basis{\*k_{T'}}(\gp{\*k'_{T'}}),
  \end{equation}
  which, by the induction hypothesis, equals $\fcnval{\*k'}$ as desired
  (if $j > 1$).
  The same reasoning for
  \eqref{eq:proofPropInvariantUnidirectionalPrinciple2} can be used
  to establish the base case for $j = 1$.
\end{proof}

\begin{shortcorollary}[%
  correctness of unidirectional principle for hierarchization%
]
  \label{cor:algUnidirectionalPrincipleCorrectness}
  \Cref{alg:unidirectionalPrinciple}
  is correct for hierarchization on full grids.
\end{shortcorollary}

\begin{proof}
  We apply \cref{prop:invariantUnidirectionalPrinciple} for $j = d$ to obtain
  $\sum_{\*k=\*0}^{\*2^\*l}
  \linout[(j)]{\*k} \basis{\*k}(\gp{\*k'})
  = \fcnval{\*k'}$
  for all $\*k' = \*0, \dotsc, \*2^\*l$, i.e.,
  the $\linout[(j)]{\*k}$ are the correct interpolation coefficients
  according to \eqref{eq:hierarchizationProblem}.
\end{proof}

\paragraph{Complexity}

We compare the complexity of the \up for hierarchization with
the direct solution of the system \eqref{eq:hierarchizationSLE} of
linear equations.
If we assume that $d$ is constant and that
$\linop$ and $\upopuv{t_j}{\lisetpole}$ apply Gaussian elimination to
solve the multivariate and univariate systems, respectively,
then directly solving \eqref{eq:hierarchizationSLE} takes
$\landauO{\ngp^2 (\ngp + d)}$ time and
$\landauO{\ngp^2}$ memory.
In contrast, the \up only requires
$\landauO{\ngp \sum_t \ngp_t^2}$ time%
\footnote{%
  There are $\ngp/\ngp_t$ poles in the
  $t$-th iteration of \cref{alg:unidirectionalPrinciple}.
  Each pole requires the solution of an $\ngp_t \times \ngp_t$ linear system,
  which takes $\landauO{\ngp_t^3}$ time.%
}
and $\landauO{\max\{N_1^2, \dotsc, N_d^2, N\}}$ memory,
where $\ngp_t$ is the grid size
$\setsize{\{k_t \mid \*k \in \liset\}}$ in dimension $t = 1, \dotsc, d$.
The dependency from the univariate grid sizes $\ngp_t$ instead of $\ngp$
makes the \up significantly less computationally expensive.
As already mentioned,
the \up is even more efficient in the piecewise linear case,
where the univariate interpolation operators can be applied
in-place.
Hence, it only needs $\landauO{Nd}$ time and
$\landauO{N}$ memory in this case.


\section{Hyperparameters}
\label{parameters}
This section proposes an equation to evaluate the build-trace trade-off of the PHR algorithm. Two possible solutions for the optimization problem are them presented in the form of grid search and Bayesian optimization. As established in the previous section, PHR depends on the parameters $\alpha$ and $\delta$ describing initial cut size and shrink rate. These parameters can be set to adjust the trade-off between build time and trace performance. However, the optimal parameters can differ between scenes and view points, especially when dealing with dynamic scenes and refitted auxiliary trees.
% TODO: Add figure that shows different optimal parameters and prove thesis
Frame size also plays a role as higher resolutions profit more from well optimized bounding volume hierarchies while the hit taken by the extended build time is not as significant given the overall higher computational effort.
\subsection{Evaluation}
\label{evaluation}
Ultimately, the overall frame time including BVH build time and render duration needs to be minimized to unlock the full potential of progressive hierarchical refinement. Stopping the actual execution times is neither efficient nor reliable enough, so an efficient metric that makes the trade-off quantifiable is needed.

A number of metrics have been proposed to estimate the quality of a given bounding volume hierarchy. A well known cost model based on the surface area heuristic, taken from \cite{meister21survey}, is given by the recurrence equation
\[
c(N) =
    \begin{cases}
        c_T + \sum_{N_c}P(N_c|N)c(N_c) &\quad\text{if }N\text{ is interior node,}\\
        c_I|N|&\quad\text{otherwise}\\
    \end{cases}
\]
where $C(N)$ is the cost of the subtree with root $N$, $N_c$ is a child of node $N$, $P(N_c|N)$ is the conditional probability of traversing node $N_c$ when $N$ is hit and $|N|$ is the number of primitives in the subtree with root $N$.

Constants $c_T$ and $c_I$ express the average cost of a traversal step and ray-primitive intersection, respectively. Utilizing the micro-benchmarking capabilities of Go revealed that a triangle intersection takes approximately 7 nanoseconds on average, while a traversal step takes around 12 nanoseconds. Consequently, these constants are set to $c_T = 2$ and $c_I = 1$, roughly representing the ratios between those values.
The conditional probabilities of traversing a node are expressed using the surface area heuristic\cite{goldsmith_automatic_1987,macdonald_heuristics_1990}
\[
P(N_c|N)^{SAH} = \frac{SA(N_c)}{SA(N)}
\]
where $SA(N)$ and $SA(N_c)$  are the bounding box surface areas of nodes $N$ and $N_c$, respectively.

Evaluating the build complexity of PHR given a set of parameters and an arbitrary scene is not as trivial, as there is no clear correlation between the parameter values and associated build times.
\begin{figure}
    \centering
    \subcaptionbox{Build Time}{\includegraphics[width=0.45\textwidth]{images/build_time.pdf}}
    \hfill
    \subcaptionbox{Build Cost}{\includegraphics[width=0.45\textwidth]{images/build_cost.pdf}}
    \caption{Both the build time and corresponding build cost for the San Miguel scene. This corresponds to a correlation of 0.994.}
    \label{fig:build_cost}
\end{figure}
For scenes with medium to high complexity, the tree size of the resulting BVH could be used as a metric with an average correlation coefficient of around 0.98 between tree size and build time. This correlation does not hold true for simpler scenes though, as trees stop growing once no favorable splits can be found anymore.
A better metric is the total sum of all cut sizes, as seen in figure \ref{fig:build_cost}, which can be obtained through a minimal adjustment of the progressive hierarchical refinement algorithm. The surface area heuristic evaluation is the most expensive part of PHR and thus the total number of evaluated nodes directly correlates with the execution duration. Using an atomic counter, the performance penalty is very marginally to non existing. Furthermore, an alternative implementation can be used for the optimization process. The correlation between this number and the BVH construction time averages to 0.99 across all tested scenes.

Let the build cost $b$ be defined as
\[
    b(\alpha, \delta) = \sum_{cut\in PHR'(\alpha, \delta)}|cut|
\]
where $b(\alpha, \delta)$ is cost of a PHR execution given the parameters $\alpha$ and $\delta$, $PHR'(\alpha, \delta)$ is a execution of the progressive hierarchical refinement algorithm resulting in a set of all cuts processed and $|cut|$ is the length of given cut.

I propose an equation to evaluate the combined cost of a frame rendered using the PHR algorithm, which also factors in the build-trace trade-off. Given a pair of parameters $\alpha$ and $\delta$, this cost is defines as
\[
    e(\alpha,\delta) = ||b(\alpha,\omega)|| + \omega^2 ||c(PHR(\alpha,\omega))||
\]
with $||b(\alpha,\delta)||$ being the normalized build cost and $||c(PHR(\alpha,\omega))||$ the normalized SAH cost of the resulting BVH. The coefficient $\omega$ functions as a weight determining to what extend trace performance should be favored over build duration. For $\omega=1$, both trace and build performance are weighed equally, $\omega>1$ favors higher trace speed and $\omega<1$ encourages faster build times. These trade-off depends on scene complexity and frame size, so $\omega$ is defined as:
\[
    \omega = (\frac{|prim|}{\omega_p})^2 * (\frac{w*h}{\omega_r})
\]
where $|prim|$ is the total number of primitives in the scene and $w$ and $h$ are width and height of the rendered frame, respectively. $\omega_p$ and $\omega_r$ are constants depending on the performance of the used path tracer and the targeted frame rate, so they stay constant across all scenes. $\omega_p$ expresses a number of primitives and $\omega_r$ a frame size at which the trade-off should be equal, i.e. $\omega=1$. Once those numbers are exceeded, BVH quality is favored increasingly.

Combining the equation with the previously mentioned metrics and given a search space over the parameters $\alpha$ and $\delta$ (e.g. $\alpha\in A=[0.4,0.6], \delta\in D=[5,10]$) leads to the expression:
\[
    e(\alpha,\delta) =
        \frac{b(\alpha, \delta)}
        {\max_{\alpha\in A,\delta\in D}b(\alpha, \delta)}
        + \omega^2
        \frac{c(PHR(\alpha, \delta))}
        {\max_{\alpha\in A,\delta\in D}c(PHR(\alpha, \delta))}
\]
where $PHR(\alpha,\delta)$ is a execution of the PHR algorithm using the parameters $\alpha$ and $\delta$ resulting in the root node of a bounding volume hierarchy.
\clearpage
\subsection{Grid Search}
The brute force approach grid search is the obvious first choice to solve this optimization problem. A number of possible values is chosen for each parameter (e.g. $\alpha\in\{0.45,0.5,0.55\}$ $\delta\in\{5,6,7,8\}$) and all possible combinations are tested and evaluated. By postponing the evaluation of $e(\alpha,\delta)$ until all individual cost evaluations $b(\alpha, \delta)$ and $c(PRH(\alpha,\delta))$ are available makes their maxima directly accessible, which is another advantage of grid search.

Usually, grid search would be too costly for such a time critical task. In this case however, only two parameters need to be optimized and the search space is relatively small making grid search a potentially viable approach.

\subsection{Bayesian Optimization}
A more efficient approach compared to grid search is Bayesian optimization\cite{pelikan99boa}, which explores the search space by taking previous observations into account. This is done by placing a prior probability distribution over the objective function $e(\alpha,\delta)$. First, parameters $\alpha$ and $\delta$ are chosen by random in the given search space. Following their evaluation, the prior is updated to form a posterior distribution. Based on this posterior distribution, an acquisition function is constructed to select the next point worth exploring.

In particular, the Bayesian optimizer\cite{ou19bo} utilizes a Gaussian process to define the prior/posterior distribution, and expected improvement is used as exploration strategy.

One issue compared to the aforementioned grid search approach is, that the maximum BVH and build costs are not directly available when evaluating $e(\alpha,\delta)$. This is solved by running PHR once with the lowest possible parameters in the search space to obtain the maximum BVH cost and once with the highest possible parameters to obtain the maximum build cost. These numbers correspond to the respective maxima very reliably and there evaluations can also be factored into the prior probability distribution.
\cleardoublepage


% This is part of the TFTB Reference Manual.
% Copyright (C) 1996 CNRS (France) and Rice University (US).
% See the file refguide.tex for copying conditions.

\markright{loctime}
\section*{\hspace*{-1.6cm} loctime}

\vspace*{-.4cm}
\hspace*{-1.6cm}\rule[0in]{16.5cm}{.02cm}
\vspace*{.2cm}

{\bf \large \sf Purpose}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
Time localization characteristics.
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf Synopsis}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
[tm,T] = loctime(x)
\end{verbatim}
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf Description}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
        {\ty loctime} computes the time localization characteristics of
        signal {\ty x}. The definition used for the averaged time
        and the time spreading are the following\,:
\begin{eqnarray*}
t_m &=& \frac{1}{E_x}\ \int_{-\infty}^{+\infty} t\ |x(t)|^2\ dt \\ T &=&
2\ \sqrt{\frac{\pi}{E_x}\ \int_{-\infty}^{+\infty} (t-t_m)^2\ |x(t)|^2\ dt}
\end{eqnarray*}
where $E_x$ is the energy of the signal. With this definition (and the one
used in {\ty locfreq}), the Heisenberg-Gabor inequality writes $B\ T\geq
1$.\\

\hspace*{-.5cm}\begin{tabular*}{14cm}{p{1.5cm} p{8.5cm} c}
Name & Description & Default value\\
\hline
   {\ty x}   & signal\\
 \hline       {\ty tm}  & averaged time center\\
        {\ty T}   & time spreading\\

\hline
\end{tabular*}

\end{minipage}
\vspace*{1cm}

{\bf \large \sf Examples}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
Here is an example of signal which corresponds to the lower bound of the
Heisenberg-Gabor inequality.
\begin{verbatim}
         z=amgauss(160,80,50);
         [tm,T]=loctime(z);
         [fm,B]=locfreq(z);
         [tm,T,fm,B,T*B]
         ans =
               80.0000   50.0000   0.0000   0.0200   1
\end{verbatim}
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf See Also}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
locfreq.
\end{verbatim}
\end{minipage}


\chapter{   Data Analytics}
% CHAPTER SETTINGS
\graphicspath{{./images/data_analytics/}

\section{Time Series analysis}

\subsection{Transforms for Time Series Data}

% for code
% https://machinelearningmastery.com/machine-learning-data-transforms-for-time-series-forecasting/

% Power Transform
% A power transform removes a shift from a data distribution to make the distribution more-normal (Gaussian).

% On a time series dataset, this can have the effect of removing a change in variance over time.

% ####

% Difference Transform

% A difference transform is a simple way for removing a systematic structure from the time series.

% For example, a trend can be removed by subtracting the previous value from each value in the series. This is called first order differencing. The process can be repeated (e.g. difference the differenced series) to remove second order trends, and so on.

% A seasonal structure can be removed in a similar way by subtracting the observation from the prior season, e.g. 12 time steps ago for monthly data with a yearly seasonal structure.

% ####

% Standardization

% Standardization is a transform for data with a Gaussian distribution.

% It subtracts the mean and divides the result by the standard deviation of the data sample. This has the effect of transforming the data to have mean of zero, or centered, with a standard deviation of 1. This resulting distribution is called a standard Gaussian distribution, or a standard normal, hence the name of the transform.

% ####

% Normalization

% ormalization is a rescaling of data from the original range to a new range between 0 and 1.

% As with standardization, this can be implemented using a transform object from the scikit-learn library, specifically the MinMaxScaler class. In addition to normalization, this class can be used to rescale data to any range you wish by specifying the preferred range in the constructor of the object.

% \subsection{Considerations for Model Evaluation}

% Considerations for Model Evaluation

% We have mentioned the importance of being able to invert a transform on the predictions of a model in order to calculate a model performance statistic that is directly comparable to other methods.

% Additionally, another concern is the problem of data leakage.

% Three of the above data transforms estimate coefficients from a provided dataset that are then used to transform the data. Specifically:

% Power Transform: lambda parameter.
% Standardization: mean and standard deviation statistics.
% Normalization: min and max values.
% These coefficients must be estimated on the training dataset only.

% Once estimated, the transform can be applied using the coefficients to the training and the test dataset before evaluating your model.

% If the coefficients are estimated using the entire dataset prior to splitting into train and test sets, then there is a small leakage of information from the test set to the training dataset. This can result in estimates of model skill that are optimistically biased.

% As such, you may want to enhance the estimates of the coefficients with domain knowledge, such as expected min/max values for all time in the future.

% Generally, differencing does not suffer the same problems. In most cases, such as one-step forecasting, the lag observations are available to perform the difference calculation. If not, the lag predictions can be used wherever needed as a proxy for the true observations in difference calculations

% \subsection{Order of Data Transforms}

% a suggested ordering for data transforms is as follows:

% Power Transform.
% Seasonal Difference.
% Trend Difference.
% Standardization.
% Normalization.
% Obviously, you would only use the transforms required for your specific dataset.

% Importantly, when the transform operations are inverted, the order of the inverse transform operations must be reversed. Specifically, the inverse operations must be performed in the following order:

% Normalization.
% Standardization.
% Trend Difference.
% Seasonal Difference.
% Power Transform.


\documentclass[../report/main.tex]{subfiles}

\begin{document}

% The asterix after \subsection disables section numbering
\subsection*{Problem 1 Part C}
Determine the number of refrigerators to be shipped from plants to warehouses, and then warehouses to retailers to minimize the cost.  \textcolor{blue}{For part C warehouse 2 is limited to just 100 units per week.  Changes to the problem statement or solution relative to Problem 1 Part A are highlighted in blue.}

\subsection*{Solution}
\begin{itemize}
	\item In all 1000 units will travel through the network at a minimum cost of \textcolor{blue}{\$18300}.
	\newline
	\item Ship 150 units from plant \#1 to warehouse \#1 at a cost of \$1500.
	\item Ship \textcolor{blue}{350} units from plant \#2 to warehouse \#1 at a cost of \textcolor{blue}{\$3850}.
	\item Ship \textcolor{blue}{100} units from plant \#2 to warehouse \#2 at a cost of \textcolor{blue}{\$800}.
	\item Ship   \textcolor{blue}{0} units from plant \#3 to warehouse \#2 at a cost of \textcolor{blue}{\$   0}.
	\item Ship \textcolor{blue}{250} units from plant \#3 to warehouse \#3 at a cost of \textcolor{blue}{\$2250}.
	\item Ship 150 units from plant \#4 to warehouse \#3 at a cost of \$1200.
	\newline
	\item Ship 100 units from warehouse \#1 to retailer \#1 at a cost of \$ 500.
	\item Ship 150 units from warehouse \#1 to retailer \#2 at a cost of \$ 900.
	\item Ship 100 units from warehouse \#1 to retailer \#3 at a cost of \$ 700.
	\item \textcolor{blue}{Ship 150 units from warehouse \#1 to retailer \#4 at a cost of \$1500}.
	\item Ship  \textcolor{blue}{50} units from warehouse \#2 to retailer \#4 at a cost of \textcolor{blue}{\$ 400}.
	\item Ship  \textcolor{blue}{50} units from warehouse \#2 to retailer \#5 at a cost of \textcolor{blue}{\$ 500}.
	\item \textcolor{blue}{Ship 150 units from warehouse \#3 to retailer \#5 at a cost of \$1800.}
	\item Ship 150 units from warehouse \#3 to retailer \#6 at a cost of \$1800.
	\item Ship 100 units from warehouse \#3 to retailer \#7 at a cost of \$ 600.
	\newline
	\item 150 total units will leave plant \#1 (capacity is 150).
	\item 450 total units will leave plant \#2 (capacity is 450).
	\item 250 total units will leave plant \#3 (capacity is 250).
	\item 150 total units will leave plant \#4 (capacity is 150).
	\newline
	\item \textcolor{blue}{500} total units will enter warehouse \#1, \textcolor{blue}{500} units will leave.
	\item \textcolor{blue}{100} total units will enter warehouse \#2, \textcolor{blue}{100} units will leave.
	\item \textcolor{blue}{400} total units will enter warehouse \#3, \textcolor{blue}{400} units will leave.
	\newline
	\item 100 total units will enter retailer \#1 (demand is 100).
	\item 150 total units will enter retailer \#2 (demand is 150).
	\item 100 total units will enter retailer \#3 (demand is 100).
	\item 200 total units will enter retailer \#4 (demand is 200).
	\item 200 total units will enter retailer \#5 (demand is 200).
	\item 150 total units will enter retailer \#6 (demand is 150).
	\item 100 total units will enter retailer \#7 (demand is 100).
\end{itemize}
\subsection*{Linear Program Formulation}
\begin{enumerate}[1.]
	\item Overall idea of problem
	\begin{itemize}
		\item Refrigerators moving from $n=4$ plants to $q=3$ warehouses to $m=7$ retailers.
		\item Not all plants deliver to all warehouses.
		\item Not all warehouses deliver to all retailers.
		\item Costs of shipping from plants to warehouses vary by pair.
		\item Costs of shipping from warehouses to retailers vary by pair.
		\item Each plant has a capacity in terms of number of refrigerators it can supply.
		\item Each retailer has a capacity in terms of number of refrigerators it demands.
		\item\textcolor{blue}{Warehouse 2 is limited to just 100 units in and out per week.}
	\end{itemize}
	\item What is the goal?  What are you trying to achieve?
	\begin{itemize}
		\item Unchanged from part A.
	\end{itemize}
	\item Identify variables
	\begin{itemize}
		\item Unchanged from part A.
	\end{itemize}
	\item Identify constraints
	\begin{itemize}
		\item All constraints from part A remain in effect with the addition of two new constraints:
		\item\textcolor{blue}{$np_{12} + np_{22} + np_{32} + np_{42} = 100$}
		\item\textcolor{blue}{$nw_{23} + nw_{24} + nw_{25} + nw_{26} = 100$}
	\end{itemize}
	\item Identify inputs and outputs that you can control
	\begin{itemize}
		\item Unchanged from part A.
	\end{itemize}
	\item Specify all quantities mathematically
	\begin{itemize}
		\item Unchanged from part A.
	\end{itemize}
	\item Check the model for completeness and correctness
	\begin{itemize}
	\item All variables are positive.
	\end{itemize}
\end{enumerate}
\subsection*{ Matlab Code}
\textcolor{blue}{Code minimally changed from part A.  Only changes are 2 additional constraints (16 total equations) in the linear equality matrix and vector.  Identical code from part A is not shown below (to save space).}
\lstinputlisting{../problem_one/partC_changes_fromA.m}
\end{document}