\chapter{Direct backward ray mapping}
\label{chap:raymapping2}
In Chapter \ref{chap:raymapping1} we introduced an inverse method based on ray mapping reconstruction in PS.
The goal was to calculate the intensity distribution at the target of an optical system. \\ \indent
The idea was to construct a map from the target \point{T} to the source \point{S} using the PS of all the optical lines, which are divided into several regions.
The method developed in the previous chapter requires that the boundaries of these regions can be determined exactly in every PS. Therefore, also the positive luminance regions were found analytically and the \textit{exact} intensity could be determined. This is only possible for systems formed by straight line segments.\\ \indent
In this chapter we modify the method to systems formed by curved lines. In this case, the boundaries of the regions in PS cannot be determined exactly.
Because of this, we need to apply a numerical procedure. In particular, we develop a method that employs only the PS of the target of the system.
% Differences
The boundaries are detected applying a bisection procedure in target PS in combination with backward ray tracing. The method is tested for two optical systems: the TIR-collimator and a parabolic reflector. The results are presented in Section \ref{sec:TIR} and \ref{sec:PR}, respectively.
\section{Bisection method and backward ray tracing}\label{sec:raymapping_explanation}
The purpose of this section is to present the direct backward ray mapping method valid for systems formed by curved lines.
Given a partition $P: -1 = \variabile{p}^0<\variabile{p}^1<\cdots<\variabile{p}^{\nbin}=1$ of the interval $[-1,1]$ with $\nbin$ the number of bins in the partitioning, the intensity in target PS is given by Equation (\ref{eta2}) for every $\dir{}{}\in P$.
Therefore, the problem reduces to calculating the coordinates
$\variabile{q}^{\textrm{\,min}}(\Pi, \variabile{p})$ and $\variabile{q}^{\textrm{\,max}}(\Pi, \variabile{p})$ of the rays on $\partial$\set{R}{}{}$(\Pi)$ for every path $\Pi$.
\\ \indent
We indicate with $(\variabile{q}^{\,\textrm{a}}, \variabile{p})= (-\variabile{b}, \variabile{p})$ and $(\variabile{q}^{\,\textrm{b}}, \variabile{p}) = (\variabile{b}, \variabile{p})$ the coordinates of the end points of \set{T}{}{} along direction $\variabile{p}$. These points are associated to two rays in PS. Consider the corresponding position coordinate $(\variabile{x}^{\textrm{a}}, \variabile{z}^{\textrm{a}})$ and $(\variabile{x}^{\textrm{b}}, \variabile{z}^{\textrm{b}})$ and angular coordinates $\optangle^{\variabile{a}}$ and $\optangle^{\variabile{b}}$ of the ray in real space, where $\variabile{x}^{\textrm{a}} = \variabile{q}^{\textrm{a}}$, $\variabile{x}^{\textrm{b}} = \variabile{q}^{\textrm{b}}$, $\variabile{z}^{\textrm{a}} = \variabile{z}^{\textrm{b}} = \variabile{h}$ where $\variabile{h}$ is the height of the target and $\optangle^{\textrm{a}}= \optangle^{\textrm{b}}=\variabile{p}$ refers to the emitted ray. Next, the rays parametrizations $\vect{r}_{\textrm{a}}(s)$ and $\vect{r}_{\textrm{b}}(s)$ are determined according to (\ref{parametrization}).
We remind the reader that, in real space, the coordinates of each ray on line $\lineai\neq \nline$ are indicated with $(\variabile{x}_{\lineai}, \variabile{z}_{\lineai})$ and $\optangle_{\lineai}$ where $(\variabile{x}_{\lineai}, \variabile{z}_{\lineai})$ are the coordinate of the intersection point between the ray and line $\lineai$ and $\optangle_{\lineai}$ is the direction of the incident ray with respect to the \textit{optical axis}.
%We remind the reader that in the previous chapter we indicated with $(\pos{t,}{\lineai},\dir{t,}{\lineai})$ the coordinates of the rays in target PS \set{T}{\lineai}{} in where the directions coordinates were expressed with respect to the \textit{normal} of line $\lineai$. Note that $\pos{\lineai}{}=\pos{t,}{\lineai}$ while $\dir{\lineai}{}\neq\dir{t,}{\lineai}$. We introduce the notation $(\pos{\lineai}{}^{\textrm{a}}, \dir{\lineai}{}^{\textrm{a}})$ for the coordinates of ray $\vect{r}_{\textrm{a}}(s)$ on line $\lineai$.\\ \indent
The procedure starts with intensity $I(\variabile{p})=0$ and the end points $(\variabile{q}^{\,\textrm{a}}, \variabile{p})= (-\variabile{b}, \variabile{p})$ and $(\variabile{q}^{\,\textrm{b}}, \variabile{p}) = (\variabile{b}, \variabile{p})$. Since the boundaries of the regions in all the phase spaces are unknown, to determine from which line $\vect{r}_{\textrm{a}}$ and $\vect{r}_{\textrm{b}}$ are emitted we apply backward ray tracing. We denote with $\nline$ the index of the target (line $\nline$) and with $\lineaj\in\{1,\cdots, \nline-1\}$ and $\lineak\in\{1,\cdots, \nline-1\}$ the lines from which the rays with parametrization $\vect{r}_{\textrm{a}}(s)$ and $\vect{r}_{\textrm{b}}(s)$ are emitted, respectively. $\Pi^\textrm{a} = (\lineaj, \nline)$ and $\Pi^\textrm{b} = (\lineak,\nline)$ are the last part of the paths followed by the two rays $\vect{r}_{\textrm{a}}$ and $\vect{r}_{\textrm{b}}$, respectively, and $(\variabile{q}^{\textrm{a}}, \variabile{p})\in\partial$\set{R}{}{}$(\Pi^{\textrm{a}})$ and $(\variabile{q}^{\textrm{b}}, \variabile{p})\in\partial$\set{R}{}{}$(\Pi^{\textrm{b}})$. At this stage we know whether the two rays are emitted from the same line or not. \\ \indent
First, assume $\lineaj= \lineak$, then $\vect{r}_{\textrm{a}}$ and $\vect{r}_{\textrm{b}}$ hit the same line before reaching the target.
In case $\lineaj = 1$ a possible path from the source to the target is found and, assuming a Lambertian source, the intensity is updated according to:
\begin{equation}\label{eq:intensity_brm}
I(\variabile{p}) = I(\variabile{p})+\variabile{q}^{\textrm{max}}(\Pi^\textrm{a}, \variabile{p})-\variabile{q}^{\textrm{min}}(\Pi^\textrm{a}, \variabile{p}),
\end{equation}
where $\variabile{q}^{\textrm{min}}(\Pi^\textrm{a}, \variabile{p}) = \variabile{q}^{\textrm{min}} = \min\{\variabile{q}^{\textrm{a}}, \variabile{q}^{\textrm{b}}\}$ and $\variabile{q}^{\textrm{max}}(\Pi^\textrm{a}, \variabile{p}) = \variabile{q}^{\textrm{max}} = \max\{\variabile{q}^{\textrm{a}}, \variabile{q}^{\textrm{b}}\}$. In case $\lineaj\neq 1$ the two rays $\vect{r}_{\textrm{a}}$ and $\vect{r}_{\textrm{b}}$ are traced back further using backward ray tracing.
\\ \indent Next, if $\lineaj\neq \lineak$ the rays $\vect{r}_\textrm{a}$ and $\vect{r}_\textrm{b}$ are emitted from two different lines, hence $\Pi^{\textrm{a}}\neq \Pi^{\textrm{b}}$ and belong to different regions \set{R}{}{}$(\Pi^{\textrm{a}})$ and \set{R}{}{}$(\Pi^{\textrm{b}})$ in \set{T}{}{}. $\variabile{q}^{\textrm{a}}$ in $\mbox{\set{R}{}{}}(\Pi^{\textrm{b}})$, the other intersection points of line
$\variabile{p}=\textrm{const}$ with the boundary $\mbox{\set{R}{}{}}(\Pi^{\textrm{b}})$ are unknown. To determine the other coordinates of the rays on the boundary $\partial$\set{R}{}{}$(\Pi^{\textrm{a}})$, the bisection method is applied to the segment $[\variabile{q}^{\,\textrm{a}}(\Pi^{\textrm{a}}, \variabile{p}), \variabile{q}^{\,\textrm{b}}(\Pi^{\textrm{b}}, \variabile{p})]$ in target PS \set{T}{}{} along direction $\variabile{p}$. Thus, this interval is repeatedly halved until the position coordinate in target PS of the ray that follows the same path $\Pi^{\textrm{a}}$ of $\vect{r}_{\textrm{a}}$ is found (the corresponding direction coordinate $\variabile{p}$ is fixed). The bisection procedure continues until the length of the segment considered becomes smaller than a fixed tolerance.
Giving as input the coordinates $\variabile{q}^{\,\textrm{a}}(\Pi^{\textrm{a}}, \variabile{p})$ and $\variabile{q}^{\,\textrm{b}}(\Pi^{\textrm{b}}, \variabile{p})$ of the rays with parametrization $\vect{r}_{\textrm{a}}(s)$ and $\vect{r}_{\textrm{b}}(s)$, the path $\Pi^\textrm{a}$ and the tolerance $\textrm{tol}= 10^{-12}$, the bisection method is implemented as in Algorithm \ref{alg:bisection}. Similarly, others paths will be obtained later using the same procedure applied to another interval in target PS.
\begin{algorithm}
\caption{Bisection($\variabile{q}^{\,\textrm{a}}(\Pi^{\textrm{a}}, \variabile{p})$, $\variabile{q}^{\,\textrm{b}}(\Pi^{\textrm{b}}, \variabile{p})$,$\vect{r}_{\textrm{a}}(s)$, $\vect{r}_{\textrm{b}}(s)$, \textrm{tol}, $\Pi^\textrm{a}$)}\label{alg:bisection}
Initialize: $\textrm{step} = 0$, $\lineai = \nline$
\begin{algorithmic}[1]
\While {$|\variabile{q}^{\,\textrm{a}}-\variabile{q}^{\,\textrm{b}}|>\textrm{tol}$}
\State $\variabile{x}_{\lineai}^{\textrm{m}}=\variabile{q}^{\,\textrm{m}}= (\variabile{q}^{\,\textrm{a}}+\variabile{q}^{\,\textrm{b}})/2,$
\State $\variabile{z}_{\lineai}^{\textrm{m}}= \variabile{z}^{\textrm{a}}$
\State $\optangle_{\lineai}^{\textrm{m}}=\variabile{p}^{\,\textrm{m}}=\variabile{p}$
\State $\Pi^{\textrm{m}}= (\nline)$
\State Consider the parametrization $\vect{r}_{\textrm{m}}$ of the ray corresponding to $(\variabile{x}_{\lineai}^{\,\textrm{m}}, \variabile{z}_{\lineai}^{\,\textrm{m}})$ and $\optangle_{\lineai}^{\textrm{m}}$,
%\State $\vect{r}_{\textrm{m}}= \variabile{q}^{\,\textrm{m}}+s \variabile{p}$ with $s>0$ the arc-length,
\While {$\textrm{step}<\mbox{length}(\Pi^\textrm{a})-1$}
\State Trace back the ray with parametrization $\vect{r}_{\textrm{m}}$ from $\lineai$
\State Find the line $\lineaj$ that the ray hits
\State Find the coordinates $(\variabile{x}^{\textrm{m}}_{\lineaj}, \variabile{z}^{\textrm{m}}_{\lineaj})$ on line $\lineaj$
\State Calculate the new direction $\optangle_{\lineai}^{\textrm{m}}$ with respect to the optical axis
\State $\Pi^{\textrm{m}}=(\lineaj,\Pi^{\textrm{m}})$.
\If {$\lineaj=1$ or $\lineaj=\nline$}
\State $\textrm{step} = \mbox{length}(\Pi^\textrm{a})$ \Comment If the source or the target are reached \\  \Comment then exit from the while loop.
\Else \State $\textrm{step}=\textrm{step}+1$
\EndIf
\EndWhile
\If {$\Pi^\textrm{a} = \Pi^\textrm{m}$}
\State $\variabile{q}^{\,\textrm{a}}= \variabile{q}^{\,\textrm{m}}$
\State $\vect{r}_{\textrm{a}}= \vect{r}_{\textrm{m}}$
\Else
\State $\variabile{q}^{\,\textrm{b}}= \variabile{q}^{\,\textrm{m}}$
%\State $\vect{r}_{\textrm{b}}\gets \vect{r}_{\textrm{m}}$
\State $\Pi^\textrm{b} =  \Pi^{\textrm{m}}$
\EndIf
\EndWhile
\State $\variabile{q}^{\,\textrm{c}}= \variabile{q}^{\,\textrm{a}}, \Pi^\textrm{c}= \Pi^{\textrm{a}}.$
\State $\variabile{q}^{\,\textrm{d}}= \variabile{q}^{\,\textrm{b}}, \Pi^\textrm{d}= \Pi^{\textrm{b}}.$
\State \Return $(\variabile{q}^{\,\textrm{c}}, \variabile{p})$, $(\variabile{q}^{\,\textrm{d}}, \variabile{p})$, $\Pi^{\textrm{c}}$ and $\Pi^{\textrm{d}}$.
\end{algorithmic}
\end{algorithm}
\\ \indent Once bisection stops, two points with coordinates $(\variabile{q}^{\,\textrm{c}}, \variabile{p})$ and $(\variabile{q}^{\,\textrm{d}}, \variabile{p})$ in \set{T}{}{} are found. The corresponding rays $\vect{r}_{\textrm{c}}$ and $\vect{r}_{\textrm{d}}$ follow path $\Pi^{\textrm{c}}=\Pi^{\textrm{a}}$ and $\Pi^{\textrm{d}}\neq\Pi^{\textrm{a}}$.
All the rays with target coordinates $(\variabile{q}, \variabile{p})$ and $\variabile{q}^{\,\textrm{a}}\leq\variabile{q}\leq\variabile{q}^{\,\textrm{c}}$ follow path $\Pi^{\textrm{a}}$, while the rays with target coordinates $(\variabile{q}, \variabile{p})$ with $\variabile{q}^{\,\textrm{d}}\leq\variabile{q}\leq\variabile{q}^{\,\textrm{b}}$ follow another path $\Pi \neq \Pi^{\textrm{a}}$ (see Figure \ref{fig:bisec}).
%To clarify the procedure, in Figure \ref{fig:bisec} we show an example of target PS of an optical system where Algorithm \ref{alg:bisection} is run for the interval $[\variabile{q}^{\textrm{a}}, \variabile{q}^{\textrm{b}}]$ along direction $\variabile{p}=0$. Applying bisection the coordinates $(\variabile{q}^{\textrm{c}}(\variabile{p}), \variabile{p})$ and $(\variabile{q}^{\textrm{d}}(\variabile{p}), \variabile{p})$ of two points are found. The corresponding rays follow the paths $\Pi^{\textrm{c}}$ and $\Pi^{\textrm{d}}$, where $\Pi^{\textrm{c}}= \Pi^{\textrm{a}}\neq\Pi^{\textrm{d}}$. If $\Pi^{\textrm{c}}$ is a path from the source to the target, Algorithm \ref{alg:bisection} is applied to the sub-interval
%$[\variabile{q}^{\textrm{a}}(\variabile{p}), \variabile{q}^{\textrm{c}}(\variabile{p})]$, otherwise the it is applied again to the sub-interval $[\variabile{q}^{\textrm{d}}(\variabile{p}), \variabile{q}^{\textrm{b}}(\variabile{p})]$. The bisection procedure is repeated for all the sub-intervals found along direction $\variabile{p}$, until the entire interval $[\variabile{q}^{\textrm{a}}(\variabile{p}), \variabile{q}^{\textrm{c}}(\variabile{p})]$ is investigated.
\begin{figure}[h]
  \begin{center}
  \includegraphics[width=0.7\textwidth]{T4_1_example}
  \end{center}
  \caption{\textbf{Bisection in target PS \set{T}{}{}.} Algorithm \ref{alg:bisection} is run for the interval $[\variabile{q}^{\textrm{a}}, \variabile{q}^{\textrm{b}}]$ along direction $\variabile{p}=0$. The coordinates $\variabile{q}^{\textrm{c}}$ and $\variabile{q}^{\textrm{d}}$ are found such that
$|\variabile{q}^{\textrm{c}}-\variabile{q}^{\textrm{d}}|<\textrm{tol}$. $\Pi^{\textrm{c}}= \Pi^{\textrm{a}}$ and $\Pi^{\textrm{d}}\neq \Pi^{\textrm{a}}$.}
\label{fig:bisec}
 \end{figure}
\\ \indent Now, if $\lineaj\neq1$ the procedure applied to the interval
$[\variabile{q}^{\textrm{a}}(\variabile{p}), \variabile{q}^{\textrm{b}}(\variabile{p})]$ needs to be applied to $[\variabile{q}^{\textrm{a}}(\variabile{p}),\variabile{q}^{\textrm{c}}(\variabile{p})]$ until the source is reached, i.e., until $\lineaj=1$. If $\lineaj=1$, the source is reached by the rays traced back from the target. This means that a possible path $\Pi^{\textrm{a}}$ from \point{S} to \point{T} is found and the position coordinates $\variabile{q}^{\textrm{\,min}}(\Pi^{\textrm{a}}, \variabile{p})$ and $\variabile{q}^{\textrm{\,max}}(\Pi^{\textrm{a}}, \variabile{p})$ are determined and the intensity is updated according to (\ref{eq:intensity_brm}).
%Next, all the possible paths along direction $\variabile{p}$ are detected by sequentially applying bisection until all the interval $[\variabile{q}^{\textrm{a}}(\variabile{p}), \variabile{q}^{\textrm{b}}(\variabile{p})]$ is investigated.
\\ \indent
Finally, to detect all possible paths that can occur along direction $\variabile{p}$ the procedure explained above is applied also to the interval $[\variabile{q}^{\textrm{d}}(\variabile{p}), \variabile{q}^{\textrm{b}}(\variabile{p})]$ along direction $\variabile{p}$ continuing until the entire interval $[\variabile{q}^{\textrm{a}}(\variabile{p}), \variabile{q}^{\textrm{b}}(\variabile{p})]$ is investigated.
The main steps of the method are outlined in the following.
\begin{enumerate}
\item Given a direction \variabile{p}, the end points $(\variabile{q}^{\,\textrm{a}}, \variabile{p})$ and $(\variabile{q}^{\,\textrm{b}}, \variabile{p})$ of the target PS \set{T}{}{}, where $\variabile{q}^{\,\textrm{a}} = -\variabile{b}$, $\variabile{q}^{\,\textrm{b}} = \variabile{b}$. Start from $\lineai=\nline$.
\item \label{ray trace} Using backward ray tracing, trace back from line $\lineai$ the rays with parametrizations $\vect{r}_{\textrm{a}}(s)$ and $\vect{r}_\textrm{b}(s)$ corresponding to the position coordinates $(\variabile{x}_{\lineai}^{\,\textrm{a}}, \variabile{z}_{\lineai}^{\,\textrm{a}})$ and $ (\variabile{x}_{\lineai}^{\,\textrm{b}}, \variabile{z}_{\lineai}^{\,\textrm{b}})$ and the angular coordinates $\optangle_{\lineai}^{\textrm{a}}$ and $\optangle_{\lineai}^{\textrm{b}}$, respectively.
%considering the directions $\variabile{p}_{\lineai}^{\,\textrm{a}}$ and $\variabile{p}_{\lineai}^{\,\textrm{b}}$ with respect to the optical axis.
%We remind the reader that we indicate with $\variabile{p}_{\lineai}^{\,\textrm{a}}$ and $\variabile{p}_{\lineai}^{\,\textrm{b}}$ on \set{T}{\lineai}{} the rays directions with respect to the normal of line $\lineai$. Hence, if $\lineai=\nline $ then $\variable{p}_{\lineai}^{\,\textrm{a}}=\variabile{p}_{\lineai}^{\,\textrm{b}}=\variabile{p}$, otherwise $\variabile{p}_{\lineai}^{\,\textrm{a}}\neq\variabile{p}_{\lineai}^{\,\textrm{b}}$.
\item Determine indices $\lineaj\neq\lineai$ and $\lineak\neq\lineai$ of the lines from which the rays with parametrizations $\vect{r}_{\textrm{a}}(s)$ and $\vect{r}_{\textrm{b}}(s)$ originated.
\item Consider the new rays parametrization $\vect{r}_{\textrm{a}}(s)$ and $\vect{r}_{\textrm{b}}(s)$ corresponding to the coordinates $(\variabile{x}_{\lineaj}^{\textrm{a}}, \variabile{z}_{\lineaj}^{\textrm{a}})$, $\optangle_{\lineaj}^{\textrm{a}}$ and $(\variabile{x}_{\lineak}^{\textrm{a}}, \variabile{z}_{\lineak}^{\textrm{b}})$, $\optangle_{\lineak}^{\textrm{b}}$, respectively.
\item Update the paths $\Pi^\textrm{a}$ and $\Pi^\textrm{b}$: $\Pi^\textrm{a} = (\lineaj, \Pi^{\textrm{a}})$ and $\Pi^\textrm{b} = (\lineak, \Pi^\textrm{b})$
\item If $\lineaj=\lineak \neq 1$ and $\lineaj=\lineak \neq \nline$
\begin{itemize}
\item Set $\lineai=\lineaj$
%\item Consider the coordinates of rays $\vect{r}_{\textrm{a}}$ and $\vect{r}_{\textrm{b}}$ on line $\lineaj$
\item Restart the procedure from point $\ref{ray trace}$
\end{itemize}
\item If $\lineaj=\lineak= 1$
\begin{itemize}
\item A relevant path $\Pi^{\textrm{a}} = \Pi^{\textrm{b}}$ is found.
\item Determine
\begin{equation*}
\begin{aligned}
\variabile{q}^{\textrm{min}}(\Pi^{\textrm{a}}, \variabile{p})&=\min\{\variabile{q}^{\textrm{a}}(\Pi^{\textrm{a}}, \variabile{p}), \variabile{q}^{\textrm{b}}(\Pi^{\textrm{b}}, \variabile{p})\}\\
\variabile{q}^{\textrm{max}}(\Pi^{\textrm{a}}, \variabile{p})&=\max\{\variabile{q}^{\textrm{a}}(\Pi^{\textrm{a}}, \variabile{p}), \variabile{q}^{\textrm{b}}(\Pi^{\textrm{b}}, \variabile{p})\}.
\end{aligned}
\end{equation*}
\item Update the intensity $$I(\variabile{p}) = I(p)+\variabile{q}^{\textrm{max}}(\Pi^{\textrm{a}}, \variabile{p})-\variabile{q}^{\textrm{min}}(\Pi^{\textrm{a}}, \variabile{p})$$
\end{itemize}
%\item If $\lineaj=\lineak= \nline$ stop the procedure (the rays reach the target again).
\item If $\lineaj\neq\lineak$
\begin{itemize}
\item Apply the bisection method to the interval $[\variabile{q}^{\textrm{a}}, \variabile{q}^{\textrm{b}}]$ along direction $\variabile{p}$, given the points $(\variabile{q}^{\,\textrm{c}}, \variabile{p})$ and $(\variabile{q}^{\,\textrm{d}}, \variabile{p})$ in target PS \set{T}{}{} such that $|\variabile{q}^{\,\textrm{c}}-\variabile{q}^{\,\textrm{d}}|<\textrm{tol}$.
\item If $\lineaj\neq\nline$
\begin{itemize}
\item Set $\variabile{q}^{\textrm{b}} = \variabile{q}^{\textrm{c}}$,
\item Set $(\variabile{x}_{\lineaj}^{\textrm{b}}, \variabile{z}_{\lineaj}^{\textrm{b}}) = (\variabile{x}_{\lineaj}^{\textrm{c}}, \variabile{z}_{\lineaj}^{\textrm{b}})$ and $\optangle_{\lineaj}^{\textrm{b}}= \optangle_{\lineaj}^{\textrm{c}}$
\item Set $\lineai = \lineaj$
\item Restart from $\ref{ray trace}$ with the updated coordinates,
\end{itemize}
\item Update $\variabile{q}^{\textrm{a}}= \variabile{q}^{\textrm{d}}$,
\item $\Pi^{\textrm{a}} = \Pi^{\textrm{c}}$
\item Update $(\variabile{x}_{\lineaj}^{\textrm{a}}, \variabile{z}_{\lineaj}^{\textrm{a}}) = (\variabile{x}_{\lineaj}^{\textrm{d}}, \variabile{z}_{\lineaj}^{\textrm{d}})$ and $\optangle_{\lineaj}^{\textrm{a}}= \optangle_{\lineaj}^{\textrm{d}}$
\item Restart from $\ref{ray trace}$
\end{itemize}
\end{enumerate}
Giving as input $I(\variabile{p})=0$ for every direction $\variabile{p}$ and the tolerance $\textrm{tol}=10^{-12}$,
the method is defined in the recursive Algorithm \ref{alg:recursiveraymapping}.
\begin{algorithm}
\caption{Recursive function for direct backward ray mapping}\label{alg:recursiveraymapping}
Initialize: $\lineai = \nline$, $\variabile{q}^{\,\textrm{a}}= \variabile{x}^{\,\textrm{a}}_{\lineai}=-\variabile{b}$, $\variabile{q}^{\,\textrm{b}}= \variabile{x}^{\,\textrm{b}}_{\lineai}=\variabile{b}$, $\dir{}{}=\optangle^{\textrm{a}}_\lineai=\optangle^{\textrm{b}}_\lineai=\textrm{const}$, $\variabile{z}_{\lineai}^{\textrm{a}}=\variabile{z}_{\lineai}^{\textrm{b}}= \variabile{h}$, $\Pi^{\textrm{a}}=({\nline})$.
\begin{algorithmic}[1]
\Procedure{Intensity computation}{$\variabile{q}^{\,\textrm{a}}$,  $\variabile{q}^{\,\textrm{b}}$, $\variabile{x}^{\,\textrm{a}}_{\lineai},$  $\variabile{x}^{\,\textrm{b}}_{\lineai},$ $\variabile{z}^{\,\textrm{a}}_{\lineai},$  $\variabile{z}^{\,\textrm{b}}_{\lineai},$ $\dir{}{}$, $\optangle^{\textrm{a}}_{\lineai}$, $\optangle^{\textrm{b}}_{\lineai}$, $\Pi^{\textrm{a}}$, $\lineai$}
\State Apply backward ray tracing to $(\variabile{x}^{\,\textrm{a}}_{\lineai},\variabile{z}^{\textrm{a}}_{\lineai})$, $\optangle_{\lineai}^{\textrm{a}}$ and $(\variabile{x}^{\,\textrm{b}}_{\lineai},\variabile{z}^{\textrm{b}}_{\lineai})$, $\optangle_{\lineai}^{\textrm{b}}$ \State Determine the lines $\lineaj\neq\lineai$ and $\lineak\neq\lineai$ from which $\vect{r}_{\textrm{a}}(s)$ and $\vect{r}_{\textrm{b}}(s)$ are emitted.
\State Update path $\Pi^{\textrm{a}}=(\lineaj, \Pi^{\textrm{a}})$
\State Calculate the position coordinates $(\variabile{x}^{\,\textrm{a}}_{\lineaj},\variabile{z}^{\textrm{a}}_{\lineaj})$, $(\variabile{z}^{\,\textrm{b}}_{\lineaj}, \variabile{p}^{\textrm{b}}_{\lineaj})$ and $\optangle_{\lineaj}^{\textrm{a}}$, $\optangle_{\lineaj}^{\textrm{b}}$.
\If {$\lineaj = \lineak$}
\If{$\lineaj\neq1$}
\State\Return{\Call{Intensity computation}{$\variabile{q}^{\,\textrm{a}},$  $\variabile{q}^{\,\textrm{b}},$ $\variabile{x}^{\,\textrm{a}}_{\lineaj},$ $\variabile{x}^{\,\textrm{b}}_{\lineaj},$ $\variabile{z}^{\,\textrm{a}}_{\lineaj},$ $\variabile{z}^{\,\textrm{b}}_{\lineaj},$ $\dir{}{}$, $\optangle^{\textrm{a}}_{\lineaj}$, $\optangle^{\textrm{b}}_{\lineaj}$, $\Pi^{\textrm{a}}$, $\lineaj$}}
\Else  \State Calculate
$\variabile{q}^{\textrm{min}} = \min\{\variabile{q}^{\textrm{a}}, \variabile{q}^{\textrm{b}}\}$, $\variabile{q}^{\textrm{max}} = \max\{\variabile{q}^{\textrm{a}}, \variabile{q}^{\textrm{b}}\}$
\State Assume Lambertian source
\State $I(\dir{}{}) = I(\dir{}{})+\variabile{q}^{\textrm{max}}(\Pi^{\textrm{a}}, \dir{}{})
-\variabile{q}^{\textrm{min}}(\Pi^{\textrm{a}}, \dir{}{})$
\EndIf
\Else
\State Apply bisection to the segment $[\variabile{q}^{\,\textrm{a}}(\Pi^\textrm{a}, \dir{}{}), \variabile{q}^{\,\textrm{b}}(\Pi^\textrm{b}, \dir{}{})]$
\State Find the target coordinates $(\variabile{q}^{\,\textrm{c}},\variabile{p})$ and $(\variabile{q}^{\,\textrm{d}},\variabile{p})$ of the rays with parame\-trization $\vect{r}_{\textrm{c}}$ and $\vect{r}_{\textrm{d}}$, such that $$|\variabile{q}^{\,\textrm{c}}-\variabile{q}^{\,\textrm{d}}|<\textrm{tol}$$
\If {$\lineaj\neq \nline$}
\State\Return{\Call{Intensity computation}{$\variabile{q}^{\,\textrm{a}},$  $\variabile{q}^{\,\textrm{c}},$ $\variabile{x}^{\,\textrm{a}}_{\lineaj},$ $\variabile{x}^{\,\textrm{c}}_{\lineaj},$ $\variabile{z}^{\,\textrm{a}}_{\lineaj},$ $\variabile{z}^{\,\textrm{c}}_{\lineaj},$ $\dir{}{}$, $\optangle^{\textrm{a}}_{\lineaj}$, $\optangle^{\textrm{c}}_{\lineaj}$, $\Pi^{\textrm{a}}$,~$\lineaj$}}
\EndIf
\State\Return{\Call{Intensity computation}{$\variabile{q}^{\,\textrm{d}},$  $\variabile{q}^{\,\textrm{b}},$ $\variabile{x}^{\,\textrm{d}}_{\lineaj},$ $\variabile{x}^{\,\textrm{b}}_{\lineaj},$ $\variabile{z}^{\,\textrm{d}}_{\lineaj},$ $\variabile{z}^{\,\textrm{b}}_{\lineaj},$ $\dir{}{}$, $\optangle^{\textrm{d}}_{\lineaj}$, $\optangle^{\textrm{b}}_{\lineaj}$, $\Pi^{\textrm{d}}$, $\lineaj$}}
\EndIf
\EndProcedure
\end{algorithmic}
\end{algorithm}
\\ \indent The procedure is able to determine all the possible paths that the rays can follow during their propagation from \set{S}{}{} to \set{T}{}{}. Also, rays located on the boundaries of the regions with positive luminance in target PS \set{T}{}{} are found.
The method is summarized by the flowchart in Figure \ref{fig:flowchart_raymapping}
\begin{figure}[t]
  \begin{center}
  \includegraphics[width=\textwidth]{flowchart_raymapping}
  \end{center}
  \caption{\textbf{Main steps of direct backward ray mapping for systems with curved lines.}}
\label{fig:flowchart_raymapping}
 \end{figure}
\\ \indent
Next, the method is applied to two optical systems formed by curved lines. In the next section we show the results for the TIR-collimator and in Section \ref{sec:PR} we provide numerical results for the parabolic reflector.
\section{Results for the TIR-collimator}\label{sec:TIR}
% Introduction
In this section we apply direct backward ray mapping to the TIR-collimator presented in Chapter \ref{chap:boundaries_alpha} and depicted in Figure \ref{fig:analyticlens}. The target PS of this system is the rectangular \set{T}{}{}$= [-\variabile{b}, \variabile{b}]\times[-1,1]$ with $\variabile{b}=9.7$. The aim is to detect all the possible path $\Pi$ and the rays located on the boundaries $\partial$\set{R}{}{}$(\Pi)$ of the corresponding regions in target PS.
\\ \indent In Chapter \ref{chap:boundaries_alpha}, we found five different paths for the TIR-collimator (see Figure \ref{fig:Tir2}). The boundaries of the corresponding regions in target PS \set{T}{}{} are in general difficult to approximate. Furthermore, along one direction $\dir{}{}$ more than two points can be located on the boundary $\partial$\set{R}{}{}$(\Pi)$ \set{R}{}{}$(\Pi)$ corresponding to a certain path $\Pi$. To determine properly all the boundaries
$\partial$\set{R}{}{}$(\Pi)$, we need to divide the interval $[-\variabile{b}, \variabile{b}]$ in
\set{T}{}{} into intervals of the same length. Hence, we consider a partitioning
$Q = -\variabile{b}=\variabile{q}^{0}<\variabile{q}^{1}<\cdots<\variabile{q}^{\textrm{Ni}}=\variabile{b}$ of $[-\variabile{b}, \variabile{b}]$ where $\textrm{Ni}$ is the total number of sub-intervals along the $\variabile{q}$-axis.
For each direction $\dir{}{}\in [-1,1]$ the procedure explained in Section \ref{sec:raymapping_explanation} is repeated for every sub-interval $[\variabile{q}^{\textrm{k}}(\dir{}{}), \variabile{q}^{\textrm{k}+1}(\dir{}{})]\subset [\variabile{q}^{\textrm{a}}(\dir{}{}), \variabile{q}^{\textrm{b}}(\dir{}{})]$ with $\textrm{k}=0, \cdots, \textrm{Ni}-1$ and $\variabile{q}^{\textrm{a}}(\dir{}{}) = -\variabile{b}$ and $\variabile{q}^{\textrm{b}}(\dir{}{}) = \variabile{b}$.\\ \indent
To establish in how many sub-intervals \textrm{Ni} we need to divide the target, we exploit \'{e}tendue conservation. We use the same idea applied to determine the value of $\alpha$ for the $\alpha$-shapes method and to provide a stopping criterion for the triangulation refinement (see Chapters \ref{chap:boundaries_alpha} and \ref{chap:triangulation}).
The source \'{e}tendue $U_1$ is calculated from (\ref{eq:Usource}), obtaining roughly $U_1 \approx 7.7$. The target \'{e}tendue $U_\textrm{t}$ is given by Equation (\ref{eq:etenduetarg1}). $U_\textrm{t}$ is calculated several times considering every time a different partitioning $Q$ for the $\variabile{q}$-axis of the target PS. Next, the absolute value of the difference between the source and target \'{e}tendue is obtained from
\begin{equation}\label{eq:delta_raymapping}
\Delta U =  \big|U_1-U_{\textrm{t}}\big|.
\end{equation}
If a small value of $\Delta U$ is obtained, then a good approximation of $U_{\textrm{t}}$ is found and therefore, the partition $Q$ used for the computation of $U_{\textrm{t}}$ is suitable for detecting correctly the boundaries $\partial$\set{R}{}{}$(\Pi)$. In Figure \ref{fig:etendue_raymapping_tir} we show how $\Delta U$ decreases by increasing the number of sub-intervals $\textrm{Ni}$ in the partitioning $Q$. Note that when $\textrm{Ni}=100$ the \'{e}tendue difference is roughly $5\cdot 10^{-3}$. This can be related to the fact that both the source \'{e}tendue $U_1$ and the target \'{e}tendue $U_{\textrm{t}}$ are approximated.
\begin{figure}[h]
  \begin{center}
  \includegraphics[width=0.7\textwidth]{etendue_raymapping_tir}
  \end{center}
  \caption{\textbf{Difference between the source and the target \'{e}tendue for the TIR-collimator.} $U_{\textrm{t}}$ is computed four times increasing every time the number of bins $\textrm{Ni}$ where $\textrm{Ni}\in\{5,10,30,100\}$. }
\label{fig:etendue_raymapping_tir}
 \end{figure}
In Figure \ref{fig:boundaries_TIR_ray_mapping} we show the distribution of the rays traced using the backward ray mapping method with $\textrm{Ni}=30$ and $\nbin = 100.$
\begin{figure}[h]
  \begin{center}
  \includegraphics[width=0.7\textwidth]{boundaries_ray_mapping_tir}
  \end{center}
  \caption{\textbf{Ray distribution at target PS of the TIR-collimator.}
 The $\variabile{q}$-axis is divided into $\textrm{Ni}=30$ bins, the $\variabile{p}$-axis is divided into $\nbin = 100$ bins. Approximately $6\cdot10^3$ rays are traced from the target to the source (red dots). Because of numerical errors, a few rays outside the region with positive luminance are found (rays rimmed in blue).}
\label{fig:boundaries_TIR_ray_mapping}
 \end{figure}
In this case, backward ray mapping detects $11$ different paths from the source to the target.
We observe that only $5$ of them are the paths that we expect from PS ray tracing, which are:
\begin{equation}\label{eq:paths_tir}
\begin{array}{llll}
\Pi_1&=(1,2,7,12), \\
\Pi_2&=(1,4,6,7,12), & \Pi_3&=(1,10,8,7,12),\\
\Pi_4&=(1,3,7,12), & \Pi_5&=(1,11,7,12).
\end{array}\end{equation}
(see Figure \ref{fig:analyticlens} for the of the line numbering of the TIR-collimator).
Backward ray mapping also detects the spurious paths:
\begin{equation}\label{eq:paths_tir}
\begin{array}{llll}
\Pi_6&=(1,2,9,8,7,12), & \Pi_7&=(1,2,5,6,7,12), \\
\Pi_8&=(1,2,2,7,12),& \Pi_9&=(1,7,12),\\
\Pi_{10}&=(1,2,4,6,7,12),& \Pi_{11}&=(1,2,10,8,7,12).
\end{array}\end{equation}
These paths are due to numerical errors, the corresponding rays are rimmed in blue in Figure \ref{fig:boundaries_TIR_ray_mapping}. Note that some rays inside different circles might correspond to the same path. The numerical error can be related to the precision of the bisection method and the numerical computation of the intersection points when backward ray tracing is used. We remark that in backward ray tracing, the intersection between the ray and the lens (line $2$) is computed using the Newton-Raphson procedure.
To detect only the boundaries of the regions formed by the rays that follow a \textit{real} path $\Pi_{\variabile{j}}$, with $\variabile{j}\in\{1, \cdots, 5\}$, we check the index of refraction that every ray has once it arrives at the source.
If this is equal to the same index of \point{S} ($\n=1$ for the TIR-collimator), then the ray follows a physical path, otherwise it follows one of the paths in (\ref{eq:paths_tir}) and, therefore, it is not considered for the intensity calculation. This gives the ray distribution at the target PS shown in Figure \ref{fig:boundaries_TIR_ray_mapping1}. We observe that, discarding those rays, $5$ different paths are found. These are the same paths we obtained using PS ray tracing.
\begin{figure}[h]
  \begin{center}
  \includegraphics[width=0.7\textwidth]{boundaries_ray_mapping_tir2}
  \end{center}
  \caption{\textbf{Ray distribution at target PS of the TIR-collimator.}
 The $\variabile{q}$-axis is divided into $\textrm{Ni}=30$ bins, the $\variabile{p}$-axis is divided into $\nbin = 100$ bins. Considering only the rays that arrive to the source with the correct index of refraction ($\n=1$), the regions with positive luminance are computed correctly.}
\label{fig:boundaries_TIR_ray_mapping1}
 \end{figure}
Figure \ref{fig:boundaries_TIR_ray_mapping1} shows that the rays on the boundaries $\partial$\set{R}{}{}$(\Pi_{\variabile{j}})$ are determined for every path $\Pi_{\variabile{j}}$ with $\variabile{j}\in\{1, \cdots, 5\}$.

The ability of the backward ray mapping method to recognize spurious paths makes it suitable to detect ghost stray light, which is unwanted light that reduce the performance of optical systems \cite{breault1995control}. Optical designers are interested in developing methods for minimizing stray light intensity \cite{grabarnik2015optical}. Backward ray mapping could be an alternative approach for such purpose.
\\ \indent Note that some rays in the interior of the regions are still traced because we divided the target PS along the $\variabile{q}$-axis into $\textrm{Ni}=30$ bins. As a consequence, also the rays located at the end points of every bin are traced. \\
\indent The target PS intensity $\hat{I}_{\textrm{PS}}$ is calculated using Equation (\ref{eq:Ips}). The profile of $\hat{I}_{\textrm{PS}}$ with $\textrm{Ni}=30$ is depicted in Figure \ref{fig:intensity_tir_raymapping} with the red line. It is compared with the reference intensity (blue line) which is given by QMC ray tracing with $10^7$ rays. The picture shows that the backward ray mapping method calculates the intensity correctly.
\begin{figure}[t]
  \begin{center}
  \includegraphics[width=0.7\textwidth]{intensity_tir_raymapping}
  \end{center}
  \caption{\textbf{Profile of the intensity for the TIR-collimator.}
 The PS intensity is calculated dividing the $\variabile{q}$-axis into $\textrm{Ni}=30$ bins. The reference intensity is obtained from QMC ray tracing with $10^7$ rays.}
\label{fig:intensity_tir_raymapping}
 \end{figure}
\\ \indent
Finally, we compare backward ray mapping with QMC ray tracing. The errors are obtained from (\ref{eq:error}) and are shown in a logarithmic scale in Figure \ref{fig:error_tir_raymapping} as a function of the CPU-time. The approximation of the PS intensity $\hat{I}_{\textrm{PS}}$ is improved by increasing the number of bins $\textrm{Ni}$ in the partitioning $Q$. The approximated QMC intensity $\hat{I}_{\textrm{QMC}}$ is calculated several times gradually increasing the number of rays $\nrays$. Both intensities are computed using the same number of bins, $\nbin=100$ in the partitioning $P$ of the $\variabile{p}$-axis. The minimum ray mapping error is obtained with $\textrm{Ni}=100$ bins, while the minimum QMC error is achieved tracing $10^6$ rays. We observe that the minimal error obtained using the backward ray mapping is of an order of magnitude of $10^{-6}$, while the minimum QMC error is of the order of $10^{-5}$. Furthermore, an extrapolation of the QMC error shows that the backward ray mapping is around than $1000$ times faster compared to QMC!
\begin{figure}[t]
  \begin{center}
  \includegraphics[width=0.7\textwidth]{Error_tir_raymapping.png}
  \end{center}
  \caption{\textbf{Errors of ray mapping and QMC ray tracing for the TIR-collimator.}
 The direct backward ray mapping method is faster and more accurate than QMC ray tracing.}
\label{fig:error_tir_raymapping}
 \end{figure}
In Table \ref{tab:ray_mapping_tir} the numerical results obtained for backward ray mapping are reported. The error values for QMC ray tracing were already reported in Chapter \ref{chap:triangulation} (Table \ref{tab:mc_error_triangulation}).
\begin{table}[t]
\centering
\caption{\bf Errors of the PS intensity for the TIR-collimator}
\begin{tabular}{llllll}
 \hline  $\textrm{Ni}$\; & $|\Delta U|$  & PS error & CPU-time (sec.)\\
  \hline
 $5$    & $1.6\cdot 10^{-1}$   & $5.19\cdot10^{-4}$     & $269$  \\
$10$    & $2.8\cdot 10^{-2}$ & $2.09\cdot 10^{-4}$   & $284$   \\
$30$   & $5.7 \cdot 10^{-3}$ & $3.15\cdot 10^{-6}$   & $313$  \\
$100$  & $5.1 \cdot 10^{-3}$ & $2.52\cdot 10^{-6}$   & $359$  \\
 \hline
 \end{tabular}
 \label{tab:ray_mapping_tir}
 \end{table}
\\ \indent In the next section we present the method for a parabolic reflector.
\section{Results for the parabolic reflector}\label{sec:PR}
In this section we provide the results for the parabolic reflector in Figure \ref{fig:PR}.
This is a very challenging example. Indeed, the rays that propagate through such a system can reflect many times along the left or the right mirror. As we have seen using PS ray tracing, this leads to many different paths. Every path corresponds to a given number of reflections with one of the two reflectors. In Chapter \ref{chap:triangulation} we found $17$ different paths for this parabolic reflector. Here, we apply the backward ray mapping method to detect all these paths. \\ \indent
The target PS of the parabolic reflector is the rectangular domain \set{T}{}{}$=[-\variabile{b}, \variabile{b}]\times[-1,1]$ where $\variabile{b}=17$. Like for the TIR-collimator, we divide the interval $[-\variabile{b}, \variabile{b}]$ in target PS into sub-intervals of the same length. Considering the partitioning $Q~:~ -\variabile{b}=\variabile{q}^{0}<\variabile{q}^{1}<\cdots<\variabile{q}^{\textrm{Ni}}=\variabile{b}$ of $[-\variabile{b}, \variabile{b}]$ and a direction $\variabile{p}\in[-1,1]$, the backward ray mapping explained in Section \ref{sec:raymapping_explanation} is applied to every sub-interval $[\variabile{q}^{\textrm{k}}, \variabile{q}^{\textrm{k+1}}]\subset[-\variabile{b}, \variabile{b}]$ with $\textrm{k}=0, \cdots, \textrm{Ni}-1$ and for every direction $\variabile{p}\in[-1,1]$. To determine how many bins $\textrm{Ni}$ are needed for a good approximation of the target photometric variables, we employ the same idea of the TIR-collimator. The source \'{e}tendue $U_1$ is compared to several approximations of the \'{e}tendue $U_{\textrm{t}}$ at the target, each of them is given by a different partitioning $Q$ of $[-\variabile{b}, \variabile{b}]$. For the parabolic reflector all the rays emitted from the source arrive at the target. Therefore, the exact \'{e}tendue as an area in PS is computed from (\ref{eq:etenduesource1}) obtaining $U=U_1=8$. The approximated target \'{e}tendue $U_{\textrm{t}}$ is given by Equation (\ref{eq:etendueintegraltarget}). In Figure \ref{fig:etendue_ray_mapping_pr_bin} we show the comparison between $U_1$ and several approximations of $U_{\textrm{t}}$ by gradually increasing the number of bins $\textrm{Ni}$ in the partitioning $Q$ while fixing the maximum number of multiple reflections to $30$. Increasing the number of bins $\textrm{Ni}$, $U_{\textrm{t}}$ increases approaching the exact value $U_1=8$. After the division into $\textrm{Ni}=4$ bins the improvement is slightly visible. Therefore, we conclude that $\textrm{Ni}=4$ bins are enough to detect $30$ multiple reflections.
\begin{figure}[t]
  \begin{center}
  \includegraphics[width=0.7\textwidth]{etendue_ray_mapping_pr_bin}
  \end{center}
  \caption{\textbf{Comparison between the exact \'{e}tendue and the approximated target \'{e}tendue by increasing $\textrm{Ni}$.}
 At most $30$ multiple reflections are considered. Increasing $\textrm{Ni}$, the target \'{e}tendue gets closer to the exact value $U_1=8$.}
\label{fig:etendue_ray_mapping_pr_bin}
 \end{figure}
\\ \indent In Figure \ref{fig:boundaries_rays_pr_raymapping} we show the ray distribution at the target PS obtained using backward ray mapping with $\textrm{Ni}=4$ and at most $30$ multiple reflections. The rays traced from the target to the source are depicted with the red dots. Most of the rays traced are located on the boundaries $\partial$\set{R}{}{}$(\Pi)$ of the regions with positive luminance.
Only few rays are traced inside those regions. These are the rays located at the end points of every sub-interval $[\variabile{q}^{\textrm{k}}, \variabile{q}^{\textrm{k}+1}]$ with $\textrm{k}=0, \cdots, \textrm{Ni}-1$.
\begin{figure}[h]
  \begin{center}
  \includegraphics[width=0.7\textwidth]{boundaries_raymapping_pr}
  \end{center}
  \caption{\textbf{Rays on the boundaries of the regions with positive luminance in target PS.}
 At most $30$ multiple reflections are considered and $\textrm{Ni}=4$ and $\nbin = 100$. Only the rays on the boundaries and on the end points of each interval are traced.}
\label{fig:boundaries_rays_pr_raymapping}
 \end{figure}
\\ \indent
The backward ray mapping method is able to detect $61$ different paths. Indeed up to $30$ multiple reflections can occur at the left reflector and the right reflector. The path that goes directly from the source to the target (no reflections) has to be added. Using PS ray tracing we found at most $17$ paths for the same parabolic reflector. Hence, we claim that ray mapping is much more accurate than PS ray tracing. Also, we observe the procedure can be stopped later to detect even more than $30$ reflections. The more reflections are considered the better the accuracy obtained. Again, to define a stopping criterion we use \'{e}tendue conservation. Fixing the number of bins $\textrm{Ni}=4$ and $\nbin=100$ and gradually increasing the number of multiple reflections we see that the approximated target \'{e}tendue $U_{\textrm{t}}$ changes as the blue line in Figure \ref{fig:etendue_pr_raymapping}. The horizontal red line represents the exact intensity $U = U_{1} = 8$. We note that the more reflections are considered the smaller the value of $\Delta U = |U_1-U_{\textrm{t}}|$ (see also Table \ref{tab:ray_mapping_pr}). We observe that after around $30$ multiple reflections there is no significant improvement in the computation of $U_{\textrm{t}}$. This is due to two main reasons. First, since only few rays follow multiple reflections, they do not give a significant contribution to the total \'{e}tendue, the regions in PS formed by those rays are very small compared to the entire PS. Second, when more paths are considered also more bins $\nbin$ and sub-intervals $\textrm{Ni}$ should be taken into account to obtain a more precise approximation of the \'{e}tendue. From this we conclude that backward ray mapping has a good accuracy when around $30$ multiple reflections and $\textrm{Ni}=4$ bins are taken into account.
\begin{figure}[h]
  \begin{center}
  \includegraphics[width=0.7\textwidth]{etendue_pr_raymapping}
  \end{center}
  \caption{\textbf{Comparison between the exact \'{e}tendue and the approximated target \'{e}tendue by increasing the number of multiple reflections.}
Fixing the number of bins along the $\variabile{q}$-axis, $\textrm{Ni}=4$, and increasing the number of reflections considered, the \'{e}tendue increases approaching to the exact value $U_1=8$.}
\label{fig:etendue_pr_raymapping}
 \end{figure}
Once a stopping criterion is established, direct backward ray mapping is run and the rays on the boundaries are determined. Finally, the target intensity is calculated from Equation (\ref{eta2}).
\\ \indent In Figure \ref{fig:intensity_pr_raymapping} both the PS intensity (red line) and the reference intensity (dotted blue line) are shown. The PS intensity $\hat{I}_{\textrm{PS}}$ is obtained considering at most $30$ multiple reflections, $\textrm{Ni}=4$ and $\nbin = 100$. The reference intensity $\hat{I}_{\textrm{ref}}$ is given by QMC ray tracing with $10^8$ rays and $\nbin = 100$. The two intensities coincide.
\begin{figure}[t]
  \begin{center}
  \includegraphics[width=0.7\textwidth]{intensity_pr_raymapping}
  \end{center}
  \caption{\textbf{Profile of the intensity for the parabolic reflector.}
The ray mapping error is calculated considering $\textrm{Ni}=4$ and at most $30$ multiple reflections. The reference intensity is obtained by running QMC ray tracing with $10^8$ rays.}
\label{fig:intensity_pr_raymapping}
 \end{figure}
\\ \indent
To conclude, we calculate the errors between the approximated intensity $\hat{I}_{\textrm{A}} (\textrm{A}=\textrm{PS}, \textrm{QMC})$ and the reference intensity $\hat{I}_{\textrm{ref}}$. From the results in Figure \ref{fig:error_raymapping_pr} we observe that the PS error (red line) converges faster than the QMC error (blues line) as long as an error of an order of $10^{-6}$ is desired. Ray mapping results to be around $1000$ times faster than QMC ray tracing! Furthermore, it is much more accurate than QMC. Our method is able to detect \textit{all} the possible paths that can occur. The procedure is stopped when $200$ multiple reflections are reached. Our expectation is that, increasing the number of multiple reflections and the number of bins $\textrm{Ni}$, the accuracy can be improved even more.
\begin{figure}[h]
  \begin{center}
  \includegraphics[width=0.7\textwidth]{error_pr_raymapping}
  \end{center}
  \caption{\textbf{Errors of ray mapping and QMC ray tracing for the parabolic reflector.}
The ray mapping error decreases by increasing the number of reflections considered.
The QMC error reduces by tracing more rays.
 The extended backward ray mapping method is significantly faster and more accurate than QMC ray tracing.}
\label{fig:error_raymapping_pr}
 \end{figure}
In Tables \ref{tab:ray_mapping_pr} and \ref{tab:qmc_raymapping_pr} these numerical results are reported.
\begin{table}[t]
\centering
\caption{\bf Errors of the PS intensity for the parabolic reflector}
\begin{tabular}{llllll}
 \hline  Number of\\
 reflections\;  & $|\Delta U|$ & PS error  & CPU-time (sec.)\\
  \hline
 $5$      & $1.74\cdot 10^{-1}$  & $3.222\cdot10^{-4}$& $3.28$  \\
$10$      & $7.52\cdot 10^{-3}$ & $1.555\cdot 10^{-5}$& $5.11$   \\
$20$      & $9.00 \cdot 10^{-4}$ & $2.059\cdot 10^{-6}$& $6.83$  \\
 $30$    & $5.00 \cdot 10^{-4}$ & $1.269\cdot 10^{-6}$ & $8.38$  \\
$100$    & $2.99 \cdot 10^{-4}$ & $1.038\cdot 10^{-6}$ & $17.49$  \\
$150$    & $2.96 \cdot 10^{-4}$ & $1.039\cdot 10^{-6}$ & $26.38$  \\
$200$    & $2.95 \cdot 10^{-4}$ & $1.039\cdot 10^{-6}$ & $32.21$  \\
 \hline
 \end{tabular}
 \label{tab:ray_mapping_pr}
 \end{table}
\begin{table}[t]
\centering
\caption{\bf Errors of the QMC intensity for the parabolic reflector}
\begin{tabular}{lllll}
 \hline  $\nrays$\;  & QMC error & CPU-time (sec.)\\
  \hline
$10^4$     & $2.05\cdot 10^{-4}$   & $2.81$  \\
$10^5$     & $2.87\cdot 10^{-5}$   & $25.81$   \\
$10^6$     & $7.18 \cdot 10^{-6}$  & $257.54$  \\
$10^7$     & $1.15 \cdot 10^{-6}$  & $2491.32$  \\
 \hline
 \end{tabular}
 \label{tab:qmc_raymapping_pr}
 \end{table}
\section{Conclusions}
In this chapter we extended the concatenated ray mapping method to systems formed by curved lines. Employing backward ray tracing and a bisection procedure in target PS, an inverse map from the target to the source was constructed such that all the possible paths that the rays can follow are determined. The direct backward ray mapping method is able to detect the rays located on the boundaries of the regions formed by rays that follow the same path. From these rays the target intensity is calculated.
\\ \indent
We presented two examples of optical systems: the TIR-collimator and the parabolic reflector. In both cases the target PS is divided into bins and the procedure is applied to each bin. A stopping criterion based on \'{e}tendue conservation is developed to determine the number of bins needed to obtain a good accuracy. For the TIR-collimator we noticed that the method is able to detect rays that follow a spurious path due to numerical error. This gives the expectation that backward ray mapping can be used for detecting stray light.
For the parabolic reflector, many paths can occur along the reflectors. Etendue conservation is used again to determine the number of multiple reflections to be considered. The target intensity is computed for both systems and is compared with a reference intensity given by QMC ray tracing with a large number of rays. The results show that the method is able to detect all the possible paths tracing a relatively small number of rays, typically around $10^3$. Comparing our method to QMC ray tracing, significant advantages are observed in accuracy and computational time for both optical systems.
% Explain the results
\\ \indent In the next chapter we explain how to apply the method to systems with Fresnel reflection. We present the method for a system formed by the source, an ideal lens and the target.


\documentclass[12pt]{article}
\usepackage[margin=1.2in]{geometry}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{tikz}
\usepackage{tikz-qtree}
\usepackage{verbatim}
\usepackage{algorithm,caption}

% \usepackage{natbib}
% \twocolumn

\newcommand{\ind}{\hspace*{1em}}
\newcommand{\kword}[1]{{\bf #1}}
\newcommand{\algblk}[2]{
  \begin{center}
    \begin{minipage}[c]{22em}
      \begin{algorithm}[H]
        \caption*{{\bf Algorithm} #1}
        #2
      \end{algorithm}
    \end{minipage}
  \end{center}
}

\title{Adding Probabilities in the Hakaru to C Compiler (HKC)}
\author{Zach Sullivan}
\date{July 30, 2016}

\begin{document}
\maketitle

\section{Probabilities in Hakaru}

Because Hakaru is a probabilistic language, we provide a type just for
probabilities. The type {\tt prob} has extra safety from underflow. The main way
we accomplish this is by storing its value as double precision floating point
numbers in the log-domain. We can do basic arithmetic on our probabilities.

\section{The ``LogSumExp Trick''}
Because our probability types are stored in the log-domain, we need to compute
\begin{displaymath}
{\rm LSE}(x) = \log\Bigg(\sum_{i=0}^{n}{e^{x_i}}\Bigg)
\end{displaymath}
often called ``LogSumExp.'' The advantage of storing probabilities in the
log-domain is lost if we simply remove them from the log-domain before doing
calculations on them. We would also add a bunch of {\tt log} and {\tt exp}
operations.

\begin{align*}
{\rm LSE^\prime}(x) &= \hat x + \log\Bigg(\sum_{i=0}^{n}{e^{x_i-\hat x}}\Bigg)\\
{\rm\bf where}&\ \hat x = {\rm max}(x)
\end{align*}

This trick preserves the value of the largest of the numbers being summed.

\section{Summation over Probabilities}

For summation over probabilities a different technique for LogSumExp will need
to be use. Since arity will not be known at runtime we need to have a function
that operates over an array.

This algorithm follows directly from the math:

\algblk{LogSumExp for Summation}
{
  \kword{input:} an index production function
  $f : \mathbb{N} \rightarrow \mathbb{R}^+$ and a size $n \in \mathbb{N}$

  \kword{output:} summation of array produced by $f$ in log-space\\

  $A[0] \leftarrow f(0)$\\
  $m \leftarrow A[0]$\\
  $s \leftarrow 0$\\
  \kword{for} $i = 0,1,...,n-1$\kword{:}\\
  \ind $A[i] \leftarrow f(i)$\\
  \ind \kword{if} $m < A[i]$ \kword{:}\\
  \ind \ind $m \leftarrow A[i]$\\
  \kword{for} $i = 0,1,...,n-1$\kword{:}\\
  \ind \ind $s \leftarrow s + \exp(A[i]-m)$\\
  \kword{return} $m + \log(s)$
}

It scales in space at $O(n)$ and in time at $O(2n)$. We generate a function
for this operation and call it when we summate over probabilities. It differs
from summation over {\tt nat}, {\tt int}, and {\tt real} which are done in
constant time.

\section{$n$-ary Probability Summation}

Algorithmically, $n$-ary operations are the same operation as summation over an
array. However, it requires different word for code generation. For summation
we do not know that arity at runtime. For $n$-ary operations, we do.

\subsection*{Max Comparison Tree}
LogSumExp safety creates a particular challenge when generating code for our
compiler. HKC compiles Hakaru to C, where we have no max function that works
on an arbitrary number of arguments. The solution is to create a tree of
comparisons using C's ternary conditional expression to find the maximum of $n$
number of arguments.

Here is an example of a max comparison tree when the length of array $x$ is
4.

\begin{center}
\Tree [ .{$x_0 > x_1$}
        [ .{$x_0 > x_2$}
          [ .{$x_0 > x_3$}
            % {$x_0$}
            % {$x_3$}
          ]
          [ .{$x_2 > x_3$}
            % {$x_2$}
            % {$x_3$}
          ]
        ]
        [ .{$x_1 > x_2$}
          [ .{$x_1 > x_3$}
            % {$x_1$}
            % {$x_3$}
          ]
          [ .{$x_2 > x_3$}
            % {$x_2$}
            % {$x_3$}
          ]
        ]
      ]
\end{center}

\subsection*{Code Generation}

After generating a max comparison tree, we can create leaves for our tree that
are different LogSumExp summations. This is rather trivial given a particular
max index.

% We use {\tt log1p} and {\tt expm1} to help prevent.

A Hakaru program that sums 4 probabilities will generate the following C
expression, where {\tt p\_a}, {\tt p\_b}, {\tt p\_c}, and {\tt p\_d} are
variables holding probabilities:

{\small
\begin{verbatim}
p_a > p_b
  ? p_a > p_c
    ? p_a > p_d
      ? p_a + log1p(expm1(p_c - p_a) + (expm1(p_d - p_a) + expm1(p_b - p_a)) + 3)
      : p_d + log1p(expm1(p_b - p_d) + (expm1(p_c - p_d) + expm1(p_a - p_d)) + 3)
    : ( p_c > p_d
      ? p_c + log1p(expm1(p_b - p_c) + (expm1(p_d - p_c) + expm1(p_a - p_c)) + 3)
      : p_d + log1p(expm1(p_b - p_d) + (expm1(p_c - p_d) + expm1(p_a - p_d)) + 3))
  : (p_b > p_c
    ? p_b > p_d
      ? p_b + log1p(expm1(p_c - p_b) + (expm1(p_d - p_b) + expm1(p_a - p_b)) + 3)
      : p_d + log1p(expm1(p_b - p_d) + (expm1(p_c - p_d) + expm1(p_a - p_d)) + 3)
    : (p_c > p_d
      ? p_c + log1p(expm1(p_b - p_c) + (expm1(p_d - p_c) + expm1(p_a - p_c)) + 3)
      : p_d + log1p(expm1(p_b - p_d) + (expm1(p_c - p_d) + expm1(p_a - p_d)) + 3)));
\end{verbatim}
}

We can use {\tt log1p} because where $x_i$ is the maximum
$e^{x_i - \hat x} = 1$. We use {\tt log1p} and {\tt expm1} because they can be
more accurate for small values.

\subsection*{Discussion}

The method presented here and used in HKC for code generation produces an amount
of code that is exponential in the number of arguments. In practice (thus far),
this is not a major issue because {\tt summate} is often used to sum a large
number of probabilities.

\end{document}


\section{Sets}\label{Sets}

This is a library on sets defined by their characteristic predicate.
It contains the following modules:

\begin{itemize}
\item {\tt Ensembles.v}
\item {\tt Constructive\_sets.v}, {\tt Classical\_sets.v}
\item {\tt Relations\_1.v}, {\tt Relations\_2.v},
 {\tt Relations\_3.v}, {\tt Relations\_1\_facts.v},  \\
 {\tt Relations\_2\_facts.v}, {\tt Relations\_3\_facts.v}
\item {\tt Partial\_Order.v}, {\tt Cpo.v}
\item {\tt Powerset.v}, {\tt Powerset\_facts.v},
 {\tt Powerset\_Classical\_facts.v}
\item {\tt Finite\_sets.v}, {\tt Finite\_sets\_facts.v}
\item {\tt Image.v}
\item {\tt Infinite\_sets.v}
\item {\tt Integers.v}
\end{itemize}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:


\documentclass[a4paper,10pt]{article}
\usepackage{graphicx}
\usepackage{lscape}
\title{Results}
\author{}
\date{\today}
\begin{document}
\begin{landscape}
\oddsidemargin 0in \topmargin 0in\maketitle
\section{Tables of Friedman, Aligned Friedman, Bonferroni-Dunn, Holm, Hochberg and Hommel Tests}
\begin{table}[!htp]
\centering
\caption{Average Rankings of the algorithms (Friedman)
}\begin{tabular}{c|c}
Algorithm&Ranking\\
\hline
 RING&4.959999999999999\\
 TREE&5.039999999999998\\
 NETA&4.959999999999999\\
 NETB&4.959999999999998\\
 TORUS&5.059999999999999\\
 GRAPH&5.3199999999999985\\
 SAME&4.959999999999999\\
 GOODBAD&4.869999999999999\\
 RAND&4.869999999999999\\
\end{tabular}
\end{table}

Friedman statistic (distributed according to chi-square with 8 degrees of freedom: 0.9853333333325054.
P-value computed by Friedman Test: 0.9983393538102286.\newline

Iman and Davenport statistic (distributed according to F-distribution with 8 and 392 degrees of freedom: 0.12100140011551622.
P-value computed by Iman and Daveport Test: 0.9984061848725897.\newline

\newpage

\begin{table}[!htp]
\centering
\caption{Average Rankings of the algorithms (Aligned Friedman)
}\begin{tabular}{c|c}
Algorithm&Ranking\\
\hline
 RING&222.76999999999984\\
 TREE&224.60999999999981\\
 NETA&221.99999999999983\\
 NETB&221.99999999999983\\
 TORUS&230.91999999999982\\
 GRAPH&255.0099999999998\\
 SAME&222.76999999999984\\
 GOODBAD&214.70999999999984\\
 RAND&214.70999999999984\\
\end{tabular}
\end{table}

Aligned Friedman statistic (distributed according to chi-square with 8 degrees of freedom: 40.986399746359325.
P-value computed by Aligned Friedman Test: 2.0968734527615496E-6.\newline

\newpage

\begin{table}[!htp]
\centering
\caption{Average Rankings of the algorithms (Quade)
}\begin{tabular}{c|c}
Algorithm&Ranking\\
\hline
 RING&4.926274509803922\\
 TREE&5.0831372549019616\\
 NETA&4.926274509803922\\
 NETB&4.926274509803923\\
 TORUS&5.12235294117647\\
 GRAPH&5.547450980392157\\
 SAME&4.926274509803923\\
 GOODBAD&4.770980392156864\\
 RAND&4.770980392156864\\
\end{tabular}
\end{table}
Quade statistic (distributed according to F-distribution with 8 and 392 degrees of freedom: 0.2788345895242685.
P-value computed by Quade Test: 0.9727048389531654.\newline

\newpage

\begin{table}[!htp]
\centering\tiny
\caption{Contrast Estimation}
\begin{tabular}{
|r|r|r|r|r|r|r|r|r|r|}
\hline
 & RING& TREE& NETA& NETB& TORUS& GRAPH& SAME& GOODBAD& RAND\\
\hline
 RING&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000\\
\hline
 TREE&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000\\
\hline
 NETA&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000\\
\hline
 NETB&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000\\
\hline
 TORUS&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000\\
\hline
 GRAPH&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000\\
\hline
 SAME&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000\\
\hline
 GOODBAD&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000\\
\hline
 RAND&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000&0.000\\
\hline

\end{tabular}
\end{table}

\newpage

\begin{table}[!htp]
\centering\scriptsize
\caption{Holm / Hochberg / Holland / Rom / Finner / Li Table for $\alpha=0.05$ (FRIEDMAN)}
\begin{tabular}{ccccccccc}
$i$&algorithm&$z=(R_0 - R_i)/SE$&$p$&Holm/Hochberg/Hommel&Holland&Rom&Finner&Li\\
\hline
8& GRAPH&0.8215838362577479&0.4113137917762598&0.00625&0.006391150954545011&0.006574125233361166&0.006391150954545011&0.0\\
7& TORUS&0.34689095308660434&0.7286732435303455&0.0071428571428571435&0.007300831979014655&0.0075128293213784685&0.012741455098566168&0.0\\
6& TREE&0.3103761159195924&0.7562749546662049&0.008333333333333333&0.008512444610847103&0.008764162596519848&0.019051173490195694&0.0\\
5& RING&0.1643167672515496&0.8694817827381617&0.01&0.010206218313011495&0.010515350115740741&0.025320565519103666&0.0\\
4& NETA&0.1643167672515496&0.8694817827381617&0.0125&0.012741455098566168&0.013109375000000001&0.031549888917161595&0.0\\
3& SAME&0.1643167672515496&0.8694817827381617&0.016666666666666666&0.016952427508441503&0.016666666666666666&0.03773939976903784&0.0\\
2& NETB&0.16431676725154795&0.869481782738163&0.025&0.025320565519103666&0.025&0.04388935252272508&0.0\\
1& RAND&0.0&1.0&0.05&0.050000000000000044&0.05&0.050000000000000044&0.05\\
\hline
\end{tabular}
\end{table}
Bonferroni-Dunn's procedure rejects those hypotheses that have a p-value $\le0.00625$.

Holm's procedure rejects those hypotheses that have a p-value $\le0.00625$.

Hommel's procedure rejects those hypotheses that have a p-value $\le0.00625$.

Holland's procedure rejects those hypotheses that have a p-value $\le0.006391150954545011$.

Finner's procedure rejects those hypotheses that have a p-value $\le0.006391150954545011$.

Li's procedure rejects those hypotheses that have a p-value $\le0.0$.

\newpage

\begin{table}[!htp]
\centering\scriptsize
\caption{Holm / Hochberg / Holland / Rom / Finner / Li Table for $\alpha=0.05$ (ALIGNED FRIEDMAN)}
\begin{tabular}{ccccccccc}
$i$&algorithm&$z=(R_0 - R_i)/SE$&$p$&Holm/Hochberg/Hommel&Holland&Rom&Finner&Li\\
\hline
8& GRAPH&1.549427092939215&0.12127908507634506&0.00625&0.006391150954545011&0.006574125233361166&0.006391150954545011&0.0\\
7& TORUS&0.6232310961921754&0.5331326698657505&0.0071428571428571435&0.007300831979014655&0.0075128293213784685&0.012741455098566168&0.0\\
6& TREE&0.3806284918138514&0.7034789365387083&0.008333333333333333&0.008512444610847103&0.008764162596519848&0.019051173490195694&0.0\\
5& RING&0.30988541858784346&0.7566480916486279&0.01&0.010206218313011495&0.010515350115740741&0.025320565519103666&0.0\\
4& SAME&0.30988541858784346&0.7566480916486279&0.0125&0.012741455098566168&0.013109375000000001&0.031549888917161595&0.0\\
3& NETA&0.2802809803356545&0.7792619417215579&0.016666666666666666&0.016952427508441503&0.016666666666666666&0.03773939976903784&0.0\\
2& NETB&0.2802809803356545&0.7792619417215579&0.025&0.025320565519103666&0.025&0.04388935252272508&0.0\\
1& RAND&0.0&1.0&0.05&0.050000000000000044&0.05&0.050000000000000044&0.05\\
\hline
\end{tabular}
\end{table}
Bonferroni-Dunn's procedure rejects those hypotheses that have a p-value $\le0.00625$.

Holm's procedure rejects those hypotheses that have a p-value $\le0.00625$.

Hommel's procedure rejects those hypotheses that have a p-value $\le0.00625$.

Holland's procedure rejects those hypotheses that have a p-value $\le0.006391150954545011$.

Finner's procedure rejects those hypotheses that have a p-value $\le0.006391150954545011$.

Li's procedure rejects those hypotheses that have a p-value $\le0.0$.

\newpage

\begin{table}[!htp]
\centering\scriptsize
\caption{Holm / Hochberg / Holland / Rom / Finner / Li Table for $\alpha=0.05$ (QUADE)}
\begin{tabular}{ccccccccc}
$i$&algorithm&$z=(R_0 - R_i)/SE$&$p$&Holm/Hochberg/Hommel&Holland&Rom&Finner&Li\\
\hline
8& GRAPH&0.6168852944521827&0.5373103871065616&0.00625&0.006391150954545011&0.006574125233361166&0.006391150954545011&0.0\\
7& TORUS&0.2791561736510878&0.7801249754918013&0.0071428571428571435&0.007300831979014655&0.0075128293213784685&0.012741455098566168&0.0\\
6& TREE&0.24800035069895815&0.8041341354591375&0.008333333333333333&0.008512444610847103&0.008764162596519848&0.019051173490195694&0.0\\
5& NETB&0.12337705889043668&0.9018085226206667&0.01&0.010206218313011495&0.010515350115740741&0.025320565519103666&0.0\\
4& SAME&0.12337705889043668&0.9018085226206667&0.0125&0.012741455098566168&0.013109375000000001&0.031549888917161595&0.0\\
3& RING&0.12337705889043597&0.9018085226206671&0.016666666666666666&0.016952427508441503&0.016666666666666666&0.03773939976903784&0.0\\
2& NETA&0.12337705889043597&0.9018085226206671&0.025&0.025320565519103666&0.025&0.04388935252272508&0.0\\
1& RAND&0.0&1.0&0.05&0.050000000000000044&0.05&0.050000000000000044&0.05\\
\hline
\end{tabular}
\end{table}
Bonferroni-Dunn's procedure rejects those hypotheses that have a p-value $\le0.00625$.

Holm's procedure rejects those hypotheses that have a p-value $\le0.00625$.

Hommel's procedure rejects those hypotheses that have a p-value $\le0.00625$.

Holland's procedure rejects those hypotheses that have a p-value $\le0.006391150954545011$.

Finner's procedure rejects those hypotheses that have a p-value $\le0.006391150954545011$.

Li's procedure rejects those hypotheses that have a p-value $\le0.0$.

\newpage

\begin{table}[!htp]
\centering\scriptsize
\caption{Adjusted $p$-values (FRIEDMAN)}
\begin{tabular}{ccccccc}
i&algorithm&unadjusted $p$&$p_{Bonf}$&$p_{Holm}$&$p_{Hoch}$&$p_{Homm}$\\
\hline
1& GRAPH&0.4113137917762598&3.2905103342100785&3.2905103342100785&1.0&1.0\\
2& TORUS&0.7286732435303455&5.829385948242764&5.100712704712418&1.0&1.0\\
3& TREE&0.7562749546662049&6.050199637329639&5.100712704712418&1.0&1.0\\
4& RING&0.8694817827381617&6.955854261905293&5.100712704712418&1.0&1.0\\
5& NETA&0.8694817827381617&6.955854261905293&5.100712704712418&1.0&1.0\\
6& SAME&0.8694817827381617&6.955854261905293&5.100712704712418&1.0&1.0\\
7& NETB&0.869481782738163&6.955854261905304&5.100712704712418&1.0&1.0\\
8& RAND&1.0&8.0&5.100712704712418&1.0&1.0\\
\hline
\end{tabular}
\end{table}

\begin{table}[!htp]
\centering\scriptsize
\caption{Adjusted $p$-values (FRIEDMAN)}
\begin{tabular}{ccccccc}
i&algorithm&unadjusted $p$&$p_{Holl}$&$p_{Rom}$&$p_{Finn}$&$p_{Li}$\\
\hline
1& GRAPH&0.4113137917762598&0.985576492323003&1.0&0.985576492323003&0.9999999999999998\\
2& TORUS&0.7286732435303455&0.9998917449008606&1.0&0.9945803593365627&1.0000000000000002\\
3& TREE&0.7562749546662049&0.9998917449008606&1.0&0.9945803593365627&1.0\\
4& RING&0.8694817827381617&0.9999621247362487&1.0&0.9945803593365627&0.9999999999999999\\
5& NETA&0.8694817827381617&0.9999621247362487&1.0&0.9945803593365627&0.9999999999999999\\
6& SAME&0.8694817827381617&0.9999621247362487&1.0&0.9945803593365627&0.9999999999999999\\
7& NETB&0.869481782738163&0.9999621247362487&1.0&0.9945803593365627&0.9999999999999999\\
8& RAND&1.0&1.0&1.0&1.0&1.0\\
\hline
\end{tabular}
\end{table}

\newpage

\begin{table}[!htp]
\centering\scriptsize
\caption{Adjusted $p$-values (ALIGNED FRIEDMAN)}
\begin{tabular}{ccccccc}
i&algorithm&unadjusted $p$&$p_{Bonf}$&$p_{Holm}$&$p_{Hoch}$&$p_{Homm}$\\
\hline
1& GRAPH&0.12127908507634506&0.9702326806107605&0.9702326806107605&0.9702326806107605&0.890585076253209\\
2& TORUS&0.5331326698657505&4.265061358926004&3.731928689060253&1.0&1.0\\
3& TREE&0.7034789365387083&5.627831492309666&4.22087361923225&1.0&1.0\\
4& RING&0.7566480916486279&6.053184733189023&4.22087361923225&1.0&1.0\\
5& SAME&0.7566480916486279&6.053184733189023&4.22087361923225&1.0&1.0\\
6& NETA&0.7792619417215579&6.2340955337724635&4.22087361923225&1.0&1.0\\
7& NETB&0.7792619417215579&6.2340955337724635&4.22087361923225&1.0&1.0\\
8& RAND&1.0&8.0&4.22087361923225&1.0&1.0\\
\hline
\end{tabular}
\end{table}

\begin{table}[!htp]
\centering\scriptsize
\caption{Adjusted $p$-values (ALIGNED FRIEDMAN)}
\begin{tabular}{ccccccc}
i&algorithm&unadjusted $p$&$p_{Holl}$&$p_{Rom}$&$p_{Finn}$&$p_{Li}$\\
\hline
1& GRAPH&0.12127908507634506&0.6445261095927789&0.9223971309589616&0.6445261095927789&1.0000000000000007\\
2& TORUS&0.5331326698657505&0.99516546853319&1.0&0.952491213461512&0.9999999999999998\\
3& TREE&0.7034789365387083&0.9993202749204726&1.0&0.9609025772024482&0.9999999999999999\\
4& RING&0.7566480916486279&0.9993202749204726&1.0&0.9609025772024482&0.9999999999999999\\
5& SAME&0.7566480916486279&0.9993202749204726&1.0&0.9609025772024482&0.9999999999999999\\
6& NETA&0.7792619417215579&0.9993202749204726&1.0&0.9609025772024482&0.9999999999999999\\
7& NETB&0.7792619417215579&0.9993202749204726&1.0&0.9609025772024482&0.9999999999999999\\
8& RAND&1.0&1.0&1.0&1.0&1.0\\
\hline
\end{tabular}
\end{table}

\newpage

\begin{table}[!htp]
\centering\scriptsize
\caption{Adjusted $p$-values (QUADE)}
\begin{tabular}{ccccccc}
i&algorithm&unadjusted $p$&$p_{Bonf}$&$p_{Holm}$&$p_{Hoch}$&$p_{Homm}$\\
\hline
1& GRAPH&0.5373103871065616&4.298483096852493&4.298483096852493&1.0&1.0\\
2& TORUS&0.7801249754918013&6.24099980393441&5.460874828442609&1.0&1.0\\
3& TREE&0.8041341354591375&6.4330730836731&5.460874828442609&1.0&1.0\\
4& NETB&0.9018085226206667&7.2144681809653335&5.460874828442609&1.0&1.0\\
5& SAME&0.9018085226206667&7.2144681809653335&5.460874828442609&1.0&1.0\\
6& RING&0.9018085226206671&7.214468180965337&5.460874828442609&1.0&1.0\\
7& NETA&0.9018085226206671&7.214468180965337&5.460874828442609&1.0&1.0\\
8& RAND&1.0&8.0&5.460874828442609&1.0&1.0\\
\hline
\end{tabular}
\end{table}

\begin{table}[!htp]
\centering\scriptsize
\caption{Adjusted $p$-values (QUADE)}
\begin{tabular}{ccccccc}
i&algorithm&unadjusted $p$&$p_{Holl}$&$p_{Rom}$&$p_{Finn}$&$p_{Li}$\\
\hline
1& GRAPH&0.5373103871065616&0.9978995226651626&1.0&0.9978995226651626&1.0\\
2& TORUS&0.7801249754918013&0.9999751554402075&1.0&0.9978995226651626&0.9999999999999999\\
3& TREE&0.8041341354591375&0.9999751554402075&1.0&0.9978995226651626&0.9999999999999999\\
4& NETB&0.9018085226206667&0.9999908721399633&1.0&0.9978995226651626&1.0\\
5& SAME&0.9018085226206667&0.9999908721399633&1.0&0.9978995226651626&1.0\\
6& RING&0.9018085226206671&0.9999908721399633&1.0&0.9978995226651626&1.0\\
7& NETA&0.9018085226206671&0.9999908721399633&1.0&0.9978995226651626&1.0\\
8& RAND&1.0&1.0&1.0&1.0&1.0\\
\hline
\end{tabular}
\end{table}

\end{landscape}\end{document}


\SecDef{dca}{Differential Computational Analysis}

I describe the general setting for our attacks. We consider \emph{a keyed symmetric primitive}, e.g. a block cipher. A \emph{white-box designer} takes a naive implementation with a hardcoded secret key and obfuscates it producing \emph{a white-box implementation}. An adversary receives the white-box implementation and her goal is to recover the secret key or a part of it. We restrict our analysis to implementations in the form of \emph{Boolean circuits}.

\begin{definition}
    \Label{def:circuit}
    A \emph{Boolean circuit} $C$ is a directed acyclic graph where each node with the indegree $k > 0$ has an associated $k$-ary symmetric Boolean function $g_v$. Nodes with the indegree equal to zero are called \emph{inputs} of $C$ and nodes with the outdegree equal to zero are called \emph{outputs} of $C$.

    Let $x = (x_1, \ldots, x_N)$ (resp. $y = (y_1, \ldots, y_M)$) be a vector of input (resp. output) nodes in a fixed order. For each node $v$ in $C$ we say that it computes a Boolean function $f_v: \field{N} \to \field{}$ defined as follows:
    \begin{itemize}
        \item for all $1 \le i \le N$ set $f_{x_i}(z) = z_i$,
        \item for all non-input nodes $v$ in $C$ set $f_{v}(z) = g_v(f_{c_1}(z), \ldots, f_{c_k}(z))$,\\
        where $c_1, \ldots, c_k$ are nodes having an outgoing edge to $v$.
    \end{itemize}

    The set of $f_v$ for all nodes $v$ in $C$ is denoted $\FUNCS(C)$ and the set of $f_{x_i}$ for all input nodes $x_i$ is denoted $\XFUNCS(C)$. By an abuse of notation we also define the function $C: \field{N} \to \field{M}$ as $C = (f_{y_1}, \ldots, f_{y_m})$.
\end{definition}

\subsubsection{Masking Schemes}
We assume that the white-box designer uses masking in some form, but we do not restrict him from using other obfuscation techniques. The only requirement is that there exists a relatively small set of nodes in the obfuscated circuit (called \emph{shares}) such that during a legitimate computation the values computed in these nodes sum to \emph{a predictable value}. We at least expect this to happen with overwhelming probability. In a more general case, we allow arbitrary functions to be used to compute the predictable value from the shares instead of plain \txor{}. We call these functions \emph{decoders}. The classic Boolean masking technique is based on the \txor{} decoder. The number of shares is denoted by $\numshares$.

I give a broad definition of a masking scheme that will be used also in \ChapRef{wbc}.

\begin{definition}[Masking Scheme]
\Label{def:masking}
An $\nsh$-bit masking scheme is defined by an encoding function $\enc: \field{} \times \field{\nrand} \to \field{\nsh}$, a decoding function $\dec: \field{\nsh} \to \field{}$ and a set of triplets $\{(\op, Eval_\op, \cir_\op), \ldots\}$ where each triplet consists of:
\begin{enumerate}
    \item a Boolean operator $\op: \field{}\times\field{} \to \field{}$,
    \item a circuit $Eval_{\op}: \field{\nsh}\times\field{\nsh}\times\field{\nrand'} \to \field{\nsh}$.
\end{enumerate}
For any $r\in\field{\nrand}$ and any $x\in\field{}$ it must hold that $\dec(\enc(x,r)) = x$. Moreover, the following equation must be satisfied for all operators $\op$ and all values $r' \in \field{\nrand'}, x_1 \in \field{\nsh}, x_2 \in \field{\nsh}$:
$$\dec(Eval_{\op}(x_1, x_2, r')) = \dec(x_1) \op \dec(x_2).$$

The degree of the masking scheme is the algebraic degree of the $\dec$ function. The masking scheme is called nonlinear if its degree is greater than 1.
\end{definition}

Note that $Eval_\op$ takes three arguments in the definition. The first two are shares of the secret values and the third one is optional randomness that must not change the secret values.

% -------------------------------------------------------------------

\subsubsection{Predictable Values}

\emph{A predictable value} typically is a value computed in the beginning or in the end of the reference algorithm such that it depends only on a few key bits and on the plaintexts/ciphertexts. In such case the adversary makes a guess for the key bits and computes the corresponding candidate for the predictable value. The total number of candidates is denoted by $\numpred$.

The obfuscation method may require random bits e.g. for splitting the secret value into random shares. Even if the circuit may have input nodes for random bits in order to achieve non-deterministic encryption, the adversary can easily manipulate them. Therefore, the obfuscation method has to rely on pseudorandomness computed solely from the input. Locating and manipulating the pseudorandomness generation is a possible attack direction. However, as we aim to study the applicability of masking schemes, we assume that the adversary can not directly locate the pseudorandomness computations and remove the corresponding nodes. Moreover, the adversary can not predict the generated pseudorandom values with high probability, i.e. such values are not predictable values.

% -------------------------------------------------------------------

\subsubsection{Window Coverage}

In a typical case shares of a predictable value will be relatively close in the circuit (for example, at the same circuit level or at a short distance in the circuit graph).  This fact can be exploited to improve efficiency of the attacks. The adversary covers the circuit by sets of closely located nodes. Any such set is called \emph{a window} (as in power analysis attack terminology e.g. from~\cite{BosApects}). The described attacks can be applied to each window instead of the full circuit. By varying the window size the attacks may become more efficient. Here we do not investigate methods of choosing windows to cover a given circuit. One possible approach is to assign each level or a sequence of adjacent levels in the circuit to a window. Choosing the full circuit as a single window is also allowed. In our attacks we assume that a coverage is already chosen. For simplicity, we describe how each attack is applied to a single window. In case when multiple windows are chosen, the attack has to be repeated for each window. The window size is denoted by $\winsize$. It is equal to the circuit size in the case of the single window coverage.

% -------------------------------------------------------------------

\subsubsection{General DCA Attack}

I would like to note that the term ``differential computation analysis'' (DCA) is very general. In~\cite{AttackBos} the authors introduced it mainly for the correlation-based attack. In fact our new attacks fit the term well and provide new tools for the ``analysis'' stage of the attack. The first stage remains the same except that we adapt the terminology for the case of Boolean circuits instead of recording the memory access traces. Our view of the procedure of the DCA attack on a white-box implementation $C$ is given in \AlgRef{dca}

\begin{algorithm}[ht]
\AlgDef{dca}{General procedure of DCA attacks on a Boolean circuit $C\colon \field{N} \to \field{M}$}

\begin{algorithmic}[1]
    \State generate a random tuple of plaintexts $\textset = (p_1, p_2, \ldots), p_i \in \field{N}$

    \ForAll{$p_i \in \textset$}
        \State compute the circuit $C$ on input $p_i$: $c_i \gets C(p_i) \in \field{M}$

        \ForAll{$j \in \seg{1}{|C|}$}
            \State $v_{j,i} \gets$ computed value in the node indexed $j$
        \EndFor

        \ForAll{$j \in \seg{1}{k}$}
            \State $\pv_{j,i} \gets$ predictable value indexed $j$
            \Statex \hspace{2.35cm} computed from plaintext $p_i$ and/or ciphertext $c_i$
        \EndFor
    \EndFor

    \State generate the list of all computed vectors:
    \Statex $\vecset \gets (v_1, \ldots, v_{|C|})$, where $v_j = (v_{j,1}, \ldots, v_{j,\numtext}) \in \field{\numtext}$

    \State generate the list of all predictable vectors:
    \Statex $\predset \gets (\pv_1, \ldots, \pv_{\numpred})$, where $\pv_j = (\pv_{j,1}, \ldots, \pv_{j,\numtext}) \in \field{\numtext}$

    \State choose a coverage $\mathcal{P}$ of $\vecset$ by windows of size $\winsize$

    \ForAll{$W \in \mathcal{P}$}
        \State\Label{dca-step-window} perform analysis on the window $W \subseteq \vecset$
        \Statex \hspace{0.5cm} using the set of predictable vectors~$\predset$
    \EndFor
\end{algorithmic}
\end{algorithm}
We remark that the correlation-based DCA attack from \cite{AttackBos} can be implemented on-the-fly, without computing the full vectors $v_j$. In contrast, most of our attacks require full vectors. Though, various optimizations are possible.

In the following two sections I describe two classes of DCA attacks: combinatorial and algebraic. They both follow the procedure described above and differ only in the analysis part (Step~\Ref{dca-step-window}). Afterwards, I describe two fault-injection attacks which allow to find locations of shares efficiently.


\documentclass[t,usenames,dvipsnames]{beamer}
\usetheme{Copenhagen}
\setbeamertemplate{headline}{} % remove toc from headers
\beamertemplatenavigationsymbolsempty

\usepackage{amsmath, xcolor, tikz, pgfplots}

\pgfplotsset{compat = 1.16}
\usetikzlibrary{arrows.meta, calc, decorations.pathreplacing}
\pgfplotsset{every axis/.append style = {axis lines = middle}}
\pgfplotsset{every tick label/.append style={font=\scriptsize}}
\everymath{\displaystyle}

\title{Graphs of Quadratic Expressions}
\author{}
\date{}

\AtBeginSection[]
{
  \begin{frame}
    \frametitle{Objectives}
    \tableofcontents[currentsection]
  \end{frame}
}

\begin{document}

\begin{frame}
    \maketitle
\end{frame}

\section{Determine the vertex and axis of symmetry of a quadratic function in standard form}

\begin{frame}{Graphing Quadratic Expressions}
We will be looking at graphing equations in the form
\[ y = ax^2 + bx + c \]
where $a$, $b$, and $c$ are real numbers with $a \neq 0$.
\end{frame}

\begin{frame}{Graph of a Quadratic Expression}
For $y= x^2$, the graph below is a \alert{parabola}.
\begin{center}
\begin{tikzpicture}[scale=0.8]
\begin{axis}[
xmin = -3.5, xmax = 3.5,
ymin = -0.5, ymax = 4.75,
xtick = {-3,-2,...,3},
ytick = {1,2,3,4}
]
\addplot[color=blue, very thick, samples=200, smooth] {x^2};
\addplot[color=blue, mark=*, only marks] coordinates {(-2,4) (-1,1) (0,0) (1,1) (2,4)};
\node at (axis cs:-2,4) [left] {$(-2,4)$};
\node at (axis cs:-1,1) [left] {$(-1,1)$};
\node at (axis cs:0,0) [below] {$(0,0)$};
\node at (axis cs:1,1) [right] {$(1,1)$};
\node at (axis cs:2,4) [right] {$(2,4)$};
\end{axis}
\end{tikzpicture}
\end{center}
\end{frame}

\begin{frame}{Graph of a Quadratic Expression}
The point $(0,0)$ is called the \alert{vertex} of the parabola and can be either a minimum (smile) or maximum point (frown). \newline\\  \pause

Through the vertex is a vertical line called the \alert{axis of symmetry} that divides the parabola into 2 equal halves.
\end{frame}

\begin{frame}{Example 1}
Find the vertex, state whether the vertex is a maximum or minimum, and find the equation of the axis of symmetry for each.   \newline\\
(a) \quad $y = x^2 + 4x + 1$  \newline\\
\begin{minipage}{0.6\textwidth}
\onslide<2->{
\begin{tikzpicture}[scale=0.8]
\begin{axis}[
xmin = -5, xmax = 1,
ymin = -4, ymax = 2,
xtick = {-4,-3,-2,-1,0},
ytick = {-3,-2,-1,0,1}
]
\addplot[color=blue, very thick, samples=200, smooth] {(x+2)^2 - 3};
\only<5->{\draw[color=red, dashed, very thick, <->, >=stealth] (axis cs: -2,-4) -- (axis cs: -2,2);}
\end{axis}
\end{tikzpicture} }
\end{minipage}
\hspace{-0.5cm}
\begin{minipage}{0.4\textwidth}
\onslide<3->{Vertex $(-2,-3)$}	\\[8pt]
\onslide<4->{Vertex is a minimum}	\\[8pt]
\onslide<5->{Axis of symmetry: $x = -2$}
\end{minipage}
\end{frame}

\begin{frame}{Example 1}
(b) \quad $y = -2x^2+12x-17$    \newline\\
\begin{minipage}{0.55\textwidth}
\onslide<2->{
\begin{tikzpicture}[scale=0.8]
\begin{axis}[
xmin = -1, xmax = 7,
ymin = -8, ymax = 2,
xtick = {-1,0,...,5},
ytick = {-7,-6,...,1}
]
\addplot[color=blue, very thick, samples=200, smooth, domain=0:6] {-2*(x-3)^2 + 1};
\only<5->{\draw[color=red, dashed, very thick, <->, >=stealth] (axis cs: 3,-8) -- (axis cs: 3,2);}
\end{axis}
\end{tikzpicture}}
\end{minipage}
\hspace{-0.25cm}
\begin{minipage}{0.4\textwidth}
\onslide<3->{Vertex $(3,1)$} \\[8pt]
\onslide<4->{Vertex is a maximum} \\[8pt]
\onslide<5->{Axis of symmetry: $x=3$}
\end{minipage}
\end{frame}

\section{Convert between standard and general form of quadratic expressions}

\begin{frame}{Standard and General Form of Quadratic Expressions}
For a quadratic function: \newline\\
\begin{itemize}
    \item<+-> The \alert{general form} is $y = ax^2+bx+c$
    \begin{itemize}
        \item<+-> $a$, $b$, and $c$ are real numbers
        \item<+-> $a \neq 0$
    \end{itemize}   \vspace{8pt}
    \item<+-> The \alert{standard form} is $y = a(x-h)^2 + k$
    \begin{itemize}
        \item<+-> Vertex is $(h,k)$
        \item<+-> $a \neq 0$
        \item<+-> $a$, $h$, and $k$ are real numbers
    \end{itemize}
\end{itemize}
\end{frame}

\begin{frame}{Converting From General to Standard Form}
    To convert from general form $y = ax^2 + bx + c$ to standard form $y = a(x-h)^2+k$ \newline\\
    \begin{enumerate}
        \onslide<2->{\item Find the vertex:}  \\[8pt]
        \begin{itemize}
            \onslide<2->{\item $x$-coordinate: $\frac{-b}{2a}$} \\[8pt]
            \onslide<2->{\item $y$-coordinate: Evaluate expression at $x$-coordinate} \\[8pt]
            \onslide<2->{\item Or use graphing technology} \\[8pt]
        \end{itemize}
        \onslide<2->{\item Use the same value of $a$}
    \end{enumerate}
\end{frame}

\begin{frame}{Example 2}
Convert each to standard form.  \newline\\
(a) \quad $y = x^2 - 4x + 3$ \newline\\
\onslide<2->{Vertex:} \newline\\
\onslide<3->{$x = \frac{-(-4)}{2(1)}$} \newline\\
\onslide<4->{$x = 2$} \newline\\
\onslide<5->{$y = 2^2 - 4(2) + 3$} \newline\\
\onslide<6->{$y = -1$}
\end{frame}

\begin{frame}{Example 2}
(a) \quad $y = x^2 - 4x + 3$ \newline\\
Vertex: $(2,-1)$    \newline\\
\onslide<2->{$a = 1$}
\onslide<3->{\[y = (x-2)^2 - 1\]}
\end{frame}

\begin{frame}{Example 2}
Convert each to standard form.  \newline\\
(b) \quad $y = 6-x-x^2$
\onslide<2->{\[y = -x^2 - x + 6\]}
\begin{minipage}{0.55\textwidth}
\onslide<3->{
\begin{tikzpicture}[scale=0.8]
\begin{axis}[
xmin = -4, xmax = 3,
ymin = -4, ymax = 7,
xtick = {-3,-2,...,2},
ytick = {-3,-2,...,6}
]
\addplot[color=blue, very thick, samples=200, smooth] {-x^2-x+6};
\addplot[color=blue, mark=*] coordinates {(-0.5,6.25)};
\end{axis}
\end{tikzpicture}}
\end{minipage}
\begin{minipage}{0.4\textwidth}
\onslide<4->{Vertex: $\left(-\frac{1}{2},\frac{25}{4}\right)$} \\[12pt]
\onslide<5->{$a = -1$} \\[12pt]
\onslide<6->{$y = -\left(x+\frac{1}{2}\right)^2 + \frac{25}{4}$}
\end{minipage}
\end{frame}

\begin{frame}{Converting From Standard to General Form}
To convert from \newline
\[ y = a(x-h)^2 + k \]	\newline
form to 	\newline
\[ y = ax^2 + bx + c\]	\newline
just {\color{blue}\textbf{do the math}} and remember your \underline{order of operations}.
\end{frame}

\begin{frame}{Example 3}
(a) \quad Convert $y = (x+2)^2 - 3$ to general form.
\begin{align*}
	\onslide<2->{y &= (x+2)^2 - 3} \\[8pt]
    \onslide<3->{y &= x^2 + 4x + 4 - 3} \\[8pt]
    \onslide<4->{y &= x^2 + 4x + 1}	\\[8pt]
    \onslide<5->{y &= x^2 + 4x + 1}
\end{align*}
\end{frame}

\begin{frame}{Example 3}
(b) \quad Convert $y = -(x-7)^2 + 10$ to general form.
\begin{align*}
	\onslide<2->{y &= -(x-7)^2 + 10} \\[8pt]
	\onslide<3->{y &= -(x^2-14x+49) + 10} \\[8pt]
	\onslide<4->{y &= -x^2 + 14x - 49 + 10} \\[8pt]
	\onslide<5->{y &= -x^2 + 14x - 39}
\end{align*}
\end{frame}

\end{document}


\section{Architecture}
\label{sec:training_of_the_cnn:architecture}

Designing a \acrlong{cnn} architecture from scratch requires choosing the types of layers and their arrangement as well as many hyperparameters.
For this reason, a lot of trial an error is involved in the design process of an adequate model \cite{training_arch_design}.
There are, however, certain design principles that work really well \cite{training_arch_hyper}:

\begin{enumerate}
  \item Starting with a low number of filters (high-level feature detection)
  \item Increasing the number of filters towards the end (low-level feature detection)
  \item Decreasing the spatial dimensions of the feature maps towards the end
  \item Using kernel sizes of $3\times 3$, $5\times 5$ or $7\times 7$ for convolutional layers
  \item Using pool sizes of $2\times 2$ or $3\times 3$ with a stride of two for max-pooling layers
  \item Adding additional layers until the model is overfitting
  \item Using state-of-the-art networks as inspiration
\end{enumerate}

A summary of all the layers of the final \acrshort{cnn} architecture is listed in table \ref{tab:arch}.
The first convolutional layer uses only \num{16} filters and a larger kernel size of $5\times 5$ due to the large dimensions of the input images.
All other convolutional layers use a kernel size of $3\times 3$ while steadily increasing the number of used filters up to \num{128}.
The designed architecture uses six max-pooling layers with a pool size of $2\times 2$ and a stride of two.
This reduces the spatial dimensions of the feature maps to $4\times 5$ (height $\times$ width).
The output of the fully-connected layer \texttt{fc8} is about the size of the output layer \texttt{fc9} squared.
This allows the output layer to combine many of the different high-level features to create a confident prediction.

Even though the model was not yet overfitting, no additional layers were added.
The reason for this is that with the current architecture the classification performance is already exceptional.

Furthermore, the number of trainable parameters (weights) is relative low compared to state-of-the-art networks like VGG, ResNet or Inception \cite{training_arch_keras}.
This increases the throughput considerably, as less mathematical operations are required.
The key to keeping the total number of trainable parameter low is evident when analyzing table \ref{tab:arch}.
A whopping \num{1311232} of the \num{1614486} total weights can be attributed to the connection between the feature maps of the last convolutional layer \texttt{conv7} and the artificial output neurons of the first dense layer \texttt{fc8}.
This accounts for \SI{81.22}{\percent} of all trainable parameters.
For this reason the spatial dimensions of the feature maps should be decreased towards the end.

\begin{table}
  \caption{Layers of the \acrshort{cnn} architecture}
  \label{tab:arch}
  \centering
  \begin{tabular}{lllllll}
    \toprule
    \textbf{Layer} & \textbf{Type} & \textbf{Activation} & \textbf{Filters} & \textbf{Kernel} & \textbf{Output Shape} & \textbf{Param \#} \\
    \midrule
    \textbf{conv1} & \texttt{Conv2D} & \acrshort{relu} & \num{16} & $5\times 5$ & $256\times 320\times 16$ & \num{1216} \\
    \textbf{pool1} & \texttt{MaxPooling2D} &  &  & $2\times 2$ & $128\times 160\times 16$ & \num{0} \\
    \midrule
    \textbf{conv2} & \texttt{Conv2D} & \acrshort{relu} & \num{32} & $3\times 3$ & $128\times 160\times 32$ & \num{4640} \\
    \textbf{pool2} & \texttt{MaxPooling2D} &  &  & $2\times 2$ & $64\times 80\times 32$ & \num{0} \\
    \midrule
    \textbf{conv3} & \texttt{Conv2D} & \acrshort{relu} & \num{32} & $3\times 3$ & $64\times 80\times 32$ & \num{9248} \\
    \textbf{pool3} & \texttt{MaxPooling2D} &  &  & $2\times 2$ & $32\times 40\times 32$ & \num{0} \\
    \midrule
    \textbf{conv4} & \texttt{Conv2D} & \acrshort{relu} & \num{64} & $3\times 3$ & $32\times 40\times 64$ & \num{18496} \\
    \textbf{pool4} & \texttt{MaxPooling2D} &  &  & $2\times 2$ & $16\times 20\times 64$ & \num{0} \\
    \midrule
    \textbf{conv5} & \texttt{Conv2D} & \acrshort{relu} & \num{64} & $3\times 3$ & $16\times 20\times 64$ & \num{36928} \\
    \textbf{pool5} & \texttt{MaxPooling2D} &  &  & $2\times 2$ & $8\times 10\times 64$ & \num{0} \\
    \midrule
    \textbf{conv6} & \texttt{Conv2D} & \acrshort{relu} & \num{128} & $3\times 3$ & $8\times 10\times 128$ & \num{73856} \\
    \textbf{pool6} & \texttt{MaxPooling2D} &  &  & $2\times 2$ & $4\times 5\times 128$ & \num{0} \\
    \midrule
    \textbf{conv7} & \texttt{Conv2D} & \acrshort{relu} & \num{128} & $3\times 3$ & $4\times 5\times 128$ & \num{147584} \\
    \midrule
    \textbf{flatten} & \texttt{Flatten} &  &  &  & \num{2560} & \num{0} \\
    \textbf{fc8} & \texttt{Dense} & \acrshort{relu} &  &  & \num{512} & \num{1311232} \\
    \textbf{fc9} & \texttt{Dense} &  &  &  & \num{22} & \num{11286} \\
    \bottomrule
  \end{tabular}
\end{table}

% ------------------------------------------------------------------------------------------------------------------------------
\subsection{Visualization}
\label{subsec:training_of_the_cnn:architecture:visualization}
Figure \ref{fig:arch} visualizes the final architecture of the \acrshort{cnn} model.
The visualization was created with Ti\textit{k}Z and the help of the open-source repository \textit{PlotNeuralNetwork} \cite{training_arch_plot}.

The boxes represent the outputs of the different layers.
On the one hand, the light orange boxes represent the feature maps of the convolutional layers and, on the other hand the purple boxes represent the artificial output neurons of the dense layers.
The darker colored bands on the boxes indicate that the \acrshort{relu} activation function is applied.
The red boxes represent max-pooling layers, which decrease the spatial dimensions.
The dashed lines between the output of \texttt{conv7} and \texttt{fc8} depict the flattening in addition to the dense connection.

\begin{figure}
  \centering
  \includegraphics[width=\textwidth]{arch}
  \caption{Final architecture of the \acrlong{cnn}}
  \label{fig:arch}
\end{figure}

% ------------------------------------------------------------------------------------------------------------------------------
\subsection{Implementation}
\label{subsec:training_of_the_cnn:architecture:implementation}
The \acrshort{cnn} architecture is defined in the Python script \texttt{cnn.py}, as shown in listing \ref{lst:arch}.
The script uses the open-source software library TensorFlow v2.2.0, along with the high-level Keras \acrshort{api} implemented in the \texttt{tf.keras} module \cite{training_arch_tf_keras}.
To define the architecture a \texttt{Sequential} model is used to arrange the desired layers in a plain stack \cite{training_arch_tf_keras_seq}.

\begin{lstlisting}[style=python, caption={Sequential model}, label=lst:arch]
# Convolutional Neural Network Architecture
# Convolution layers
model = models.Sequential()
model.add(layers.Conv2D(16, (5, 5), padding='same', activation='relu', input_shape=fh.inf_shape))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(32, (3, 3), padding='same', activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(32, (3, 3), padding='same', activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), padding='same', activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), padding='same', activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), padding='same', activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), padding='same', activation='relu'))

# Dense layers
model.add(layers.Flatten())
model.add(layers.Dense(512, activation='relu'))
model.add(layers.Dense(22))
\end{lstlisting}


\documentclass[letter]{article}
\renewcommand{\baselinestretch}{1.25}

\usepackage[margin=1in]{geometry}
\usepackage{physics}
\usepackage{amsmath}
\usepackage{graphicx}
%\usepackage{pythonhighlight}
\usepackage{hyperref}
\usepackage{fancyvrb}

% MATLAB Formating Code
\usepackage[numbered,framed]{matlab-prettifier}
\lstset{style=Matlab-editor,columns=fullflexible}
\renewcommand{\lstlistingname}{Script}
\newcommand{\scriptname}{\lstlistingname}

% Command for easier minimization problem def
\newcommand{\optpblm}[3][eq:default]{
	\begin{equation}\label{#1}
% Array method... more centered
%		\begin{array}{rl}
%			\text{minimize}  \hspace{0.2in} &#2 \vspace{5pt}\\
%			\text{subject to} \hspace{0.2in} &#3
%		\end{array}
% Aligned method... left aligned... idk if its better
		\begin{aligned}
			\text{minimize} \hspace{0.5in} &#2\vspace{5pt}\\
			\text{subject to \hspace{0.5in}} &#3
		\end{aligned}
	\end{equation}
}

\newcommand{\maxpblm}[3][eq:default]{
	\begin{equation}\label{#1}
		\begin{aligned}
			\text{maximize} \hspace{0.5in} &#2\vspace{5pt}\\
			\text{subject to \hspace{0.5in}} &#3
		\end{aligned}
	\end{equation}
}
\allowdisplaybreaks

\title{MECH 6327 - Homework 4}
\author{Jonas Wagner}
\date{2021, April 12}

\begin{document}

\maketitle

\newpage
\tableofcontents

\newpage
\section*{BV Textobook Problems}
\subsection{Problem 5.43}
The dual a SOCP defined as:
\optpblm{f^T x}{\norm{A_i x + b_i}_2 \leq c_i^T x + d_i, \ i = 1,\dots,m}
with $x \in \real^n$ can be expressed as:
\maxpblm{\sum_{i=1}^m \qty(b_i^T u_i - d_i v_i)}{
	  \sum_{i=1}^m \qty(A_i^T u_i - c_i v_i) + f = 0\\
	& \norm{u_i}_2 \leq v_i, \ i = 1,\dots,m}
with variables $u_i \in \real^n_i$, $v_i \in \real, \ i=1,\dots,m$ and problem data $f\in \real^n, A_i \in \real^{n_i \cross n}, b_i \in \real^{n_i}, c_i \in \real, \ i = 1,\dots,m$.\\

\newpage
\subsubsection{Part a}
\textbf{Problem:}
Derive the dual by defining $y_i \in \real^{n_i}$ and $t_i \in \real$ and the equalities $y_i = A_i x + b_i, \ t_i = c_i^T x + d_i$ then deriving the Lagrange dual.\\

\noindent
\textbf{Solution:}
The problem can first be written in a standard form as:
\optpblm{f^T}{
	y_i = A_i x + b_i\\
	&t_i = c_i^T x + d_i\\
	&\norm{y_i}_2 \leq t_i, \ \forall i = 1, \dots, m}

The lagrange can then be defined with
\begin{align}
	L(x,y,t, \lambda_i, \nu_i, \mu_i)
	&= f^T
	+ \sum_{i=1}^m \lambda_i \qty(\norm{y_i}_2 - t_i)
	+ \sum_{i=1}^m \mu_i^T (t_i - C_i x + d_i)\\
	&= \qty(f + \sum_{i=1}^m \qty(A_i^T \mu_i - c_i \nu_i))^T x
	+ \sum_{i=1}^m \lambda_i \norm{y_i}_2 + \mu_i^T y_i \nonumber\\
	&\ \ \ + \sum_{i=1}^m \qty(-\lambda_i + \nu_i) t_i
	- \qty(\sum_{i=1}^m b_i^T \mu_i - d_i \nu_i)
\end{align}

Since the definition of the dual optimization problem is to maximize $$g(\lambda_i, \nu_i, \mu_i) = \inf_{x,y_i,t_i} L(x,y,t, \lambda_i, \nu_i, \mu_i)$$ the $\inf$ can be found by determining when a min/max would occur for each of the variables.

For the critical point on $x$ the direvative can be set to zero and thus the following equality must hold:
\begin{equation}
	f + \sum_{i=1}^m \qty(A_i^T \mu_i - c_i \nu_i) = 0
\end{equation}

For the $y_i$ related term, it is known $$\sum_{i=1}^m \lambda_i \norm{y_i}_2 + \mu_i^T y_i$$ will be bounded below if it is within the cone defined by $\lambda_i \norm{y_i}_2 \geq \norm{\mu_i}_2 y_i$ which can be rewritten as: $$\norm{\mu_i}_2 \leq \lambda_i$$

For the critical point over $t_i$ the equality $\nu_i = \lambda_i$.

From this the dual problem can be obtained when the quantity $\qty(\sum_{i=1}^m b_i^T \mu_i - d_i \nu_i)$ is maximized.

Thus the dual problem is defined as:
\maxpblm{\sum_{i=1}^m \qty(b_i^T u_i - d_i v_i)}{
	\sum_{i=1}^m \qty(A_i^T u_i - c_i v_i) + f = 0\\
	& \norm{u_i}_2 \leq v_i, \ i = 1,\dots,m}

\newpage
\subsubsection{Part b}
\textbf{Problem:}
Start with the conic formulation of the SOCP and use the conic dual to prove the equivalence. Use the fact that the second-order dual is self-dual.\\

\noindent
\textbf{Solution:}
Starting with the SOCP given as
\optpblm{f^T x}{\norm{A_i x + b_i}_2 \leq c_i^T x + d_i, \ i = 1,\dots,m}

a standard form can be defined by
\optpblm{f^T x}{\qty(A_i x + b_i, c_i^T x + d_i) \preceq_{2} 0}

Since the conic dual transformation is known to transform
\optpblm{f^T x}{- \qty(A^T x + b, c^T x + d) \preceq_{K} 0}

into its dual according to its dual cone definition
\maxpblm{b^T u + d \ v}{A^T u + v c = f\\ &(u,v) \succeq_{K^*} 0, \ i = 1,\dots,m}

and from the fact that the 2-norm is self-dual, the dual program is given as
\maxpblm{\sum_{i=1}^m \qty(b_i^T u_i - d_i v_i)}{
	- \sum_{i=1}^m \qty(A_i^T u_i - c_i v_i) = f\\
	& (u_i, v_i) \succeq_{2} 0, \ i = 1,\dots,m}

or equivalently
\maxpblm{\sum_{i=1}^m \qty(b_i^T u_i - d_i v_i)}{
	\sum_{i=1}^m \qty(A_i^T u_i - c_i v_i) + f = 0\\
	& \norm{u_i}_2 \leq v_i, \ i = 1,\dots,m}

\newpage
\section{Problem 1: Robust control design}
For the standard DT dynamical system defined as:
\begin{equation}
	x_{t+1} = A x_t + B u_t
\end{equation}
with dynamic matrix $A$ unknown but assumed to belong to a set:
\begin{equation}
	A \in \mathcal{A} = \text{conv}(A_1,\dots,A_m)
\end{equation}
with $A_i$ and $B$ known.\\

\textbf{Problem:}
For a state-feedback controller $u_t = K x_t$ use Lyapunov techniques to design it so the system is globally asymptotically stable (GAS) by solving a semi-definite program (SDP).\\

\textbf{Solution:}
The closed-loop system for the DT dynamical system can be defined by the dynamics
\begin{equation}
	x_{t+1} = \hat{A} x = (A + B K) x
\end{equation}
where $\hat{A} = A + B K$.

In order for the closed-loop system to be Globally Asymptotically Stable, a quadratic Lyapnov Function could be used to prove that if the following inequality is true then the system is GAS:
\begin{equation}
	\hat{A}^T P \hat{A} - P \prec 0
\end{equation}

Since the system dynamics themselves are uncertain, this inequality will not be enough to prove GAS. This can be be address, however, by considering all $A \in \mathcal{A}$ to be a linear combination of the individual corner matrices. Since this is a convex hull, it is known that following this to its conclusion, GAS can be guaranteed for all $A \in \mathcal{A}$ if $A_i$ is GAS $\forall i = 1, \dots, m$.

Following this, a stabalizing gain can then be found as follows:
\begin{align}
	\hat{A}_i^T P \hat{A}_i - P \prec 0
\end{align}
recognizing the Schur's compliment form, the following is true
\begin{align}
	\mqty[P & \hat{A}_i^T\\ \hat{A}_i P^{-1}] \succ 0\\
	\mqty[P & (A_i + B K)^T\\ (A_i + B K) &P^{-1}] \succ 0
\end{align}

A SDP feasibility problem can then  be done to solve the problem such that
\begin{equation}
	\begin{aligned}
		\mqty[P & \hat{A}_i^T\\ \hat{A}_i P^{-1}] \succ 0\\
		B^{-1} \qty(\hat{A}_i - A_i) - K = 0, \ \forall i = 1, \dots, m
	\end{aligned}
\end{equation}
with variables $P$, $\hat{A}_i$, and $K$, along with problem data $A_i$ and $B$.
This is a problem that can now be easily implemented using CVX or YALMIP in MATLAB for given problem data.

\newpage
\section{Problem 2: Nonnegative and sum of squares polynomials}
The Motzkin polynomial is defined as:
\begin{equation}
	M(x,y) = x^2 y^4 + x^4 y^2 + 1 - 3 x^2 y^2
\end{equation}

\textbf{Problem:}
Show that the Motzkin polynomial is nonnegative but can be expressed as sum of squares. It is sufficient to show this using numerical and/or symbolic solvers.\\

\textbf{Solution:}
Nonegativity of the Motzkin polynomial can be proven using the AM-GM inequality using $n=3$ with $x^4y^2, x^2 y^4, 1$.
\begin{align}
	\sqrt[n]{\prod_{i=1}^{n} x_i} &\leq \frac{1}{n} \sum_{i=1}^n x_i\\
	\sqrt[3]{(x^4 y^2)(x^2 y^4)(1)} &\leq \frac{1}{3} (x^4 y^2 + x^2 y^4 + 1)\\
	x^2 y^2 &\leq \frac{1}{3} (x^4 y^2 + x^2 y^4 + 1)\\
	3 x^2 y^2 &\leq x^4 y^2 + x^2 y^4 + 1\\
	x^4 y^2 + x^2 y^4 + 1 - 3 x^2 y^2 &\geq 0
\end{align}
Therefore the Motzkin polynomial is non-negative.

However, it is not possible to put this into sum of square form using the standard solver as demonstrated by the infeasability result from the solvesos() command in yalmip (shown in \appendixname \ref{apx:matlab})

\newpage
\appendix
\section{MATLAB Code:}\label{apx:matlab}
All code I write in this course can be found on my GitHub repository:\\
\href{https://github.com/jonaswagner2826/MECH6337}{https://github.com/jonaswagner2826/MECH6327}
\lstinputlisting[caption={MECH6327\_HW4},label={script:HW4}]{MECH6327_HW4.m}

\newpage
\textbf{Refrences:} * not bibtex becouse of time...

https://people.eecs.berkeley.edu/~elghaoui/Teaching/EE227A/lecture10.pdf

https://people.orie.cornell.edu/miketodd/iccopt.pdf

\end{document}