\subsubsection{Support Vector Regression}
\label{svm}

\cite{vapnik1963} and \cite{vapnik1964} introduce the so-called support vector
    machine (SVM) model, and \cite{vapnik2013} summarizes the research
    conducted since then.
In its basic version, SVMs are linear classifiers, modeling a binary
    decision, that fit a hyperplane into the feature space of $\mat{X}$ to
    maximize the margin around the hyperplane seperating the two groups of
    labels.
SVMs were popularized in the 1990s in the context of optical character
    recognition, as shown in \cite{scholkopf1998}.

\cite{drucker1997} and \cite{stitson1999} adapt SVMs to the regression case,
    and \cite{smola2004} provide a comprehensive introduction thereof.
\cite{mueller1997} and \cite{mueller1999} focus on SVRs in the context of time
    series data and find that they tend to outperform classical methods.
\cite{chen2006a} and \cite{chen2006b} apply SVRs to predict the hourly demand
    for water in cities, an application similar to the UDP case.

In the SVR case, a linear function
    $\hat{y}_i = f(\vec{x}_i) = \langle\vec{w},\vec{x}_i\rangle + b$
    is fitted so that the actual labels $y_i$ have a deviation of at most
    $\epsilon$ from their predictions $\hat{y}_i$ (cf., the constraints
    below).
SVRs are commonly formulated as quadratic optimization problems as follows:
$$
\text{minimize }
\frac{1}{2} \norm{\vec{w}}^2 + C \sum_{i=1}^m (\xi_i + \xi_i^*)
\quad \text{subject to }
\begin{cases}
y_i - \langle \vec{w}, \vec{x}_i \rangle - b \leq \epsilon + \xi_i
\text{,} \\
\langle \vec{w}, \vec{x}_i \rangle + b - y_i \leq \epsilon + \xi_i^*
\end{cases}
$$
$\vec{w}$ are the fitted weights in the row space of $\mat{X}$, $b$ is a bias
    term in the column space of $\mat{X}$, and $\langle\cdot,\cdot\rangle$
    denotes the dot product.
By minimizing the norm of $\vec{w}$, the fitted function is flat and not prone
    to overfitting strongly.
To allow individual samples outside the otherwise hard $\epsilon$ bounds,
    non-negative slack variables $\xi_i$ and $\xi_i^*$ are included.
A non-negative parameter $C$ regulates how many samples may violate the
    $\epsilon$ bounds and by how much.
To model non-linear relationships, one could use a mapping $\Phi(\cdot)$ for
    the $\vec{x}_i$ from the row space of $\mat{X}$ to some higher
    dimensional space; however, as the optimization problem only depends on
    the dot product $\langle\cdot,\cdot\rangle$ and not the actual entries of
    $\vec{x}_i$, it suffices to use a kernel function $k$ such that
    $k(\vec{x}_i,\vec{x}_j) = \langle\Phi(\vec{x}_i),\Phi(\vec{x}_j)\rangle$.
Such kernels must fulfill certain mathematical properties, and, besides
    polynomial kernels, radial basis functions with
    $k(\vec{x}_i,\vec{x}_j) = exp(\gamma \norm{\vec{x}_i - \vec{x}_j}^2)$ are
    a popular candidate where $\gamma$ is a parameter controlling for how the
    distances between any two samples influence the final model.
SVRs work well with sparse data in high dimensional spaces, such as
    intermittent demand data, as they minimize the risk of misclassification
    or predicting a significantly far off value by maximizing the error
    margin, as also noted by \cite{bao2004}.


\numberwithin{equation}{section}

\section{Deep Convolutional Neural Fields}

In this Appendix, we show some technical details about the proposed deep convolutional field model.

Let $\x$ be an image and $\y=[y_1,\ldots, y_n]^{\T} \in \Real^n$ be
%the continuous real-valued vectorized depth map of
a vector of continuous depth values corresponding to
 all $n$ superpixels in $\x$.
 %Continuous \crf (CCRF)
 Similar to conventional CRF,
 we model the conditional probability distribution of the data with the following density function:
\begin{equation}\label{eq:prob}
\begin{aligned}
\Pr(\y|\x) = \frac{1}{\mathrm{Z}(\x)} \exp \{ -E(\y, \x) \},
\end{aligned}
\end{equation}
where $E$ is the energy function and $Z(\x)$ the partition function, defined respectively as:
\begin{equation}\label{eq:feature}
E(\y, \x) = \sum_{p \in {\cal N} } (y_p - \regress_p)^2
+ \sum_{(p,q) \in {\cal S}} \half \pws_{pq} (y_p - y_q)^2,
\end{equation}
\begin{equation}\label{eq:partition}
\mathrm{Z}(\x) = \int_{\y} \exp \{ -E(\y, \x) \}\mathrm{d}\y,
\end{equation}
in which,
\begin{equation} \label{eq:def_R}
\pws_{pq} = \sum_{k=1}^K \beta_k S_{pq}^{(k)},
\end{equation}
where $\regress$ is the regressed depths parametrized by $\btheta$ (namely, $\regress$ is the abbreviation of $\regress(\btheta)$), $\bbeta=[\beta_1, \ldots, \beta_K]$ the pairwise parameter, $\S^{(k)}$ the $k$-th similarity matrix (which is symmetric) and $K$ the number of pairwise terms considered.
To guarantee $Z(\x)$ (Eq. \eqref{eq:partition}) is integrable, $\bbeta_k > 0$ are required.
We aim to jointly learn $\regress(\btheta)$ and $\bbeta$ here.

By expanding Eq. \eqref{eq:feature}, we then have:
\begin{align} \label{eq:feature_expand}
E(\y, \x) &= \sum_p y_p^2 - 2\sum_p y_p \regress_p  + \sum_p \regress_p^2 + \half \sum_{pq} \pws_{pq} y_p^2 - \sum_{pq} \pws_{pq} y_p y_q + \half \sum_{pq} \pws_{pq} y_q^2 \notag \\
	&= \y^\T \y - 2\bregress^\T \y  + \bregress^\T \bregress +  \y^\T \D \y -  \y^\T \bpws \y  \notag \\
	&= \y^\T (\I+ \D - \bpws) \y - 2\bregress^\T \y + \bregress^\T \bregress \notag \\
	&= \y^\T \A \y - 2\bregress^\T \y + \bregress^\T \bregress,
\end{align}
%\bd = 2\regress
%\underbrace{(2\I+2\bbeta \D -2\bbeta \S)}_\text{\Sigma^{-1}}
where
\begin{align} \label{eq:def_A}
\A = \I+ \D - \bpws.
\end{align}
Here, $\I$ is the n$\times$n identity matrix; $\D$ is a diagonal matrix with $\D_{pp} = \sum_q \pws_{pq}$.
Since $\beta_k \geq 0$ are enforced, $\A$ is ensured to be positive definite ($\A$ is symmetric, and strictly diagonally dominant with positive diagonal entries). We can then calculate the partition function according to the Gaussian integral formula as:
\begin{align} \label{eq:part_expand}
Z(\x) &=\int_\y \exp \Big\{ -E(\y, \x) \Big\}\mathrm{d}\y   \notag \\
	&=  \int_{\y} \exp \Big\{ - \y^\T \A \y + 2\bregress^\T \y - \bregress^\T \bregress \Big\}\mathrm{d}\y  \notag  \\
	&=\exp \{  - \bregress^\T \bregress \} \int_{\y} \exp \Big\{ - \y^\T \A \y + 2\bregress^\T \y \Big\}\mathrm{d}\y \notag  \\
	&= \exp \{  - \bregress^\T \bregress \}\sqrt{\frac{{(2\pi)}^n}{|2\A|}} \exp \{{ \bregress ^\T \A^{-1} \bregress}\} \notag  \\
	&= \frac{{(\pi)}^{\frac{n}{2}}}{{|\A|}^{\frac{1}{2}}} \exp \{{ \bregress ^\T \A^{-1} \bregress   - \bregress^\T \bregress }\} ,
\end{align}
where $|\A|$ denotes the determinant of the matrix $\A$, and $\A^{-1}$ the inverse of $\A$.
From Eq. \eqref{eq:prob}, \eqref{eq:feature_expand}, \eqref{eq:part_expand}, we can write the probability density function as:
\begin{align} \label{eq:prob_gaussian}
\Pr(\y|\x) &=\frac{\exp \Big\{ -E(\y, \x) \Big\}} { Z(\x)}  \\ \notag
	&= \frac{\exp \Big\{ - \y^\T \A \y + 2\bregress^\T \y - \bregress^\T \bregress \Big\}}{\frac{{(\pi)}^{\frac{n}{2}}}{{|\A|}^{\frac{1}{2}}} \exp \{{ \bregress ^\T \A^{-1} \bregress   - \bregress^\T \bregress }\}}  \\ \notag
	&=\frac{ |\A|^{\half}} {{\pi^{\frac{n}{2}}}} \exp \Big\{ - \y^\T \A \y + 2\bregress^\T \y -  \bregress ^\T \A^{-1} \bregress \Big\}. \\ \notag
\end{align}
%%
%
%
According to Eq. \eqref{eq:prob_gaussian}, we can then rewrite the negative log-likelihood $-\log\Pr(\y|\x)$ as:
\begin{align} \label{eq:log-likelihood}
-\log\Pr(\y|\x)&= \y^\T \A \y - 2\bregress^\T \y + \bregress ^\T \A^{-1} \bregress - \half\log(|\A|) + \frac{n}{2}\log(\pi).
\end{align}

%In learning, CCRF minimizes the negative conditional log-likelihood of the training data,
%\begin{equation} \label{eq:ccrf}
%(\btheta^{\star}, \bbeta^{\star}) = \argmin_{\btheta, \bbeta} -L(\btheta, \bbeta) = \argmin_{\btheta, \bbeta} -\sum_{i=1}^N \log\Pr(\y^{(i)} | \x^{(i)}; \btheta, \bbeta)
%\end{equation}
In learning, we minimizes the negative conditional log-likelihood of the training data. Adding regularization to $\btheta$, $\bbeta$, we then arrive at the final optimization:
\begin{align} \label{eq:ccrf_final}
\min_{\btheta, \bbeta \geq \zeros} &-\sum_{i=1}^N \log\Pr(\y^{(i)} | \x^{(i)}; \btheta, \bbeta) + \frac{\lambda_1}{2} \fnorm \btheta + \frac{\lambda_2}{2} \fnorm \bbeta,
\end{align}
where $\x^{(i)}$, $\y^{(i)}$ denote the $i$-th training image and the corresponding depth map; $N$ is the number of training images; $\lambda_1$ and $\lambda_2$ are weight decay parameters.

For the unary part,
here we calculate the partial derivatives of $-\log\Pr(\y|\x)$ with respect to $\theta_l$ (one element of the network parameters $\btheta$ for the unary part ). Recall that $\A = \I+ \D - \bpws$ (Eq. \eqref{eq:def_A}), $\A^{\T}=\A$, $(\A^{-1})^{\T}=\A^{-1}$, $|\A^{-1}|=\frac{1}{|\A|}$, we have:
\begin{align}  \label{eq:derive_z}
\frac{\partial \{ -\log\Pr(\y|\x)\} }{\partial \theta_l} &= \frac{\partial \{-2\bregress^\T \y + \bregress ^\T \A^{-1} \bregress\}}{\partial \theta_l}   \\ \notag
&= \frac{\partial \{-2\bregress^\T \y \} } {\partial \theta_l} + \frac{\partial \{ \bregress ^\T \A^{-1} \bregress\}}{\partial \theta_l}   \\ \notag
&= -2 \frac{\partial \{ \sum_p \regress_p y_p \} } {\partial \theta_l} + \frac{\partial \{ \sum_{pq} \regress_p \regress_q A^{-1}_{pq} \}}{\partial \theta_l}   \\ \notag
&= -2 \sum_p \Big ( y_p\frac{\partial \regress_p } {\partial \theta_l} \Big ) + \sum_{pq} \Big ( z_p \frac{\partial  \regress_q} {\partial \theta_l} + z_q \frac{\partial  \regress_p} {\partial \theta_l} \Big ) A^{-1}_{pq} \\ \notag
&= -2 \y^{\T} \frac{\partial \bregress } {\partial \theta_l} + 2\bregress^{\T} \A^{-1} \frac{\partial \bregress } {\partial \theta_l}\\ \notag
&= 2 (\A^{-1} \bregress - \y)^{\T} \frac{\partial \bregress}{\partial \theta_l} .
\end{align}

%For the pairwise part, we first calculate the partial derivatives of $-\log\Pr(\y|\x)$ with respect to $\pws_{pq}$:
%\begin{align}  \label{eq:derive_R}
%\frac{\partial \{ -\log\Pr(\y|\x)\} }{\partial \pws_{pq}} &= \frac{\partial \{\y^\T \A \y + \bregress ^\T \A^{-1} \bregress - \half\log(|\A|)\}}{\partial \pws_{pq}}   \notag \\
%&= \frac{\partial \{ \y^\T \A \y \}}{\partial \pws_{pq}} +  \frac{\partial \{\bregress ^\T \A^{-1} \bregress\}}{\partial \pws_{pq}} - \half \frac{\partial \log(|\A|)}{\partial \pws_{pq}}, \notag \\
%&=\y^\T\frac{\partial  \A }{\partial \pws_{pq}}\y - \bregress^\T \A^{-1} \frac{\partial \A }{\partial \pws_{pq}} \A^{-1} \bregress  - \half \frac{1}{|\A|} \frac{\partial \{|\A|\} }{\partial \pws_{pq}}, \notag \\
%&=\y^\T \frac{\partial  \A }{\partial \pws_{pq}} \y - \bregress^\T \A^{-1} \frac{\partial \A }{\partial \pws_{pq}} \A^{-1} \bregress  - \half \trace \Big ( \A^{-1}  \frac{\partial \A} {\partial \pws_{pq}} \Big ), \notag \\
%&=\y^\T \G \y - \bregress^\T \A^{-1} \G \A^{-1} \bregress  - \half \trace \Big ( \A^{-1}  \G\Big ).
%\end{align}
%Here we introduce a matrix $\G=\frac{\partial  \A }{\partial \pws_{pq}}$. Each element of $\G$ is:
%\begin{align} \label{eq:def_G}
%G_{pq} &= \frac{\partial A_{pq} }{\partial \pws_{pq}}  \notag \\
%&= \frac{\partial \{  D_{pq} - \pws_{pq} \} }{\partial \pws_{pq}}  \notag \\
%%&= \frac{\partial \{  \sum_q \pws_{pq} - \pws_{pq} \} }{\partial \pws_{pq}}  \notag \\
%&= -\delta(p \neq q),
%\end{align}
%where $\delta(\cdot)$ is the indicator function, which equals 1 if $p \neq q$ is true and 0 otherwise.

Next, for the pairwise part, we calculate the  partial derivatives of $-\log\Pr(\y|\x)$ with respect to $\beta_k$ as:
\begin{align}  \label{eq:derive_beta}
\frac{\partial \{ -\log\Pr(\y|\x)\} }{\partial \beta_k} &= \frac{\partial \{\y^\T \A \y + \bregress ^\T \A^{-1} \bregress - \half\log(|\A|)\}}{\partial \beta_k}   \notag \\
&= \frac{\partial \{ \y^\T \A \y \}}{\partial \beta_k} +  \frac{\partial \{\bregress ^\T \A^{-1} \bregress\}}{\partial \beta_k} - \half \frac{\partial \log(|\A|)}{\partial \beta_k}, \notag \\
&=\y^\T\frac{\partial  \A }{\partial \beta_k}\y - \bregress^\T \A^{-1} \frac{\partial \A }{\partial \beta_k} \A^{-1} \bregress  - \half \frac{1}{|\A|} \frac{\partial \{|\A|\} }{\partial \beta_k}, \notag \\
&=\y^\T \frac{\partial  \A }{\partial \beta_k} \y - \bregress^\T \A^{-1} \frac{\partial \A }{\partial \beta_k} \A^{-1} \bregress  - \half \trace \Big ( \A^{-1}  \frac{\partial \A} {\partial \beta_k} \Big ).
\end{align}
We here introduce matrix $\J$ to denote $\frac{\partial  \A }{\partial \beta_k}$. Each element of $\J$ is:
\begin{align}  \label{eq:def_J}
J_{pq} &= \frac{\partial A_{pq}}{\partial \beta_k}  \notag \\
&= \frac{\partial \{ D_{pq} - R_{pq} \} }{\partial \beta_k}  \notag \\
&= \frac{\partial D_{pq}  }{\partial \beta_k}  - \frac{\partial R_{pq}  }{\partial \beta_k} \notag \\
&= - \frac{\partial  \pws_{pq} }{\partial \beta_k} + \delta(p=q) \sum_q \frac{\partial  \pws_{pq} }{\partial \beta_k},
\end{align}
where $\delta(\cdot)$ is the indicator function, which equals 1 if $p=q$ is true and 0 otherwise.
From Eq. \eqref{eq:derive_beta}, Eq. \eqref{eq:def_J}, we can see that our framework is general, therefore more complicated networks for the pairwise part can be seamlessly incorporated.
Here, in our case, with the definition of $\pws_{pq}$ in Eq. \eqref{eq:def_R}, we have $\frac{\partial  \pws_{pq} }{\partial \beta_k}=S_{pq}^{(k)}$.

According to Eq. \eqref{eq:derive_beta} and the definition of  $\J$ in \eqref{eq:def_J}, we can now write the partial derivative of $-\log\Pr(\y|\x)$ with respect to $\beta_k$  as:
\begin{equation}  \label{eq:derive_beta_final}
\frac{\partial \{ -\log\Pr(\y|\x)\} }{\partial \beta_k} =  \y^{\T} \J \y - \bregress^{\T} \A^{-1} \J \A^{-1}\bregress - \half \trace \Big( \A^{-1} \J \Big).
\end{equation}
%%
%Using the chain rule, we can write the partial derivatives of  $-\log\Pr(\y|\x)$ with respect to $\theta_l$ as:
%\begin{align}
%\frac{\partial \{ -\log\Pr(\y|\x) \} } {\partial \theta_l}
% = \; & \frac{\partial \{ -\log\Pr(\y|\x) \} }{\partial \bregress} \frac{\partial \bregress }{\partial \theta_l}   \notag \\
% = \; & \Big( 2 \A^{-1} \bregress - 2\y \Big ) \frac{\partial \bregress }{\partial \theta_l} , \label{eq:derive_z}
%\end{align}

\paragraph{Depth prediction}
Predicting the depths of a new image is to solve the MAP inference.
Because of the quadratic form of $\y$ in Eq. \eqref{eq:log-likelihood}, closed form solutions exist (details refer to supplementary): % \cite{ccrf_nips08}:
\begin{align} \label{eq:inf_solution1}
\y^{\star}&=\argmax_{\y} \Pr(\y|\x)  \notag \\
&=\argmax_{\y} \log \Pr(\y|\x) \notag   \\
&= \argmax_{\y} -\y^\T \A \y + 2\bregress^\T \y .
\end{align}
With the definition of $\A$ in Eq.\eqref{eq:def_A}, $\A$ is symmetric.
Then by setting the partial derivative of $-\y^\T \A \y + 2\bregress^\T \y$ with respect to $\y$ to $\zeros$ ($\zeros$ is an n$\times$1 column vector with all elements being 0), we have
\begin{align} \label{eq:inf_deriv}
&\frac{\partial \{ -\y^\T \A \y + 2\bregress^\T \y \} } {\partial \y} =0  \notag  \\
\Rightarrow \;\;&-(\A + \A^{\T} ) \y +2 \bregress=0  \notag  \\
\Rightarrow \;\;&-2\A \y +2 \bregress=0  \notag  \\
\Rightarrow \;\;&\y = \A^{-1} \bregress.
\end{align}
Now we can write the solution for the MAP inference in Eq. \eqref{eq:inf_solution1} as:
\begin{align} \label{eq:inf_solution2}
\y^{\star}&= \A^{-1} \bregress
\end{align}

%Because of the quadratic form of $\y$ in Eq. \eqref{eq:log-likelihood}, there exists closed form solution for the inference problem \cite{ccrf_nips08}:
%\begin{equation} \label{eq:inference}
%\y^{\star}=\argmax_{\y} \Pr(\y|\X) = \argmax_{\y} -\y^\T \A \y + 2\regress^\T \y  = \A^{-1} \regress.
%\end{equation}
%If the pairwise terms are ignored, namely $\bbeta=\zeros$, then Eq. \eqref{eq:inference} degenerates to $\y^{\star}=\regress$, which is a conventional regression model.

\section{Experiments}
To show how the superpixel number affects the performance of our model, we add an experiment to evaluate the root mean square (rms) errors and the training time of our pre-train model on the Make3D dataset by varying the superpixel number per image.
Fig. \ref{fig:rmsVSspnum} shows the results.
As we can see, increasing the number of supperpixel per image yields further decrease in the rms error, but at the cost of more training time.
We use $\sim 700$ superpixels per image in all other experiments in this paper, therefore by increasing it, we can expect better results.

\begin{figure} \center
     \includegraphics[width=0.38\textwidth, height=0.28\textwidth]{./fig/Make3D/rmsVSspNum.pdf}
     \includegraphics[width=0.38\textwidth, height=0.28\textwidth]{./fig/Make3D/timeVSspNum.pdf}
\caption{Left: Root mean square (C2 rms)  errors  \vs  varying superpixel numbers on the Make3D dataset.
Right: Training time \vs varying superpixel numbers per image on the Make3D dataset.
Clearly, increasing the number of supperpixels per image, we can further improve the results but at the cost of more training time.}  \label{fig:rmsVSspnum}
\end{figure}


The goal of this document is to make you acquainted with the concept of inference systems. You will learn various aspects of logic and inference system, which we will use to give rise to definitions of concepts such as numbers, boolean expressions, etc. As an advanced example you will also learn how to model programming languages constructs into inference systems. This leads to an important end result: you will learn the \textit{what} of programming languages, in contrast to the \textit{how} that you have studied so far. As a short conclusion we will discuss the strong relationship between logic and programming, and draw some conclusions about computability.

\subsection{Why bother?}
Try drawing a circle or a similar shape. The circle starts out as a single ``piece''. Choose two points along the border and draw a connecting line between the two: the circle is now split into two pieces. Chose now a third point and connect it to the other two points: the circle is now split into four pieces. A clear pattern is emerging: every point we add, we get twice as many pieces. This suggests that by \texttt{k} points we get $2^{k-1}$ pieces.

This seems all quite logical, but there is a catch. Try now doing this with six points along the border of the circle: instead of the expected 32 pieces, we will end up with 31. This means that the original conclusion was definitely not correct.

What is the moral of this story? If you want (or need) to be absolutely sure about some assertion (often called \textit{proposition} or \textit{hypothesis}), then you cannot trust a few examples and an intuitively appealing story to show that some pattern will always hold true. In the practice, and certainly in that of informatics or medical technology, there are many situations where a few tests and an intuition of why the system works are sufficient. Would you dare fly in an airplane where the programmer concluded after a dozen of tests that ``it all works, send it to production''? Would you dare step into an MRI machine (which throws radioactive isotopes at your body) knowing that the software was quickly written during a weekend with a bit of unit testing? Of course not\footnote{as a side note, if you are comfortable with these scenarios, then perhaps the motivation for the rest of the document will feel quite weak, so you might better go back to your vacation in Chernobyl or flying in planes held together by duct tape.}.

The use of logic as a foundation for programming does not automatically solve all our problems. It is very well possible (we will get there) that there is no single mathematical formalism or piece of software that will entirely remove the need for human intellect when writing software. \textbf{In any case, the use of logic and formality when decomposing problems can greatly help in writing software that is better thought out and over which it is far easier to reason and prove properties.}

\subsection{Structure of the document}
In the rest of this document we will:
\begin{inparaenum}[\itshape i\upshape)]
\item introduce informal logical reasoning in Section \ref{sec:informalIntroduction};
\item introduce a more formal logical formalism in Section \ref{sec:inferenceSystems};
\item give a first example of logic in action to define basic boolean operators in Section \ref{sec:booleanExpressions};
\item use logic to define \textit{apparently atomic} and unrelated concepts such as numbers in Chapter \ref{chap:numbers};
\item construct complex data structures such as lists, binary search trees, and even balanced binary search trees in Chapter \ref{chap:dataStructures};
\item build a small programming language in Chapter \ref{chap:aPL};
\item we conclude with a short and woefully incomplete presentation of fragments of computability in Chapter \ref{chap:closingRemarks}.
\end{inparaenum}


\documentclass[twoside]{MATH77}
\usepackage{multicol}
\usepackage[fleqn,reqno,centertags]{amsmath}
\begin{document}
\hyphenation{SSVDRS ISCALE}
\begmath 4.3 Singular Value Decomposition and Analysis

\silentfootnote{$^\copyright$1997 Calif. Inst. of Technology, \thisyear \ Math \`a la Carte, Inc.}

\subsection{Purpose}

Any M$\times $N matrix, $A$, has a Singular Value Decomposition (SVD) of the
form
\begin{equation*}
A = USV^t
\end{equation*}
where $U$ is an M$\times $M orthogonal matrix, $V$ is an N$\times $N
orthogonal matrix, and $S$ is an M$\times $N matrix having nonnegative
elements on the diagonal and zeros elsewhere. It is customary to arrange
that the diagonal elements of $S$, called the singular values of $A$, are in
decreasing order. This is done in the software described here.

The SVD can be useful in analyzing a linear least-squares problem or other
matrix problems, particularly when there is reason to believe that the model
is ill-conditioned. The SVD also has a role as a component in many
specialized algorithms of linear algebra and statistics.

Here we describe three subroutines to facilitate use of the SVD:

(1) Let a matrix, $A$, and a matrix or vector, $B$, be given. Denote the SVD
of $A$ by $A$ $= USV^t$. Subroutine SSVDRS computes the SVD of a matrix, $A$%
, and returns $S$, $V$, and the product, $U^tB$. If one needs to obtain the
matrix $U$ explicitly, call SSVDRS with $B$ set to be the $M^{th}$ order
identity matrix.

(2) Subroutine SSVA uses SSVDRS and produces a report of quantities useful
for the singular value analysis of a least-squares problem, $A{\bf x} \simeq
{\bf b}.$

(3) Subroutine SCOV3 can be used following SSVDRS to compute a covariance
matrix.

\subsection{Usage}

\subsubsection{Singular Value Decomposition with Computation of $U^tB$}

Compute the Singular Value Decomposition, $USV^t$, of a matrix, $A$, and
optionally return the product, $U^tB$, where $B$ is another given matrix.

\paragraph{Program Prototype, Single Precision}

\begin{description}
\item[INTEGER]  \ {\bf LDA, M, N, LDB, NB}

\item[REAL]  \ {\bf A}(LDA, $\geq $N){\bf , SING}($\geq $N){\bf , D}($\geq $%
N){\bf ,\\ WORK}($\geq 2\times $N){\bf , B}(LDB, $\geq $NB) or {\bf B}($\geq
$M)
\end{description}

Assign values to A(,), LDA, M, N, B(), LDB, and NB.

\begin{center}
\fbox{\begin{tabular}{@{\bf }c}
CALL SSVDRS (A, LDA, M, N, B,\\
LDB, NB, SING, WORK)\\
\end{tabular}}
\end{center}

Computed quantities are returned in A(,), B(), and SING().

\paragraph{Argument Definitions}

\begin{description}
\item[A(,)]  \ [inout] On entry, contains an M$\times $N matrix, $A$. On
return contains the N$\times $N matrix, $V$, such that $A=USV^t.$

\item[LDA]  \ [in] First dimensioning parameter for A(,). Require LDA $\geq
\max (\text{M},\text{N}).$

\item[M,N]  \ [in] Number of rows and columns, respectively, in the given
matrix, $A$. Either M $>$ N or M $\leq $ N is permitted. Require M $>0$ and
N $>0.$

\item[B()]  \ [inout] On entry, contains an M-vector, ${\bf b}$, or an M$%
\times $NB matrix, $B$. On return, contains the M-vector, ${\bf g}=U^t{\bf b}
$, or the M$\times $NB matrix, $G=U^tB$, where $U$ satisfies $USV^t=A.$ Note
that if $B$ is the $M^{th}$ order identity matrix on entry, then on return
the array B(,) will contain $U^t.$

\item[LDB]  \ [in] First dimensioning parameter for the array B(,). Require
LDB $\geq \max (\text{M},\text{N})$ when NB $\geq 1$ and LDB $\geq 1$ when
NB = 0.

\item[NB]  \ [in] Number of columns in the input matrix, $B$. Require NB $%
\geq 0$. If NB = 1, the array B() may be singly or doubly subscripted. If NB
$>1$, the array B(,) must be doubly subscripted. If NB = 0, the array B()
will not be referenced.

\item[SING()]  \ [out] On return, contains the singular values of $A$, in
descending order, in locations indexed 1 through N. If M $<$ N, SING(M+1)
through SING(N) will be set to zero.

\item[WORK()]  \ [scratch] Work space of length at least 2$\times $N.
\end{description}

\subsubsection{Singular Value Analysis}

Computes quantities useful for the singular value analysis of a
least-squares problem, $A{\bf x} \simeq {\bf b}$. Optionally produces a
report, with options to select the full report or only parts of it, to
select the output unit for the report, and to specify the display width
available for the report.

\paragraph{Program Prototype, Single Precision}

\begin{description}
\item[INTEGER]  \ {\bf LDA, M, N, MDATA, ISCALE, KPVEC}(4)

\item[REAL]  \ {\bf A}(LDA, $\geq $N){\bf , B}($\geq $M){\bf , SING}($\geq $%
N){\bf , D}($\geq $N){\bf , WORK}($\geq 2 \times \text{N}$)

\item[CHARACTER]  \ {\bf NAMES}($\geq N)*($lennam)\\
{}[lennam $\geq $ 1]
\end{description}

Assign values to A(,), LDA, M, N, MDATA, B(), KPVEC(), NAMES(), ISCALE, and
optionally to D().

\begin{center}
\fbox{\begin{tabular}{@{\bf }c}
CALL SSVA (A, LDA, M, N, MDATA,\\
B, SING, KPVEC, NAMES,\\
ISCALE, D, WORK)\\
\end{tabular}}
\end{center}

Computed quantities are returned in A(,), B(), SING(), and optionally in
D(). A report is produced if selected by KPVEC().

\paragraph{Argument Definitions}

\begin{description}
\item[A(,)]  \ [inout] On entry, contains the M$\times $N matrix, $A$, of the
least-squares problem to be analyzed. This could be a matrix obtained by
preliminary orthogonal transformations applied to the actual problem matrix
which may have had more rows (See MDATA below.) On return, contains an $%
N^{th}$ order matrix in which the $j^{th}$ column is the $j^{th}$ candidate
solution for the least-squares problem, $i.e$. the solution computed as
though singular values $j+1$ through N were zero.

\item[LDA]  \ [in] First dimensioning parameter for A(,). Require LDA $\geq
\max (\text{M},\text{N}).$

\item[M,N]  \ [in] Number of rows and columns, respectively, in the matrix, $A$.
Either M $>$ N or M $\leq $ N is permitted. Require M $>0$ and N $>0.$

\item[MDATA]  \ [in] Number of rows in actual least-squares problem.
Generally MDATA\ $\geq $ M. MDATA is used only in computing statistics for
the report and is not used as a loop count or array dimension.

\item[B()]  \ [inout] On entry, contains the right-side M-vector, ${\bf b}$,
of the least-squares problem. On return, contains the M-vector, ${\bf g}=U^t%
{\bf b}$, where $U$ comes from the singular value decomposition of $A$.

\item[SING()]  \ [out] On return, contains the singular values of $A$, in
descending order, in locations indexed 1 through N. If M $<$ N, SING(M+1)
through SING(N) will be set to zero.

\item[KPVEC()]  \ [in] Option array controlling report generation. If
KPVEC(1) = 0, default settings will be used, producing the full report,
sending it to the standard system output unit, formatted with a maximum line
length of~79. If KPVEC(1) = 1, the contents of KPVEC(I), I = 2,...,~4, set
options for the report, as follows:
\begin{description}
\item[KPVEC(2)] \ The decimal representation of KPVEC(2) must be at
most 6~digits, each being 0~or~1. The decimal digits will be interpreted as
independent on/off flags for the 6 possible blocks of the report. Examples:
101010 selects the 1st, 3rd, and 5th~blocks, 111111 selects all blocks, 0
suppresses the whole report, etc. The default value is 111111. The six
blocks are:

\begin{itemize}
\item[1.]  Header, with size and scaling option parameters.

\item[2.]  $V$-matrix.

\item[3.]  Singular values and related quantities.

\item[4.]  Listing of YNORM and RNORM and their logarithms.

\item[5.]  Levenberg-Marquardt analysis.

\item[6.]  Candidate solutions.
\end{itemize}

\item[KPVEC(3)] \ Define UNIT = KPVEC(3). If UNIT $\geq $ 0, UNIT will be
used as the output unit number. If UNIT $=-1$, output will be written to
the ``$\ast $'' output unit, $i.e.$, the standard system output unit.
The default value is $-$1. The calling program unit is responsible for
opening and/or closing the selected output unit if the host system
requires these actions.

\item[KPVEC(4)] Determines the width of blocks~2, 3, and~6 of the output report.
Define WIDTH = KPVEC(4). The default value is~79. Each output line will have
a leading blank for Fortran ``carriage control'' with line widths as
follows: Output blocks~1, 4, and~5 always have 63, 66, and 66~character
positions respectively. Output blocks~2 and~6 will generally have at most
WIDTH character positions. One output line will contain a row number, a name
from NAMES(), and from one to eight floating point numbers. The space
allocated for a name will be that needed for the longest name in NAMES(),
which may be less than the declared length of elements of NAMES(). The line
length will only exceed WIDTH if this is necessary to accommodate the row
number and name plus one floating-point number. Output block~3 will have
69~character positions if WIDTH $<95$ and otherwise will have 95~character
positions.
\end{description}

\item[NAMES()]  \ [in] NAMES$(j)$, for $j=1$, ..., N, may contain a name for
the $j^{th}$ component of the solution vector. If NAMES(1) contains only
blank characters, it will be assumed that no names have been provided, and
this subroutine will not access the NAMES() array beyond the first element.

\item[ISCALE]  \ [in] Set by the user to~1, 2, or~3 to select the column
scaling option.

\begin{itemize}
\item[1]  The subroutine will use identity scaling and ignore the D() array.

\item[2]  The subroutine will scale nonzero columns of $A$ to have unit
Euclidean length, and will store reciprocal lengths of the original nonzero
columns in D().

\item[3]  User supplies column scaling factors in D(). The subroutine will
multiply column $j$ by D($j)$, and remove the scaling from the solution at
the end.
\end{itemize}

\item[D()]  \ [ignored or out or in] Usage of D() depends on ISCALE as
described above. When used, its length must be at least N.

\item[WORK()]  \ [scratch] Scratch work space.
\end{description}

\subsubsection{Computation of the Covariance Matrix}

Subroutine, SCOV3, can be used to compute the covariance matrix for the
solution vector of a least-squares problem, $A{\bf x} \simeq {\bf b}$,
following use of SSVDRS in cases in which M $>$ N and all N singular values
of $A$ are nonzero.

\paragraph{Program Prototype, Single Precision}

\begin{description}
\item[INTEGER]  \ {\bf LDA, N, IERR}

\item[REAL]  \ {\bf A}(LDA, $\geq $N){\bf , SING}($\geq $N){\bf , VAR,\\ WORK%
}($\geq $N)
\end{description}

On entry, the arguments, A(,), LDA, N, and SING() should contain the same
values as on return from a previous call to SSVDRS.

\begin{center}
\fbox{\begin{tabular}{@{\bf }c}
CALL SCOV3( A, LDA, N, SING,\\
VAR, WORK, IERR)\\
\end{tabular}}
\end{center}

Computed quantities are returned in A(,) and IERR.

\paragraph{Argument Definitions}

\begin{description}
\item[A(,)]  \ [inout] On entry, contains the N$\times $N matrix, $V$,
computed by a previous call to SSVDRS. (Note that the quantities left in the
array A() by SSVA are not appropriate for use as input to SCOV3.) On return
contains the N$\times $N symmetric covariance matrix, $C$, for the solution
vector of the least-squares problem, $A{\bf x}\simeq {\bf b}.$

\item[LDA]  \ [in] First dimensioning parameter for A(,). Must be the
same value used when SSVDRS was called to compute the $V$ matrix.

\item[N]  \ [in] Number of rows and columns of the matrix, $V$, contained in
the array, A(,). Must be the same value used when SSVDRS was called
to compute the $V$ matrix.

\item[SING()]  \ [in] Contains the singular values of $A$ in locations indexed
1 through N. Must all be nonzero.

\item[VAR]  \ [in] User-supplied estimate of the variance of error in the
vector, ${\bf b}$, of the least-squares problem.

\item[WORK()]  \ [scratch] Work space of length at least N.

\item[IERR]  \ [out] Set to~0 if all of the N given singular values are
nonzero. Otherwise IERR will be set to the index of the first zero element
of the array, SING(). In this latter case the covariance matrix cannot be
computed and the contents of A(,) on return will be meaningless.
\end{description}

\subsubsection{Modifications for Double Precision}

For double precision usage change all REAL type statements to DOUBLE
PRECISION, and change the subroutine names from SSVA, SSVDRS, and SCOV3 to
DSVA, DSVDRS, and DCOV3, respectively.

\subsection{Examples and Remarks}

The program, DRDSVA, illustrates the use of DSVA to compute and report
quantities for the singular value analysis of a $15\times 5$ least-squares
problem.  This is an artificially constructed problem that was used as an
example in \cite{Lawson:1974:SLS}.  The output of this example is shown in
ODDSVA.

\subsubsection{Large problems}

If M $>>$ N and storage limitations make it awkward or impossible to
allocate M$\times $N locations for the array, A(,), one can use sequential
accumulation of the rows of data to produce a smaller matrix to which SSVDRS
or SSVA can then be applied. When the ratio M\,/\,N is sufficiently large this
approach will be faster than direct application of SSVDRS to the original
matrix. See Chapter~4.4 for sequential accumulation.

\subsubsection{The pseudoinverse of $A$}

The pseudoinverse of $A$, conventionally denoted by $A^+$, can be computed
using results produced by SSVDRS. Set $B=I$, the $\text{M}^{th}$ order identity
matrix. Then call SSVDRS obtaining $S$, $V$, and $U^t$. Since $S$ is diagonal,
its pseudoinverse, $S^+$, is its transpose with nonzero elements replaced by
their reciprocals. Depending on the application, it may be appropriate to
treat nonzero singular values that are smaller than some problem-related
threshold as though they were zero. After defining $S^+$, one can compute $%
A^+ = VS^+U^t.$

\subsubsection{Solution of the least-squares problem,\protect\\ $A{\bf x}
\simeq {\bf b}$}

From $A$ and ${\bf b}$, SSVDRS can compute $V$, $S$, and ${\bf g} = U^t{\bf b}$.
The user can then determine $S^+$ from $S$ as described in the preceding
paragraph and compute the solution vector as ${\bf x} = VS^+{\bf g}.$

\subsection{Functional Description}

\subsubsection{Method in SSVDRS}

Subroutine SSVDRS computes the singular value decomposition by an
algorithm due to G.  Golub and W.  Kahan, as described in
\cite{Lawson:1974:SLS}.  This method uses approximately 2N Householder
orthogonal transformations to reduce the given $A$ matrix to a bidiagonal
matrix and then calls SQRBD to apply a specialized version of the QR
eigenvalue algorithm to the bidiagonal matrix to complete the reduction to
the diagonal $S$ matrix.  The QR algorithm generally requires about 2N
iterations to reach convergence to machine accuracy.  The product of the
orthogonal transformations is accumulated to produce the $V$ matrix and
the product, $G = U^tB.$

Results are permuted so the singular values are in decreasing order.
The largest singular value will be accurate to nearly the machine accuracy.
Other singular values will have about the same absolute accuracy as the
largest one. The matrices $V$ and $U$ will be orthogonal to nearly machine
accuracy.

This implementation gives special treatment to any column of $A$ that is
exactly zero. If M $>$ N each such column will give rise to an exactly zero
singular value in the SING() array. This is convenient if one wishes to
remove a variable from a problem by just zeroing its column in $A$.

\subsubsection{Method in SSVA}

When the matrix of a least-squares problem is ill-conditioned it will
frequently be true that the vector ${\bf x}$ that minimizes the residual
norm $\|A{\bf x} - {\bf b}\|$ is undesirably large, $i.e.$, $\|{\bf x}\|$ is
large. It is also generally true that there will be other vectors, ${\bf x}$%
, that are significantly smaller in norm than the exact solution vector,
with only a small increase in the residual norm. One of these
vectors may be preferable to the true solution as an operational solution
for the problem.

There are various ways to define either a discrete or a continuous family of
candidate solutions for a least-squares problem that range from the true
solution through ${\bf x}$'s having smaller norms but allowing
larger residual norms. This subroutine gives information on two such
families of candidate solutions.

To simplify the discussion we assume M $>$ N; however the subroutine also
handles M $\leq $ N. A useful discrete family of candidate solutions is
obtained by defining ${\bf x}^{(i)}$ to be the solution obtained when all
singular values following the $i^{th}$ are set to zero. Then ${\bf x}^{(%
\text{N})}$ is the true solution, ${\bf x}^{(0)}$ is the zero vector, and
the intermediate candidate solution vectors satisfy the monotonicity
conditions, $\Vert {\bf x}^{(i)}\Vert \leq \Vert {\bf x}^{(i+1)}\Vert $ and $%
\Vert A{\bf x}^{(i)}-{\bf b}\Vert \geq \Vert A{\bf x}^{(i+1)}-{\bf b}\Vert .$

A useful continuous family of candidate solutions is defined by letting $%
{\bf x}^\lambda $, for $\lambda \geq 0$, be the vector that minimizes $\Vert
A{\bf x}-{\bf b}\Vert ^2+\lambda ^2\Vert {\bf x}\Vert ^2$. This is
equivalent to asking for the solution of the augmented least-squares problem:%
\begin{equation*}
\left[
\begin{array}{c}
A \\
\lambda I
\end{array}
\right] {\bf x}^\lambda \simeq \left[
\begin{array}{c}
{\bf b} \\ {\bf 0}
\end{array}
\right]
\end{equation*}
where I denotes the $N^{th}$ order identity matrix. If $\lambda <\mu $ then $%
\Vert {\bf x}^\lambda \Vert \geq \Vert {\bf x}^\mu \Vert $ and $\Vert A{\bf x%
}^\lambda -{\bf b}\Vert \leq \Vert A{\bf x}^\mu -{\bf b}\Vert $. This family
of candidate solutions is discussed in the literature under various names,
such as ridge regression, damped least-squares, and Levenberg-Marquardt
stabilization.

Since the units in which variables are expressed is arbitrary and the value
of $\|{\bf x}\|$, and thus the family of candidate solutions, depends on the
choice of units, this subroutine gives the user the ability, via the
parameters ISCALE and D(), to scale the columns of $A$ and thus the
components of ${\bf x}.$

Given $A$ and ${\bf b}$ defining a least-squares problem, $A{\bf x} \simeq
{\bf b}$, SSVA performs the following steps, doing the indicated printing
only if selected by the settings of KPVEC().

\begin{itemize}
\item[1a.]  As specified by the user's setting of ISCALE and D(),
introduce a nonsingular diagonal scaling matrix, $D$, reformulating the
problem as\ $(AD)(D^{-1}{\bf x})\simeq {\bf b}$. Let ${\bf y}=D^{-1}{\bf x}$
so the problem can be written as $(AD){\bf y}\simeq {\bf b}.$

\item[1b.]  Use SSVDRS to compute the singular value decomposition of $AD$,
obtaining $U$, $S$, and $V$ satisfying $AD=USV^t$, and ${\bf g}=U^t{\bf b%
}$. Let $s_i$ denote the $i^{th}$ diagonal element of $S$, $i.e$. the $i^{th}$
singular value of $AD$. The scaled solution vector, ${\bf y}$, is given by $%
{\bf y}=VS^{+}U^t{\bf b}$, and thus may be computed in two steps as ${\bf p}%
=S^{+}{\bf g}$ and ${\bf y}=V{\bf p}$. Specifically the components of the
N-vector ${\bf p}$ are computed as $p_i=g_i/s_i$ if $s_i\neq 0$ and $p_i=0$ if $%
s_i=0.$

\item[2.]  Print the $V$ matrix.

\item[3.]  For $i=1$, ..., N, print $s_i$, $p_i$, $1/s_i$, $g_i$,
and $g_i^2.$

Define $\rho _j^2=\Vert A{\bf x}^{(j)}-{\bf b}\Vert ^2$. This
quantity is computed as%
\begin{equation*}
\rho _j^2=\sum_{i=j+1}^Ng_i^2
\end{equation*}
and is printed with the heading ``Cumulative Sum of Squares."

An estimate of the variance of the errors in the data vector ${\bf b}$,
under the assumption that the singular values following the $j^{th}$ are
zero, is given by $\sqrt{\rho _j^2/(\text{M}-j)}$. This quantity is printed
with the heading ``Scaled Sqrt of Cum. S.S." for Scaled Square Root of the
Cumulative Sum of Squares.

\item[4.]  The quantities $\Vert {\bf y}^{(j)}\Vert $ are computed
(using the $p_i$'s) and printed along with the corresponding values
of $\rho _j$, with the headings YNORM and RNORM.

\item[5.]  A range of values of the Levenberg-Marquardt parameter, $%
\lambda $, and associated $\Vert {\bf y}^\lambda \Vert $ and $\Vert AD{\bf y}%
^\lambda -{\bf b}\Vert $ are computed and printed. These quantities are
computed from formulas involving $s_i$'s, ${p_i}$'s,
and ${g_i}$'s.

\item[6.]  The candidate solutions, ${\bf x}^{(j)}=D{\bf y}^{(j)}$, are
computed and printed.
\end{itemize}

SSVA calls SPRTSV to print the $V$ matrix and the candidate solutions.

\subsubsection{Method for SCOV3}

If the variance of the error in the data vector, ${\bf b}$, is VAR, the
covariance matrix, $C$, for the solution vector, ${\bf x}$, is conventionally
defined as $C = \text{VAR}\times (A^tA)^{-1}$. Using the singular value
decomposition, $A = USV^t$, one can write $C =\text{VAR}\times
(VS^tSV^t)^{-1} = \text{VAR}\times VS^+(VS^+)^t$. SCOV3 computes $C$ using
this latter formula.

\bibliography{math77}
\bibliographystyle{math77}

\subsection{Error Procedures and Restrictions}

In SSVA, if M $\leq 0$ or N $\leq 0$ an immediate return will be made.

SSVDRS will issue an error message, set SING$(1) = -1.0$, and do an
immediate return if any of the following conditions are noted.

\hspace{.2in}M $< 1$, N $< 1$, NB $< 0$, LDA $< \max (\text{M},\text{N}),$

\hspace{.2in}LDB $<$ M when NB $> 0$, or LDB $< 1$ when NB $= 0.$

SCOV3 requires N nonzero singular values. If this is satisfied it will set
IERR = 0. Otherwise it will set IERR to the index of the first zero singular
value and the results returned in A(,) will be meaningless.

The subroutine SCOV3 is intended for use following SSVDRS. It cannot be used
following SSVA since SSVA does not leave the $V$ matrix in the A(,) array on
return.

\subsection{Supporting Information}

The source language is ANSI Fortran~77.

These subroutines were adapted from \cite{Lawson:1974:SLS} for use with
Fortran~77 by C.  L.  Lawson and S.  Y.  Chiu, May~1986, June~1987.
Altered March~1989 by Lawson to introduce the vector KPVEC() to provide
more report options.

\begin{tabular}{@{\bf}l@{\hspace{5pt}}l}
\bf Entry & \hspace{.35in} {\bf Required Files}\vspace{2pt} \\
DCOV3 & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
DCOPY, DCOV3, DDOT, DSCAL, ERFIN, ERMSG, IERM1, IERV1\rule[-5pt]{0pt}{8pt}}\\
DSVA & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, DAXPY, DCOPY, DDOT, DHTCC, DHTGEN, DNRM2, DPRTSV, DQRBD, DROT, DROTG, DSVA, DSVDRS, DSWAP, ERFIN, ERMOR, ERMSG, IERV1\rule[-5pt]{0pt}{8pt}}\\
DSVDRS & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, DAXPY, DCOPY, DDOT, DHTCC, DHTGEN, DNRM2, DQRBD, DROT, DROTG, DSVDRS, DSWAP, ERFIN, ERMOR, ERMSG, IERV1\rule[-5pt]{0pt}{8pt}}\\
SCOV3 & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
ERFIN, ERMSG, IERM1, IERV1, SCOPY, SCOV3, SDOT, SSCAL\rule[-5pt]{0pt}{8pt}}\\
SSVA & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, ERFIN, ERMOR, ERMSG, IERV1, SAXPY, SCOPY, SDOT, SHTCC, SHTGEN, SNRM2, SPRTSV, SQRBD, SROT, SROTG, SSVA, SSVDRS, SSWAP\rule[-5pt]{0pt}{8pt}}\\
SSVDRS & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, ERFIN, ERMOR, ERMSG, IERV1, SAXPY, SCOPY, SDOT, SHTCC, SHTGEN, SNRM2, SQRBD, SROT, SROTG, SSVDRS, SSWAP}\\
\end{tabular}

\begcode

\medskip\
\lstset{language=[77]Fortran,showstringspaces=false}
\lstset{xleftmargin=.8in}

\centerline{\bf \large DRDSVA}\vspace{10pt}
\lstinputlisting{\codeloc{dsva}}
\newpage
\enlargethispage*{8pt}
\centerline{\bf \large ODDSVA}\vspace{5pt}
\lstset{language={}}
\lstinputlisting{\outputloc{dsva}}
\end{document}


\documentclass[a4paper]{article}

\usepackage[margin=3cm]{geometry}
\usepackage{fontspec}
\usepackage{amsmath, amsfonts}
\usepackage{enumitem}
\usepackage{algorithm}
\usepackage{algorithmicx}
\usepackage{algpseudocode}
\PassOptionsToPackage{hyphens}{url}\usepackage{hyperref}

\setlength{\parindent}{0pt}
\setlength{\parskip}{0.5em}
\def\arraystretch{1.5}

\algnewcommand\algorithmicinput{\textbf{Input:}}
\algnewcommand\INPUT{\item[\algorithmicinput]}
\algnewcommand\algorithmicoutput{\textbf{Output:}}
\algnewcommand\OUTPUT{\item[\algorithmicoutput]}

\title{A public-key criptosystem}
\author{Arnau \and Joan \and Marc}

\begin{document}
\maketitle
\section*{Statement}
Let us consider the following public-key cryptosystem based on \textsc{Subset Sum}:

\begin{enumerate}[label=\alph*)]
	\item The \emph{private key} consists of a \emph{superincreasing} sequence $E = (e_1, ..., e_n)$ of integers, an integer $m$ greater that the sum of all the elements $e_1, ..., e_n$ and an integer $w$ that is relatively prime to $m$. (A sequence $e_1, ..., e_n$ is called \emph{superincreasing} if each element $e_i$ is strictly greater than the sum of all previous elements).
	\item The \emph{public key} is a sequence $H = (h_1, ..., h_n)$ derived from the private key via $h_i = (we_i) \pmod{m}$.
	\item The encryption of an $n$-bit message $X = (x_1, ..., x_n)$ is the number $c = HX = \sum_{i=1}^n h_ix_i$.
	\item Decrypting the message amounts to solving $c = HX$ for $X \in \{0, 1\}^n$, which is of course equivalent to solving \textsc{Subset Sum} for H with target sum $c$. The owner of the private key, however, can simplify the decryption considerably by calculating $w^{-1}c \equiv_m EX$.

	(Now the condition $m > \sum_{i=1}^n e_i$ allows us to replace '$\equiv_m$' by '$=$')
\end{enumerate}

\section*{Solution}
Let us see that the problem of, given an encrypted message $c$ and a private key $\langle E = (e_1, ..., e_n), m, w \rangle$, decrypting $c$, can be solved in polynomial time.
\begin{enumerate}[label=\roman*)]
	\item Show that $w^{-1}c = EX$.
\end{enumerate}
We want to prove that $w^{-1}c = EX$ so, using all the previous definitions:
\begin{gather*}
	w^{-1}c = EX \pmod{m} \iff w^{-1}c = EX \iff w^{-1}c = \sum_{i=1}^{n} e_i x_i \iff \\
	\iff w^{-1} \sum_{i=1}^n h_i x_i = \sum_{i=1}^n e_i x_i \iff \sum_{i=1}^n \underbrace{w^{-1}w}_1 e_i x_i = \sum_{i=1}^n e_i x_i
\end{gather*}

\begin{enumerate}[resume, label=\roman*)]
	\item Prove that the following problem is a polynomial time computable:

	Given integers $w$ and $m$ satisfying \emph{a)}, and an integer $c$ satisfying \emph{c)}, compute $(w^{-1}c) \pmod{m}$.
\end{enumerate}
Taking into account that $(w^{-1}c)\equiv_m(w^{-1}\mod{m})·(c \mod{m})$, we know we can compute $w^{-1}\mod{m}$ in $O(n^3)$ using the Extended Euclid algorithm:

 \textbf{EXT-EUCLID}(w,m) returns (d,x,y) where $d=GCD(w,m)=wx + my=1$ because we know $w$ and $m$ are relatively prime. Working in modular arithmetic $my=_m0$, so $wx=_m 1$, and as a result $x=_m w^{-1}$.

 Being $c$ a number, computing $c \mod{N}$ is also polynomial, and afterwards the only step left is to compute the product of the two results previously obtained and apply the module one last time in $(w^{-1}c) \pmod{m}$, which are two polynomial operations too. AND WE ARE DONE.

\begin{enumerate}[resume, label=\roman*)]
	\item Show that \textsc{Subset Sum} can be solved in polynomial time if the input sequence of integers is superincreasing. Recall that \textsc{Subset Sum} is defined by:

	Given a sequence of integers $e_1, ..., e_n$, and a target integer $W$, compute $X = \{0, 1\}^n$ such that $W = EX$ where $E = (e_1, ..., e_n)$. (\emph{Hint}: be greedy).

	Hence, given the superincreasing sequence $E$ and $W = (w^{-1}c) \pmod{m}$, we can compute the decrypted original message $X$ in polynomial time.
\end{enumerate}

The following greedy algorithm solves the problem in linear time.

\begin{algorithm}[H]
	\caption{\textsc{Subset Sum} with superincreasing list}
%	\KwData{}
%	\KwResult{  }
\begin{algorithmic}
	\INPUT $E= (e_1, ... , e_n), W $
	\OUTPUT $ X = \{0,1\}^n $ such that $W=EX$
	\State \Comment Each variable $x_j$ is boolean in spirit: it indicates if you include $e_j$ in the sum or not.
	\State $s := 0$ \Comment Sum variable
	\State $j := n$ \Comment Index variable
	\While{$ j>0 $ \and $ s<W $}
		\If{$ s + e_j \leq W $} \Comment Ifincluding  $e_j$ will not create overflow, we include it.
			\State $s := s + e_j$
			\State $x_j := 1$
		\Else
			\State $x_j := 0$
		\EndIf
		\State $ j := j-1 $
	\EndWhile
\end{algorithmic}
\end{algorithm}

Reference: algorithm explained here \rightarrow \url{https://math.stackexchange.com/questions/378148/greedy-optimized-subset-sum-problem}.

This is a greedy algorithm that solves the \textsc{Subset Sum} problem without guaranteeing an optimal solution (assuming that $E$ is still an ordered sequence). However, if the input list is superincreasing, the optimal is guaranteed.

\end{document}


% !TeX root = ./main.tex
% chktex-file 46
% !TeX spellcheck = en-GB
% !TeX encoding = utf8

Our novel graph-based approach to model the dynamics of the agents' health states incorporates communal effects and temporal decease effects through a graph contribution and an individual health contribution term, respectively.

Our main propagation rule is based on the definition of a graph convolution shown in Eq.~\eqref{eq:graph_convolution} and reads as follows in component notation
\begin{equation}
	\label{eq:state_propagation}
	h_{v_i, m}^{(l+1)}
	=
	\underbrace{
		\sum_k \frac{\hat{A}_{v_i, k}^{(l)}}{\sum_j \hat{A}_{v_i, j}^{(l)}} h_{k, m}^{(l)} \delta_{m, 1}
	}_{\text{\textcolor{red}{Graph}}}
	+
	\underbrace{
		{(h_{v_i}^{(l)}\cdot T)}_m
	}_{\text{\textcolor{blue}{Temporal}}}
\end{equation}
with $m$ being the health state index ranging from $0$ to $2$, $\hat{A}_{v_i, k}^{(l)}$ infection-adjusted adjacency matrix component, $\delta$ the Kronecker delta and $T$ as temporal transition matrix.

The propagation consists of two parts, first the graph contribution and second the temporal contribution. While the former captures the dynamics of infections based on the social contacts between agents, the former ensures that an infected agent heals over time and becomes resistant against the Corona virus. Figure~\ref{fig:state_propagation} visualises the propagation rule and the two subsequent sections explain the terms in greater detail.

\begin{figure}[H]
	\centering
	\includegraphics[width=0.8\columnwidth]{img/state_propagation.pdf}
	\caption{Visualisation of propagation rule from equation~\eqref{eq:state_propagation}. The blue connection visualises the temporal term and the red connections visualise the graph term. The dashed red line is does not contribute to the graph term as the two nodes are not connected according to $\hat{A}(l)$.}%
	\label{fig:state_propagation}
\end{figure}

\subsection{Explanation of the graph contribution term}

The graph contribution models how infected agents spread the disease through contacts with susceptible agents. This is modeled by the term

\begin{equation}
	(h_{v_i, m}^{(l+1)})_{\text{Graph}} =
	\sum_k \textcolor{cyan}{\frac{\hat{A}_{v_i, k}^{(l)}}{\sum_j \hat{A}_{v_i, j}^{(l)}}} h_{k, m}^{(l)} \textcolor{orange}{\delta_{m, 1}}.
\end{equation}

A sum over all neighbouring agents' features $h_{k,m}^{(l)}$ is weighted by the normalised infection-adjusted graph connections as shown in cyan. The Kronecker delta, as shown in orange, ensures that only the I feature is added as this is the only one that matters during social contacts between agents.

The infection-adjusted adjacency matrix $\hat{A}$ is constructed from $A$ and $I$ which are the regular continuous adjacency matrix and the infection matrix, respectively. These three quantities are explained in the following:

\begin{itemize}
	\item The adjacency matrix $A$ is time dependent, $A^{(l)}$, and inferred from data. In our use case, $A_{ij} = \frac{1}{dist(v_i, v_j)+\epsilon}$, hence $A_{ij}$ is large when persons $i$ and $j$ have been in contact. $\epsilon$ serves as regularization for small distances.
	\item The infection matrix is constructed as
	\begin{equation}
	I =
	\begin{pmatrix}
	0     &  0  & 0 \\
	\beta &  0  & \alpha \\
	0     &  0  & 0
	\end{pmatrix}
	=
	{(I_{ij})}_{i,j}
	\end{equation}
	with $i$ as the index of the host state and $j$ is the index of the contact person state. The states that we consider here are ordered as follows: susceptible, infected, recovered. $\beta$ denotes the probability of infection  after contact (also known as attack rate). $\alpha$ models the probability of being reinfected, which we assume to be zero ($\alpha=0$) based upon current medical~\cite{Bao2020.03.13.990226}.
	\item $\hat{A}$, with $\hat{A}_{ij}\in [0, 1]$, is the infection-adjusted adjacency matrix that takes the infection interactions into account and is computed as follows
	\begin{equation}
	\hat{A}_{ij} = A_{ij}\cdot \frac{ h_{v_1}^T I h_{v_2} + h_{v_2}^T I h_{v_1} }{\beta}.
	\end{equation}
	The weighted scalar product of the health states of agents $i$ and $j$ is used to evaluate whether the edge is relevant for the infection dynamics. Only when an infected person and a susceptible have contact, the edge $A_{ij}$ should be considered, otherwise it should be dropped.	The sum in the denominator comes from the fact that both, agent $i$ and $j$, can act as host during a contact. The division by $\beta$ normalises the factor to one to ensure $\hat{A}_{ij} \in [0, 1]$. Since $I$ is not symmetric, $p_a$ is a proper normalization because the sum is in $\{0, p_a\}$. Note that the fraction has the desired properties for pure $S$-, $I$- and $R$-persons.
\end{itemize}

Figure~\ref{fig:state_propagation} visualises the influence of the infection-adjusted adjacency matrix to the agents' states at the next time step. The two solid red lines contribute directly while the dashed red lines does not.

\subsubsection{Explanation of the temporal contribution term}

The transition of a person's health state $h_{v_i}^{(l)}$ is determined by three rules that are stated in the following:

\begin{itemize}
	\item A susceptible person always stays susceptible.
	\item An infected person has a probability $\gamma$, called recovery rate, to recover. The complementary probability $1-\gamma$ denotes that the person remains sick.
	\item A recovered person could have a probability to be re-infected, but we assume this to be zero throughout this work. Thus a recovered person always stays recovered~\cite{Bao2020.03.13.990226}.
\end{itemize}

These three rules are combined into a temporal transition matrix $T$, which describes the health state of an agent as time passes. This matrix reads as

\begin{equation}
	T =
	\begin{pmatrix}
		1 &     0    & 0      \\
		0 & 1-\gamma & \gamma \\
		0 &     0    & 1      \\
	\end{pmatrix}.
\end{equation}

The temporal update rule based on the health status thus becomes

\begin{equation}
	H^{(l+1)} = H^{(l)} T.
\end{equation}

Figure~\ref{fig:state_propagation} visualises the influence of the temporal component as red line connecting agents at subsequent time steps.


%---------------------------Oddy-----------------------------
\section{Oddy}

Let $\vec L_4 = \vec L_0$. The Oddy metric is then defined as
\[
q = \max_{i\in\{0,1,2,3\}}\left\{
    \frac{(\normvec{L_i}^2 - \normvec{L_{i+1}}^2)^2
    + 4 (\vec L_i \cdot \vec L_{i+1})^2}
    {2 \normvec{N_{i+1}}^2 }
  \right\}.
\]
This metric measures the maximum deviation of the metric tensor at the corners of the quadrilateral.

Note that if $\normvec{N_{i+1}}^2 < DBL\_MIN$, we set $q = DBL\_MAX$.

\quadmetrictable{Oddy}%
{$1$}%                                      Dimension
{$[0,0.5]$}%                                Acceptable range
{$[0,DBL\_MAX]$}%                           Normal range
{$[0,DBL\_MAX]$}%                           Full range
{$0$}%                                      Unit square
{\cite{odd:88}}%                            Citation
{v\_quad\_oddy}%                            Verdict function name


\lesson{4}{Dec 01 2021 Wed (18:08:39)}{Solving and Graphing Polynomial Functions}{Unit 3}

A function written in factored form gives us the roots of the equation. The roots and the end behavior rules below can be used to create a graph for the function.

Some \bf{Polynomial Functions} are far more complicated and should be graphed using graphing technology such as graphing software or a graphing calculator. The real number solutions of a \bf{Polynomial Equation} can be found by identifying the \bf{x-intercepts} from the graph of the function. Graph the function, using graphing technology and find the point(s) of intersection between the graph and the x-axis.

\begin{marginfigure}
    \centering
    \incfig{graph-technology}
    \sidecaption{$0.8x^{4} \times 8x^{3} + 2x^{2} - 7x - 1$ Graphed.}
    \label{fig:graph-technology}
\end{marginfigure}

\begin{definition}[Average Rate of Change]
    \begin{align}
        m = \frac{y_2 - y_1}{x_2 - x_1}
    \end{align}

    \paragraph{Increasing versus Decreasing}

    \begin{itemize}
        \item
        \item If the \bf{Average Rate of Change} is \bf{Positive}, then the function is considered to be \bf{Increasing}.
        \item If the \bf{Average Rate of Change} is \bf{Negative}, then the function is considered to be \bf{Decreasing}.
    \end{itemize}

    \paragraph{Minimums and Maximums}

    \begin{itemize}
        \item
        \item \bf{Minimums} are located on the \bf{Lowest Point of a Graph}. A \bf{Local Minimum} is the bottom of a \bf{Valley} or \bf{Turning Point} where the graph goes from \bf{Decreasing} to \bf{Increasing}.
        \item \bf{Maximums} are located on the \bf{Highest Point of a Graph}. A \bf{Local Maximum} is the top of a \bf{Hill} or \bf{Turning Point} where the graph goes from \bf{Increasing} to \bf{Decreasing}.
    \end{itemize}
\end{definition}

\begin{example}[Example 1]
    Determine the zeros of the function $f(x) = (x - 2)(x - 5)(x + 1)$, and describe the end behavior of the graph.
    The zeros of the function represent the values of $x$ that make the function equal to $0$. Replace the function notation with $0$ to determine the zeros of the function.

    \begin{align}
        f(x) &= (x - 2)(x - 5)(x + 1) \\
        0 &= (x - 2)(x - 5)(x + 1) \\
        0 &= x^3 - 6 \times x^2 + 3x + 10 \\
        0 &= x^3 - 6 \times x^2 + 3x + 10 \\
        x &= 0, x = 5, x = -1 \\
    \end{align}

    So, from this, we know that the zeros function from this function are: $0, 5, -1$, which means that's where they will intersect the $x-axis$.

    The function has an \bf{Odd Degree}, so the ends will travel in the opposite directions. The leading coefficient is positive, so the left side continues down and the right side continues up.
\end{example}

\newpage


\documentclass{article}
\usepackage{tabularx}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage[top = 2cm, bottom = 2cm, right = 2cm, left = 2cm]{geometry}
\usepackage{cite}
\usepackage[final]{hyperref}
\usepackage{listings}
\hypersetup{
	colorlinks=true,
	linkcolor=blue,
	citecolor=blue,
	filecolor=magenta,
	urlcolor=blue
}

\begin{document}

\title{Practicle 4\\Raytracer - Sphere}
\date{23/01/19}
\maketitle

\begin{abstract}
	The last practicals focused on sending rays. Here, we will see how to grab information from a virtual scene and display it.
\end{abstract}

\section{Intersect a sphere}
To compute the intersection between a ray and a sphere we'll use an analytic method. The implicit function of a sphere is: $x^2+y^2+z^2 = R^2$ with $P(x, y, z)$ a point in the surface sphere centered in the origin with a rayon of R. We want only points on the ray so we just need to replace $P$ by the ray equation : \\
$|O+Dt|^2 = R^2,$\\
$O^2 + 2DtO + (Dt)^2 = R^2,$\\
$O^2 + D^2t^2 + 2ODt = R^2,$\\
$O^2 + D^2t^2 + 2ODt - R^2 = 0,$\\
We want to find the $t$ value for the ray.\\
$ D^2t^2 + 2ODt + O^2 - R^2 = 0,$\\
with $a = D^2$, $b = 2OD$ and $c = O^2 - R^2$ we get a quadratic function ($f(t) = at^2+bt+c$)\\
If we want to move the sphere we just need to subtract the center $C$ to $P$ and $O$. The equation becomes: $|(O-C)+Dt|^2 = R^2;$
We get $a = D^2$, $b = 2(O-C)D$ and $c = (O-C)^2 - R^2$. Because the product for vector is the dot product and D is normalized, $a=1$\\
Remember that quadratic function can be solved in this way: $t_i = \frac{-b\pm \sqrt{b^2-4ac}}{2a}$

\section{C++ implementation}
Create a Sphere class into the device side for storing the center and the radius information. Create now the intersect function:
\begin{lstlisting}
	__device__ bool Sphere::intersect(const Ray& ray, float& t) {
		float t0, t1;
		Vector3GPU OC = ray.getOrigin() - center_;
		float b = 2.f*dot(oc, ray.getDirection());
		float c = dot(oc, oc) - radius_*radius_;
		// ...
	}
\end{lstlisting}
if the discriminant is negative you just have to return false and don't modify the final image buffer. If there is a collision, just return the red color. For information the smallest value of t is the nearest intersection point.
\begin{figure}[h]
	\centering
	\includegraphics[scale=0.6]{figures/intersect.png}
	\caption{Intersection ray sphere}
\end{figure}

\newpage
\section{Compute and show normal information}
When you intersect the sphere you can compute some geometry information like the normal of the point from the intersection point.
\begin{figure}[h]
	\centering
	\includegraphics[scale=0.4]{figures/normal.png}
	\caption{Normal of a sphere}
\end{figure}
\begin{lstlisting}
	Vector3 intersection = ray.getOrigin()+ray.getDirection()*t;
	Vector3 normal = (intersection-world[wid].center_);
\end{lstlisting}

To be sure our normal is correctly computed we can show them on the sphere. We just have to be careful because normal are between (-1, -1, -1) and (1, 1, 1). So we just have to multiply the normal by 0.5 and add the vector (0.5, 0.5, 0.5) to match the $[0, 1]^3$ range.
\begin{figure}[h]
	\centering
	\includegraphics[scale=0.6]{figures/intersectnormal.png}
	\caption{Intersection ray sphere}
\end{figure}
\newpage
\section{A world of spheres}
To avoid the creation of the world inside each kernel we can generate it one on the host side. Add the keyword \_\_host\_\_ to the constructor of the Sphere class and create a buffer of spheres.
\begin{lstlisting}
std::vector<Sphere> spheres;
world.push_back(Sphere(Vector3(0.f, 0.f, -3.f), 0.5f));
world.push_back(Sphere(Vector3(0.f, -100.5f, -3.f), 100.f));
Sphere* spheresGPU;
cudaMalloc(&spheresGPU, sizeof(Sphere)*spheres.size());
cudaMemcpy(spheresGPU, &spheres[0], sizeof(Sphere)*spheres.size(), cudaMemcpyHostToDevice);
\end{lstlisting}
\begin{figure}[h]
	\centering
	\includegraphics[scale=0.45]{figures/intersectnormaltwosphere.png}
	\caption{Intersection ray two spheres}
\end{figure}

For advenced groups, you can take a look in other intersection at this address: http://iquilezles.org/www/articles/intersectors/intersectors.htm

\newpage
\section{CUDA errors}
Each CUDA function return error value. We can store it in a cudaError\_t.
\begin{lstlisting}
	cudaError_t err = cudaMalloc(...);
	if (err != cudaSuccess) {
		std::cout<<cudaGetErrorString(err);
	}
\end{lstlisting}
The kernel computation don't return value. So if there is an error in the kernel we have to wait the end of the kernel execution and synchronize the device with the host and grab the last error.\\
Most of the time, when we compute heavy algorithm we get some error about kernel configuration. Commun errors are:
\begin{itemize}
	\item Too Many Resources Requested for Launch (That mean each thread need more registe that the SM can provide. Reducing the number of thread per block solve this error)
	\item an illegal memory access was encountered. You may read values outside your buffers or you stack pointer (GPU threads are designed to be short process. If you use recursive function you may saturate one of your callstack and get this error. If you really need to call a lot of function, you can increase the size of your callstack using cudaDeviceSetLimit)
\end{itemize}
\begin{lstlisting}
	cudaError_t err = cudaGetLastError();
	if (err != cudaSuccess) {
		std::cout<<cudaGetErrorString(err);
	}
\end{lstlisting}

\end{document}


\chapter{Preliminaries}\label{chap:preliminaries}

\section{General Notation}

We'll refer to \(\reals\) for the reals, \(\utri\) represents
the unit triangle (or unit simplex) in \(\reals^2\):
\(\utri = \left\{(s, t) \mid 0 \leq s, t, s + t \leq 1\right\}\).
When dealing with sequences with multiple indices, e.g.
\(s_{m, n} = m + n\), we'll use bold symbols to represent
a multi-index: \(\bm{i} = (m, n)\). We'll use \(\left|\bm{i}\right|\) to
represent the sum of the components in a multi-index.
The binomial coefficient
\(\binom{n}{k}\) is equal to \(\frac{n!}{k! (n - k)!}\) and the trinomial
coefficient \(\binom{n}{i, j, k}\) is equal to \(\frac{n!}{i! j! k!}\)
(where \(i + j + k = n\)). The notation \(\delta_{ij}\) represents the
Kronecker delta, a value which is \(1\) when \(i = j\) and \(0\)
otherwise.

\section{Floating Point and Forward Error Analysis}

We assume all floating point operations obey
\begin{equation}
  a \star b = \fl{a \circ b} = (a \circ b)(1 + \delta_1) =
  (a \circ b) / (1 + \delta_2)
\end{equation}
where \(\star \in \left\{\oplus, \ominus, \otimes, \oslash\right\}\), \(\circ
\in \left\{+, -, \times, \div\right\}\) and \(\left|\delta_1\right|,
\left|\delta_2\right| \leq \mach\). The symbol \(\mach\) is the unit round-off
and \(\star\) is a floating point operation, e.g.
\(a \oplus b = \fl{a + b}\). (For IEEE-754 floating point double precision,
\(\mach = 2^{-53}\).) We denote the computed result of
\(\alpha \in \reals\) in floating point arithmetic by
\(\widehat{\alpha}\) or \(\fl{\alpha}\) and use \(\floats\) as the set of
all floating point numbers (see \cite{Higham2002} for more details).
Following \cite{Higham2002}, we will use the following classic properties in
error analysis.

\begin{enumerate}
  \item If \(\delta_i \leq \mach\), \(\rho_i = \pm 1\), then
      \(\prod_{i = 1}^n (1 + \delta_i)^{\rho_i} = 1 + \theta_n\),
  \item \(\left|\theta_n\right| \leq \gamma_n \coloneqq
      n \mach / (1 - n \mach)\),
  \item \((1 + \theta_k)(1 + \theta_j) = 1 + \theta_{k + j}\),
  \item \(\gamma_k + \gamma_j + \gamma_k \gamma_j \leq \gamma_{k + j}
    \Longleftrightarrow (1 + \gamma_k)(1 + \gamma_j) \leq 1 + \gamma_{k + j}\),
  \item \((1 + \mach)^j \leq 1 / (1 - j \mach) \Longleftrightarrow
  (1 + \mach)^j - 1 \leq \gamma_j\).
\end{enumerate}

\section{B\'{e}zier Curves}

A \emph{B\'{e}zier curve} is a mapping from the unit interval
that is determined by a set of control points
\(\left\{\bm{p}_j\right\}_{j = 0}^n \subset \reals^d\).
For a parameter \(s \in \left[0, 1\right]\), there is a corresponding
point on the curve:
\begin{equation}
b(s) = \sum_{j = 0}^n \binom{n}{j} (1 - s)^{n - j} s^j \bm{p}_j \in
  \reals^d.
\end{equation}
This is a combination of the control points weighted by
each Bernstein basis function
\(B_{j, n}(s) = \binom{n}{j} (1 - s)^{n - j} s^j\).
Due to the binomial expansion
\(1 = (s + (1 - s))^n = \sum_{j = 0}^n B_{j, n}(s)\),
a Bernstein basis function is in
\(\left[0, 1\right]\) when \(s\) is as well. Due to this fact, the
curve must be contained in the convex hull of it's control points.

\subsection{de Casteljau Algorithm}

Next, we recall\footnote{We have used slightly non-standard notation for the
terms produced by the de Casteljau algorithm: we start the superscript at
\(n\) and count down to \(0\) as is typically done when describing Horner's
algorithm. For example, we use \(b_j^{(n - 2)}\) instead of
\(b_j^{(2)}\).} the de Casteljau algorithm:

\begin{breakablealgorithm}
  \caption{\textit{de Casteljau algorithm for polynomial evaluation.}}
  \label{alg:de-casteljau}

  \begin{algorithmic}
    \Function{\(\mathtt{result} = \mathtt{DeCasteljau}\)}{$b, s$}
      \State \(n = \texttt{length}(b) - 1\)
      \State \(\widehat{r} = 1 \ominus s\)
      \\
      \For{\(j = 0, \ldots, n\)}
        \State \(\widehat{b}_j^{(n)} = b_j\)
      \EndFor
      \\
      \For{\(k = n - 1, \ldots, 0\)}
        \For{\(j = 0, \ldots, k\)}
          \State \(\widehat{b}_j^{(k)} = \left(
              \widehat{r} \otimes \widehat{b}_j^{(k + 1)}\right) \oplus
              \left(s \otimes \widehat{b}_{j + 1}^{(k + 1)}\right)\)
        \EndFor
      \EndFor
      \\
      \State \(\mathtt{result} = \widehat{b}_0^{(0)}\)
    \EndFunction
  \end{algorithmic}
\end{breakablealgorithm}

\begin{theorem}[\cite{Mainar1999}, Corollary 3.2]
If \(p(s) = \sum_{j = 0}^n b_j B_{j, n}(s)\) and \(\mathtt{DeCasteljau}(p, s)\)
is the value computed by the de Casteljau algorithm then\footnote{In the
original paper the factor on \(\widetilde{p}(s)\) is \(\gamma_{2n}\),
but the authors did not consider round-off when computing
\(1 \ominus s\).}
\begin{equation}
\left|p(s) - \mathtt{DeCasteljau}(p, s)\right| \leq \gamma_{3n}
\sum_{j = 0}^n \left|b_j\right| B_{j, n}(s).
\end{equation}
\end{theorem}

The relative condition number of the evaluation of \(p(s) = \sum_{j = 0}^n
b_j B_{j, n}(s)\) in Bernstein form used in this work is (see
\cite{Mainar1999, Farouki1987}):
\begin{equation}
\cond{p, s} = \frac{\widetilde{p}(s)}{\left|p(s)\right|},
\end{equation}
where
\(\widetilde{p}(s) \coloneqq \sum_{j = 0}^n \left|b_j\right| B_{j, n}(s)\).

To be able to express the algorithm in matrix form, we define
the vectors
\begin{equation}
b^{(k)} = \left[\begin{array}{c c c} b_0^{(k)} & \cdots &
b_k^{(k)}\end{array}\right]^T, \quad
\widehat{b}^{(k)} = \left[\begin{array}{c c c} \widehat{b}_0^{(k)} & \cdots &
    \widehat{b}_k^{(k)}\end{array}\right]^T
\end{equation}
and the reduction matrices:
\begin{equation}
U_k = U_k(s) = \left[\begin{array}{c c c c c c}
    1 - s  & s      & 0      & \cdots & \cdots & 0      \\
    0      & 1 - s  & s      & \ddots &        & \vdots \\
    \vdots & \ddots & \ddots & \ddots & \ddots & \vdots \\
    \vdots &        & \ddots & \ddots & \ddots & 0 \\
    0      & \cdots & \cdots & 0      & 1 - s  & s
\end{array}\right] \in \reals^{k \times (k + 1)}.
\end{equation}
With this, we can express (\cite{Mainar1999}) the de Casteljau algorithm as
\begin{equation}\label{eq:matrix-de-casteljau}
b^{(k)} = U_{k + 1} b^{(k + 1)}
\Longrightarrow b^{(0)} = U_1 \cdots U_n b^{(n)}.
\end{equation}

In general, for a sequence \(v_0, \ldots, v_n\) we'll refer to \(v\)
as the vector containing all of the values:
\(v = \left[\begin{array}{c c c} v_0 & \cdots &
    v_n\end{array}\right]^T.\)

\section{B\'{e}zier Triangles}

A \emph{B\'{e}zier triangle} (\cite[Chapter~17]{Farin2001}) is a
mapping from the unit triangle
\(\utri\) and is determined by a control net
\(\left\{\bm{p}_{i, j, k}\right\}_{i + j + k = n} \subset \reals^d\).
A B\'{e}zier triangle is a particular kind of B\'{e}zier surface, i.e. one
in which there are two cartesian or three barycentric input parameters.
Often the term B\'{e}zier surface is used to refer to a tensor product or
rectangular patch.
For \((s, t) \in \utri\) we can define barycentric weights
\(\lambda_1 = 1 - s - t, \lambda_2 = s, \lambda_3 = t\) so that
\begin{equation}
1 = \left(\lambda_1 + \lambda_2 + \lambda_3\right)^n =
  \sum_{\substack{i + j + k = n \\ i, j, k \geq 0}} \binom{n}{i, j, k}
  \lambda_1^i \lambda_2^j \lambda_3^k.
\end{equation}
Using this we can similarly define a (triangular) Bernstein basis
\begin{equation}
B_{i, j, k}(s, t) = \binom{n}{i, j, k} (1 - s - t)^i s^j t^k
  = \binom{n}{i, j, k} \lambda_1^i \lambda_2^j \lambda_3^k
\end{equation}
that is in \(\left[0, 1\right]\) when \((s, t)\) is in \(\utri\).
Using this, we define points on the B\'{e}zier triangle as a
convex combination of the control net:
\begin{equation}
b(s, t) = \sum_{i + j + k = n} \binom{n}{i, j, k}
  \lambda_1^i \lambda_2^j \lambda_3^k
  \bm{p}_{i, j, k} \in \reals^d.
\end{equation}

\begin{figure}
  \includegraphics{../images/preliminaries/main_figure01.pdf}
  \centering
  \captionsetup{width=.75\linewidth}
  \caption{Cubic B\'{e}zier triangle}
  \label{fig:cubic-bezier-example}
\end{figure}

\noindent Rather than defining a B\'{e}zier triangle by the control net, it can
also be uniquely determined by the image of a standard lattice of
points in \(\utri\): \(b\left(j/n, k/n\right) = \bm{n}_{i, j, k}\);
we'll refer to these as \emph{standard nodes}.
Figure~\ref{fig:cubic-bezier-example} shows these standard nodes for
a cubic triangle in \(\reals^2\). To see the correspondence,
when \(p = 1\) the standard nodes \emph{are} the control net
\begin{equation}
b(s, t) = \lambda_1 \bm{n}_{1, 0, 0} +
\lambda_2 \bm{n}_{0, 1, 0} + \lambda_3 \bm{n}_{0, 0, 1}
\end{equation}
and when \(p = 2\)
\begin{multline}
b(s, t) = \lambda_1\left(2 \lambda_1 - 1\right) \bm{n}_{2, 0, 0} +
\lambda_2\left(2 \lambda_2 - 1\right) \bm{n}_{0, 2, 0} +
\lambda_3\left(2 \lambda_3 - 1\right) \bm{n}_{0, 0, 2} + \\
4 \lambda_1 \lambda_2 \bm{n}_{1, 1, 0} +
4 \lambda_2 \lambda_3 \bm{n}_{0, 1, 1} +
4 \lambda_3 \lambda_1 \bm{n}_{1, 0, 1}.
\end{multline}
However, it's worth noting that the transformation between
the control net and the standard nodes has condition
number that grows exponentially with \(n\) (see \cite{Farouki1991}, which
is related but does not directly show this).
This may make working with
higher degree triangles prohibitively unstable.

A \emph{valid} B\'{e}zier triangle is one which is
diffeomorphic to \(\utri\), i.e. \(b(s, t)\) is bijective and has
an everywhere invertible Jacobian. We must also have the orientation
preserved, i.e. the Jacobian must have positive determinant. For example, in
Figure~\ref{fig:inverted-element}, the image of \(\utri\) under
the map \(b(s, t) = \left[\begin{array}{c c} (1 - s - t)^2 + s^2 & s^2 + t^2
\end{array}\right]^T\) is not valid because the Jacobian is zero along
the curve \(s^2 - st - t^2 - s + t = 0\) (the dashed line). Elements that
are not valid are called \emph{inverted} because they have regions with
``negative area''. For the example, the image \(b\left(\utri\right)\)
leaves the boundary determined by the edge curves: \(b(r, 0)\),
\(b(1 - r, r)\) and \(b(0, 1 - r)\) when \(r \in \left[0, 1\right]\).
This region outside the boundary is traced twice, once with
a positive Jacobian and once with a negative Jacobian.
\begin{figure}
  \includegraphics{../images/preliminaries/inverted_element.pdf}
  \centering
  \captionsetup{width=.75\linewidth}
  \caption{The B\'{e}zier triangle given by \(b(s, t) = \left[
    (1 - s - t)^2 + s^2 \; \; s^2 + t^2 \right]^T\) produces an
    inverted element. It traces the same region twice, once with
    a positive Jacobian (the middle column) and once with a negative
    Jacobian (the right column).}
  \label{fig:inverted-element}
\end{figure}

\section{Curved Elements}\label{sec:curved-elements}

We define a curved mesh element \(\mathcal{T}\) of degree \(p\)
to be a B\'{e}zier triangle in \(\reals^2\) of the same degree.
We refer to the component functions of \(b(s, t)\) (the map that
gives \(\mathcal{T} = b\left(\utri\right)\)) as \(x(s, t)\) and \(y(s, t)\).

This fits a typical definition (\cite[Chapter~12]{FEM-ClaesJohnson})
of a curved element, but gives a special meaning to the mapping from
the reference triangle. Interpreting elements as B\'{e}zier triangles
has been used for Lagrangian methods where
mesh adaptivity is needed (e.g. \cite{CardozeMOP04}). Typically curved
elements only have one curved side (\cite{McLeod1972}) since they are used
to resolve geometric features of a boundary. See also
\cite{Zlmal1973, Zlmal1974}.
B\'{e}zier curves and triangles have a number of mathematical properties
(e.g. the convex hull property) that lead to elegant geometric
descriptions and algorithms.

Note that a B\'{e}zier triangle can be
determined from many different sources of data (for example the control net
or the standard nodes). The choice of this data may be changed to suit the
underlying physical problem without changing the actual mapping. Conversely,
the data can be fixed (e.g. as the control net) to avoid costly basis
conversion; once fixed, the equations of motion and other PDE terms can
be recast relative to the new basis (for an example, see \cite{Persson2009},
where the domain varies with time but the problem is reduced to
solving a transformed conservation law in a fixed reference configuration).

\subsection{Shape Functions}\label{subsec:shape-functions}

When defining shape functions (i.e. a basis with geometric meaning) on a
curved element there are (at least) two choices. When the degree of the
shape functions is the same as the degree of the function being
represented on the B\'{e}zier triangle,
we say the element \(\mathcal{T}\) is \emph{isoparametric}.
For the multi-index
\(\bm{i} = (i, j , k)\), we define \(\bm{u}_{\bm{i}} =
\left(j/n, k/n\right)\) and the corresponding standard node
\(\bm{n}_{\bm{i}} = b\left(\bm{u}_{\bm{i}}\right)\).
Given these points, two choices for shape functions present
themselves:
\begin{itemize}
  \itemsep 0em
  \item \emph{Pre-Image Basis}:
    \(\phi_{\bm{j}}\left(\bm{n}_{\bm{i}}\right) =
      \widehat{\phi}_{\bm{j}}\left(\bm{u}_{\bm{i}}\right) =
      \widehat{\phi}_{\bm{j}}\left(b^{-1}\left(
      \bm{n}_{\bm{i}}\right)\right)\)
    where \(\widehat{\phi}_{\bm{j}}\) is a canonical basis function
    on \(\utri\), i.e.
    \(\widehat{\phi}_{\bm{j}}\) a degree \(p\) bivariate polynomial and
    \(\widehat{\phi}_{\bm{j}}\left(\bm{u}_{\bm{i}}\right) =
    \delta_{\bm{i} \bm{j}}\)
  \item \emph{Global Coordinates Basis}:
    \(\phi_{\bm{j}}\left(\bm{n}_{\bm{i}}\right) =
    \delta_{\bm{i} \bm{j}}\), i.e. a canonical basis function
    on the standard nodes \(\left\{\bm{n}_{\bm{i}}\right\}\).
\end{itemize}

\noindent For example, consider a quadratic B\'{e}zier triangle:
\begin{gather}
b(s, t) = \left[ \begin{array}{c c}
    4 (s t + s + t) & 4 (s t + t + 1)
  \end{array}\right]^T \\
\Longrightarrow
\left[ \begin{array}{c c c c c c}
    \bm{n}_{2, 0, 0} &
    \bm{n}_{1, 1, 0} &
    \bm{n}_{0, 2, 0} &
    \bm{n}_{1, 0, 1} &
    \bm{n}_{0, 1, 1} &
    \bm{n}_{0, 0, 2}
  \end{array}\right] = \left[ \begin{array}{c c c c c c}
    0 & 2 & 4 & 2 & 5 & 4 \\
    4 & 4 & 4 & 6 & 7 & 8
  \end{array}\right].
\end{gather}
In the \emph{Global Coordinates Basis}, we have
\begin{equation}
\phi^{G}_{0, 1, 1}(x, y) = \frac{(y - 4) (x - y + 4)}{6}.
\end{equation}
For the \emph{Pre-Image Basis}, we need the inverse
and the canonical basis
\begin{equation}
b^{-1}(x, y) = \left[ \begin{array}{c c}
    \frac{x - y + 4}{4} & \frac{y - 4}{x - y + 8}
  \end{array}\right] \quad \text{and} \quad
\widehat{\phi}_{0, 1, 1}(s, t) = 4 s t
\end{equation}
and together they give
\begin{equation}
\phi^{P}_{0, 1, 1}(x, y) = \frac{(y - 4) (x - y + 4)}{x - y + 8}.
\end{equation}
In general \(\phi_{\bm{j}}^P\) may not even be a rational bivariate
function; due to composition with \(b^{-1}\) we can only guarantee that
it is algebraic (i.e. it can be defined as the zero set of polynomials).

\subsection{Curved Polygons}\label{subsec:curved-polygons}

\begin{figure}
  \includegraphics{../images/preliminaries/main_figure26.pdf}
  \centering
  \captionsetup{width=.75\linewidth}
  \caption{Intersection of B\'{e}zier triangles form a curved polygon.}
  \label{fig:bezier-triangle-intersect}
\end{figure}

When intersecting two curved elements, the resulting surface(s) will
be defined by the boundary, alternating between edges of each
element.
For example, in Figure~\ref{fig:bezier-triangle-intersect}, a
``curved quadrilateral'' is formed when two B\'{e}zier triangles
\(\mathcal{T}_0\) and \(\mathcal{T}_1\) are intersected.

A \emph{curved polygon} is defined by a collection of B\'{e}zier curves
in \(\reals^2\) that determine the boundary. In order to be
a valid polygon, none of the boundary curves may cross, the
ends of consecutive edge curves must meet and the curves must be right-hand
oriented. For our example in
Figure~\ref{fig:bezier-triangle-intersect}, the triangles
have boundaries formed by three B\'{e}zier curves:
\(\partial \mathcal{T}_0 = b_{0, 0} \cup b_{0, 1} \cup b_{0, 2}\) and
\(\partial \mathcal{T}_1 = b_{1, 0} \cup b_{1, 1} \cup b_{1, 2}\).
The intersection \(\mathcal{P}\) is defined by four boundary
curves: \(\partial \mathcal{P} =
C_1 \cup C_2 \cup C_3 \cup C_4\). Each boundary
curve is itself a B\'{e}zier curve\footnote{A specialization of a
B\'{e}zier curve \(b\left(\left[a_1, a_2\right]\right)\)
is also a B\'{e}zier curve.}:
\(C_1 = b_{0, 0}\left(\left[0, 1/8\right]\right)\),
\(C_2 = b_{1, 2}\left(\left[7/8, 1\right]\right)\),
\(C_3 = b_{1, 0}\left(\left[0, 1/7\right]\right)\) and
\(C_4 = b_{0, 2}\left(\left[6/7, 1\right]\right)\).

Though an intersection can be described in terms of the B\'{e}zier triangles,
the structure of the control net will be lost. The region will not in general
be able to be described by a mapping from a simple space like
\(\utri\).

\section{Error-Free Transformation}

An error-free transformation is a computational method where both
the computed result and the round-off error are returned. It
is considered ``free'' of error if the round-off can be represented
exactly as an element or elements of \(\floats\).
The error-free transformations used in this work are
the \texttt{TwoSum} algorithm by Knuth (\cite{Knuth1997}) and
\texttt{TwoProd} algorithm by Dekker (\cite{Dekker1971}, Section 5),
respectively.

\begin{theorem}[\cite{Ogita2005}, Theorem 3.4]\label{thm:eft}
For \(a, b \in \floats\) and \(P, \pi, S, \sigma \in \floats\),
\texttt{TwoSum} and \texttt{TwoProd} satisfy
\begin{alignat}{4}
\left[S, \sigma\right] &= \mathtt{TwoSum}(a, b), & \, S &= \fl{a + b},
  S + \sigma &= a + b, \sigma &\leq \mach \left|S\right|,
  & \, \sigma &\leq \mach \left|a + b\right| \\
\left[P, \pi\right] &= \mathtt{TwoProd}(a, b),
  & \, P &= \fl{a \times b}, P + \pi &= a \times b,
  \pi &\leq \mach \left|P\right|,
  & \, \pi &\leq \mach \left|a \times b\right|.
\end{alignat}
The letters \(\sigma\) and \(\pi\) are used to indicate that the
errors came from sum and product, respectively. See
Appendix~\ref{chap:appendix-algo} for implementation details.
\end{theorem}


% Part: first-order-logic
% Chapter: model-theory
% Section: partial-iso

\documentclass[../../../include/open-logic-section]{subfiles}

\begin{document}

\olfileid{mod}{bas}{dlo}
\section{Dense Linear Orders}

\begin{defn}
  A \emph{dense linear ordering without endpoints} is !!a{structure}
  $\Struct{M}$ for the !!{language} containg a single 2-place
  !!{predicate}~$<$ satisfying the following sentences:
  \begin{enumerate}
  \item $\lforall[x][\lnot x < x]$;
  \item $\lforall[x][\lforall[y][\lforall[z][(x < y \lif (y < z \lif x
    <z ))]]]$;
  \item $\lforall[x][\lforall[y][(x< y \lor \eq[x][y] \lor y < x)]]$;
  \item $\lforall[x][\lexists[y][x < y]]$;
  \item $\lforall[x][\lexists[y][y < x]]$;
  \item $\lforall[x][\lforall[y][(x < y \lif \lexists[z][(x < z \land
        z < y))]]]$.
 \end{enumerate}
\end{defn}

\begin{thm}\ollabel{thm:cantorQ}
  Any two !!{enumerable} dense linear orderings without
  endpoints are isomorphic.
\end{thm}

\begin{proof}
  Let $\Struct{M_1}$ and $\Struct{M_2}$ be !!{enumerable} dense linear
  orderings without endpoints, with ${<_1} = \Assign{<}{M_1}$ and ${<_2} =
  \Assign{<}{M_2}$, and let $\PIso{I}$ be the set of all partial
  isomorphisms between them. $\PIso{I}$ is not empty since at least
  $\emptyset \in \PIso{I}$. We show that $\PIso{I}$ satisfies the
  Back-and-Forth property.  Then $\Struct{M_1} \iso[p] \Struct{M_2}$,
  and the theorem follows by \olref[pis]{thm:p-isom1}.

  To show $\PIso{I}$ satisifes the Forth property, let $p \in
  \PIso{I}$ and let $p(a_i) = b_i$ for $i = 1$, \dots,~$n$, and
  without loss of generality suppose $a_1 <_1 a_2 <_1 \cdots <_1
  a_n$. Given $a \in \Domain{M_1}$, find $b \in \Domain{M_2}$ as
  follows:
  \begin{enumerate}
  \item if $a <_2 a_1$ let $b \in \Domain{M_2}$ be such that $b <_2
    b_1$;
  \item if $a_n <_1 a$ let $b \in \Domain{M_2}$ be such that $b_n <_2 b$;
 \item if $a_i <_1 a <_1 a_{i+1}$ for some $i$, then let $b \in
   \Domain{M_2}$ be such that $b_i <_2 b <_2 b_{i+1}$.
  \end{enumerate}
  It is always possible to find a $b$ with the desired property since
  $\Struct{M_2}$ is a dense linear ordering without endpoints. Define
  $q = p \cup \{ \langle a, b \rangle \}$ so that $q \in \PIso{I}$ is
  the desired extension of $p$. This establishes the Forth
  property. The Back property is similar. So $\Struct{M_1} \iso[p]
  \Struct{M_2}$; by \olref[pis]{thm:p-isom1}, $\Struct{M_1} \iso
  \Struct{M_2}$.
\end{proof}

\begin{prob}
  Complete the proof of \olref[mod][bas][dlo]{thm:cantorQ} by
  verifying that $\PIso{I}$ satisfies the Back property.
\end{prob}

\begin{rem}
  Let $\Struct{S}$ be any !!{enumerable} dense linear ordering without
  endpoints. Then (by \olref{thm:cantorQ}) $\Struct{S} \iso
  \Struct{Q}$, where $\Struct{Q} = (\Rat, <)$ is the !!{enumerable}
  dense linear ordering having the set $\Rat$ of the rational numbers
  as its domain. Now consider again the !!{structure} $\Struct{R} =
  (\Real, <)$ from \olref[thm]{remark:R}. We saw that there is
  !!a{enumerable} !!{structure} $\Struct{S}$ such that $\Struct{R}
  \elemequiv \Struct{S}$. But $\Struct{S}$ is !!a{enumerable} dense
  linear ordering without endpoints, and so it is isomorphic (and
  hence elementarily equivalent) to the !!{structure}~$\Struct{Q}$. By
  transitivity of elementary equivalence, $\Struct{R} \elemequiv
  \Struct{Q}$. (We could have shown this directly by establishing
  $\Struct{R} \iso[p] \Struct{Q}$ by the same back-and-forth
  argument.)
\end{rem}
\end{document}


\problemname{Mixing Drinks}
Pia is a famous bartender at the hip Stockholm night club Supernova.
One of her most impressive feats is the mixing a series of drinks using each of the $N$ distinct drink ingredients in the bar exactly once.
She does this in the following way.

First, Pia chooses a number of drinks to make.
Each of the drink ingredients are then lined up in front of her in order $1, 2, \dots, N$.
For the first drink, she uses some positive number $K$ of ingredients starting from the left, i.e. $1, 2, ..., K$.
For the next drink, she uses some positive number $L$ of ingredients starting from the first unused ingredient, i.e. $K + 1, K + 2, \dots, K + L$.
She continues this process until the final drink, which uses some set of ingredients $N - M, N - M + 1, \dots, N$.

However, not every pair of ingredients work well in a drink.
For example, milk and water would not go very well together.
She may not include a bad pair of ingredients in any drink.

So far, she has managed to make a different set of drinks every night.
For how many nights can she mix a new set of drinks?
We call two sets of drinks different if they do not consist of the exact same drinks (though they are allowed to have drinks in common).

\section*{Input}
The first line of the input contains two integers $1 \le N \le 100\,000$ and $0 \le P \le 100\,000$, the number of ingredients and bad pairs of ingredients.

Each of the next $P$ lines contains two integers $1 \le a \not= b \le N$, two ingredients that do not work well together in a drink.
The same pair of ingredients may appear multiple times in this list.

\section*{Output}
Output a single integer, the number of nights Pia can construct a different set of drinks.
Since this number may be large, output the remainder when divided by $10^9 + 7$.