\chapter{The Cox Proportional Hazards Model \label{chapter:cox}}

We just encountered the Kaplan-Meier estimate of the survival function in Chapter~\ref{chapter:km}. Now we are going to talk about models that essentially treat the survival function as the \emph{outcome} in a supervised learning problem. These models are called \textbf{Cox proportional hazards models}.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Survival and Hazard Functions}

Consider a situation where we have some process that generates events, and we're trying to model the time to first event. The probability density for when the event occurs is $f(t)$. The cumulative probability of the event's having occurred by time $t$, also called the \textbf{cumulative incidence}, is
$$ F(t) = \int_0^t f(t) dt.$$
The probability of an individual's \emph{not} having experienced the event by time $t$ is
$$ S(t) = 1 - F(t), $$
which we call the \textbf{survival}. The probability of experiencing the event in an infinitesimally small interval starting at $t$, given that one has not experienced it by time $t$, is:
$$ \lambda(t) = \frac{f(t)}{S(t)} $$
and is called the \textbf{hazard}. The \textbf{cumulative hazard} function is equal to
$$ \Lambda(t) = \int_0^t \lambda(t') dt' = -\log S(t) $$
so $S(t) = \exp(-\Lambda(t))$.

Here is a graphical representation of some of these quantities. Remember that the probability distribution $f(t)$ must integrate to one.

\begin{center}
\includegraphics[width=0.85\textwidth]{img/survival-function-example-fx.png}
\end{center}

\vspace{3mm}

\begin{question}{}
What's the interpretation of the hazard, $\lambda(t)$, on this image? What happens to the hazard if $S(t)$ is low vs. high for the same $f(x)$?
\end{question}

\begin{question}{}
Assume $f(t)$ is exponential: $f(t) = b \cdot \exp(-bt)$, where $b$ is constant. Then $F(t) = 1 - \exp(-bt)$ and $S(t) = \exp(-bt)$. What is the hazard, $\lambda(t)$? What is the cumulative hazard, $\Lambda(t)$?
\end{question}

\vspace{3mm}

\begin{question}{}
The concept of a ``cumulative hazard'' is pretty weird. How should this quantity be interpreted?
% 1. The total amount of risk accumulated up to time t.
% 2. The number of times per subject that we would expect to observe failures over a given period if the failure event were repeatable.
% 3. It is useful in terms of allowing us to test assumptions of Cox models. Sort of like the logit in that way. It pops out of the modeling, so we care about it, but it isn't what we usually think about.
\end{question}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Estimating Survival and Cumulative Hazard}

The Kaplan-Meier estimate of survival (Chapter~\ref{chapter:km}) is the most common estimate of the survival function. One can estimate the cumulative hazard using a couple of different methods.
\begin{enumerate}
\item Take the negative log of the Kaplan-Meier estimate of survival:
\begin{align*} \hat{\Lambda}_{KM}(t) &= -\log \hat{S}_{\text{KM}}(t) \\
&= -\sum_{i:t_i < t} \log \left( 1 - \frac{d_i}{n_i} \right) \end{align*}
\item Use the \textbf{Nelson-Aalen estimator}
$$ \hat{\Lambda}_{NA}(t) = \sum_{i:t_i < t} \frac{d_i}{n_i} $$
\end{enumerate}

One thing that is confusing about the R \texttt{survival} package's output is that is uses the Kaplan-Meier estimate for survival by default, but it uses the Nelson-Aalen estimator for cumulative hazard by default. So if you exponentiate the negative cumulative hazard that comes out of \texttt{survfit}, it won't match the survival estimate. The two are close in most cases, though. Here are some pictures for the ovarian cancer survival dataset we discussed in Chapter~\ref{chapter:km}:

\begin{center}
\includegraphics[width=0.85\textwidth]{img/ovarian-overall-survival.png}\\[3mm]
\includegraphics[width=0.85\textwidth]{img/ovarian-overall-cumhaz.png}
\end{center}

% From survfit.formula docs: "The routine returns both an estimated probability in state and an estimated cumulative hazard estimate. The cumulative hazard estimate is the Nelson-Aalen (NA) estimate or the Fleming-Harrington (FH) estimate, the latter includes a correction for tied event times. The estimated probability in state can estimated either using the exponential of the cumulative hazard, or as a direct estimate using the Aalen-Johansen approach. For single state data the AJ estimate reduces to the Kaplan-Meier and the probability in state to the survival curve; for competing risks data the AJ reduces to the cumulative incidence (CI) estimator. For backward compatability the type argument can be used instead."

\newpage

\begin{question}{}
Here is a plot showing the function $-\log(1-x)$ vs. $x$. Look at the expressions for the two estimators for the cumulative hazard, above. Under what conditions will they be similar? Under what conditions will they be different?
\begin{center}
\includegraphics[width=0.6\textwidth]{img/survival-na-km-comparison.png}
\end{center}
\end{question}

\vspace{3mm}

\begin{question}{}
The curvature of the Nelson-Aalen estimator gives you an idea of how the hazard varies with time. A concave shape is an indication of \emph{deceleration} of the hazard; for example, if the event in question is death and time is patient age, this would represent higher infant/childhood mortality than adult mortality. A convex shape is an indication of \emph{acceleration} of the hazard; in the death/age example, this would represent a process that accelerates as one ages (so called ``wear-out mortality'').

Looking at the graph of the overall cumulative hazard, what do you notice about how the hazard for ovarian cancer changes since the initiation of treatment?
\end{question}

\newpage

\begin{question}{}
Here are the raw data from treatment group $1$ of the \texttt{ovarian} dataset. These are the same data we looked at when building the Kaplan-Meier curve in Chapter~\ref{chapter:km}, Question~\ref{question:kmo1}. Using these data, fill in the remaining cells of the table below. Here we are using the Nelson-Aalen estimator for the cumulative hazard.
{\footnotesize
\begin{center}
\begin{tabular}{rlrr}
  \toprule
 & rx & futime & fustat \\
  \midrule
  1 & 1 & 59 & 1 \\
  2 & 1 & 115 & 1 \\
  3 & 1 & 156 & 1 \\
  4 & 1 & 268 & 1 \\
  5 & 1 & 329 & 1 \\
  6 & 1 & 431 & 1 \\
  7 & 1 & 448 & 0 \\
  8 & 1 & 477 & 0 \\
  9 & 1 & 638 & 1 \\
  10 & 1 & 803 & 0 \\
  11 & 1 & 855 & 0 \\
  12 & 1 & 1040 & 0 \\
  13 & 1 & 1106 & 0 \\
  \bottomrule
\end{tabular}
\end{center}
}
\vspace{-5mm}
{\footnotesize
\begin{center}
\begin{tabular}{rrrrll}
  \toprule
$j$ & $t_j$ & $n_j$ & $d_j$ & $\hat{\Lambda}(t_j)$ & Calculation \\
  \midrule
  0 & 0 & 13 & 0 & $0.000$ & $\frac{0}{13}$ \\
  1 & 59 & 13 & 1 & $0.077$ & $\hat{\Lambda}(t_0) + \frac{1}{13}$ \\
  2 & 115 & 12 & 1 & $0.160$ & $\hat{\Lambda}(t_1) + \frac{1}{12}$ \\[2mm]
  3 & 156 & \\[2mm] % 11 & 1 & $0.251$ & $\hat{\Lambda}(t_2) + \frac{1}{11}$ \\
  4 & 268 & \\[2mm] % 10 & 1 & $0.351$ & $\hat{\Lambda}(t_3) + \frac{1}{10}$ \\
  5 & 329 & 9 & 1 & $0.462$ & $\hat{\Lambda}(t_4) + \frac{1}{9}$ \\
  6 & 431 & 8 & 1 & $0.587$ & $\hat{\Lambda}(t_5) + \frac{1}{8}$ \\
  7 & 448 & 7 & 0 & $0.587$ & $\hat{\Lambda}(t_6) + \frac{0}{7}$ \\
  8 & 477 & 6 & 0 & $0.587$ & $\hat{\Lambda}(t_7) + \frac{0}{6}$ \\
  9 & 638 & 5 & 1 & $0.787$ & $\hat{\Lambda}(t_8) + \frac{1}{5}$ \\[2mm]
  10 & 803 & 4 & 0 & \\[2mm] % $0.787$ & $\hat{\Lambda}(t_9) + \frac{0}{4}$ \\
  11 & 855 & 3 & 0 & \\[2mm] % $0.787$ & $\hat{\Lambda}(t_{10}) + \frac{0}{3}$ \\
  12 & 1040 & 2 & 0 & \\[2mm] % $0.787$ & $\hat{\Lambda}(t_{11}) + \frac{0}{2}$ \\
  13 & 1106 & 1 & 0 & \\[2mm] % $0.787$ & $\hat{\Lambda}(t_{12}) + \frac{0}{1}$ \\
  \bottomrule
\end{tabular}
\end{center}
}
\end{question}

\newpage

\begin{question}{}
Here are plots of the cumulative hazard (Nelson-Aalen estimator) for patients by sex and by ECOG performance score status:
\begin{center}
\includegraphics[width=0.45\textwidth]{img/ovarian-rx-cumhaz.png}
\hspace{4mm}
\includegraphics[width=0.45\textwidth]{img/ovarian-ecog-cumhaz.png}
\end{center}
How does treatment group appear to impact the cumulative hazard? What about ECOG score? Do the cumulative hazards appear proportional (i.e., related by a common multiplier) in each case?
\end{question}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Deriving the Cox Model}

We have spent considerable time on the hazard and cumulative hazard because these play important roles in what is perhaps the most famous survival analysis tool: the Cox proportional hazards model (``Cox model'', for short), developed by D.R. Cox in 1972. The model has the following form:
$$ \lambda(t|x) = \lambda_0(t)\exp(\beta_1 x_1 + \beta_2 x_2 + \dots + \beta_p x_p) $$
and another way of writing it is:
$$ \log \left( \frac{\lambda(t|x)}{\lambda_0(t)} \right) = \beta_1 x_1 + \beta_2 x_2 + \dots + \beta_p x_p. $$
Note that for now we are assuming that the covariates do not depend on time. There is a variant of the Cox model called the \textbf{extended Cox model} that allows time-dependent covariates. For now, we will just consider the fixed covariate case.

\vspace{3mm}

\begin{question}{}
Compare the Cox model to a logistic regression model. What is the same? What is different?
\end{question}

Here $\lambda_0(t)$ is called the \textbf{baseline hazard}. As usual, we symbolize the linear sum of the $\beta$s as $\beta^Tx$. Importantly, the baseline hazard can take any shape as a function of time. The $\lambda_0(t)$ part of the equation is, therefore, referred to as the ``nonparametric'' part, while the $\beta^Tx$ part is called the ``parametric'' part. The overall model is referred to as \textbf{semiparametric}.

The ratio of the hazards for two different sets of covariates, $x$ and $z$, is
$$ \frac{\lambda(t|x)}{\lambda(t|z)} = \frac{\lambda_0(t)\exp(\beta^Tx)}{\lambda_0(t)\exp(\beta^Tz)} = \exp(\beta^T (x-z)). $$
Taking the log of this, we arrive at
$$ \log \left( \frac{\lambda(t|x)}{\lambda(t|z)} \right) = \beta^T (x-z). $$
A single coefficient, $\beta_j$, is therefore the \textbf{hazard ratio} when the corresponding predictor, $x_j$, increases by one. This ratio is assumed to be constant over time. The hazard ratio is also called the \textbf{relative risk}.

\vspace{5mm}

\begin{question}{}
Why doesn't the Cox model have an intercept, $\beta_0$?
\end{question}

\vspace{1mm}

\begin{question}{}
Compare the interpretation of the coefficients in a Cox model to their interpretation in a logistic regression model.
\end{question}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Fitting the Cox Model \label{section:coxfit}}

Fitting a Cox model means taking a sample of possibly right-censored data and deriving estimates for the parameters, $\beta_1, \dots, \beta_p$. Cox models are fit using a variant of maximum likelihood estimation called \textbf{partial likelihood estimation}.

Let's consider all of the unique times, $t_i$, that events are observed. For now, we will assume that exactly one event is observed at each of these times (no ties). We will use $R(t_i)$ to refer to the set of subjects who are ``at risk'' (i.e., not censored) just prior to time $t_i$. At each failure time, $t_i$, the contribution to the partial likelihood is:
\begin{align*} \mathcal{L}_i(\beta) &= \frac{P(\text{person $i$ experiences event|still around at $t_i$)}}{\sum_{l \in R(t_i)} P(\text{person $l$ experiences event|still around at $t_i$)}} \\
&= \frac{\lambda(t_i|x_i)}{\sum_{l \in R(t_i)} \lambda(t_i|x_l)} = \frac{\exp(\beta^Tx^{(i)})}{\sum_{l \in R(t_i)} \exp( \beta^Tx^{(l)})} \end{align*}
The complete partial likelihood over all $K$ observed event times, $t_i = t_1, \dots, t_K$, is:
$$ \mathcal{L}(\beta) = \prod_{i=1}^K \frac{\exp(\beta^Tx^{(i)})}{\sum_{l \in R(t_i)} \exp( \beta^Tx^{(l)})}.  $$
This is the thing that the model fitting process is trying to maximize. It is optimized numerically, using the same types of optimization procedures used by logistic regression and other generalized linear models.

\vspace{3mm}

\begin{question}{}
Why is this quantity called a ``partial likelihood'', instead of just a ``likelihood''?
\end{question}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Interpreting Cox Models}

Here is a simple Cox model for survival time vs. age, residual disease, ECOG score, and treatment group in the ovarian cancer dataset.

\begin{center}
\includegraphics[width=0.9\textwidth]{img/ovarian-coxph-model.png}
\end{center}

\vspace{3mm}

\begin{question}{}
Interpret the coefficients, exponentiated coefficients, standard errors of the coefficients, Z scores, and $p$-values in this model. Also try your hand at interpreting the second block of model output, which includes the exponentiated coefficients (again), the exponentiated negative coefficients, and the lower and upper bounds of a 95\% confidence interval for the exponentiated coefficients.
\end{question}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Making Predictions with Cox Models}

The Cox model fitted using the \texttt{coxph} function can be used to make various predictions on the original dataset. This yields a lot of output. All of the available outputs for the \texttt{ovarian} dataset are shown below.

{\scriptsize
\begin{center}
\begin{tabular}{rrrrrrrrrrrrr}
  \toprule
 & age & lp\_age & resid.ds & lp\_resid.ds & ecog.ps & lp\_ecog.ps & rx & lp\_rx & lp & risk & expected & surv \\
  \midrule
  1 & 72.33 & 9.03 & 2 & 0.83 & 1 & 0.00 & 1 & 0.00 & 2.67 & 14.43 & 0.16 & 0.85 \\
  2 & 74.49 & 9.30 & 2 & 0.83 & 1 & 0.00 & 1 & 0.00 & 2.94 & 18.90 & 0.46 & 0.63 \\
  3 & 66.47 & 8.30 & 2 & 0.83 & 2 & 0.34 & 1 & 0.00 & 2.27 & 9.71 & 0.40 & 0.67 \\
  4 & 53.36 & 6.66 & 2 & 0.83 & 1 & 0.00 & 2 & -0.91 & -0.61 & 0.54 & 0.11 & 0.89 \\
  5 & 50.34 & 6.28 & 2 & 0.83 & 1 & 0.00 & 1 & 0.00 & -0.08 & 0.93 & 0.25 & 0.78 \\
  6 & 56.43 & 7.04 & 1 & 0.00 & 2 & 0.34 & 1 & 0.00 & 0.19 & 1.21 & 0.33 & 0.72 \\
  7 & 56.94 & 7.11 & 2 & 0.83 & 2 & 0.34 & 2 & -0.91 & 0.17 & 1.18 & 0.40 & 0.67 \\
  8 & 59.85 & 7.47 & 2 & 0.83 & 2 & 0.34 & 2 & -0.91 & 0.53 & 1.71 & 0.70 & 0.49 \\
  9 & 64.18 & 8.01 & 2 & 0.83 & 1 & 0.00 & 1 & 0.00 & 1.65 & 5.21 & 2.15 & 0.12 \\
  10 & 55.18 & 6.89 & 1 & 0.00 & 2 & 0.34 & 2 & -0.91 & -0.88 & 0.42 & 0.24 & 0.79 \\
  11 & 56.76 & 7.08 & 1 & 0.00 & 2 & 0.34 & 1 & 0.00 & 0.24 & 1.27 & 0.94 & 0.39 \\
  12 & 50.11 & 6.25 & 1 & 0.00 & 1 & 0.00 & 2 & -0.91 & -1.84 & 0.16 & 0.12 & 0.89 \\
  13 & 59.63 & 7.44 & 2 & 0.83 & 2 & 0.34 & 2 & -0.91 & 0.51 & 1.66 & 1.23 & 0.29 \\
  14 & 57.05 & 7.12 & 2 & 0.83 & 1 & 0.00 & 2 & -0.91 & -0.15 & 0.86 & 0.63 & 0.53 \\
  15 & 39.27 & 4.90 & 1 & 0.00 & 1 & 0.00 & 1 & 0.00 & -2.28 & 0.10 & 0.08 & 0.93 \\
  16 & 43.12 & 5.38 & 1 & 0.00 & 2 & 0.34 & 1 & 0.00 & -1.47 & 0.23 & 0.17 & 0.84 \\
  17 & 38.89 & 4.85 & 2 & 0.83 & 2 & 0.34 & 1 & 0.00 & -1.17 & 0.31 & 0.23 & 0.79 \\
  18 & 44.60 & 5.57 & 1 & 0.00 & 1 & 0.00 & 1 & 0.00 & -1.62 & 0.20 & 0.15 & 0.86 \\
  19 & 53.91 & 6.73 & 1 & 0.00 & 1 & 0.00 & 2 & -0.91 & -1.37 & 0.25 & 0.19 & 0.83 \\
  20 & 44.21 & 5.52 & 2 & 0.83 & 1 & 0.00 & 2 & -0.91 & -1.76 & 0.17 & 0.13 & 0.88 \\
  21 & 59.59 & 7.44 & 1 & 0.00 & 2 & 0.34 & 2 & -0.91 & -0.33 & 0.72 & 0.53 & 0.59 \\
  22 & 74.50 & 9.30 & 2 & 0.83 & 2 & 0.34 & 1 & 0.00 & 3.28 & 26.49 & 1.65 & 0.19 \\
  23 & 43.14 & 5.38 & 2 & 0.83 & 1 & 0.00 & 1 & 0.00 & -0.97 & 0.38 & 0.04 & 0.96 \\
  24 & 63.22 & 7.89 & 1 & 0.00 & 2 & 0.34 & 2 & -0.91 & 0.13 & 1.14 & 0.18 & 0.84 \\
  25 & 64.42 & 8.04 & 2 & 0.83 & 1 & 0.00 & 2 & -0.91 & 0.77 & 2.16 & 0.45 & 0.64 \\
  26 & 58.31 & 7.28 & 1 & 0.00 & 1 & 0.00 & 2 & -0.91 & -0.82 & 0.44 & 0.09 & 0.91 \\
   \bottomrule
\end{tabular}
\end{center}
}

The term \texttt{age} is the raw value for age for each patient, and the term \texttt{lp\_age} is the linear predictor, $\beta_{\text{age}} x_{\text{age}}^{(i)}$, for patient $i$. The same is true for the other predictors. The term \texttt{lp} is the entire linear predictor, $\beta^Tx$, for each $x^{(i)}$. Confusingly, it has been centered, so its value is shifted from the sum of columns 2, 4, 6, and 8 by a fixed amount (Exercise: What is this amount?).  The \texttt{risk} term is just the overall risk score, $\exp(\texttt{lp})$. The \texttt{expected} term is the expected number of events given the covariates and follow-up time. The survival probability, \texttt{surv}, for each subject is $\exp(-\texttt{expected})$. % answer: 7.184752

Cox models make no assumptions about the shape of the baseline hazard, so to make predictions, they simply use the empirical survival (or cumulative hazard) curve for the entire dataset and then adjust it up or down depending on the values of the covariates. The way R does this is super confusing - it estimates the baseline hazard at the means of the covariates after centering, so the baseline hazard is not very interpretable. You can get it out of the model using the \texttt{basehaz} function. In any case, here's what you get when you ask the model to predict the survival and cumulative hazard curves for all of the patients in the training set:

\begin{center}
\includegraphics[width=0.9\textwidth]{img/ovarian-survplot-bypatient.png}\\
\includegraphics[width=0.9\textwidth]{img/ovarian-cumhaz-bypatient.png}
\end{center}

\newpage

\begin{question}{}
Which patients have the lowest and highest risk scores? Where do they appear on the patient-level survival and cumulative hazard graphs?
\end{question}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Testing the Proportional Hazards Assumption}

The Cox model makes three important assumptions:
\begin{enumerate}
\item \emph{Common baseline hazard.} At any time, $t$, all individuals experience the same baseline hazard, $\lambda_0(t)$.
\item \emph{Proportional hazards.} The hazard for one individual is proportional to the hazard of any other individual.
\item \emph{Time-invariance.} The constant of proportionality between the hazards of any two individuals does not depend on time.
\end{enumerate}
All of them are potentially problematic. In particular, it's hard to come up with situations in which the hazards for \emph{any} two individuals can reasonably be assumed to be proportional. Thus, it's important to check this assumption.

There are whole book chapters and papers devoted to model diagnostics for the Cox model. I will present a couple of common methods here and leave the others to the course website.

\subsection{Schoenfeld Residuals}

In Section~\ref{section:coxfit}, we saw how the Cox model was fit using maximization of the partial likelihood. The quantity
$$ \frac{\exp(\beta^Tx^{(i)})}{\sum_{l \in R(t_i)} \exp( \beta^Tx^{(l)})} $$
was important because it gave us the probability, according to the model, that the person observed to experience the event at time $t_i$ would experience it, given all the people in the risk set just prior to time $t_i$. The \textbf{Schoenfeld residual} capitalizes on this idea.

\begin{quote}
\textbf{Schoenfeld residual:} The covariate value $x_j^{(i)}$ for the person ($i$) who actually experienced the event at time $t_i$, minus the expected value of the covariate for the risk set at $t_i$. Or:
$$ \text{residual} = x_j^{(i)} - \sum_{l \in R(t_i)} x_j^{(l)} \frac{\exp(\beta^Tx^{(l)})}{\sum_{m \in R(t_i)} \exp( \beta^Tx^{(m)})} $$
\end{quote}

There is one Schoenfeld residual for each combination of observed event and covariate. Typically, they will be plotted against time to assess if there is a trend. The test for trend comes from a simple linear regression model of the residuals against time, conducted separately for each covariate.

\begin{center}
\includegraphics[width=0.9\textwidth]{img/ovarian-cox-diagnostics-schoenfeld.png}
\end{center}

\begin{question}{}
How would you conduct the test for trend for each covariate using a simple linear regression model?
\end{question}

\begin{question}{}
Try calculating the Schoenfeld residual for a single covariate and failure time (your choice). All of the information you need is in the table below. The \texttt{risk} column is the same as in the previous table. It is $\exp{(\beta^Tx)}$.
{\small
\begin{center}
\begin{tabular}{rrrrrrrr}
  \hline
 & futime & fustat & age & resid.ds & ecog.ps & rx & risk \\
  \hline
1 & 59 & 1 & 72.33 & 2 & 1 & 1 & 14.43 \\
  2 & 115 & 1 & 74.49 & 2 & 1 & 1 & 18.90 \\
  3 & 156 & 1 & 66.47 & 2 & 2 & 1 & 9.71 \\
  22 & 268 & 1 & 74.50 & 2 & 2 & 1 & 26.49 \\
  23 & 329 & 1 & 43.14 & 2 & 1 & 1 & 0.38 \\
  24 & 353 & 1 & 63.22 & 1 & 2 & 2 & 1.14 \\
  25 & 365 & 1 & 64.42 & 2 & 1 & 2 & 2.16 \\
  26 & 377 & 0 & 58.31 & 1 & 1 & 2 & 0.44 \\
  4 & 421 & 0 & 53.36 & 2 & 1 & 2 & 0.54 \\
  5 & 431 & 1 & 50.34 & 2 & 1 & 1 & 0.93 \\
  6 & 448 & 0 & 56.43 & 1 & 2 & 1 & 1.21 \\
  7 & 464 & 1 & 56.94 & 2 & 2 & 2 & 1.18 \\
  8 & 475 & 1 & 59.85 & 2 & 2 & 2 & 1.71 \\
  9 & 477 & 0 & 64.18 & 2 & 1 & 1 & 5.21 \\
  10 & 563 & 1 & 55.18 & 1 & 2 & 2 & 0.42 \\
  11 & 638 & 1 & 56.76 & 1 & 2 & 1 & 1.27 \\
  12 & 744 & 0 & 50.11 & 1 & 1 & 2 & 0.16 \\
  13 & 769 & 0 & 59.63 & 2 & 2 & 2 & 1.66 \\
  14 & 770 & 0 & 57.05 & 2 & 1 & 2 & 0.86 \\
  15 & 803 & 0 & 39.27 & 1 & 1 & 1 & 0.10 \\
  16 & 855 & 0 & 43.12 & 1 & 2 & 1 & 0.23 \\
  17 & 1040 & 0 & 38.89 & 2 & 2 & 1 & 0.31 \\
  18 & 1106 & 0 & 44.60 & 1 & 1 & 1 & 0.20 \\
  19 & 1129 & 0 & 53.91 & 1 & 1 & 2 & 0.25 \\
  20 & 1206 & 0 & 44.21 & 2 & 1 & 2 & 0.17 \\
  21 & 1227 & 0 & 59.59 & 1 & 2 & 2 & 0.72 \\
   \hline
\end{tabular}
\end{center}
}
\end{question}

\begin{question}{}
The R function \texttt{cox.zph} performs the test of trend for all predictors as well as a global test of trend using ANOVA (don't worry, we'll get to this later). Here is the output for this model:
\begin{center}
\includegraphics[width=0.4\textwidth]{img/ovarian-coxzph.png}
\end{center}
For which predictors is there a potentially worrying association between the Schoenfeld residuals and time?
\end{question}

\subsection{What to do if Violations are Found}

Interpretation of the Cox model is relatively insensitive to deviations from proportionality, especially for large sample sizes. However, if nonproportionality is a huge issue for one or more predictors, there are a few strategies to deal with it.

\begin{enumerate}
\item \emph{Stratify.} One can stratify the model by different levels of the problematic predictor(s), essentially building separate models for the other covariates at each different level of the problematic predictor(s). This only works for predictors that have discrete levels, however; otherwise, one would need to discretize. A potential downside is that stratification eliminates the model's ability to quantify the effect of the stratification variable(s).
\item \emph{Partition the time axis.} Sometimes proportionality holds for the first part of the time axis but falls apart at the end. In that case, one can analyze the data from the first part of the study separately. The disadvantage, of course, is that one must throw out information from later parts of the study.
\item \emph{Add a nonlinear effect term.} Continuous covariates with nonlinear effects on the outcome may lead to nonproportional hazards. Including transformations of these covariates may help to alleviate the nonproportional hazards.
\end{enumerate}


%Background (a brief technical discussion of the D* Lite search method you implemented); figures may be helpful here;

\section{Background}
	This section will contain a brief technical discussion and intuition for each of the three approaches.

	\subsection{Naive Re-planning A*}
	    The NRA* algorithm is an adaption of the classic A* Search to an environment that is partially observable. In an A* Search, a rational agent prioritizes nodes $s$ in the fringe based on a value $f(s) = g(s)+h(s)$, which corresponds to the cost to reach the current node, plus the heuristic cost to reach the goal. In NRA*, the only information that the agent can perceive are the four adjacent cells in the assumed grid-world environment. Therefore, the NRA* agent generates an optimal path, and follows it until one of two things occur: (a) it has reached the goal or (b) it encounters a previously-unobserved obstacle. In this second case, it updates the edge weights into the obstacle square, re-runs A* search from its current location, and repeats the process as before. N.B. that when this re-planning takes place, any previous route knowledge that might have been computed previously does not carry over to the next search, resulting in significant recomputation.

	\subsection{Lifelong Planning A*}
	    The LPA* search algorithm is based on an incremental heuristic search \cite{koenig2002d} \cite{koenig2002improved}. Unlike NRA*, LPA* is able to use information from previous routefinding computations to inform new searches. To do so, LPA* keeps track of $g(s), rhs(s)$ 2-tuples for each node $s$ in the graph. Intuitively, $g(s)$ is an estimate of the cost so far, and corresponds to the same value from A* search. $rhs(s)$ is a one-step look-ahead estimate that takes into consideration the predecessors $s'$ of $s$, potentially providing a better estimate than $g(s)$. More formally, the $rhs$ is defined as 0 if $s=s_{start}$ and $min_{s'\in pred(s)} g(s')+c(s',s)$ otherwise. The queuing strategy for fringe nodes is more complex than in (NR)A*, as it uses a tuple of keys that establish a dual-priority system. Since the $rhs(s)$ values are a ``potentially better informed'' version of the $g(s)$ values, a node is called locally inconsistent if $g(s) \neq rhs(s)$. When a node becomes locally inconsistent, either through initialization or replanning, it is added to this queue.

	    Initially, only the start state is locally inconsistent. By making the nodes locally consistent, and therefore making sure that the $rhs$ values are precisely the same as the $g$ values, an optimal path can be found based on the currently-known edge weights. Upon discovering environmental changes which affect the edge weights between nodes, it updates those affected nodes, and makes them locally inconsistent (or in other words, adds them to the priority queue) if necessary. It then repeats the pathfinding process until the graph is once again consistent. Note that edge weights can increase (locally overconsistent) or decrease (locally underconsistent) in general, but in our example scenario, edges can only increase. This corresponds to the agent encountering an impassable barrier. Once the graph is locally consistent, the shortest path to the goal can be found by starting at the goal state, transitioning back to the predecessor through the nodes which minimize the cost, continuing until the start state is reached.

		A key drawback for LPA* is that it does not allow for changing start positions. For our motivational scenario, where the agent must directly observe the updated edge weights, this can cause significant backtracking.

	\subsection{D* Lite}
		 D* Lite \cite{koenig2004lifelong} \cite{simmons1995probabilistic} aims to improve on this by only updating nodes on the path from the in-progress current position to the goal state. To do so, modest changes to the LPA* algorithm are required; however, the core replanning approach of LPA* is maintained.

		LPA* iteratively attempts to find shortest paths from the start to the goal, incorporating new edge costs as it observes them through the local consistency approach described above. In contrast, D* Lite iteratively attempts to find the shortest path from the $current$ node to the goal, also incorporating new edge costs as it observes them, but only towards the goal. There are some other differences as well. In particular, the search direction in now switched, and the $g(s)$ values now represent the goal distance, not the backwards cost. Similarly, the $rhs$ values are now forward looking, i.e. one step look-ahead with respect to successors and their respective goal distances. Similar to LPA*, D* Lite also prioritizes nodes based on a tuple that is defined in terms of  $g(s), rhs(s)$. However, the definition of this tuple differs from that of LPA* in that its priorities are effectively lower bounds on those priorities from LPA*. It also maintains a priority offset $k_m$, which allows it to bound the replanning propagation. This re-formulation, along with a less strict terminating condition allow D* Lite to have better performance than LPA*.

		The framework for D* Lite is as follows: it first attempts to compute the shortest path from the start to the goal node. It then takes steps along this path, and makes any changes to the edge costs and associated priority queue values when necessary. It then computes the shortest path using this new information, and repeats. The currently-optimal shortest path to the goal can be found by starting at the current start state, transitioning to the successor which minimizes the $rhs$ condition, i.e. one step look-ahead, and continuing until the goal state is reached. However, to extract the actual path taken by the agent including replanning, this information must be extracted retrospectively, by recording the steps taken.


\section{Logistic Regression II}

\subsection{Parameterization}

\begin{frame}
  \frametitle{Parameterization}

  \begin{itemize}
    \item Until now, $F(\vec x)$ was some arbitrary function in $\vec x$ \\[.1cm]
      \structure{Example:} $ F(\vec x) = \vec x^T \mat A \vec x + \vec \alpha^T\vec x + \alpha_0 $ with components defined by Gaussian distributions \\[.25cm] \pause
    \item We can express a nonlinear $F(\vec x)$ as a scalar product by lifting $\vec x$ into a higher dimensional space: \\
      Given \\
      \hspace{0.5cm} $\vec x = \left(x_1, x_2\right)^T \in \real^2,~ \mat A = \left( \begin{array}{cc} a_{11} & a_{12} \\ a_{21} & a_{22} \end{array} \right), ~ \vec \alpha = \left(\alpha_1, \alpha_2 \right)^T, ~\alpha_0 $\,, \\
      then \\
      \hspace{0.5cm} $F(\vec x) = a_{11} x_1^2 + (a_{12} + a_{21}) x_1 x_2 + a_{22} x_2^2 + \alpha_1 x_1 + \alpha_2 x_2 + \alpha_0$\,.\\[.25cm] \pause
    \item Rewrite $F(\vec x) = \vec \theta^T \vec x'$ with $\vec \theta, \vec x' \in \real^6$: \\ \pause
      \hspace{0.5cm} $\vec \theta = (a_{11}, a_{12} + a_{21}, a_{22}, \alpha_1, \alpha_2, \alpha_0)^T$ \\
      \hspace{0.5cm} $\vec x' = ( x_1^2, x_1 x_2, x_2^2, x_1, x_2, 1)^T$
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Parameterization \cont}

  \begin{citeblock}{Definition}

    We write the parameterized logistic function in the following:
    \begin{eqnarray*}
      g(\vec \theta^T \vec x) &=& \frac{1}{1+e^{-\vec \theta^T \vec x}}
    \end{eqnarray*}
    where $\vec \theta, \vec x$ are the lifted parameters of the original decision function $F$ \\
    (if it was not already linear).
  \end{citeblock}
\end{frame}

\subsection{Learning in Logistic Regression}

\subsubsection{Log-Likelihood Function}

\begin{frame}
  \frametitle{Log-Likelihood Function}

  \begin{itemize}
    \item Let us assume the posteriors are given by
      \begin{eqnarray*}
        p(y=0|\vec x) &=&  1-g(\vec \theta^T\vec x) \\
        p(y=1|\vec x) &=&  g(\vec \theta^T\vec x)
      \end{eqnarray*}
      where $g(\vec \theta^T\vec x)$ is the sigmoid function parameterized in $\vec \theta$. \\[.3cm]
    \item The parameter vector $\vec{\theta}$ has to be estimated from a set $S$ of $m$ training samples:
      \begin{eqnarray*}
        S &=& \{ (\vec x_1, y_1), (\vec x_2, y_2), (\vec x_3, y_3), \dots, (\vec x_m, y_m) \}\quad .
      \end{eqnarray*}
      \pause
    \item Method of choice: Maximum Likelihood Estimation
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Log-Likelihood Function \cont}

  Before we work on the formulas of the ML-estimator, we rewrite the posteriors
  using Bernoulli probability:

  \begin{eqnarray*}
    p(y|\vec x) &=& \pause g(\vec \theta^T\vec x)^{y}(1-g(\vec \theta^T\vec x))^{1-y}\\[.3cm]
  \end{eqnarray*}

  which shows the great benefit of the chosen notation for class numbers.
\end{frame}

\begin{frame}
  \frametitle{Log-Likelihood Function \cont}

  Now we can compute the log-likelihood function \\
  (assuming that the training samples are mutually independent):

  \begin{eqnarray*}
    \mathcal{L} (\vec \theta) &=& \log \left( \prod_{i=1}^m p(y_i|\vec x_i) \right) \\ \pause
                              &=& \sum_{i=1}^m \log \left( g(\vec \theta^T\vec x_i)^{y_i}\,\big(1-g(\vec \theta^T\vec x_i)\big)^{1-y_i} \right) \\ \pause
                              &=& \sum_{i=1}^m \left(y_i \log g(\vec \theta^T\vec x_i) + (1-y_i)\log\big(1-g(\vec \theta^T\vec x_i)\big) \right)
  \end{eqnarray*}
\end{frame}

\begin{frame}
  \frametitle{Log-Likelihood Function \cont}

  Simplification of the log-likelihood function:

  \begin{eqnarray*}
    \mathcal{L} (\vec \theta)
      &=& \sum_{i=1}^m \left( y_i \log g(\vec \theta^T\vec x_i) + (1-y_i)\log\big(1-g(\vec \theta^T\vec x_i)\big) \right) \\ \pause
      &=& \sum_{i=1}^m \left( y_i \log \frac{e^{\vec \theta^T \vec x_i}}{1 + e^{\vec \theta^T \vec x_i}} + (1 - y_i) \log \frac{1}{1 + e^{\vec \theta^T \vec x_i}} \right) \\ \pause
      &=& \sum_{i=1}^m \left( y_i \vec \theta^T \vec x_i + \log \frac{1}{1 + e^{\vec \theta^T \vec x_i}} \right) \\ \pause
      &=& \sum_{i=1}^m \left( y_i \vec \theta^T \vec x_i + \log \big( 1 - g(\vec \theta^T \vec x_i) \big) \right)
  \end{eqnarray*}
\end{frame}

\begin{frame}
  \frametitle{Log-Likelihood Function \cont}

  \structure{Notes for the expert:}

  \begin{itemize}
    \item The negative of the log-likelihood function is the cross entropy of\\
      $y$ and $g(\vec \theta^T\vec x)$. \\[.5cm]
    \item The negative of the log-likelihood function is a convex function.
  \end{itemize}
\end{frame}

\input{nextTime.tex}

\subsubsection{Newton-Raphson Iteration}

\begin{frame}
  \frametitle{Maximization of the log-likelihood function}

  \begin{itemize}
    \item The log-likelihood function is concave.
    \item We use the \point\href{http://www.stat.washington.edu/quinn/classes/560/Newton.pdf}{\structure{Newton-Raphson}} algorithm to solve the unconstrained optimization problem:
      \spread

      For the $(k+1)$-st iteration step, we get:
%
      \begin{eqnarray*}
        \vec \theta^{(k+1)} &=& \vec \theta^{(k)} - \left(  \frac{\partial^2}{\partial\vec \theta \partial \vec \theta^T} \mathcal{L} \left(\vec \theta^{(k)} \right)  \right)^{-1}
       \frac{\partial}{\partial\vec \theta} \mathcal{L}\left(\vec \theta^{(k)}\right)
      \end{eqnarray*}
  \end{itemize}
  \spread

  \structure{Note:} If you write the Newton-Raphson iteration in matrix form, you will end up with a weighted least squares iteration scheme.
\end{frame}

\begin{frame}
  \frametitle{Newton-Raphson Iteration}

  \structure{Taylor's Theorem:}\\[.3cm]

  Approximation of a $k$-times differentiable function $f(x)$ \\
  around a given point $x_0$:

  {\small
  \begin{displaymath}
    f(x_0+h) = f(x_0) + f'(x_0) h + \frac{f''(x_0)}{2!} h^2 + \ldots +
           \frac{f^{(k)}(x_0)}{k!} h^k + r_k(x_0 + h) h^k,
           \quad \lim_{h \rightarrow 0} r_k(x_0 + h) = 0
  \end{displaymath}
  }
  \pspread

  \structure{Second order Taylor polynomial:}
  \begin{displaymath}
    f(x_0 + h) \approx f(x_0) + f'(x_0) h + \frac{1}{2} f''(x_0) h^2
  \end{displaymath}
\end{frame}

\begin{frame}
  \frametitle{Newton-Raphson Iteration \cont}

  \structure{Extremum:}

  \begin{eqnarray*}
    f'(x_0 + h)         & = & f'(x_0) + f''(x_0) h ~\stackrel{!}{=}~ 0 \\[.3cm]
                \hat{h} & = & - \frac{f'(x_0)}{f''(x_0)} \\[.3cm]
    x_1 = x_0 + \hat{h} & = & x_0 - \frac{f'(x_0)}{f''(x_0)}
  \end{eqnarray*}
\end{frame}

\begin{frame}
  \frametitle{Newton-Raphson Iteration \cont}

  \begin{center}
    \resizebox{.7\linewidth}{!}{
      \alt<8->{
        \input{\texfigdir/newton-raphson8.pstex_t}
      }{\alt<7>{
        \input{\texfigdir/newton-raphson7.pstex_t}
      }{\alt<6>{
        \input{\texfigdir/newton-raphson6.pstex_t}
      }{\alt<5>{
        \input{\texfigdir/newton-raphson5.pstex_t}
      }{\alt<4>{
        \input{\texfigdir/newton-raphson4.pstex_t}
      }{\alt<3>{
        \input{\texfigdir/newton-raphson3.pstex_t}
      }{\alt<2>{
        \input{\texfigdir/newton-raphson2.pstex_t}
      }{
        \input{\texfigdir/newton-raphson1.pstex_t}
      }}}}}}}
    }
  \end{center}
\end{frame}

\begin{frame}
  \frametitle{Gradient of the Log-Likelihood Function}

  \structure{The gradient:}
 %
  {\small
  \begin{eqnarray*}
    \frac{\partial}{\partial \theta_j} \mathcal{L}(\vec \theta)
      &=& \frac{\partial}{\partial \theta_j} \left( \sum_{i=1}^m \left( y_i \vec \theta^T \vec x_i + \log \big( 1 - g(\vec \theta^T \vec x_i) \big) \right) \right)\\ \pause
      &=& \sum_{i=1}^m \left( y_i x_{i,j} - \frac{1}{1-g(\vec \theta^T\vec x_i)} \frac{\partial}{\partial \theta_j}g(\vec \theta^T \vec x_i) \right)
  \end{eqnarray*}
  }
  \pause
%
  Now we use the derivative of the sigmoid function and get
%
  {\small
  \begin{eqnarray*}
     \frac{\partial}{\partial \theta_j} \mathcal{L}(\vec \theta)
       &=& \sum_{i=1}^m \left( y_i x_{i,j} - \frac{1}{1-g(\vec \theta^T\vec x_i)} g(\vec \theta^T \vec x_i) \big(1-g(\vec \theta^T \vec x_i)\big) x_{i,j} \right) \\ \pause
       &=& \sum_{i=1}^m \left( y_i - g(\vec \theta^T\vec x_i) \right) x_{i,j}
  \end{eqnarray*}
  }
%
   where $x_{i,j}$ is the $j$-th component of the $i$-th training feature vector.
\end{frame}

\begin{frame}
  \frametitle{Gradient of the Log-Likelihood Function \cont}

  Finally, we have a quite simple gradient:

  {\small
  \begin{eqnarray*}
    \frac{\partial}{\partial \theta_j} \mathcal{L}(\vec \theta)
    &=& \sum _{i=1}^m  \left( y_i - g(\vec \theta^T\vec x_i) \right) x_{i,j}
  \end{eqnarray*}
  }

  where $x_{i,j}$ is the $j$--th component of the $i$--th training feature vector. \\[.3cm]

  Or in vector notation:
  {\small
  \begin{eqnarray*}
    \frac{\partial}{\partial\vec \theta} \mathcal{L}(\vec \theta)
    &=& \sum _{i=1}^m  \left( y_i-g(\vec \theta^T\vec x_i) \right)\vec x_{i}
  \end{eqnarray*}
  }
\end{frame}

\begin{frame}
  \frametitle{Hessian of the Log-Likelihood Function}

  \begin{itemize}
    \item The Newton-Raphson algorithm requires the Hessian matrix.
    \item Remember the derivative of the sigmoid function!
  \end{itemize}

  \begin{eqnarray*}
    \frac{\partial^2}{\partial\vec \theta \partial \vec \theta^T} \mathcal{L}(\vec \theta) &=& -
    \sum _{i=1}^m  g(\vec \theta^T\vec x_i) \left(1-g(\vec \theta^T\vec x_i)\right)\vec x_i \vec x_i^T
  \end{eqnarray*}
\end{frame}

\subsection{Perceptron and Logistic Regression}

\begin{frame}
   \frametitle{Perceptron and Logistic Regression}

   \begin{center}
     \resizebox{.7\linewidth}{!}{
       \input{\texfigdir/perceptron.pstex_t}
     }
   \end{center}
\end{frame}

\subsection{Lessons Learned}

\begin{frame}
  \frametitle{Lessons Learned}

  \begin{itemize}
    \item Posteriors can be rewritten in terms of a logistic function.\\[.5cm]
    \item Given the decision boundary $F(\vec x)=0$, we can write down the posterior $p(y|\vec x)$ right away.\\[.5cm]
    \item Decision boundary for normally distributed feature vectors for each class is a quadratic function.\\[.5cm]
    \item If Gaussians share the same covariances, the decision boundary is a linear function.
  \end{itemize}
\end{frame}

\input{nextTime.tex}

\subsection{Further Readings}

\begin{frame}
  \frametitle{Further Readings}

  \begin{itemize}
    \item T. Hastie, R. Tibshirani, and J. Friedman: \\
      \structure{The Elements of Statistical Learning --}\\
      \structure{ Data Mining, Inference, and Prediction},\\
      2nd edition, Springer, New York, 2009. \\[.3cm]
    \item David W. Hosmer, Stanley Lemeshow: \\
      \structure{Applied Logistic Regression}, 2nd Edition, \\
      John Wiley \& Sons, Hoboken, 2000.
  \end{itemize}
\end{frame}

\subsection{Comprehensive Questions}

\begin{frame}
  \frametitle{Comprehensive Questions \cont}

  \begin{itemize}
    \item How can a nonlinear function be written as a scalar product? \\[.7cm] \pause
    \item What is the objective function for the ML-estimation of the logistic regression parameters? \\[.7cm] \pause
    \item What is the difference between a gradient descent and Newton-Raphson numerical optimization scheme? \\[.7cm] \pause
    \item What is the parameter update rule for the logistic regression parameters using the Newton-Raphson scheme?
  \end{itemize}
\end{frame}


\chapter{One-dimensional spaces}

We can divide one-dimensional spaces into two categories:
\emph{lines} and \emph{curves}.
Lines are straight.
Curves may bend.
Thus every line is a curve.

\ResearchQuestion{%
Drawing a line is easy,
but how do we describe a line \emph{algebraically}?
}

\ResearchQuestion{%
A line is a straight one-dimensional space.
\emph{Straight} is defined by the ambient space that contains the line.
How do we define \emph{straight}?
}

\section{Defining a line}

If two points \(a\) and \(b\) are on a line,
then their midpoint \((a + b) / 2\) is also on the line.

(These aren't obvious to the uninitiated?)

If both \(a\) and \(b\) are on a line,
then the point \(k \cdot (b - a) + a\) is also on the line, for every \(k:\Real\).

If three points \(a,b,c\) are on a line,
then the displacements \(b-a\) and \(c-b\) are parallel.

\paragraph{Infinite extension of a line segment}
A \emph{line segment} is what is drawn using a straightedge.
A \emph{line} is obtained by infinitely extending a line segment in both directions.

\paragraph{Embedding of \(\Real\)}
\emph{A line is a straight embedding of \(\Real\).}
A line is something straight-shaped and isomorphic to \(\Real\).

But there is a problem with that definition.
That definition may include a pathological sheet, which should be a two-dimensional object.
This is a mapping from \(\Real^2\) to \(\Real\):
Let there be two real numbers \(a\) and \(b\).
Define the number \(c\) as \(\ldots a_1 b_1 a_0 b_0 . a_{-1} b_{-1} a_{-2} b_{-2} \ldots\).%
\footnote{\url{https://math.stackexchange.com/questions/75107/injective-map-from-mathbbr2-to-mathbbr}}%
\footnote{\url{https://math.stackexchange.com/questions/183361/examples-of-bijective-map-from-mathbbr3-rightarrow-mathbbr}}

We can define a line by a parametric equation: \( x(k) = k g + p \).

We can define a line by an algebraic equation: \( a \cdot x + b = 0 \).

\subsection{Defining a line as a curve with constant velocity}

The \emph{velocity} of the curve \( x : \Real \to \Real^n \) is the derivative of \(x\).
The velocity of \(x\) is the rate of change of \(x\).

\emph{A line is a curve whose velocity is constant.}

\subsection{Defining a line as a geodesic}

\section{Describing lines in a two-dimensional ambient space}

Every line can be described as the set
\( \{ (x,y) ~|~ (x,y) \in \Real^2, ~ a x + b y = c \} \).
(Why?)

\subsection{Describing a line that passes two points}

Describe a line that passes \((x_1,y_1)\) and \((x_2,y_2)\).

The description is
\begin{align*}
    a x_1 + b y_1 &= c
    \\
    a x_2 + b y_2 &= c
\end{align*}
Rearrange:
\begin{align*}
    x_1 a + y_1 b &= c
    \\
    x_2 a + y_2 b &= c
\end{align*}
Solve for \(a,b,c\).
\begin{align*}
    \Matrix{x_1 & y_1 \\ x_2 & y_2} \Matrix{a \\ b} = \Matrix{c \\ c}
\end{align*}
We can solve it using GNU Octave by typing \verb@[x1,y1;x2,y2] \ [c;c]@
but we have to substitute the variables with numbers first.

\subsection{Finding the angle formed by two lines}

\subsection{Translating lines and describing parallel lines}

Translating a line produces another line that is parallel to the original line.

Two lines \(ax+by=c\) and \(a'x+b'y=c'\) are parallel iff \(\abs{a/b} = \abs{a'/b'}\)?

\subsection{Describing orthogonal lines}

This is important for tangents, normals, and osculating circles.

\section{Describing higher-dimensional lines}

Every \(n\)-dimensional line can be described as
\( \{ k g + p ~|~ k \in \Real \} \)
if \(g, p : \Real^n\).

Describe a line that passes \(x_1\) and \(x_2\).
The description is
\begin{align*}
    x_1 &= k_1 g + p
    \\
    x_2 &= k_2 g + p
\end{align*}

\begin{align*}
    x_i - p &= k_i g
\end{align*}

How do we solve the equation \(a = kb\) if \(a,b\) are vectors and \(k\) is a scalar?


\section*{Probabilities}
$\mathbb{E}_x[X] = \begin{cases}
   \int x \cdot p(x) \partial x  &|\mathbb{E}_x[f(x)] =\\
   \sum_x x \cdot p(x) &|\int f(x) \cdot p(x) \partial x
  \end{cases}$\\
%$\mathbb{E}_x[f(x)] = \int f(x) \cdot p(x) \partial x $\\
$Var[X] = \mathbb{E}[(X-\mu_X)^2] = \mathbb{E}[X^2] - \mathbb{E}[X]^2$\\
$P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}$; $p(Z|X,\theta) = \frac{p(X,Z|\theta)}{p(X|\theta)}$\\
$P(x,y) = P(x \cap y)= P(y|x) \cdot P(x) = P(x|y) \cdot P(y)$

%\subsection*{Linearity of expectation}
%$X, Y$ rand. var., $a, b \in \mathbb{R}$:\\
%$\mathbb{E}_{x,y}[aX + bY] = a\mathbb{E}_x[X] + b \mathbb{E}_y[Y]$


\begin{intro}
  \putindex{multigrid} Multigrid methods avoid the problems discussed
  in the section on two-level Schwarz methods by using not only two,
  but a whole hierarchy of mesh levels. On each level, an approximate
  solver, a so called smoother is employed, which improves the error
  somewhat, and then an approximation on a coarser level is used to
  improve further. This is done down to the coarsest level, where we
  assume that the solution process is cheap.
\end{intro}

\begin{Definition}{hierarchy-spaces}
  A \define{hierarchy of spaces} $\{V_\ell\}_0\le \ell \le L$ is a
  sequence of the form
  \begin{gather}
    V_0 \subset V_1 \subset \dots \subset V_L.
  \end{gather}
  We assume that $V_L = V$ is the high resolution space on which we
  want to solve~\eqref{eq:itintro:1}, but where the condition number
  of the matrix $\mat A$ is bad. On the other end of the spectrum, we
  assume that the solution of~\eqref{eq:itintro:1} on $V_0$ is easily
  possible.
\end{Definition}

\begin{Definition}{multigrid-method}
  A \define{multigrid method} consists of the following components:
  \begin{enumerate}
  \item A \define{smoother} $R_\ell$ acting on the level space
    $V_\ell$, usually an iterative method like Richardson, Jacobi,
    Gauss-Seidel or a Schwarz method.
  \item A \define{coarse grid solver} solving the problem on $V_0$
    exactly.
  \item Transfer operators between the levels $V_{\ell}$ and
    $V_{\ell+1}$. For standard finite element methods, this is
    typically the embedding operator. The transfer in opposite
    direction is achieved by the $L^2$-projection.
  \end{enumerate}

  On a given level $V_{\ell}$, the multigrid level consists of an
  alternating sequence of \putindex{smoothing step}s and
  \putindex{coarse grid correction}s, where the latter consist of a
  projection of the residual to the space $V_{\ell-1}$ and then
  recursive application of the same sequence. This is easiest
  described by the function
  \begin{subequations}
    \label{eq:mg:5}
  \begin{gather}
    \label{eq:mg:1}
    u_1 = MG_\ell(u^{(0)}, g),
  \end{gather}
  which takes an initial value $u^{(0)}$ and computes an approximation
  $u_1$ to the solution to $A_\ell u = g$ by the following scheme:
  first, for $\ell = 0$ let
  \begin{gather*}
    MG_0(u^{(0)}, g) = A_0^{-1} g_0.
  \end{gather*}
  On levels $\ell \neq 0$, perform the three steps
  \begin{itemize}
  \item Pre-smoothing: apply $m_{\text{pre}}$ steps of a Richardson
    iteration preconditioned with the smoother $R_\ell$:
    \begin{gather}
      \label{eq:mg:2}
      u^{(k+1)} = u^{(k)} - R_{\ell}^{-1} \left(A_\ell u^{(k)} - g_\ell\right),
      \qquad 0 \le k < m_{\text{pre}}.
    \end{gather}
    \item Coarse grid correction: let $v^{(0)} \in V_{\ell-1}$ and
      $g_{\ell-1} \in V_{\ell-1}^*$ such that
      \begin{gather}
        \label{eq:mg:3}
        g_{\ell-1} = \Pi_{\ell-1}^T \left(g_\ell - A_\ell u^{(m_{\text{pre}})}\right),
        \qquad
        v^{(0)} = 0.
      \end{gather}
      Then, compute
      \begin{gather}
        \label{eq:mg:4}
        v^{(k+1)} = MG_{\ell-1}(v^{(k)}, g_{\ell-1}),
      \qquad 0 \le k < m_{\text{coarse}}.
      \end{gather}
      Let $w^{(0)} \in V_{\ell}$ be given by $w^{(0)} =
      u^{(m_{\text{pre}})} + v^{(m_{\text{coarse}})}$.
  \item Post-smoothing: apply $m_{\text{post}}$ steps of a Richardson
    iteration preconditioned with the smoother $R_\ell$:
    \begin{gather}
      \label{eq:mg:2a}
      w^{(k+1)} = w^{(k)} - R_{\ell}^{-1} \left(A_\ell w^{(k)} - g_\ell\right),
      \qquad 0 \le k < m_{\text{post}}.
    \end{gather}
    Assign $MG(u^{(0)}, g_\ell) = w^{(m_{\text{post}})}$.
  \end{itemize}
  \end{subequations}

  This method has three parameters, the numbers of pre- and post
  smoothing steps $m_{\text{pre}}$ and $m_{\text{post}}$ as well as
  the number of coarse grid iterations $m_{\text{coarse}}$. Here, it
  is the last one which has a strong impact on the structure of the
  iteration. It defines what is called the \define{cycle type}, which
  is either \define{V-cycle} for $m_{\text{coarse}} = 1$ or
  \define{W-cycle} for $m_{\text{coarse}} = 2$. The structure of the
  cycles can be seen in Figure~\ref{fig:mg:1}.
\end{Definition}
\begin{figure}[tp]
  \centering
  \begin{minipage}[t]{.49\linewidth}
  \begin{tikzpicture}[thick,scale=.7]
    % Levels
    \draw[dotted](0,0) node[anchor=east]{$V_0$} -- (7,0);
    \draw[dotted](0,1) node[anchor=east]{$V_1$} -- (7,1);
    \draw[dotted](0,2) node[anchor=east]{$V_1$} -- (7,2);
    \draw[dotted](0,4) node[anchor=east]{$V_{L-2}$} -- (7,4);
    \draw[dotted](0,5) node[anchor=east]{$V_{L-1}$} -- (7,5);
    \draw[dotted](0,6) node[anchor=east]{$V_L$} -- (7,6);

    % Transfers
    \draw(0.5,6) -- (1.5,4);
    \draw[dashed](1.5,4) -- (2.5,2);
    \draw(2.5,2) -- (3.5,0) -- (4.5,2);
    \draw[dashed](4.5,2) -- (5.5,4);
    \draw(5.5,4) -- (6.5,6);

    % Smoothers and coarse grid correction
    \node at (0.5,6) [circle,draw=blue,fill=blue] {};
    \node at (1.0,5) [circle,draw=blue,fill=blue] {};
    \node at (1.5,4) [circle,draw=blue,fill=blue] {};
    \node at (2.5,2) [circle,draw=blue,fill=blue] {};
    \node at (3.0,1) [circle,draw=blue,fill=blue] {};
    \node at (4.0,1) [circle,draw=blue,fill=blue] {};
    \node at (4.5,2) [circle,draw=blue,fill=blue] {};
    \node at (5.5,4) [circle,draw=blue,fill=blue] {};
    \node at (6.0,5) [circle,draw=blue,fill=blue] {};
    \node at (6.5,6) [circle,draw=blue,fill=blue] {};
    \node at (3.5,0) [rectangle,draw=red,fill=red] {};
  \end{tikzpicture}
  \end{minipage}
  \begin{minipage}[t]{.49\linewidth}
  \begin{tikzpicture}[thick,yscale=.7,xscale=.6]
    % Levels
    \draw[dotted](0,0) node[anchor=east]{$V_0$} -- (8.0,0);
    \draw[dotted](0,1) node[anchor=east]{$V_1$} -- (8.0,1);
    \draw[dotted](0,2) node[anchor=east]{$V_1$} -- (8.0,2);
    \draw[dotted](0,4) node[anchor=east]{$V_{L-2}$} -- (8.0,4);
    \draw[dotted](0,5) node[anchor=east]{$V_{L-1}$} -- (8.0,5);
    \draw[dotted](0,6) node[anchor=east]{$V_L$} -- (8.0,6);

    % Transfers and smoothers
    \draw(0.2,6) node[circle,draw=blue,fill=blue] {}
    -- (0.4,5) node[circle,draw=blue,fill=blue] {}
    -- (0.6,4) node[circle,draw=blue,fill=blue] {};
    \draw[dashed](0.6,4) -- (1.0,2);
    \draw (1.0,2) node[circle,draw=blue,fill=blue] {}
    -- (1.2,1) node[circle,draw=blue,fill=blue] {}
    -- (1.4,0) node[rectangle,draw=red,fill=red] {}
    -- (1.6,1)
    -- (1.8,0) node[rectangle,draw=red,fill=red] {}
    -- (2.0,1) node[circle,draw=blue,fill=blue] {}
    -- (2.2,2)
    -- (2.4,1) node[circle,draw=blue,fill=blue] {}
    -- (2.6,0) node[rectangle,draw=red,fill=red] {}
    -- (2.8,1)
    -- (3.0,0) node[rectangle,draw=red,fill=red] {}
    -- (3.2,1) node[circle,draw=blue,fill=blue] {}
    -- (3.4,2) node[circle,draw=blue,fill=blue] {};
    \draw[dashed](3.4,2) -- (3.5,3) -- (3.7,0) -- (3.8,4);
    \draw(3.8,4) node[circle,draw=blue,fill=blue] {}
    -- (4.0,5)
    -- (4.2,4) node[circle,draw=blue,fill=blue] {};
    \draw[dashed] (4.2,4) -- (4.4,0) -- (4.5,3)
    -- (4.6,2)
    ;
    \draw (4.6,2) node[circle,draw=blue,fill=blue] {}
    -- (4.8,1) node[circle,draw=blue,fill=blue] {}
    -- (5.0,0) node[rectangle,draw=red,fill=red] {}
    -- (5.2,1)
    -- (5.4,0) node[rectangle,draw=red,fill=red] {}
    -- (5.6,1) node[circle,draw=blue,fill=blue] {}
    -- (5.8,2)
    -- (6.0,1) node[circle,draw=blue,fill=blue] {}
    -- (6.2,0) node[rectangle,draw=red,fill=red] {}
    -- (6.4,1)
    -- (6.6,0) node[rectangle,draw=red,fill=red] {}
    -- (6.8,1) node[circle,draw=blue,fill=blue] {}
    -- (7.0,2) node[circle,draw=blue,fill=blue] {};
    \draw[dashed] (7.0,2) -- (7.4,4);
    \draw (7.4,4) node[circle,draw=blue,fill=blue] {}
    -- (7.6,5) node[circle,draw=blue,fill=blue] {}
    -- (7.8,6) node[circle,draw=blue,fill=blue] {};
  \end{tikzpicture}
  \end{minipage}
  \caption{Smoothing and grid transfer of the V-cycle (left) and
    W-cycle (right). Black lines indicate grid transfer, blue dots are
  smoothing operations and red squares are coarse grid
  solvers. ``Time'' is left to right.}
  \label{fig:mg:1}
\end{figure}

\begin{remark}
  Figure~\ref{fig:mg:1} shows that the recursive structure of the
  W-cycle is much more complex than that of the V-cycle. The
  complexity analysis below will show that higher values of
  $m_{\text{coarse}}$ do not lead to efficient algorithms.
\end{remark}

\begin{Definition}{variable-v-cycle}
  If the numbers of pre- and post smoothing steps in the V-cycle are
  dependent on the level $\ell$, we speak of the \define{variable
    V-cycle}. A typical choice is $m_\ell = 2^{L-\ell} m_L$, thus
  doubling the number of smoothing steps whenever stepping down one
  level.
\end{Definition}

\begin{remark}
  The variable V-cycle with $m_\ell$ as mentioned in the previous
  definition has as many smoothing steps per iteration as the W-cycle.
\end{remark}

\begin{remark}
  If we use an additive or multiplicative Schwarz method (omitting the
  coarse grid) as our smoother $R_\ell$, it should be possible in
  principle to use the analytical tools of
  Chapter~\ref{cha:iteration:schwarz-methods}. The difficulty then
  consists in ensuring that the spectral radius of the iteration
  matrix does not grow towards one if we proceed upwards on our scale
  of spaces $V_\ell$. This remark is a todo for the author and an
  encouragement for the reader. Hints may be found
  in~\cite{GriebelOswald95,Xu92}.
\end{remark}

\begin{remark}
  It turns out that the techniques used for the analysis of the
  V-cycle and the W-cycle, respectively are quite
  different. Therefore, we separate them into two sections.
\end{remark}

\begin{Theorem}{mg-complexity}
  Let $n_\ell$ be the dimension of $V_\ell$. Assume that the effort
  needed to for the operations in equations~\eqref{eq:mg:5}b/c/e is
  linear in $n_\ell$ and assume that $ n_{\ell+1}/n_\ell \approx 2^d$,
  where $d$ is the space dimension of the grid. Assume that the effort
  for the coarse grid solver is negligible. Then, the effort for
  one step of the V-cycle is of order $n_L$. The effort for one step
  of the W-cycle is of order $n_L$ for $d \ge 2$, while it is of order
  $n_L \log(n_L)$ in one dimension.
\end{Theorem}

\begin{proof}
  Start the recursion on level $L$ with the function $MG_L(0,
  g)$. This function calls $MG_{L-1}(\ldots)$ $m_{\text{coarse}}$
  times. Thus, by recursion, $MG_{\ell}(\ldots)$ is executed
  $m_{\text{coarse}}^{L-\ell}$ times.

  By our assumptions, the amount of operations $\breve N_\ell$ in $MG_\ell(\ldots)$
  without the coarse grid correction is linear in $n_\ell$, say
  bounded by $Cn_\ell$. Then, the overall effort $N_L$ on level $L$ is
  \begin{gather}
    \label{eq:mg:6}
    N_L \le C \sum_{\ell=1}^L n_\ell m_{\text{coarse}}^{L-\ell} \le C
    \sum_{\ell=1}^L n_L 2^{d(l-L)}m_{\text{coarse}}^{L-\ell}
    = C n_L \sum_{\ell=1}^L \left(\frac{m_{\text{coarse}}}{2^d}\right)^l.
  \end{gather}
  It remains to notice that the sum converges and is bounded
  independent of $L$ if and only if $m_{\text{coarse}}/2^d < 1$. The
  statements of the theorem follow immediately, observing that $L
  \simeq \log n_L$.
\end{proof}

\begin{Lemma}{mg-error-step}
  Let $B_\ell^{-1}$ be the operator associated with the action of the
  multigrid preconditioner on level $\ell$ for
  $\ell=0,\ldots,L$. Then, the error after one step of the multigrid
  method has the form
  \begin{gather}
    \label{eq:mg:7}
    u^{(k+1)} - u = E_L \left(u^{(k)} - u \right),
  \end{gather}
  where for $\ell=0,\ldots,L$ we denote by $E_\ell$ the
  \putindex{error propagation operator}
  \begin{gather}
    \label{eq:mg:8}
    E_\ell = \left(I-R_\ell^{-1} A_\ell\right)^{m_{\text{post}}}
    \left(I-B_{\ell-1}^{-1}A_{\ell-1} P_{\ell-1}\right)^{m_{\text{coarse}}}
     \left(I-R_\ell^{-1} A_\ell\right)^{m_{\text{pre}}}
  \end{gather}
\end{Lemma}

\begin{proof}
  For the smoother, we use the standard technique for Richardson's
  method outlined in Lemma~\ref{lemma:richardson:1}. For the coarse
  grid correction, we use Lemma~\ref{lemma:schwarz:2}.
\end{proof}

\begin{remark}
  The structure of the error propagation operator~\eqref{eq:mg:8}
  already suggests the course of the multigrid analysis (as well as
  the design of smoothers). Namely, we will have to decompose $V_l$
  into $V_{l-1}$ and its $A$-orthogonal complement. Then, we use the
  induction argument that $I-B_{\ell-1}^{-1}A_{\ell-1}$ is is small on
  $V_{l-1}$, while bounded on its complement. Vice versa,
  $I-R_\ell^{-1} A_\ell$ must be bounded on all of $V_\ell$, while
  providing good reduction properties on the complement of $V_{l-1}$.
\end{remark}

\section{The V-cycle}

\begin{assumption}
  \label{assumption:mg:1}
  Let the smoother $R_\ell$ be symmetric, positive definite and let
  the following two conditions hold, the second for some positive
  constant $\alpha$ independent of $\ell$:
  \begin{subequations}
    \label{eq:mg:9}
    \begin{xalignat}{2}
      \label{eq:mg:10}
      a\bigl((I-R_\ell^{-1}A_\ell)v,v\bigr) & \ge 0
      & \forall v &\in V_\ell, \\
      \label{eq:mg:20}
      r(w,w) &\le \alpha a(w,w)
      & \forall v &\in V_\ell, \quad w = (I-P_{\ell-1}) v
    \end{xalignat}
  \end{subequations}
\end{assumption}

\begin{Theorem}{v-cycle-convergence}
  Let $a(.,.)$ be symmetric, positive definite and let
  Assumption~\ref{assumption:mg:1} hold. Then, the V-cycle operator
  with $m_{\text{pre}} =m_{\text{post}} = m$ admits the estimate
  \begin{gather}
    \label{eq:mg:11}
    0 \le a(\bigl((I-B_\ell^{-1}A_\ell)v,v\bigr) \le \delta a(v,v),
    \qquad \forall v \in V_\ell,
  \end{gather}
  where
  \begin{gather}
    \label{eq:mg:12}
    \delta = \frac{\alpha}{\alpha+2m}.
  \end{gather}
  In particular, the contraction number of the multigrid method is
  bounded by a number less than 1, independent of the level.
\end{Theorem}

\begin{proof}
  \footnote{This version of the proof is taken
    from~\cite{ArnoldFalkWinther97Hdiv}. It can also be found
    in~\cite{BraessHackbusch83,Bramble93}.}
  First, abbreviate $K_{\ell} = I-R_{\ell}^{-1} A_\ell$, the error
  propagation operator of a smoothing step. Now we will prove the
  theorem by induction over $\ell$. First, since $B_0 = A_0$, it holds
  on level zero. For higher levels, we derive from~\eqref{eq:mg:8} the
  relation
  \begin{gather}
    \label{eq:mg:13}
    E_\ell = I-B_\ell^{-1} A_\ell = K_\ell^m\Bigl(
    (I-P_{\ell-1}) + (I-B_{\ell-1}^{-1} A_{\ell-1}) P_{\ell-1}
    \Bigr) K_\ell^m.
  \end{gather}
  Non-negativity follows readily by the induction argument and
  the same properties of the smoother and the Ritz-projection. For the
  upper bound, let $w = K_\ell^m v$ to obtain by the induction
  hypothesis
  \begin{multline}
    \label{eq:mg:14}
    a(E_\ell v,v) \le a\bigl((I-P_{\ell-1}) w,w\bigr) + \delta
    a(P_{\ell-1}w,w)
    \\
    = (1-\delta) a\bigl((I-P_{\ell-1}) w,w\bigr)
   + \delta a(w,w).
  \end{multline}
  Now we use the smoothing hypothesis anf the
  Bunyakovsky-Cauchy-Schwarz inequality for the bilinear form $r(.,.)$
  associated with the smoothing operator $R_\ell$ to obtain
  \begin{align*}
    a\bigl((I-P_{\ell-1}) w,w\bigr)
    &= \scal({(I-P_{\ell-1}) w}, A_\ell w) \\
    &= \scal(R_\ell {(I-P_{\ell-1})} w, R_\ell^{-1} A_\ell w)
    \\
    &= r\bigl((I-P_{\ell-1}) w, R_\ell^{-1} A_\ell w\bigr) \\
    &\le \sqrt{r\bigl((I-P_{\ell-1}) w,(I-P_{\ell-1}) w\bigr)}
    \sqrt{r\bigl(R_\ell^{-1} A_\ell w, R_\ell^{-1} A_\ell w\bigr)}
    \\
    &\le \sqrt{\alpha a\bigl((I-P_{\ell-1}) w,(I-P_{\ell-1}) w\bigr)}
    \sqrt{a\bigl(R_\ell^{-1} A_\ell w, w\bigr)}
  \end{align*}
  Using the projection property of $I-P_{\ell-1}$, we obtain
  \begin{gather}
    \label{eq:mg:15}
    a\bigl((I-P_{\ell-1}) w,w\bigr) \le \alpha a\bigl(R_\ell^{-1} A_\ell
    w, w\bigr)
    = \alpha a\bigl((I-K_\ell) K_\ell^{2m} v,v\bigr).
  \end{gather}

  The smoothing assumption also guarantees that the spectrum of
  $K_\ell$ is contained in the interval $[0,1]$. Therefore,
  \begin{gather}
    \label{eq:mg:16}
    a\bigl((I-K_\ell) K_\ell^{2m} v,v\bigr)
    \le a\bigl((I-K_\ell) K_\ell^{i} v,v\bigr),
    \qquad i=0,\dots,2m,
  \end{gather}
  yielding by deflating the telescoping sum
  \begin{gather}
    \label{eq:mg:17}
    a\bigl((I-K_\ell) K_\ell^{2m} v,v\bigr)
    \le \frac1{2m} \sum_{i=0}^{2m-1}
    a\bigl((I-K_\ell) K_\ell^{i} v,v\bigr)
    = \frac1{2m} a\bigl((I-K_\ell^{2m}) v,v\bigr)
  \end{gather}
  Combining~\eqref{eq:mg:14}, \eqref{eq:mg:15}, and~\eqref{eq:mg:17},
  we obtain
  \begin{gather}
    \begin{split}
      a(E_\ell v,v) &\le (1-\delta)\frac\alpha{2m}
      a\bigl((I-K_\ell^{2m}) v,v\bigr)
      + \delta a(K_\ell^m v,K_\ell^m v)
      \\
      &= (1-\delta)\frac\alpha{2m} a(v,v)
      + \left(\delta - (1-\delta)\frac\alpha{2m}\right)
      a(K_\ell^m v,K_\ell^m v).
    \end{split}
  \end{gather}
  Finally, we enter $\delta=\alpha/(\alpha+2m)$ to obtain
  \begin{gather*}
    \delta - (1-\delta)\frac\alpha{2m} = 0,
  \end{gather*}
  and thus
  \begin{gather}
    \label{eq:mg:18}
    a(E_\ell v,v) \le \left(1-\frac\alpha{\alpha+2m}\right)\frac\alpha{2m}
    a(v,v) = \frac\alpha{\alpha+2m}a(v,v).
  \end{gather}
\end{proof}

\begin{Lemma}{mg-additive}
  Let $R_\ell$ be the scaled additive Schwarz method
  \begin{gather}
    \label{eq:mg:19}
    R_a^{-1} = \omega \sum_{j=1}^J P_j A^{-1},
  \end{gather}
  where the subspaces $V_j$ are defined by overlapping subdomains
  $\Omega_j$ as in~\eqref{eq:schwarz:9}. Not that we do not include
  the coarse space here. Then, for $\omega$ sufficiently small, this
  smoother fulfills Assumption~\ref{assumption:mg:1}.
\end{Lemma}

\begin{proof}
  The positive definiteness and symmetry of the smoother have been
  proven in an abstract way in Lemma~\ref{lemma:schwarz:3}. In order
  to prove estimate~\eqref{eq:mg:10}, we observe that by
  Lemma~\ref{lemma:schwarz:5} and Lemma~\ref{lemma:schwarz:6}
  \begin{gather*}
    r(v,v) = \omega \min_{v=\sum v_j} a(v_j, v_j) \gtrsim \omega a(v,v),
  \end{gather*}
  where the implicit constant depends on the number of overlaps in
  Definition~\ref{definition:schwarz:finite-covering}. Thus, we can
  choose $\omega$ independent of $\ell$ such that
  \begin{gather*}
    r(v,v) \ge a(v,v) \qquad \forall v\in V_\ell.
  \end{gather*}
  Accordingly, $R_\ell - A_\ell$ is positive definite, and since
  $R_\ell^{-1}$ is as well, so is $I-R_\ell^{-1} A_\ell$.
  It remains to prove~\eqref{eq:mg:20}, but this is exactly the second
  half of the proof of Lemma~\ref{lemma:schwarz:stable-decomposition},
  if we replace the Clément interpolant into the coarse space by the
  Ritz projection.
\end{proof}

%\section{The W-cycle and two-grid convergence}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "main"
%%% End:


%begin-include

\section{Predicates}

\begin{para}
Until now, we have been concerned with propositions --- statements that were either true or false --- and how we could manipulate them with using logical connectives.
We will now take one step further and take into consideration the particular objects which we make statements about.
Welcome to the world of predicate logic.
\end{para}

\begin{para}[Predicates]
Just as propositional logic was all about propositions, predicate logic is all about \emph{predicates}.
An $n$-ary predicate $P(x_1,\ldots,x_n)$, where $n$ is zero or a natural number, is a statement that depends on $n$ variables $x_1,\ldots,x_n$, known as \emph{arguments}, that is either true or false depending on the values that those variables take.
The set of values the variables are allowed to take is called the \emph{domain of discourse} (or \emph{domain}, for short).
Of course, nullary predicates --- which are mere propositions --- need not be followed by parentheses since they do not take any arguments.

For example, if we were working with the set of all animals as domain, we could define a unary predicate $P(x)$ as ``$x$ can fly'' in such a way that $P(\tx{cow})$ would be false but $P(\tx{bird})$ would be true.

Notice how the proposition that arises after any assignment of values to the arguments of a predicate is written substituting the different variable symbols by the values they take, just as we did in $P(\tx{cow})$ for $P(x)$.

\end{para}

\begin{para}
Predicates, just like propositions, can be combined with the connectives from propositional logic in order to form new predicates.
Nonetheless, being able to work directly with the objects we are ``saying things'' about opens up a new world of possibilities.
One of them is the ability to incorporate functions and constants into our predicates, but the most notable of all is the use of \emph{quantifiers}.

As their name implies, quantifiers allow us to create new predicates stating the quantity of elements in the domain that verify a certain predicate. The two most fundamental are the \emph{universal} quantifier and the \emph{existential} quantifier.
The universal quantifier is used to state that a certain predicate $P(x_1,\ldots,x_n)$ is true for every value taken by a variable $x_i$ (with $i\in\{1,\ldots,n\}$) in the domain. This is written as $(\forall x_i)P(x_1,\ldots,x_n)$ and is read as ``for all $x_i$, $P(x_1,\ldots,x_n)$.''
On the other hand, we use the existential quantifier to state that a certain property\footnote{``Property'' can be used to mean ``predicate.''} $P(x_1,\ldots,x_n)$ is verified by, at least, one assignment to $x_i$ of an element in the domain. We write this as $(\exists x_i)P(x_1,\ldots,x_n)$ and read it as ``there exists an $x_i$ such that $P(x_1,\ldots,x_n)$.''
The predicate $P$ over which we are quantifying is said to be the \emph{scope} of the quantifier.

A variable $x$ appearing in the scope of a quantifier $(\forall x)$ or $(\exists x)$ is said to be a \emph{bound} variable. Variables that are not bound are called \emph{free} variables. For example, if we let $P(x)$ and $Q(x)$ be predicates, in the predicate $P(x) \land (\forall x)Q(x)$, the first occurrence of $x$ is free but the second is bound.
Nevertheless, it should be taken into account that it is a pretty poor notational choice to use the same symbol to represent a free and a bound variable in the same predicate. This last predicate could have been better written as $Q(y) \land (\forall x)Q(x)$, but, from a purely formal point of view, there is nothing wrong with the original formulation.
\end{para}

\begin{para}[Order matters]
We are now ready to explore our first application of predicate logic; isn't that exciting!?
And that application goes far beyond any mathematics that you will ever study: it is \href{https://www.youtube.com/watch?v=-mLpe7KUg9U}{love}.

Yesterday, I was scrolling through my social media feed and I found this statement: ``No matter who you are, there exists someone who will love you''.
Just when I was going to click the ``don't show me posts like this in the future'' button, I had a brilliant idea: that was a perfect sentence waiting for us --- you, my dear reader, and me --- to analyse. So let us get to it!

It should be clear that, if we take the set of all human beings as our domain and let $\heartsuit(x,y)$ denote the predicate ``$y$ loves $x$'', the statement ``no matter who you are, love has already found someone for you'' can be written as $(\forall x)(\exists y) \heartsuit(x,y)$.
Now, I have a simple question: what would happen if you changed the order of the quantifiers and wrote $(\exists y)(\forall x)\heartsuit(x,y)$?
What does this sentence mean?
It means that there exists an individual who we all happen to be in love with!
The first sentence read ``for every person $x$, there exists a person $y$ who will love them'', but now it reads ``there exists a person $x$ who is loved by every person $y$''.
Do you have the slightest idea of the mess you have made by swapping two quantifiers? You have taken a cheesy sentence and transformed its meaning to postulate the existence of a love monster!

The moral of the story is simple: do not swap quantifiers.
Nevertheless, if the same quantification is being used on two different variables consecutively, such as in $(\exists x)(\exists y)\heartsuit(x,y)$, there is obviously no harm in swapping the order of quantifiers; in fact, that last sentence would be often written as $(\exists x,y)\heartsuit(x,y)$.

Hey, I know you might be angry at me for having created hype with applications and all that stuff and having only given you a cheesy sentence. What can I say? I needed your attention! I hope you will forgive me.
\end{para}

\begin{para}[Notation]
\label{notaquan}
There is a little bit of freedom in the way that quantifiers can be written and each alternative has its own advantages. The following are just some examples of the notation that one can find in the literature:
\[ \forall x, \exists y, P(x,y),\qquad (\forall x)(\exists y) : P(x,y),\qquad \forall x : \exists y : P(x,y).\]

I see myself as a liberal person when it comes to notation, but there is something I beg you to do: do not EVER write quantifiers after their scope. Every time you write $P(x),\forall x$ a cute kitten cries immersed in sadness, so, please do not do that.
This is not a matter of taste; it is a matter of readability. When you are about to read an expression, what variables are being quantified is the first thing you should know, not the last!

Furthermore, as we have just seen, the order in which quantifiers are set is very, very important; thus, if you write them at the end\ldots what order are they in? Should they be read from left to right or from right to left? I mean, it is just inelegant and clumsy. Do not do that, please.

If you really want to do things right, you should extend this idea to your writing.
I know; saying, for instance, ``$P(x)$ is true for all $x$'' sounds natural and harmless, but as the number of quantifiers increases --- and, believe me, it will increase and not just in logic, but in ordinary mathematics --- you better have those quantifiers right at the start.
As a rule of thumb, I would only put a quantifier at the end of a predicate in written text if it is a single universal quantifier, and I would never ever write a quantifier at the end in a symbolic expression.

Now that we are dealing with notation, let me draw your attention to an important issue.
In \ref{hierarchy}, we said that, in propositional logic, there was no harm in getting rid of the parentheses that surround a full proposition, so there was no need to write $(p \land q)$ and we could just write $p\land q$.
In predicate logic, however, we need to be somewhat careful with this idea.
If, for instance, I wrote $(\forall x) P(x) \land Q(x)$, I would not mean the same as if I had written $(\forall x)(P(x) \land Q(x))$.
In the former case, the scope of the quantifier is $P(x)$, whereas in the latter it is $P(x)\land Q(x)$.

In order to save ourselves some parentheses and make everything a little bit cleaner, we will use the following convention.
If a sequence of quantifiers is followed by a dot, it will mean that its scope is the remainder of the predicate unless an opening parenthesis is present before the quantifiers; if this happens, the scope will end at the point where the matching closing parenthesis is located.
Thus, in $(\forall x)(\exists y)\qsep P(x)\land Q(y)$, the scope of the first quantifier is $(\exists y)(P(x) \land Q(y)$ and that of the second is $P(x) \land Q(y)$.
On the other hand, in $( (\forall x)\qsep P(x) ) \land Q(x)$, the scope of the quantifier is $P(x)$.
\end{para}

\begin{para}[Negating predicates]
If I told you that every single human loves mathematics, what would you have to do to prove me wrong?
You would simply need to show the existence of a person who does not like mathematics.
Therefore, the negation of the sentence ``for all $x$, $P(x)$ holds'' is ``there exists an $x$ such that $P(x)$ does not hold.''
In other words, $\lnot (\forall x) P(x)$ is the same as $(\exists x)(\lnot P(x))$.

Now, let us just say that I postulate the existence of a person who has the superpower of flying.
If you wanted to show that my statement is false, you would have to prove that no person has the superpower of flying or, equivalently, that ``for every person $x$, $x$ cannot fly''.
What is the moral of the story? That $\lnot(\exists x)P(x)$ equivalent to $(\forall x)(\lnot P(x))$.

Putting together all that we have learnt: how would you write the predicate $\lnot(\forall x)(\exists y)P(x,y)$ without having a negation connective before any quantifier?
Take a sheet of paper and write down the result.

[Spoiler alert]
If you have understood this part, you should have reasoned as follows: $\lnot(\forall x)(\exists y) P(x,y)$ is the same as $(\exists x)(\lnot (\exists y) P(x,y))$, which is equivalent to $(\exists x)(\forall y) \lnot P(x,y)$. If you got this right, congrats! You are on the right track.
If you did not, do not worry: make yourself a good cup of tea, go through this material again and give it some thinking.

As you can see, when negating an expression involving the existential and universal quantifiers, the only thing we need to do is ``swap them and negate what is inside them''.
That is a pretty easy rule, but, as always happens with this kind of shortcuts, you should only apply it if you really know what is going on underneath the hood.
\end{para}

\begin{para}[Defining quantifiers]
There is something kind of significant in our analysis of the negation of predicates. What we have shown --- probably without your noticing --- is that one of our quantifiers is redundant. The predicate $(\exists x) P(x)$ is equivalent to $\lnot (\forall x)(\lnot P(x))$, so, in a way, there was no need to introduce the existential quantifier once we had the universal one. Conversely, $(\forall x) P(x)$ is equivalent to $\lnot(\exists x)(\lnot P(x))$, hence the existential quantifier could also serve us as our only quantifier.

Despite the redundancy, we introduced both of them for an obvious reason:
for us humans, it is more natural to think ``every $x$ verifies $P(x)$'' than ``there does not exits an $x$ not verifying $P(x)$''.
\end{para}

\begin{para}[Pseudo-quantifiers]
\label{pseudoquan}
We will now introduce some ``pseudo quantifiers''.
I have given them this name --- which is, by the way, not standard whatsoever --- because they are just constructions that help us quantify over things without being proper quantifiers.
Instead, they are mere logical artefacts that are limited to a certain kind of theories.

Some theories (in fact, most theories) define a predicate $E$ that is meant to represent equality. In these theories, we can define a predicate $(\exists! x)P(x)$ meaning ``there exists a unique $x$ verifying $P(x)$''. The equivalent real predicate behind $(\exists! x)P(x)$ would be
\[(\exists x)(\forall y) (P(x) \land (P(y)\limplies E(x,y))),\]
or, in English, ``there exists an $x$ verifying $P(x)$ and such that, if any $y$ verifies $P(y)$, then $y$ is equal to $x$''.

On some occasions, one may wish to quantify only over the set of elements in the domain that verify a certain predicate $P$.
This is very easy to do. If we wanted to restrict the quantification of $(\forall x)Q(x)$ or $(\exists x)Q(x)$ only to the elements $x$ verifying $P(x)$, we would just need
\[ (\forall x)(P(x)\limplies Q(x)) \quad \tx{and} \quad  (\exists x)(P(x) \land Q(x))\]
respectively.
You see, saying ``for all $x$ verifying $P(x)$, $Q(x)$ is true'' is the same as saying ``for all $x$, if $x$ verifies $P(x)$, then $Q(x)$ is true'', and analogously for the existential quantifier.

Many theories include binary predicates $P(x,y)$ that can be written as $xPy$ --- for instance, the inclusion predicate $\in$ that we studied in our elementary treatment of set theory, --- and, given a fixed $y$, one may wish to quantify over all the elements $x$ verifying $xPy$.
Of course, one could write $(\forall x)(xPy\limplies Q(x))$ or $(\exists x)(xPy\land Q(x))$, but, instead of wasting time and ink with such lengthy expressions, it is common to simply use $(\forall x P y)Q(x)$ and $(\exists x P y) Q(x)$.
Thus, if we wanted to postulate the existence of an element $X$ inside a set $Y$ verifying a property $P(x)$, we would simply have to write $(\exists X \in Y)P(X)$.
It is important to keep in mind that this is just shorthand notation, and we should always understand what is really going on.

Just to finish with this, I have an innocent question for you: let us assume that we are in the context of set theory and, given a set $X$ and a predicate $P(x)$, we formulate the sentence $(\exists! x \in X) P(x)$.
Is this expression well-formed and unambiguous? Take your time to think about it.

[Spoiler alert] Turns out that this expression is ambiguous. What do you mean: that there exists an element $x\in X$ verifying $P(x)$ that is unique among all the elements, or unique among all the elements in $X$? As far as I know, there is no widespread convention on which of these two possible interpretations is correct, so it is better not to use this construction to avoid ambiguity.
If you wanted to say that it is unique among all the elements of $X$, you could write $(\exists! x)(x\in X \land P(x))$. If, on the other hand, you meant that it is unique amongst all the elements, you could write $(\exists x\in X)(P(x)\land (\forall y)(P(y)\limplies y =x))$.
\end{para}

\begin{para}[Higher-order logic]
Predicate logic is also known as \emph{first-order logic}. Higher-order logic is an extension of first-order logic that not only allows quantification over variables, but also over predicates about variables, over predicates about predicates about variables, and so on.

For example, the statement ``for every property $P$, there exists an element $x$ such that $P(x)$ is true'' would be a statement in second-order logic.
\end{para}

\begin{para}[Mathematical induction]
And now, let us close this chapter with a fundamental tool that we will be using extensively from now on, the principle of mathematical induction.
What this principle states is the following: if a subset $A$ of the natural numbers contains $1$ and, for every $n \in A$, $n+1 \in A$, then $A$ is the set of natural numbers.
This is kind of trivial when you think about it.
We already know that $A \subseteq \mathbb{N}$ by hypothesis, so, in order to prove that $A = \mathbb{N}$, we just need to show that $\mathbb{N} \subseteq A$.
Thus, let $n\in\mathbb{N}$ and let us prove that $n\in A$.
We know, by hypothesis that $1 \in A$ and --- since, for every $n\in A$, $n+1\in A$, --- then $1 + 1 \in A$ and, therefore, $1 + 1 + 1 \in A$. If we apply this reasoning $n$ times, we are led to
\[ n = 1 + \overset{n}{\cdots} + 1 \in A.\]

And how does this relate to predicates?
Well, let us say that we want to show that a predicate $P(n)$ is true for all natural numbers $n$.
Let $A$ be the set of natural numbers $m$ such that $P(m)$ is true.
If we show that $1 \in A$ and that, given any $m\in A$, $m+1 \in A$, we will have shown that $A = \mathbb{N}$.
In other words, if we show that $P(1)$ is true and that, assuming $P(n)$ to be true, $P(n+1)$ is true, then we will know that the predicate is true for all natural numbers.

This principle can be extended to what is known as \emph{strong induction} (we will refer to it as induction too).
If, given a subset $A\subseteq \mathbb{N}$, we know that $1\in A$ and that, assuming every natural number $i$ such that $0\leq i \leq n$ to belong to $A$, we have $n+1 \in A$, then $A = \mathbb{N}$.
The reasoning that justifies this is essentially the same as that for normal induction.
Of course, strong induction can also be applied to show that a certain property holds for every natural number.
\end{para}


\section{Integration}
\subsection{The Riemann Integral}
Our goal is the following: for a ``sufficiently nice'' function $f:[a,b]\to\mathbb R$ we want to define its integral
$$\int_a^bf(x)\,\mathrm dx$$
We would like to think of this as the area under the graph of $y=f(x)$, which motivates the Riemann integral.
\begin{definition}
    We say $f:[a,b]\in\mathbb R$ is bounded if $\exists M\in\mathbb R,|f(x)|\le M$ for all $x\in [a,b]$.
\end{definition}
\begin{definition}
    A dissection of the interval $[a,b]$ is a finite subset $D\subset [a,b]$ such that $a,b\in D$.
    If $D$ is a dissection, we can write $D=\{a_0,\ldots,a_n\}$ where $a=a_0<a_1<\cdots<a_n=b$.
\end{definition}
Note that if $D,D'$ are both dissections, so is $D\cup D'$.
\begin{definition}
    If $f:[a,b]\to\mathbb R$ is bounded and a dissection $D$ consists of $a=a_0<a_1<\cdots<a_n=b$, then we define the upper and lower Riemann sums wrt $D$ by
    $$U(f,D)=\sum_{i=1}^n(a_i-a_{i-1})\sup_{a_{i-1}<x<a_i}f(x),L(f,D)=\sum_{i=1}^n(a_i-a_{i-1})\inf_{a_{i-1}<x<a_i}f(x)$$
    respectively.
    They exists as $f$ is bounded.
\end{definition}
Since $f$ is bounded, $U,L$ are bounded over all dissections $D$.
\begin{lemma}
    If $D\subset D'$ are dissections of $[a,b]$, then $U(f,D)\ge U(f,D')$ and $L(f,D)\le L(f,D')$.
\end{lemma}
\begin{proof}
    Obvious.
\end{proof}
\begin{lemma}
    If $D_1,D_2$ are dissections of $[a,b]$, then $U(f,D_1)\ge L(f,D_2)$.
\end{lemma}
\begin{proof}
    Trivial.
\end{proof}
Consider the nonempty sets
$$U=\{U(f,D):D\text{ dissects }[a,b]\},L=\{L(f,D):D\text{ dissects }[a,b]\}$$
It then follows by the preceding lemma that $U$ is bounded below and $L$ is bounded above, so
\begin{definition}
    We define
    $$U(f)=\inf U=\inf_{D\text{ dissects }[a,b]}U(f,D),L(f)=\sup L=\sup_{D\text{ dissects }[a,b]}L(f,D)$$
    as the upper and lowe Riemann sums of $f$ over $[a,b]$.
\end{definition}
\begin{lemma}
    $U(f)\ge L(f)$.
\end{lemma}
\begin{proof}
    Follows from definition.
\end{proof}
So we will want to define
\begin{definition}
    If $L(f)=U(f)$, then we say $f$ is (Riemann) integrable on $[a,b]$ and write
    $$\int_a^bf(x)\,\mathrm dx=L(f)=U(f)$$
\end{definition}
\begin{example}
    1. Consider $f(x)=c$ for a constant $c$ for all $x\in [a,b]$, then consider $D=\{a,b\}$, then $U(f,D)=c(b-a)=L(f,D)$, hence $U(f)=c(b-a)=L(f)$, therefore $f$ is Riemann integrable and
    $$\int_a^bf(x)\,\mathrm dx=c(b-a)$$
    2. Take $f:[a,b]\to\mathbb R$ by $f(x)=0$ if $x\in\mathbb Q$ and $f(x)=1$ if $x\in\mathbb R\setminus\mathbb Q$.
    Note that if $a_{i-1}<a_i$, then $[a_{i-1},a_i]$ contains both rational and irrational numbers, so it follows that $U(f,D)=b-a$ and $L(f,D)=0$ for any dissection $D$ of $[a,b]$, therefore $U(f)=b-a\neq 0=L(f)$, hence $f$ is not Riemann integrable.
\end{example}
\subsection{Properties of the Integral}
\begin{proposition}
    $f:[a,b]\to\mathbb R$ is integrable on $[a,b]$ and
    $$\int_a^bf(x)\,\mathrm dx=I$$
    iff for every $\epsilon>0$, there is a dissection $D$ of $[a,b]$ with $U(f,D)< I+\epsilon$ and $L(f,D)>I-\epsilon$
\end{proposition}
\begin{proof}
    Immediate.
\end{proof}
\begin{proposition}
    $f:[a,b]\to\mathbb R$ is integrable on $[a,b]$ iff for any $\epsilon>0$, there is a dissection $D$ of $[a,b]$ such that $U(f,D)-L(f,D)<\epsilon$
\end{proposition}
\begin{proof}
    Consider $\inf_D U(f,D)-L(f,D)$.
\end{proof}
\begin{definition}
    If $D=\{a=a_0<a_1<\ldots<a_n=b\}$ is a dissection, then the mesh of $D$ is the difference $m(D)=\max\{a_i-a_{i-1}\}$.
\end{definition}
Observe that if $\epsilon>0$, then one can find dissection with mesh less than $\epsilon$.
\begin{theorem}
    If $f:[a,b]\to\mathbb R$ is increasing, then $f$ is integrable.
\end{theorem}
\begin{proof}
    Since $f$ is increasing, $f(a_i)\ge f(x)$ whenever $x\le a_i$.
    So
    $$\sup_{a_{i-1}<x<a_i}f(x)\le f(a_i),\inf_{a_{i-1}<x<a_i}f(x)\ge f(a_{i-1})$$
    Hence
    $$U(f,D)=\sum_{i=1}^n(a_i-a_{i-1})\sup_{a_{i-1}<x<a_i}f(x)\le\sum_{i=1}^n(a_i-a_{i-1})f(a_i)$$
    and
    $$L(f,D)=\sum_{i=1}^n(a_i-a_{i-1})\inf_{a_{i-1}<x<a_i}f(x)\ge\sum_{i=1}^n(a_i-a_{i-1})f(a_{i-1})$$
    So by substracting
    $$U(f,D)-L(f,D)\le \sum_{i=1}^n(a_i-a_{i-1})(f(a_i)-f(a_{i-1}))\le m(D)(f(b)-f(a))<\epsilon$$
    If $f(b)=f(a)$ then $f$ is constant and we know that it is integrable.
    Otherwise we take $D$ with $m(D)<\epsilon/(f(b)-f(a))$, then by the preceding lemma, $f$ is integrable on $[a,b]$.
\end{proof}
\begin{proposition}
    Suppose $f,g:[a,b]\in\mathbb R$ are integrable, then\\
    1. If $\lambda\in\mathbb R$, then $f+\lambda g$ is integrable and
    $$\int_a^bf(x)+\lambda g(x)\,\mathrm dx=\int_a^bf(x)\,\mathrm dx+\lambda\int_a^bg(x)\,\mathrm dx$$
    2. For any $c\in [a,b]$, $f|_{[a,c]},f|_{[c,b]}$ are integrable and
    $$\int_a^cf(x)\,\mathrm dx+\int_c^bf(x)\,\mathrm dx=\int_a^bf(x)\,\mathrm dx$$
    3. If $f\ge g$ on $[a,b]$, then
    $$\int_a^bf(x)\,\mathrm dx\ge\int_a^bg(x)\,\mathrm dx$$
    4. $|f|$ is integrable and
    $$\left|\int_a^bf(x)\,\mathrm dx\right|\le\int_a^b|f(x)|\,\mathrm dx$$
\end{proposition}
\begin{proof}
    Trivial but need patience.
\end{proof}
\subsection{Fundamental Theorem of Calculus}
Recall that continuous function on closed interval is bounded.
\begin{theorem}\label{cont_int}
    If $f:[a,b]\to\mathbb R$ is continuous, then $f$ is integrable.
\end{theorem}
\begin{lemma}
    Suppose $U(f,D)-L(f,D)>c>0$, then there is $a_i\in D$ such that $\exists x,y\in (a_i,a_{i+1})$ with $f(x)-f(y)\ge c/(b-a)$.
\end{lemma}
\begin{proof}
    Basically just pigeonhole principle.
\end{proof}
\begin{proof}[Proof of Theorem \ref{cont_int}]
    If $f$ is not integrable, then there is some $\epsilon>0$ such that $U(f,D)-L(f,D)>\epsilon$ for all dissections $D$ of $[a,b]$.
    Let $0<\alpha<\epsilon/(b-a)$.
    By the preceding lemma, if $D$ is a dissection of $[a,b]$, then there is some $a_i\in D$ such that $\exists x,y\in (a_i,a_{i+1}),f(x)-f(y)\ge\alpha$
    Consider the dissections $D_n$ with mesh $m(D_n)<1/n$, then we can find $a_i^{(n)}\in D_n$ and $x_n,y_n\in (a_i^{(n)},a_{i+1}^{(n)}),f(x_n)-f(y_n)\ge\alpha$.
    Since $m(D_n)<1/n$, we know that $|x_n-y_n|<1/n$, so $x_n-y_n\to 0$.
    Now by Bolzano-Weierstrass, we can find a converging subsequence $x_{n_k}\to x$ of $x_n$.
    Also $x\in [a,b]$ since $[a,b]$ is closed.
    Note also that since $x_n-y_n\to 0$, we have $y_{n_k}\to x$.
    But $f$ is continuous, so $0<\alpha\le f(x_{n_k})-f(y_{n_k})\to 0$, contradiction.
\end{proof}
\begin{theorem}[Fundamental Theorem of Calculus, Version 1]
    Suppose $f:[a,b]\to\mathbb R$ is continuous, then define
    $$G(x)=\int_a^xf(t)\,\mathrm dt$$
    for $x\in [a,b]$.
    Then $G$ is differentiable and $G^\prime=f$.
\end{theorem}
Note that given $x\in [a,b]$, since $f$ is continuous in $[a,b]$, then it is continuous and hence integrable in $[a,x]$, so $G(x)$ is always well-defined.
\begin{proof}
    We need to show that, as $h\to 0$,
    $$\frac{G(x+h)-G(x)}{h}\to f(x)$$
    for fixed $x\in[a,b]$.
    Assume for the moment that $h>0$, then
    $$G(x+h)-G(x)=\int_x^{x+h}f(t)\,\mathrm dt$$
    Given $\epsilon>0$, we can find $\delta>0$ such that $|f(t)-f(x)|<\epsilon$ whenever $|t-x|<\delta$.
    So whenever $0<h<\delta$, we have $f(x)-\epsilon<f(t)<f(x)+\epsilon$ for $t\in [x,x+h]$, so
    $$f(x)h-\epsilon h=\int_x^{x+h}f(x)-\epsilon\,\mathrm dt\le\int_{x}^{x+h}f(t)\,\mathrm dt\le\int_x^{x+h}f(x)-\epsilon\,\mathrm dt=f(x)h+\epsilon h$$
    hence
    $$\left|\frac{G(x+h)-G(x)}{h}-f(x)\right|\le\epsilon$$
    for $0<h<\delta$, hence
    $$\lim_{h\to 0^+}\frac{G(x+h)-G(x)}{h}=f(x)$$
    Using exactly the same way, we also have
    $$\lim_{h\to 0^-}\frac{G(x+h)-G(x)}{h}=f(x)$$
    Combining them gives the result.
\end{proof}
\begin{definition}
    If $F,f:[a,b]\to\mathbb R$ with $F^\prime=f$, we say $F$ is an antiderivative of $f$.
\end{definition}
So the theorem says every continuous function on closed interval has an antiderivative.
\begin{theorem}[Fundamental Theorem of Calculus, Version 2]
    If $f:[a,b]\to\mathbb R$ is continuous and $F:[a,b]\to\mathbb R$ is differentiable with $F^\prime=f$, then
    $$\int_a^bf(x)\,\mathrm dx=F(b)-F(a)$$
\end{theorem}
\begin{proof}
    Let
    $$G(x)=\int_a^xf(t)\,\mathrm dt$$
    Then $G$ is differentiable and $G^\prime=f=F^\prime$, so $(G-F)^\prime=0$, hence $G-F$ is constant hence and equals $G(a)-F(a)=-F(a)$.
    Hence
    $$\int_a^bf(x)\,\mathrm dx=G(b)=F(b)+G(b)-F(b)=F(b)-F(a)$$
    Which is what we want.
\end{proof}
Note that we have used here many theorems about the real numbers, like the mean value theorem.
\begin{corollary}
    Suppose $f:[a,b]\to\mathbb R$ is continuous and $g:[c,d]\to\mathbb R$ is $C^1$, and $g(c)=a, g(d)=b$, then
    $$\int_a^bf(t)\,\mathrm dt=\int_c^df(g(s))g^\prime(s)\,\mathrm ds$$
\end{corollary}
\begin{proof}
    Chain rule and FTC.
\end{proof}
\begin{corollary}[Integration by Part]
    Suppose $f,g:[a,b]\to\mathbb R$ are both $C^1$, then
    $$\int_a^bg^\prime(t)h(t)\,\mathrm dt=g(b)h(b)-g(a)h(a)-\int_a^bg(t)h^\prime(t)\,\mathrm dt$$
\end{corollary}
\begin{proof}
    Product rule and FTC.
\end{proof}
Recall that if $f:[a,b]\to\mathbb R$ is $C^k$, the $k^{th}$ Taylor polynomial centered at $a$ is
$$P_k(x)=\sum_{i=0}^k\frac{f^{(i)}(a)}{i!}(x-a)^i$$
\begin{theorem}[Integral Form of Taylor's Theorem]
    If $f:[a,b]\to\mathbb R$ is $C^k$ and $x\in[a,b]$, then $R_k(x)=f(x)-P_k(x)$ has
    $$R_k(x)=\int_a^x\frac{(x-t)^{k-1}}{(k-1)!}f^{(k)}(t)\,\mathrm dt$$
\end{theorem}
\begin{proof}
    Induction by using integration by part.
\end{proof}
\begin{corollary}
    If $f:[a,b]\to\mathbb R$ is $C_k$, then there is a constant $M\in\mathbb R$ such that $|f(x)-P_{k-1}(x)|\le M|x-a|^k$.
\end{corollary}
\begin{proof}
    Since $f$ is $C^k$, $|f^{(k)}|$ is bounded.
    The rest follows instantly.
\end{proof}
If $x>0,s\in\mathbb R$, we defined $x^s=\exp(s\log x)$, so $\mathrm dx^s/\mathrm dx=sx^{s-1}$.
Now consider $f(x)=(1+x)^s$.
It has Taylor polynomials (centered at $0$)
$$P_k(x)=\sum_{i=0}^k\binom{s}{i}x^i,\binom{s}{i}=\frac{1}{i!}\prod_{j=0}^{i-1}(s-j)$$
We write $P(x)$ to be the formal sum when $k\to\infty$.
By the ratio test, $P$ converges whenever $|x|<1$.
\begin{theorem}[Newton's Binomial Theorem]\label{gen_bin}
    For $x\in (-1,1)$, $P(x)$ converges to $(1+x)^s$.
\end{theorem}
\begin{lemma}
    $$k\binom{s}{k}=s\binom{s-1}{k-1},\binom{s}{k-1}+\binom{s}{k}=\binom{s+1}{k}$$
\end{lemma}
\begin{proof}
    Trivial.
\end{proof}
\begin{proof}[Proof of Theorem \ref{gen_bin}]
    By termwise differentiation,
    $$(1+x)P^\prime(x)=(1+x)\sum_{i=0}^\infty i\binom{s}{i}x^{i-1}=sP(x)$$
    by the preceding lemma.
    So consider $h(x)=(1+x)^{-s}P(x)$, so $h^\prime(x)=0$ by the differential equation we obtained above, hence $h$ is constant in $(-1,1)$, hence $h\equiv h(0)=1$.
    Therefore $P(x)=(1+x)^s$ for each $x\in (-1,1)$.
\end{proof}
\subsection{Improper Integral}
Our definition of integral
$$\int_a^bf(x)\,\mathrm dx$$
requires $a,b<\infty$ and $f$ is defined and bounded in $[a,b]$.
But we'd also like to think about things like
$$\int_1^\infty\frac{\mathrm dx}{x^2+1},\int_0^1\frac{\mathrm dx}{\sqrt{x}}$$
which are not defined in our previous definition of integral.
Suppose $f:[a,b)\to\mathbb R$ is continuous and $b\in\mathbb R\cup\{\infty\}$.
Then if $x\in [a,b)$, then
$$\int_a^xf(t)\,\mathrm dt$$
is well-defined since $f$ is continuous on $[a,x]$.
\begin{definition}
    If $f:[a,b)\to\mathbb R$ is continuous and
    $$\lim_{x\to b^-}\int_a^xf(t)\,\mathrm dt$$
    exists and equals to some $c\in\mathbb R$, we define
    $$\int_a^bf(t)\,\mathrm dt=c$$
    and say that this integral converges.
    If this limit does not exist, we say the integral diverges.\\
    Similarly, if $f:(a,b]\to\mathbb R$ is continuous and
    $$\lim_{x\to a^+}\int_x^bf(t)\,\mathrm dt$$
    exists and equals to $c$, then we define
    $$\int_a^bf(t)\,\mathrm dt=c$$
    And say that this integral converges.
    Otherwise we say this integral diverges.
\end{definition}
Note that by FTC this definition (whenever the integral converges) does coincide with our original definition of integrals.
\begin{example}
    Consider the integral
    $$\int_1^\infty t^{-s}\,\mathrm dt$$
    Then for $s>1$, then integral converges and equals $1/(s-1)$, and it diverges for $s\le 1$.
    Similarly
    $$\int_0^1 t^{-s}\,\mathrm dt$$
    converges iff $s<1$.
\end{example}
More generally,
\begin{definition}
    If $f:(a,b)\to\mathbb R$ is continuous, then choose $c\in (a,b)$, then if
    $$\int_a^cf(t)\,\mathrm dt,\int_c^bf(t)\,\mathrm dt$$
    both converge, then we say the integral of $f(t)$ over $(a,b)$ converges and
    $$\int_a^bf(t)\,\mathrm dt=\int_a^cf(t)\,\mathrm dt+\int_c^bf(t)\,\mathrm dt$$
\end{definition}
Note that this definition (both the convergence and the value of the integral) does not depend on the choice of $c$.
\begin{example}
    We have
    $$\int_{-\infty}^\infty\frac{\mathrm dx}{1+x^2}=\int_{-\infty}^0\frac{\mathrm dx}{1+x^2}+\int_{0}^\infty\frac{\mathrm dx}{1+x^2}=\pi$$
    But
    $$\int_{-\infty}^\infty t\,\mathrm dt$$
    does not converge since
    $$\int_0^\infty f(t)\,\mathrm dt$$
    does not converge.
    $$\int_0^\infty t^{-s}\,\mathrm dt=\int_0^1 t^{-s}\,\mathrm dt+\int_1^\infty t^{-s}\,\mathrm ds$$
    never converges for any value of $s$.
\end{example}
\begin{proposition}[The Integral Test]
    Suppose $f:[1,\infty)\to[0,\infty)$ is continuous and decreasing, then
    $$\sum_{n=1}^\infty f(n)$$
    converges iff
    $$\int_1^\infty f(t)\,\mathrm dt$$
    converges.
\end{proposition}
\begin{example}
    $\sum_nn^{-s}$ converges iff $s>1$ due to our discussion on the integral of $t^{-s}$ over $[1,\infty)$.
\end{example}
\begin{proof}
    Since $f(x)\ge 0$,
    $$F(x)=\int_1^x f(t)\,\mathrm dt$$
    is increasing.
    Let $A=F([1,\infty))$, then if $A$ is bounded above iff the integral converges.
    Similarly since $F(n)\ge 0$, the sequence $s_n$ of partial sums is increasing, so the series converges iff $B=\{s_n\}$ is bounded above.
    Now let $D_n=\{1,2,\ldots,n\}$ be a dissection of $[1,n]$, then
    $$s_n-f(1)=L(f,D_n)\le F(n)\le U(f,D_n)=s_n-f(n)\le s_n-c$$
    where $c=\inf_x f(x)$ (exists as $f$ is bounded below by $0$) since $f$ is decreasing.
    This implies the claim.
\end{proof}
\begin{remark}
    As we see in the proposition, improper integrals behave like series in a certain sense.
    We also have a comparison test for improper integrals.
    If $f,g:[a,b)\to\mathbb R$ satisfies $|g|\le f$ and
    $$\int_a^bf(t)\,\mathrm dt$$
    converges, then
    $$\int_a^bg(t)\,\mathrm dt$$
    converges.
\end{remark}
Using improper integrals, we can define certain interesting functions.
\begin{definition}
    The gamma function $\Gamma:(0,\infty)\to\mathbb R$ is defined by
    $$\Gamma(s)=\int_0^\infty x^{s-1}e^{-x}\,\mathrm dx$$
\end{definition}
Note that the integral always converges by whatever triviality, so this function is always well-defined.
Note also that by integration by part we know that for $s>1$, we have $\Gamma(s+1)=s\Gamma(s)$, hence $\Gamma(n+1)=n!$ for $n\in\mathbb Z_{\ge 1}$.\\
One of the motivation of defining this gamma function is because of the following (here many informal arguments are used):
Now consider an $n$-dimensional sphere $S^{n-1}=\{v\in\mathbb R^n:|v|=1\}$.
Let $V(S^{n-1})$ be the volume (or area, etc.) of $S^{n-1}$, so for example $V(S_1)=2\pi$.
Observe that
$$\int_{\mathbb R^n}f(|v|)\,\mathrm dv=\int_0^\infty V(S^{n-1})r^{n-1}f(r)\,\mathrm dr$$
Now consider $f(r)=e^{-r^2}$ and let $c_n$ be the integral as above, then
$$c_n=\prod_{i=1}^n\int_\mathbb Re^{-x^2}\,\mathrm dx=(c_1)^n$$
But on the other hand $c_n$ equals to
$$\int_0^\infty V(S^{n-1})r^{n-1}e^{-r^2}\,\mathrm dr=\frac{V(S^{n-1})}{2}\Gamma\left( \frac{n}{2} \right)$$
So
$$V(S^{n-1})=\frac{2c_1^n}{\Gamma(n/2)}$$
We know that $c_1=\sqrt{\pi}$ since we know $V(S_1)=2\pi$, and this gives the formula.


\documentclass[12pt,a4paper]{article}

\usepackage{polyglossia}
\setdefaultlanguage{english}

\usepackage{fontspec}
\setmainfont{Gentium Basic}

%\linespread{1.3}
\setlength{\parindent}{0em}
\setlength{\parskip}{5pt}

\usepackage{amssymb}
\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{mathabx}
\usepackage{eulervm}

% theorem, definition, remark
\theoremstyle{plain}
\newtheorem{mytheo} {Theorem}

\begin{document}

\title{Cycle detection problem}
\author{}
\date{}
\maketitle

\section{The problem}

Let function $ f $ maps finite set $ S $ to itself:
\begin{equation}
f: S \to S
\end{equation}

Choose initial value $ a \in S $ and build a sequence by applying $ f $ iteratively:
\begin{gather}
x_0 = a \\
x_{i+1} = f(x_i), \quad i = 0, 1, \ldots
\end{gather}

Since $ S $ is finite the sequence gets back to the older value:
\begin{equation}
x_{\nu} = x_{\mu} \quad (\nu > \mu)
\end{equation}
and then cyclically repeats values $ x_{\mu}, \ldots, x_{\nu-1} $.

Mathematical details are presented below. The problem in question is to find loop parameters: cycle start index $ \mu $ and its length $ \lambda $.

\section{Mathematical analysis}

Let $ \nu $ be \emph{the largest} index such that values

\begin{equation}
x_0, x_1, \ldots, x_{\nu-1}
\end{equation}
are all different. This means the value $ x_{\nu} $ already appeared in the sequence before, at some index $ \mu < \nu $:

\begin{equation}
x_{\nu} = x_{\mu}
\end{equation}

Let
\begin{equation}
\lambda = \nu - \mu
\end{equation}

By applying function $ f $ to both sides of equation
\begin{gather}
x_{\mu + \lambda} = x_{\mu}
\end{gather}

we have
\begin{gather}
x_{\mu + \lambda + 1} = x_{\mu + 1}
\end{gather}

Apply function $ f $ again:
\begin{gather}
x_{\mu + \lambda + 2} = x_{\mu + 2}
\end{gather}

By induction we conclude
\begin{equation}\label{eq:period}
x_{i + \lambda} = x_{i}
\end{equation}
for any index $ i \geq \mu $.

For future reference let us prove the following statement:

\begin{mytheo}
Given two indices $ i < j $ we have: $ x_i = x_j $ if and only if

\begin{equation}
i \geq \mu
\end{equation}
and
\begin{equation}
\lambda \, | \, j - i
\end{equation}

\end{mytheo}

\section{Floyd's hare and tortoise algorithm}

\section{Brent's algorithm}

\end{document}