\chapter{Problem description}
\label{chapter:TSPdescription}
\begin{figure}[h]
	\centering
	\includegraphics[width=.5\columnwidth]{img/tsp225_plot}
	\caption{The \textit{tsp225.tsp} instance from \textit{tsplib} resolved with \texttt{subtour} method described in cap \ref{sec:subtour} plotted with Gnuplot.}
	\label{fig:tsp225}
\end{figure}

The Traveling Salesman Problem is a well-known NP-hard problem. In short, given a set of cities it's required to find the shortest tour that visit exactly one time each city and return to the first one.
In this report two version of the TSP will be analyzed in details:
\begin{itemize}
	\item symmetric TSP (sTSP): with bidirectional edges;
 	\item asymmetric TSP (aTSP): with oriented arcs.
\end{itemize}

\section{Symmetric TSP}

\begin{figure}[h]
	\centering
	\includegraphics[width=.3\columnwidth]{img/symTSP_example.png}
	\caption{Example of tour. Only the selected edges are visible.}
	\label{fig:symTSP}
\end{figure}

Given a complete graph $ G = (V, E) $, a distance function $ d $ where:
\begin{itemize}
	\item $ V := \{1, 2, .., n\}$ is a set of nodes;
	\item $ E $ is the set of edges between each node pairs $ i,j \in V, i \ne j $ represented with $ (i,j) $ or for a generic edge $ e $, note that for sTSP $ (i,j) = (j,i) $ therefore the number of edges $ |E| = \frac{n(n-1)}{2} $;
	\item $ d: (i,j) = e \to d(i,j) = c_e $ where $ c_e $ is the cost associated to $ e \in E $;
\end{itemize}
and defining the decision variable:
\[
x_e := \begin{cases}
	1 & \text{if $ e $ is in the tour,} \\
	0 & \text{otherwise}
\end{cases}
\]
the sTSP can be represented with linear programming system in \ref{eq:sTSP_LP}.
\begin{equation}
\begin{cases}
		min \sum_{ e\in E } c_ex_e & \text{Cost function} \\
		\sum_{e\in \delta (v) } x_e = 2, \forall v \in V  & \text{Degree constraint} \\
		\sum_{e\in E(S) } x_e \le |S|-1, \forall S \subset V, |S| \ge 3  & \text{Subtour elimination} \\
		x_e \in \{0,1\}, \forall e \in E & \text{Domain constraint}
\end{cases}
\label{eq:sTSP_LP}
\end{equation}
As usual it is required to find the value of $ x_e, \forall e \in E $ that minimize the \textit{cost function} and verify the constraints.
For the \textit{degree constrain}, the degree of each $ v \in V $ ($ |\delta(v)| $) must be equal to 2. A system with only the degree constrain (and the domain constraint) would probably have a set of subtours with size three or more, however by adding the \textit{subtour elimination} constraints  it's impose to have only one tour.

The number of possible subset $ S \subset V $ is exponential ($ 2^{n} - \binom{n}{3} -\binom{n}{2} -\binom{n}{1} - 1 = O(2^n)$), therefore even a graph with 100 nodes has huge matrix dimension. Note that $ E(S) := \{ e = (i,j): i,j \in S \subset V \}$ is the set of edges with both vertices in $ S $.\\
Considering the example in fig. \ref{fig:symTSP}, the value of the solution is represented in the matrix in tab. \ref{tab:symTSP_solution} where $ x_{ij} = x_{ji} $ for each $ i,j \in V, i \ne j $ but only the cell with $ i < j $ have been completed to enhance the number of variables that are used in practice.

\begin{table}[h!]
	\begin{center}
		\caption{Decision variable matrix. It show the value $ x_{ij} $ of the solution represented in figure \ref{fig:symTSP}. Due to the symmetry of the matrix, only the cell with $ i < j $ have a value, the others can be checked from the first.}
		\label{tab:symTSP_solution}
		\begin{tabular}{cc|c|c|c|c|c|}
			 \multicolumn{2}{c}{} & \multicolumn{5}{c}{j} \\ % <-- Combining two cells with alignment c| and content 12.
			& \multicolumn{1}{c}{} & \multicolumn{1}{c}{1} & \multicolumn{1}{c}{2} & \multicolumn{1}{c}{3} & \multicolumn{1}{c}{4} & \multicolumn{1}{c}{5} \\ \cline{3-7}
			\multirow{5}{*}{i} 	& 1 & \cellcolor{Black} & 0 & 1 & 0 & 1 \\ \cline{3-7}
								& 2 &  & \cellcolor{Black} & 1 & 1 & 0 \\ \cline{3-7}
								& 3 &  &  & \cellcolor{Black} & 0 & 0 \\ \cline{3-7}
								& 4 &  &  &  & \cellcolor{Black} & 1 \\ \cline{3-7}
								& 5 &  &  &  &  & \cellcolor{Black} \\ \cline{3-7}
		\end{tabular}
	\end{center}
\end{table}

\section{Asymmetric TSP}
Given a complete graph $ G = (V,A) $, a distance funciton  $ d $ where:
\begin{itemize}
	\item $ V:= \{1, 2, .., n\} $ is the set of nodes;
	\item $ A := $ the set arcs between each nodes $ i,j \in V, i \ne j$ represented as $ (i,j) $, note that in the aTSP $ (i,j) \ne (j,i) $ and the number of arcs is $ |A| = n (n-1) $ which is double of the sTSP;
	\item $ d: (i,j) \rightarrow d(i,j) = c_{ij} $ where $ c_{ij} $ is the cost associated to the arc $ (i,j) $.
\end{itemize}
As in the sTSP, it is defined the decision variable $ x_{ij} $ which tell if the arc $ (i,j) $ is in the tour:
\[
x_{ij} := \begin{cases}
1 & \text{if $ (i,j) $ is in the tour,} \\
0 & \text{otherwise}
\end{cases}
\]

The aTSP can be defined in different ways, in this report two version will be considered in detail: Miller Tucker Zemlin \cite{miller1960integer} and Flow1 by Gavish and Graves \cite{gavish1978travelling}.

\subsection{Miller Tucker Zemlin}

\begin{figure}[h]
	\centering
	\includegraphics[width=.45\columnwidth]{img/asymTSP_MTZ_example.png}
	\caption{Example of tour with MTZ model. Only the selected arcs are visible and for each node $ k $, the decision variable $ u_k $ represent the posizion of the node in the tour.}
	\label{fig:asymTSP_MTZ}
\end{figure}

Miller Tucker Zemlin (MTZ) model impose the "subtour elimination" constraints with the help of another decision variable $ u_k $. For each node $ k = 2,..,n $, $ u_k $ is the position of the node $ k $ in the tour that start from the node $ 1 $. As an example in fig \ref{fig:asymTSP_MTZ} it is shown that $ u_1 = 0 $ and than $ u_k $ increase until $ n-1 $ along the tour.

The proposed model is shown in the system of equations \ref{eq:aTSP_MTZ}. The cost function and the degree constraints are adapted to the asymmetric problem, moreover MTZ constraint is derived from $ u_j \ge u_i+1-M(1-x_{ij}), \forall i,j \in V \backslash \{1\}, M \gg 0 $. Considering an arc $ (i,j) \in A \backslash \{1\} $, if it is in the tour then $ u_j \ge u_i + 1 $ because $ M(1-x_{ij}) = 0 $, else $ u_j \ge u_i + 1 - M < 0 \implies u_j \ge 0 $. For example for the arc $ (4,2) $ in fig \ref{fig:asymTSP_MTZ}, $ 3 \ge 2 + 1 - M(1-1) \iff 3 \ge 3 $. \\
Note that, if the value of M is less than $|V| - 1$  then the constraint is incorrect, otherwise a value of M greater than $|V| - 1 $ makes the constraint redundant, therefore the best value of M is $|V| - 1$, as implemented. \\
\begin{equation}
\begin{cases}
min \sum_{i\in V}\sum_{j\in V} c_{ij}x_{ij} & \text{Cost function} \\
\sum_{i\in V} x_{ih} = 1, \forall h \in V  & \text{$ x $ Outer arcs degree} \\
\sum_{i\in V} x_{hi} = 1, \forall h \in V  & \text{$ x $ Inner arcs degree} \\
u_i -u_j +M x_{ij} \le M - 1, \forall i,j \in V \backslash \{1\} & \text{MTZ constraints ($ O(n^2) $)} \\
0 \le x_{ij} \le 1, Integer, \forall i,j \in N  & \text{$ x $ Domain} \\
0 \le x_{ii} \le 0, Integer, \forall i \in N  & \text{} \\
0 \le u_{i} \le n-2, Integer, \forall i \in N \backslash \{1\} & \text{$ u $ Domain}
\end{cases}
\label{eq:aTSP_MTZ}
\end{equation}

Note that the number of MTZ constraints is $ O(n^2) $, one for each arc, therefore much less respect sTSP model.\\
The number of decision variables instead is more than the double $ n(n-1) + n = n^2 $. Computational consideration will be done in the next chapter.

\begin{table}[h!]
	\begin{center}$  $
		\caption{Decision variable matrix. It show the value $ x_{ij} $ and $ u_k $ of the solution represented in figure \ref{fig:asymTSP_MTZ}. }
		\label{tab:asymTSP_MTZ_solution}
		\begin{tabular}{cc|c|c|c|c|c|}
			 $ x_{ij} $ & \multicolumn{1}{c}{} & \multicolumn{5}{c}{j} \\ % <-- Combining two cells with alignment c| and content 12.
			& \multicolumn{1}{c}{} & \multicolumn{1}{c}{1} & \multicolumn{1}{c}{2} & \multicolumn{1}{c}{3} & \multicolumn{1}{c}{4} & \multicolumn{1}{c}{5} \\ \cline{3-7}
			\multirow{5}{*}{i} 	& 1 & \cellcolor{Black} & 0 & 0 & 0 & 1 \\ \cline{3-7}
			& 2 & 0 & \cellcolor{Black} & 1 & 0 & 0 \\ \cline{3-7}
			& 3 & 1 & 0 & \cellcolor{Black} & 0 & 0 \\ \cline{3-7}
			& 4 & 0 & 1 & 0 & \cellcolor{Black} & 0 \\ \cline{3-7}
			& 5 & 0 & 0 & 0 & 1 & \cellcolor{Black} \\ \cline{3-7}
			\multicolumn{7}{c}{} \\

			\multicolumn{2}{c}{} & \multicolumn{5}{c}{$ k $} \\
			& \multicolumn{1}{c}{} & \multicolumn{1}{c}{1} & \multicolumn{1}{c}{2} & \multicolumn{1}{c}{3} & \multicolumn{1}{c}{4} & \multicolumn{1}{c}{5} \\ \cline{3-7}
 			$ u_k $ &  & 0 & 3 & 4 & 2 & 1 \\ \cline{3-7}
		\end{tabular}
	\end{center}
\end{table}

\subsection{Flow1}

\begin{figure}[h]
	\centering
	\includegraphics[width=.4\columnwidth]{img/asymTSP_F1_example.png}
	\caption{Example of tour with Flow1 model. Only the selected arcs are visible. $ y_{ij} $ is the decision variable that represent the decreasing flow of each arcs of the tour. Note that $ y_{ij} = 0 $ for each unselected arcs and in the figure they are not shown.}
	\label{fig:asymTSP_F1}
\end{figure}

The Flow1 model impose the "subtour elimination" constraints with a decision variable $ y_{ij} $ which is associated to each arc $ (i,j) \in V $ and represent a flow that decrease from $ n-1 $ to $ 0 $ for each node of the tour and is $ 0 $ if it is not in the tour. As shown in fig \ref{fig:asymTSP_F1} starting from the node $ 1 $ the flow $ y_{15} = 4 $ than decrease each step by $ 1 $ unit until the last arc of the tour where $ y_{31} = 0 $. Note that the last arc $ y_{31} $ has the same value of the unselected one, this is because the last arc of the tour is implicit. \\
The proposed LP model is in the system of equations \ref{eq:aTSP_F1}. The outer flow of $ 1 $ is the first constraint, than to make the flow decrease it is imposed the "flow equilibrium" where each inner flow must be equal to the outer flow $ +1 $ and, in the end, the \textit{coupling constraints} imposed that only the arcs in the tour ($ x_{ij} > 0 $) can have $ y_{ij} > 0 $ otherwise must be equal to $ 0 $. The last set of constraint force the flow to move along the tour and decrease each step.

\begin{equation}
\begin{cases}
min \sum_{i\in V}\sum_{j\in V} c_{ij}x_{ij}  & \text{Cost function} \\
\sum_{i\in V} x_{ih} = 1, \forall h \in V  & \text{$ x $ Outer arcs} \\
\sum_{i\in V} x_{hi} = 1, \forall h \in V  & \text{$ x $ Inner arcs} \\
\sum_{j\in V} y_{1j} = n-1  & \text{$ 1 $ Outer flow} \\
\sum_{j\in V} y_{hj} = \sum_{i \in V}y_{ih} - 1, \forall h \in V \backslash \{1\}  & \text{Flow equilibrium} \\
y_{ij} - x_{ij}(n-1) \le 0, \forall i \in V, \forall j \in V \backslash \{1\}  & \text{Coupling constraints} \\
0 \le x_{ij} \le 1, Integer, \forall i,j \in N  & \text{$ x $ Domain} \\
0 \le x_{ii} \le 0, Integer, \forall i \in N & \text{} \\
0 \le y_{i1} \le 0, \forall i \in V  & \text{$ y $ Domain} \\
0 \le y_{ii} \le 0, \forall i \in V  & \text{} \\
0 \le y_{ij} \le n-1, Integer, \forall i,j \in V  & \text{}
\end{cases}
\label{eq:aTSP_F1}
\end{equation}

The number of F1 constraint is equal to $ O(1) + O(n-1) + O(n^2) = O(n^2) $. The tables of variable are shown in tab \ref{tab:asymTSP_F1_solution} with the associated solution example in fig \ref{fig:asymTSP_F1}. It's easy to see that the number of variable is $ 2n(n-1) = O(n^2) $\\

\begin{table}[h!]
	\begin{center}
		\caption{Decision variable matrix. It show the value $ x_{ij}, y_{ij} $ of the solution represented in figure \ref{fig:asymTSP_F1}.}
		\label{tab:asymTSP_F1_solution}
		\begin{tabular}{cc|c|c|c|c|c|}

$ x_{ij} $ & \multicolumn{1}{c}{} & \multicolumn{5}{c}{j} \\ % <-- Combining two cells with alignment c| and content 12.
& \multicolumn{1}{c}{} & \multicolumn{1}{c}{1} & \multicolumn{1}{c}{2} & \multicolumn{1}{c}{3} & \multicolumn{1}{c}{4} & \multicolumn{1}{c}{5} \\ \cline{3-7}
\multirow{5}{*}{i} 	& 1 & \cellcolor{Black} & 0 & 0 & 0 & 1 \\ \cline{3-7}
& 2 & 0 & \cellcolor{Black} & 1 & 0 & 0 \\ \cline{3-7}
& 3 & 1 & 0 & \cellcolor{Black} & 0 & 0 \\ \cline{3-7}
& 4 & 0 & 1 & 0 & \cellcolor{Black} & 0 \\ \cline{3-7}
& 5 & 0 & 0 & 0 & 1 & \cellcolor{Black} \\ \cline{3-7}
\multicolumn{7}{c}{} \\

$ y_{ij} $ & \multicolumn{1}{c}{} & \multicolumn{5}{c}{j} \\ % <-- Combining two cells with alignment c| and content 12.
& \multicolumn{1}{c}{} & \multicolumn{1}{c}{1} & \multicolumn{1}{c}{2} & \multicolumn{1}{c}{3} & \multicolumn{1}{c}{4} & \multicolumn{1}{c}{5} \\ \cline{3-7}
\multirow{5}{*}{i} 	& 1 & \cellcolor{Black} & 0 & 0 & 0 & 4 \\ \cline{3-7}
					& 2 & 0 & \cellcolor{Black} & 0 & 0 & 0 \\ \cline{3-7}
					& 3 & 1 & 0 & \cellcolor{Black} & 0 & 0 \\ \cline{3-7}
					& 4 & 0 & 2 & 0 & \cellcolor{Black} & 0 \\ \cline{3-7}
					& 5 & 0 & 0 & 0 & 3 & \cellcolor{Black} \\ \cline{3-7}

		\end{tabular}
	\end{center}
\end{table}


\documentclass{article}
\usepackage{amsmath}
\begin{document}

\section{Integrals}

\[
    \int_{a}^{b} x^{x^2} \,dx
\]
and multiple integrals:
\begin{gather*}
    \iint_V \mu(u,v) \,du\,dv
\\
    \iiint_V \mu(u,v,w) \,du\,dv\,dw
\\
    \iiiint_V \mu(t,u,v,w) \,dt\,du\,dv\,dw
\\
    \idotsint_V \mu(u_1,\dots,u_k) \,du_1 \dots du_k
\\
    \oint_V f(s) \,ds
\end{gather*}

\section{Sums and products}

\[
    \sum_{n=1}^{\infty} 2^{-n} = 1
\]

\[
    \prod_{i=a}^{b} f(i)
\]

\section{Limits}

\[
    \lim_{x\to\infty} f(x)
\]

\end{document}

LaTeX/06 Mathematical Equations Tutorial/07 Mathematical Equations.tex


\documentclass{ximera}
\input{../preamble}
\title{Exercises: Series}
%%%%%\author{Philip T. Gressman}

\begin{document}
\begin{abstract}
Exercises relating to fundamental properties of series.
\end{abstract}
\maketitle

\begin{exercise}
Compute the sum of the series:
\begin{hint} Split the series into two separate geometric series and evaluate each separately. \end{hint}
\[ \sum_{n=0}^\infty \frac{3^n - 2^n}{4^n}  = \answer{2}. \]
\end{exercise}

\begin{exercise}
Compute the sum of the series:
\begin{hint} One way to proceed is to reindex the series so that it starts at $n = 0$. \end{hint}
\[ \sum_{n=3}^\infty \frac{(-1)^{n+1} 2^n}{3^n} = \answer{ \frac{8}{45} } \]
\end{exercise}

\begin{exercise}
Compute the sum of the series:
\[ \sum_{n=1}^\infty \frac{3^{2n+1}}{2^{3n} \cdot 9^n} = \answer{\frac{3}{7}} \]
\begin{hint} Simplify as much as possible and write in standard form for a geometric series.\end{hint}
\end{exercise}

\begin{exercise}
\begin{itemize}
\item Give an example of a geometric series whose first term is $3$ and which sums to $6$. If no such series exists, enter N/A in both spaces below:
\[ 6 = \sum_{n=0}^\infty \answer{3} \left( \answer{\frac{1}{2}} \right)^n \]
\item Give an example of a geometric series whose first term is $3$ and which sums to $2$. If no such series exists, enter N/A in both spaces below:
\[ 2 = \sum_{n=0}^\infty \answer{3} \left( \answer{- \frac{1}{2}} \right)^n \]
\item Give an example of a geometric series whose first term is $3$ and which sums to $1$. If no such series exists, enter N/A in both spaces below:
\[ 1 = \sum_{n=0}^\infty \answer{N/A} \left( \answer{N/A} \right)^n \]
\end{itemize}
\end{exercise}

\begin{exercise}
Find the correct value of the constant in the blank below which makes the sum of the series equal to zero:
\[ 0 = \sum_{n=1}^\infty \frac{(-1)^{n-1}\answer{8} - 2^n}{3^n} \]
\begin{hint} One reasonable strategy is to separate the series into two geometric series and reindex to apply the standard formula. \end{hint}
\end{exercise}

\begin{exercise}
Compute the sum of the infinite series below:
\[ \sum_{n=1}^\infty \frac{1}{n(n+1)(n+2)} = \answer{\frac{1}{4}}. \]
\begin{hint}
There is a closely-related telescoping series:
\[ \frac{1}{n(n+1)} - \frac{1}{(n+1)(n+2)} = \frac{2}{n(n+1)(n+2)} \]
\end{hint}
\end{exercise}

\begin{exercise}
Compute the sum of the infinite series below.
\[ \sum_{n=1}^\infty \frac{2}{n(n+2)} = \answer{\frac{3}{2}} \]
\begin{hint}
Expand the terms using partial fractions and compute several partial sums by hand. What you get is something like a telescoping series, but cancellations occur in a slightly different way than usual.
\begin{hint}
\[ \frac{2}{n(n+2)} = \frac{1}{n} - \frac{1}{n+2}. \]
\begin{hint}
The formula for a general partial sum is
\[ \frac{2}{1 \cdot 3} + \cdots + \frac{2}{N \cdot (N+2)} = 1 + \frac{1}{2} - \frac{1}{N+1} - \frac{1}{N+2} \]
\end{hint}
\end{hint}
\end{hint}
\end{exercise}

\begin{exercise}
\begin{itemize}
\item Use your answer to the previous exercise to compute the sum of the series:
\[ \sum_{n=3}^\infty \frac{1}{n(n-2)} = \answer{\frac{3}{4}} \]
\item Use the answer you just found to compute the sum of the series:
\[ \sum_{n=4}^\infty \frac{1}{n(n-2)} = \answer{\frac{5}{12}} \]
\begin{hint}
To drop the $n=3$ term, you can just calculate it and then subtract it from the previous series.
\end{hint}
\end{itemize}
\end{exercise}

\section*{Sample Quiz Questions}

\begin{question}%%%%%[2019Telescope1]

Compute the exact value of the infinite series
\[ \sum_{n=1}^\infty \ln \left( \frac{1 + n^{-1}}{1 + (n+1)^{-1}} \right). \]
\begin{multiplechoice}
\choice[correct]{\(\ln 2\)}
\choice{\(\ln 3\)}
\choice{\(\ln 4\)}
\choice{\(\ln 5\)}
\choice{\(\ln 6\)}
\choice{\(\ln 7\)}
\end{multiplechoice}
\begin{feedback}
The series is not a geometric series or Taylor series, we compute the first few partial sums:
\[
\begin{aligned}
S_1 & = \ln \left( \frac{1 + 1}{1 + 2^{-1}} \right)  = \ln \left( \frac{2}{\frac{3}{2}} \right) \\
S_2 & = \ln \left( \frac{1 + 1}{1 + 2^{-1}} \right)  + \ln \left( \frac{1 + 2^{-1}}{1 + 3^{-1}} \right)  = \ln \left( \frac{1 + 1}{1 + 3^{-1}} \right) = \ln \left( \frac{2}{\frac{4}{3}} \right) \\
S_3 & = \ln \left( \frac{2}{1 + 3^{-1}} \right)  + \ln \left( \frac{1 + 3^{-1}}{1 + 4^{-1}} \right)  = \ln \left( \frac{2}{1 + 4^{-1}} \right) = \ln \left( \frac{2}{\frac{5}{4}} \right) \\
& \ \vdots  \\
S_n & = \ln \left( \frac{2}{1 + (n+1)^{-1}} \right).
 \end{aligned}
\]
In particular, writing the sum of logarithms as a logarithm of a product leads to substantial cancellation.  By letting \(n \rightarrow \infty\), we get \(S_n \rightarrow \ln 2\).
\end{feedback}

\end{question}

\end{document}


\subsection{Differentiability}\label{subsec:differentiability}

Let \( X \) and \( Y \) be Hausdorff \hyperref[def:topological_vector_space]{topological vector spaces} and let \( U \subseteq X \) be an open set.

\begin{definition}\label{def:differentiability}
  Out goal is to study the following \hyperref[def:partial_function]{partial} operator:
  \begin{balign}\label{def:differentiability/partial_operator}
     & \partial: \cat{Set}(U, Y) \times U \times X \to Y                             \\
     & \partial(f, x, h) \coloneqq \lim_{t \downarrow 0} \frac {f(x + th) - f(x)} t.
  \end{balign}

  We implicitly assume that \( t \neq 0 \) because otherwise the definition would not make sense.

  We only use the operator \( \partial \) inside this definition. See \fullref{rem:derivative_notation} for a discussion of derivative notation.

  The quotient under the limit sign is called a \term{difference quotient}.

  For each function \( f: U \to Y \), each point \( x_0 \in X \) and each \enquote{direction} vector \( x_0 \in X \), we want to obtain a value in \( Y \), which we will call the \term{directional derivative} of \( f \) at \( x_0 \) in the direction \( h \). Note that \( h \) is allowed to range over \( X \).

  The existence of a directional derivative is already a harsh condition, however we impose even harsher restrictions

  \begin{thmenum}
    \thmitem{def:differentiability/first_variation}\mcite[sec. 0.2.1]{ИоффеТихомиров1974}If, for fixed \( f \) and \( x_0 \), the directional derivative \( \partial(f, x_0, h) \) exists for all directions \( h \), we define the \term{first variation} of \( f \) at \( x_0 \) as
    \begin{balign*}
       & \delta f(x_0): X \to Y                            \\
       & [\delta f(x_0)](h) \coloneqq \partial(f, x_0, h).
    \end{balign*}

    Within its domain of definition of \( \delta \), which is stricter than that of \( \partial \), the operator \( \delta \) is a \hyperref[def:function/currying]{currying} of \( \partial \). We are interested in how the operator \( \delta f(x) \) varies as \( x \) varies.

    Note that \( \delta f(x_0) \) is an operator from \( X \) to \( Y \) even if \( f \) is a function from \( U \subsetneq X \) to \( Y \).

    Note that, in general, the first variation operator \( \delta f(x_0) \) is not linear - for example, by \fullref{thm:convex_one_sided_derivatives_sublinear}, the first variation of a general convex functions is, at most, sublinear. See \fullref{subsec:nonsmooth_derivatives} for how \enquote{nonlinear derivatives} are handled.

    \thmitem{def:differentiability/gateaux}\mcite[sec. 0.2.1]{ИоффеТихомиров1974}If the first variation \( \delta f(x_0) \) is a continuous linear operator, we say that \( f \) is \term{Gateaux differentiable} or \term{weakly differentiable} at \( x_0 \) with \term{Gateaux derivative} \( f'_G(x_0) \coloneqq \delta f(x_0) \).

    Since \( f'(x_0) \) is linear in \( h \), we can replace \( t \downarrow 0 \) with \( t \to 0 \) in \fullref{def:differentiability/partial_operator} and reformulate this condition of Gateaux differentiability as the existence of a continuous linear operator \( \Lambda: X \to Y \) such that
    \begin{equation}\label{def:differentiability/gateaux/condition}
      \Lambda(h) = \lim_{t \to 0} \frac {f(x_0 + h) - f(x_0)} t.
    \end{equation}

    If \( \Lambda \) exists, we usually denote it by \( D_G f(x_0) \) or \( f_G'(x_0) \) and call it the \term{Gateaux derivative} of \( f \) at \( x_0 \). See \fullref{rem:derivative_notation} for a discussion of the notation.

    \thmitem{def:differentiability/frechet}\mcite[sec. 0.2.1]{ИоффеТихомиров1974}We now restrict our attention to \hyperref[def:banach_space]{Banach spaces}. We say that \( f \) is \term{Frechet differentiable} or \term{strongly differentiable} at \( x_0 \) if there exists a continuous linear operator \( \Lambda: X \to Y \) such that
    \begin{equation}\label{def:differentiability/frechet/condition}
      \lim_{h \to 0} \frac {\norm{f(x_0 + h) - f(x_0) - \Lambda(h)}_Y} {\norm{h}_X} = 0.
    \end{equation}

    If \( \Lambda \) exists, we usually denote it by \( D f(x_0) \) or \( f(x_0) \) and call it the \term{Frechet derivative} of \( f \) at \( x_0 \). See \fullref{rem:derivative_notation} for a discussion of the notation.

    Note that \fullref{def:differentiability/gateaux/condition} uses convergence in the topology of \( Y \) while \fullref{def:differentiability/frechet/condition} uses convergence in \( \BbbR \). We discuss in \fullref{rem:gateaux_vs_frechet} how Frechet differentiability is a special \enquote{uniform} case of Gateaux differentiability.

    \thmitem{def:differentiability/strict}\mcite[33]{DontchevRockafellar2014}If there exists a continuous linear operator \( \Lambda \) such that
    \begin{equation}\label{def:differentiability/strict/condition}
      \lim_{\substack{y \to x_0 \\ z \to x_0}} \frac {\norm{f(y) - f(z) - \Lambda (y - z)}_Y} {\norm{y - z}_X} = 0,
    \end{equation}
    we say that \( f \) is \term{strictly differentiable} at \( x_0 \).
  \end{thmenum}
\end{definition}

\begin{remark}\label{rem:derivative_notation}
  The following are standard notations for derivatives (some of the comments are based on \cite[146]{Фихтенгольц1968Том2}):
  \begin{thmenum}
    \thmitem{rem:derivative_notation/lagrange} We already used \term{Lagrange's notation} \( f'(x_0) \) and \( f_G'(x_0) \) in \fullref{def:differentiability}. Brevity is the only benefit of this notation. It becomes convenient when the functions have no name is a burden for directional derivatives.

    The second and third derivatives of \( f \) at \( x_0 \) are denoted as \( f^{''}(x_0) \) and \( f^{'''}(x_0) \) and the \( n \)-th derivative of is denoted as \( f^{(n)} \).

    See \fullref{def:nonsmooth_derivatives} for variations of this notation.

    \thmitem{rem:derivative_notation/newton} Newton's notation is similar to that of Leibniz, but depends on placing dots on top of \( f \), e.g. \( \ddot{f}(x_0) \coloneqq f''(x_0) \). This is used in areas like mathematical physics, however it has not become standard in more pure areas of analysis.

    \thmitem{rem:derivative_notation/euler} We use \term{Euler's notation} \( Df(x_0) \coloneqq f'(x_0) \) for more complicated expressions, e.g. \fullref{thm:derivative_limit_exchange}. The main benefit of this notation is that is allows to express differentiation as an operator, similar to what we defined in \fullref{def:differentiability}. The directional derivative of \( f \) at \( x_0 \) in the direction \( h \) is denoted as \( D_h f(x_0) \). Iterated differentiation corresponds to the standard notation for group composition: the \( n \)-th derivative at \( x_0 \) is denoted as \( D^n f(x_0) \).

    We also use other letters in the superscripts like \( D^G f(x_0) \) for Gateaux derivatives, \( D^\circ f(x_0) \) for Clarke's generalized derivatives, etc.

    \thmitem{rem:derivative_notation/phelps} Some authors like \cite{Phelps1993} use a variation of Euler's notation with \( \partial \) instead of \( D \). For example, directional derivatives are introduced as \( \partial^+ f(x_0)(h) \) in \cite[lemma 1.2]{Phelps1993}. This is consistent with the standard notation for subdifferentials - see \fullref{def:subdifferentials}, however Euler's notation appears to be more widely adoped.

    \thmitem{rem:derivative_notation/leibniz} The Leibniz notation for the derivative \( f'(x_0) \) is
    \begin{equation*}
      \diff f x (x_0) \coloneqq D f(x_0).
    \end{equation*}

    This notation is used extensively in integral calculus, however it is often confusing when manipulating derivatives. The fraction notation is unjustified in anything, but trivial cases and the partial derivative notation
    \begin{equation*}
      \diffp f x (x_0) \coloneqq D_x f(x_0)
    \end{equation*}
    is even more confusing.

    Note also that this depends on the convention of having variable names.
  \end{thmenum}
\end{remark}

\begin{remark}\label{rem:gateaux_vs_frechet}
  We will compare Gateaux \hyperref[def:differentiability/gateaux]{differentiability} with Frechet \hyperref[def:differentiability/frechet]{differentiability}. Let \( X \) and \( Y \) be Banach spaces, let \( U \subseteq X \) be an open set and let \( f: U \to Y \) be an arbitrary function. Fix a point \( x_0 \in U \).

  The continuous linear operator \( \Lambda: X \to Y \) is a Gateaux derivative if, for every \( \varepsilon > 0 \) and every direction \( h \in X \) there exists \( \delta_G^h > 0 \) such that
  \begin{equation}\label{rem:gateaux_vs_frechet/gateaux}
    \norm{\frac {f(x_0 + th) - f(x_0)} t - \inprod \Lambda h}_Y < \varepsilon \quad\forall t \in (0, \delta_G^h).
  \end{equation}

  In order for \( \Lambda \) to be a Frechet derivative, for every \( \varepsilon > 0 \) there must exist a \( \delta_F > 0 \), so that
  \begin{equation*}
    \frac{\norm{f(x_0 + h) - f(x_0) - \inprod \Lambda h}_Y} {\norm{h}_X} < \varepsilon \quad\forall h \in B(0, \delta_F) \setminus \{ 0 \},
  \end{equation*}
  which can be restated as
  \begin{equation}\label{rem:gateaux_vs_frechet/frechet}
    \norm{\frac {f(x_0 + th) - f(x_0)} t - \inprod \Lambda h}_Y < \varepsilon \quad\forall t \in (0, \delta_F) \ \forall h \in S_X.
  \end{equation}

  By comparing \fullref{rem:gateaux_vs_frechet/gateaux} to \fullref{def:differentiability/frechet}, we conclude that \( f \) is Frechet differentiable at \( x_0 \) if \( \inf_{h \in S_X} \delta^h_G > 0 \), that is, if \( f \) is Gateaux differentiable and the convergence of the Gateaux derivative is uniform on \( h \in S_X \).

  In particular, Frechet differentiability implies Gateaux differentiability.
\end{remark}


%---------------------------Relative Size-----------------------------
\section{Relative Size Squared\label{s:quad-rel-size-squared}}

The relative size squared metric is defined as
\[
q = \left( \min\left\{ \frac{A}{\overline{A}}, \frac{\overline{A}}{A} \right\} \right)^2
\]
where $A$ is the area of the element as defined in \S\ref{s:quad-area}
and $\overline{A}$ is the average of $A$ over all of the elements in the
ensemble of elements being considered.
It is the square of the minimum of the ratio of quad area to the average quad area and its inverse.

Note that if $\overline{A} < DBL\_MIN$ or $A < DBL\_MIN$, we take $q = 0$.

\quadmetrictable{relative size squared}%
{$1$}%                                      Dimension
{$[0.3, 1]$}%                               Acceptable range
{$[0,1]$}%                                  Normal range
{$[0,1]$}%                                  Full range
{Dependent on $\overline{A}$}%              Unit square
{\cite{knu:03}}%                            Citation
{v\_quad\_relative\_size\_squared}%         Verdict function name


\subsection{Numerical Approximation and Analysis}\label{sec:set-approx}
In the \emph{set-based} inversion framework, there are two primary sources of approximation error:
\begin{description}[leftmargin=!, labelwidth=0.7in]
  \item[(1)] partitioning the parameter space $\pspace$ to approximate events in $\cborel$ (and consequently, $\pborel$), and
  \item[(2)] partitioning the data space $\dspace$ to approximate events in $\dborel$.
\end{description}
In practice, we must rely on a finite numerical approximation of the (often uncountable) events in the $\sa$s $\dborel$, $\cborel$, and $\pborel$.
Owing to the inclusion $\cborel \subset \pborel$, it is possible to approximate events in both of these $\sa$s simultaneously.

Assume we fix some collection of sets $\set{D_\idisc}_{\idisc=1}^{\ndiscs} \subset \dborel$ to partition $\dspace$ (independent of any specification of $\qoi$), and that we partition $\pspace$ with $\set{\VV_\iparam}_{\iparam=1}^{\nsamps}$, where $\VV_\iparam \in \pborel$.
Both sets yield implicitly-defined Voronoi-cell partitions given by a finite sampling of each space.

In order to approximate $\paramP(\VV_\iparam)$ for $\iparam=1,\hdots,\nsamps$, we must determine which collection of $\VV_\iparam$'s approximate $\qoi^{-1}(D_\idisc)$ for each $\idisc=1,\hdots,\ndiscs$, and then apply the ansatz on this approximation of the contour event with known probability $\dataP(D_\idisc)$.
Algorithm~\ref{alg:inv_density} was proven to converge to the solution $\paramP$ in \cite{BET+14-arxiv} as the discretizations of $\pspace$ and $\dspace$ (as ${\nsamps \text{ and } \ndiscs}$ tend to $\infty$, respectively.