\documentclass[11pt, oneside]{article}   	% use "amsart" instead of "article" for AMSLaTeX format
\usepackage{geometry}                		% See geometry.pdf to learn the layout options. There are lots.
\geometry{letterpaper}                   		% ... or a4paper or a5paper or ...
%\geometry{landscape}                		% Activate for rotated page geometry
%\usepackage[parfill]{parskip}    		% Activate to begin paragraphs with an empty line rather than an indent
\usepackage{graphicx}				% Use pdf, png, jpg, or eps§ with pdflatex; use eps in DVI mode
								% TeX will automatically convert eps --> pdf in pdflatex
\usepackage{amssymb, amsmath}

%SetFonts

%SetFonts

\title{Secure MPC Protocol for (Sample) Standard Deviation}
\begin{document}

\maketitle

\section{Introduction}
Recall that for a database of size $n$ with data points $x_i$ indexed by $i \in {1, \dots, n},$ the sample variance $\sigma^2$ is defined to be:
$$\sigma^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})^2$$
where $\bar{x}$ is the sample mean:
$$ \bar{x} = \frac{1}{n} \sum_{i=1}^n x_i$$
The sample standard deviation of a database, $\sigma$, is the square root of the sample variance. \\

In the setting of MPC, rather than considering a single database of entries, we consider the case in which there is a set of parties where each party $P_i$ holds entry $x_i$. The parties wish to compute the standard deviation of their inputs, but do not wish to reveal their inputs to anyone. The inputs are allowed to be negative or positive, and they may not be whole numbers.

\paragraph{MPC-friendly formulation} However, there exists a well-known alternative formulation of the equation above, that show that $\sigma^2$ can be written as:
$$ \sigma^2 = \text{E}[X^2] -(\text{E}[X])^2$$
$$ = \frac{\sum{X^2}}{n} - \frac{(\sum{X})^2}{n^2}$$
$$ = \frac{n\sum{X^2} - (\sum{X})^2}{n^2} $$

\section{Secure Building Blocks}
The following building blocks that are implemented in JIFF are useful to us here:

\begin{itemize}
\item share(input): secret-shares input between all parties
\item share1.sadd(share2): secure addition of shares, which returns a share of results.
\item share1.ssub(share2): secure subtraction of shares, which returns a share of results.
\item share1.smult(share2, $<$op\_id$>$, $<$truncate$>$): secure multiplication of shares, which returns a share of results.

The fixed-point extension supports the optional $<$truncate$>$ parameter, which determines whether the result of the multiplication is truncated to within the precision or not.

It is unsafe to use a non-truncated value in a subsequent multiplication or division without truncating it, as well as in an addition or subtraction where the two operands do not have the same truncation/precision.
\item share.cmult(n, $<$op\_id$>$, $<$truncate$>$): secure multiplication of share by a constant, with a similar truncate optional parameter to the above.
\end{itemize}

\section{Basic Implementation}
We use the alternative MPC-friendly formulation from above. This alternative formulation is friendlier for MPC for the following reasons:
\begin{enumerate}
    \item It has only a single division by a public constant, furthermore the division is the very last operation. Since division by a public constant is reversible, we can have our MPC computation output the numerator only, and perform the division after the MPC concludes locally at every party. This avoids performing an MPC division which is one of the most expensive operations under MPC.
    \item The formulation contains $2 + n$ multiplications: (1) $n$ of them for squaring independent inputs, which can be performed locally by every party prior to sharing their input. (2) a multiplication by a public constant $n$ which is efficient to perform under MPC. (3) Squaring the average: this is the only secret multiplication that needs to be performed.
    \item Since we allow decimals as inputs, we must use the fixed-point extension of JIFF (as well as the negative number extension for negative inputs). This changes the performance behavior of the underlying primitives: namely, every multiplication by a secret requires an internal division to shift the decimal point to the left and truncate the portion of the result that is beyond the specified accuracy. However, we are in luck: the only such multiplication here is the one for squaring the average. The result of that multiplication is not used for any division (the division is in the clear!) or multiplication. So we can instruct JIFF to not shift the decimal point so that the expensive division is skipped. In order for the following subtraction operation to be meaningful, the decimal point on both operands need to be at the same location, we can shift the decimal point to the right for the minuend, which is a communication-free operation, making both operands consistent, and then truncate and shift to the left locally after the output of the MPC was revealed to the parties.
\end{enumerate}

Hence, our entire protocol operates as follows:
\paragraph{Local stage}
Every party $i$ shares both $X_i$ and $X_i^2$

\paragraph{MPC} The parties together compute and reveal the following quantity:
$$\Delta = (n \otimes \sum{X_i^2}) - (\sum{X_i} \otimes \sum{X_i})$$
Where $\otimes$ is the un-truncated fixed-point multiplication operator:
$$ x \otimes y = x \times y \times 10^{\text{precision}}$$

\paragraph{Local stage} Every party receives revealed $\Delta$ and computes:
$$\sigma = \sqrt{\frac{\Delta}{n \times 10^\text{precision}}}$$

\subsection{Decimal Errors}
All the operations used through out the protocol, both under MPC and locally, except for $\otimes$ are fixed point operations. They are guaranteed to satsify the following relation:
$$x \bigoplus y = \text{truncate}(x \oplus y, \text{precision})$$

Where truncate floors to the nearest fixed-point number towards $-\infty$. \\

\noindent In other words, they satisfy the following error bound:
$$(x \bigoplus y) - (x \oplus y) < 10^\text{precision}$$

\noindent Finally, the non-truncated multiplication is a perfectly accurate operation and introduces no error in-of-itself. Instead, it introduces an implicit error since it requires a division by an additional $10^\text{precision}$ afterwards. Note that this additional division is the only operation added to our MPC protocol compared to the fixed-point non-MPC version, which does not perform that division and uses truncated fixed point multiplication instead. \\

\noindent This means that the overall error introduced by our protocol is smaller or equal to that introduced by the non-MPC fixed point version of it.

\subsection{Security} The only thing learned by any semi-honest party after the execution of our protocol is the standard deviation. The intermediate value revealed by our MPC computation is deducible from the standard deviation knowing the number of parties, which is public information. Nothing else is revealed.

\end{document}


\chapter{Homotopy images}\label{chap:image}

We have observed in \cref{eg:rcoeq} that the reflexive coequalizer of the pre-kernel of a map $f:A\to X$ is the fiberwise join $\join[X]{A}{A}$, i.e.~we have a reflexive coequalizer diagram
\begin{equation*}
\begin{tikzcd}
A\times_X A \arrow[r,yshift=1ex] \arrow[r,yshift=-1ex] & A \arrow[l] \arrow[r] & \join[X]{A}{A}.
\end{tikzcd}
\end{equation*}
It can also be shown that the colimit of the 2-pre-kernel of a map $f:A\to X$ is the triple fiberwise join $\join[X]{A}{\join[X]{A}{A}}$, i.e.~we have a colimiting
\begin{equation*}
\begin{tikzcd}
A\times_{X} A \times_{X} A \arrow[r,yshift=2ex] \arrow[r] \arrow[r,yshift=-2ex] & A \times_X A \arrow[l,yshift=1ex] \arrow[l,yshift=-1ex] \arrow[r,yshift=1ex] \arrow[r,yshift=-1ex] & A \arrow[l] \arrow[r] & \join[X]{A}{\join[X]{A}{A}},
\end{tikzcd}
\end{equation*}
although one has to take care to use the right amount of coherence data.
These results suggest that geometric realization of the Cech nerve of a map $f:A\to X$, i.e.~the homotopy image of $f$, is the sequential colimit of the type sequence
\begin{equation*}
\begin{tikzcd}
A \arrow[r] & \join[X]{A}{A} \arrow[r] & \join[X]{A}{(\join[X]{A}{A})} \arrow[r] & \cdots
\end{tikzcd}
\end{equation*}
and its sequential colimit. We do not have a way of presenting the Cech nerve of a map in type theory, due to the infinite coherence problem of presenting simplicial types in type theory. Nevertheless, we \emph{can} analyze this type sequence and its colimit in homotopy type theory, since it is constructed entirely in terms of known operations. We will show in \cref{thm:image} that for any map $f:A\to X$, the infinite fiberwise join-power $\join[X]{\cdots}{\join[X]{A}{\join[X]{A}{A}}}$ is the image of $f$. We conclude that the image of a map always exists in univalent type theory with homotopy pushouts. We note that an earlier construction of the propositional truncation in a similar setting is due to van Doorn \cite{vanDoorn2016}, and another one is due to Kraus \cite{Kraus2016}. The present construction of the image of $f$ is called the join construction implies, as we show in \cref{thm:image_small}, that the image of an essentially small type mapping into a locally small type is again essentially small. In particular, the image of a map from a small type into the universe is essentially small. This corollary should be viewed as a type theoretic replacement axiom. This fact has the important consequence that any connected component of the universe is essentially small. We also note that the join construction leads to new constructions of set quotients and of Rezk completions, see \cite{joinconstruction}, where also a construction of the $n$-truncations is given as an application of the join construction.

\section{The universal property of the image}
\begin{defn}\label{defn:image_up}
Consider a commuting triangle
\begin{equation*}
\begin{tikzcd}[column sep=small]
A \arrow[rr,"i"] \arrow[dr,swap,"f"] & & U \arrow[dl,"m"] \\
& X
\end{tikzcd}
\end{equation*}
with $I:f\htpy m\circ i$, and where $m$ is an embedding\index{embedding}.
We say that $m$ has the \define{universal property of the image of $f$}\index{universal property!of the image|textit} if the map
\begin{equation*}
(i,I)^\ast : \mathrm{hom}_X(m,m')\to\mathrm{hom}_X(f,m')
\end{equation*}
defined by $(i,I)^\ast(h,H)\defeq (h\circ i,\ct{I}{(i\cdot H)})$,
is an equivalence for every embedding $m':U'\to X$.
\end{defn}

\begin{lem}
For any $f:A\to X$ and any embedding\index{embedding} $m:U\to X$, the type $\mathrm{hom}_X(f,m)$ is a proposition.
\end{lem}

\begin{proof}
From \cref{cor:fib_triangle} we obtain that the type $\mathrm{hom}_X(f,m)$ is equivalent to the type
\begin{equation*}
\prd{x:X}\fib{f}{x}\to\fib{m}{x},
\end{equation*}
so it suffices to show that this is a proposition.
Recall from \cref{thm:prop_emb} that a map is an embedding if and only if its fibers are propositions.
Thus we see that the type $\prd{x:X}\fib{f}{x}\to\fib{m}{x}$ is a product of propositions, so it is a proposition by \cref{thm:prop_pi}.
\end{proof}

\begin{cor}\label{cor:image_up}
Consider a commuting triangle
\begin{equation*}
\begin{tikzcd}[column sep=small]
A \arrow[rr,"i"] \arrow[dr,swap,"f"] & & U \arrow[dl,"m"] \\
& X
\end{tikzcd}
\end{equation*}
with $I:f\htpy m\circ i$, and where $m$ is an embedding. Then $m$ satisfies the universal property of the image of $f$ if and only if the implication
\begin{equation*}
\mathrm{hom}_X(f,m')\to\mathrm{hom}_X(m,m')
\end{equation*}
holds for every embedding $m':U'\to X$.
\end{cor}

Recall that embeddings into the unit type are just propositions.
Therefore, the universal property of the image of the map $A\to\unit$ is a proposition $P$ satisfying the universal property of the propositional truncation:

\begin{defn}
Let $A$ be a type, and let $P$ be a proposition that comes equipped with a map $f:A\to P$. We say that $f:A\to P$ satisfies the \define{universal property of propositional truncation}\index{universal property!of propositional truncation|textit} if for every proposition $Q$, the precomposition map
\begin{equation*}
\blank\circ f:(P\to Q)\to (A\to Q)
\end{equation*}
is an equivalence.
\end{defn}

\section{The join construction}
\subsection{Step one: constructing the propositional truncation}

\begin{lem}\label{lem:extend_join_prop}
Suppose $f:A\to P$, where $A$ is any type, and $P$ is a proposition.
Then the map
\begin{equation*}
(\join{A}{B}\to P)\to (B\to P)
\end{equation*}
given by $h\mapsto h\circ \inr$ is an equivalence, for any type $B$.
\end{lem}

\begin{proof}
Since both types are propositions by \cref{thm:prop_pi} it suffices to construct a map
\begin{equation*}
(B\to P)\to (\join{A}{B}\to P).
\end{equation*}
Let $g:B\to P$. Then the square
\begin{equation*}
\begin{tikzcd}
A\times B \arrow[r,"\proj 2"] \arrow[d,swap,"\proj 1"] & B \arrow[d,"g"] \\
A \arrow[r,swap,"f"] & P
\end{tikzcd}
\end{equation*}
commutes since $P$ is a proposition. Therefore we obtain a map $\join{A}{B}\to P$ by the universal property of the join.
\end{proof}

The idea of the construction of the propositional truncation is that if we are given a map $f:A\to P$, where $P$ is a proposition, then it extends uniquely along $\inr:A\to \join{A}{A}$ to a map $\join{A}{A}\to P$. This extension again extends uniquely along $\inr:\join{A}{A}\to \join{A}{(\join{A}{A})}$ to a map $\join{A}{(\join{A}{A})}\to P$ and so on, resulting in a diagram of the form
\begin{equation*}
\begin{tikzcd}
A \arrow[dr] \arrow[r,"\inr"] & \join{A}{A} \arrow[d,densely dotted] \arrow[r,"\inr"] & \join{A}{(\join{A}{A})} \arrow[dl,densely dotted] \arrow[r,"\inr"] & \cdots \arrow[dll,densely dotted,bend left=10] \\
& P
\end{tikzcd}
\end{equation*}

\begin{defn}
The \define{join powers} $A^{\ast n}$ of a type $X$ are defined by
\begin{align*}
A^{\ast 0} & \defeq \emptyt \\
A^{\ast 1} & \defeq A \\
A^{\ast (n+1)} & \defeq \join{A}{A^{\ast n}}.
\end{align*}
Furthermore, we define $A^{\ast\infty}$ to be the sequential colimit of the type sequence
\begin{equation*}
\begin{tikzcd}
A^{\ast 0} \arrow[r] & A^{\ast 1} \arrow[r,"\inr"] & A^{\ast 2} \arrow[r,"\inr"] & \cdots.
\end{tikzcd}
\end{equation*}
\end{defn}

Our goal is now to show that $A^{\ast\infty}$ is a proposition and satisfies the universal property of the propositional truncation.

\begin{lem}
Consider a type sequence
\begin{equation*}
\begin{tikzcd}
A_0 \arrow[r,"f_0"] & A_1 \arrow[r,"f_1"] & A_2 \arrow[r,"f_2"] & \cdots
\end{tikzcd}
\end{equation*}
with sequential colimit $A_\infty$, and let $P$ be a proposition. Then the map
\begin{equation*}
\seqin^\ast: (A_\infty\to P)\to \Big(\prd{n:\N}A_n\to P\Big)
\end{equation*}
given by $h\mapsto \lam{n}(h\circ \seqin_n)$ is an equivalence.
\end{lem}

\begin{proof}
By the universal property of sequential colimits established in \cref{thm:sequential_up} we obtain that $\coconemap$ is an equivalence. Note that we have a commuting triangle
\begin{equation*}
\begin{tikzcd}[column sep=tiny]
\phantom{\Big(\prd{n:\N}A_n\to P\Big).} & P^{A_\infty} \arrow[dl,swap,"\coconemap"] \arrow[dr,"\seqin^\ast"] & \phantom{\cocone(P)} \\
\cocone(P) \arrow[rr,swap,"\proj 1"] & & \Big(\prd{n:\N}A_n\to P\Big)
\end{tikzcd}
\end{equation*}
Note that for any $g:\prd{n:\N}A_n\to P$ the type
\begin{equation*}
\prd{n:\N} g_n\htpy g_{n+1}\circ f_n
\end{equation*}
is a product of contractible types, since $P$ is a proposition. Therefore it is contractible by \cref{thm:funext_wkfunext}, and it follows that the projection is an equivalence. We conclude by the 3-for-2 property of equivalences that $\seqin^\ast$ is an equivalence.
\end{proof}

\begin{lem}\label{lem:infjp_up}
Let $A$ be a type, and let $P$ be a proposition. Then the function
\begin{equation*}
\blank\circ \seqin_0: (A^{\ast\infty}\to P)\to (A\to P)
\end{equation*}
is an equivalence.
\end{lem}

\begin{proof}
We have the commuting triangle
\begin{equation*}
\begin{tikzcd}[column sep=0]
& P^{A^{\ast\infty}} \arrow[dl,swap,"\seqin^\ast"] \arrow[dr,"\blank\circ\seqin_0"] & \phantom{\Big(\prd{n:\N}A^{\ast n} \to P\Big)} \\
\Big(\prd{n:\N}A^{\ast n} \to P\Big) \arrow[rr,swap,"\lam{h}h_0"] & & P^A.
\end{tikzcd}
\end{equation*}
Therefore it suffices to show that the bottom map is an equivalence. Since this is a map between propositions, it suffices to construct a map in the converse direction. Let $f:A\to P$. We will construct a term of type
\begin{equation*}
\prd{n:\N}A^{\ast n} \to P
\end{equation*}
by induction on $n:\N$. The base case is trivial. Given a map $g:A^{\ast n}\to P$, we obtain a map $g:A^{\ast(n+1)}\to P$ by \cref{lem:extend_join_prop}.
\end{proof}

\begin{lem}\label{lem:seqcolim_contr}
Consider a type sequence
\begin{equation*}
\begin{tikzcd}
A_0 \arrow[r,"f_0"] & A_1 \arrow[r,"f_1"] & A_2 \arrow[r,"f_2"] & \cdots
\end{tikzcd}
\end{equation*}
and suppose that each $A_n$ is equipped with a base point $a_n:A_n$, and each $f_n$ is equipped with a homotopy $H_n:\mathsf{const}_{a_{n+1}}\htpy f_n$. Then the sequential colimit $A_\infty$ is contractible.\qed
\end{lem}

\begin{lem}\label{lem:isprop_infjp}
The type $A^{\ast\infty}$ is a proposition for any type $A$.
\end{lem}

\begin{proof}
By \cref{lem:prop_char} it suffices to show that $A^{\ast\infty}\to \iscontr(A^{\ast\infty})$, and by \cref{lem:infjp_up} it suffices to show that
\begin{equation*}
A\to \iscontr(A^{\ast\infty}),
\end{equation*}
because $\iscontr(A^{\ast\infty})$ is a proposition.

Let $x:A$. To see that $A^{\ast\infty}$ is contractible it suffices by \cref{lem:seqcolim_contr} to show that $\inr:A^{\ast n}\to A^{\ast(n+1)}$ is homotopic to the constant function $\const_{\inl(x)}$. However, we get a homotopy $\const_{\inl(x)}\htpy \inr$ immediately from the path constructor $\glue$.
\end{proof}

\begin{thm}
For any type $A:\UU$ there is a proposition $\brck{A}:\UU$ that comes equipped with a map $\eta:A\to \brck{A}$, and satisfies the universal property of propositional truncation.
\end{thm}

\begin{proof}
Let $A$ be a type. Then we define $\brck{A}\defeq A^{\ast\infty}$, and we define $\eta\defeq \seqin_0:A\to A^{\ast\infty}$. Then $\brck{A}$ is a proposition by \cref{lem:isprop_infjp}, and $\eta:A\to \brck{A}$ satisfies the universal property of propositional truncation by \cref{lem:infjp_up}.
\end{proof}

\subsection{Step two: constructing the image of a map}\label{sec:join_stage2}
Following Definition 7.6.3 of \cite{hottbook}, we recall that the image of a map $f:A\to X$ can be defined using the propositional truncation:
\begin{defn}
For any map $f:A\to X$ we define the \define{image}\index{image|textbf} of $f$ to be the type
\begin{equation*}
\im(f) \defeq \sm{x:X}\brck{\fib{f}{x}}
\end{equation*}
and we define the \define{image inclusion} to be the projection $\proj 1 :\im(f)\to X$.
\end{defn}
However, the construction of the fiberwise join in \cref{defn:fib_join} suggests that we can also define the image of $f$ as the infinite join power $f^{\ast\infty}$, where we repeatedly take the fiberwise join of $f$ with itself. Our reason for defining the image in this way is twofold:
\begin{itemize}
\item We use this construction to show that the image of a map $f:A\to B$ from an essentially small type $A$ into a locally small type $B$ is again essentially small.
\item Some interesting types, such as the real and complex projective spaces, appear in specific instances of this construction.
\end{itemize}

\begin{lem}
Consider a map $f:A\to X$, an embedding $m:U\to X$, and $h:\mathrm{hom}_X(f,m)$. Then the map
\begin{equation*}
\mathrm{hom}_X(\join{f}{g},m)\to \mathrm{hom}_X(g,m)
\end{equation*}
is an equivalence for any $g:B\to X$.
\end{lem}

\begin{proof}
Note that both types are propositions, so any equivalence can be used to prove the claim. Thus, we simply calculate
\begin{align*}
\mathrm{hom}_X(\join{f}{g},m) & \eqvsym \prd{x:X}\fib{\join{f}{g}}{x}\to \fib{m}{x} \\
& \eqvsym \prd{x:X}\join{\fib{f}{x}}{\fib{g}{x}}\to\fib{m}{x} \\
& \eqvsym \prd{x:X}\fib{g}{x}\to\fib{m}{x} \\
& \eqvsym \mathrm{hom}_X(g,m).
\end{align*}
The first equivalence holds by \cref{cor:fib_triangle}; the second equivalence holds by \cref{defn:join-fiber}; the third equivalence holds by \cref{lem:extend_join_prop}; the last equivalence again holds by \cref{cor:fib_triangle}.
\end{proof}

For the construction of the image of $f:A\to X$ we observe that if we are given an embedding $m:U\to X$ and a map $(i,I):\mathrm{hom}_X(f,m)$, then $(i,I)$ extends uniquely along $\inr:A\to \join[X]{A}{A}$ to a map $\mathrm{hom}_X(\join{f}{f},m)$. This extension again extends uniquely along $\inr:\join[X]{A}{A}\to \join[X]{A}{(\join[X]{A}{A})}$ to a map $\mathrm{hom}_X(\join{f}{(\join{f}{f})},m)$ and so on, resulting in a diagram of the form
\begin{equation*}
\begin{tikzcd}
A \arrow[dr] \arrow[r,"\inr"] & \join[X]{A}{A} \arrow[d,densely dotted] \arrow[r,"\inr"] & \join[X]{A}{(\join[X]{A}{A})} \arrow[dl,densely dotted] \arrow[r,"\inr"] & \cdots \arrow[dll,densely dotted,bend left=10] \\
& U
\end{tikzcd}
\end{equation*}

\begin{defn}
Suppose $f:A\to X$ is a map. Then we define the \define{fiberwise join powers}
\begin{equation*}
f^{\ast n}\defeq A_X^{\ast n}\to X.
\end{equation*}
\end{defn}

\begin{proof}[Construction]
Note that the operation $(B,g)\mapsto (\join[X]{A}{B},\join{f}{g})$ defines an endomorphism on the type
\begin{equation*}
\sm{B:\UU}B\to X.
\end{equation*}
We also have $(\emptyt,\ind{\emptyt})$ and $(A,f)$ of this type. For $n\geq 1$ we define
\begin{align*}
A_X^{\ast (n+1)} & \defeq \join[X]{A}{A_X^{\ast n}} \\
f^{\ast (n+1)} & \defeq \join{f}{f^{\ast n}}.\qedhere
\end{align*}
\end{proof}

\begin{defn}
We define $A_X^{\ast\infty}$ to be the sequential colimit of the type sequence
\begin{equation*}
\begin{tikzcd}
A_X^{\ast 0} \arrow[r] & A_X^{\ast 1} \arrow[r,"\inr"] & A_X^{\ast 2} \arrow[r,"\inr"] & \cdots.
\end{tikzcd}
\end{equation*}
Since we have a cocone
\begin{equation*}
\begin{tikzcd}
A_X^{\ast 0} \arrow[r] \arrow[dr,swap,"f^{\ast 0}" near start] & A_X^{\ast 1} \arrow[r,"\inr"] \arrow[d,swap,"f^{\ast 1}" near start] & A_X^{\ast 2} \arrow[r,"\inr"] \arrow[dl,swap,"f^{\ast 2}" xshift=1ex] & \cdots \arrow[dll,bend left=10] \\
& X
\end{tikzcd}
\end{equation*}
we also obtain a map $f^{\ast\infty}:A_X^{\ast\infty}\to X$ by the universal property of $A_X^{\ast\infty}$.
\end{defn}

\begin{lem}\label{lem:finfjp_up}
Let $f:A\to X$ be a map, and let $m:U\to X$ be an embedding. Then the function
\begin{equation*}
\blank\circ \seqin_0: \mathrm{hom}_X(f^{\ast\infty},m)\to \mathrm{hom}_X(f,m)
\end{equation*}
is an equivalence.
\end{lem}

\begin{thm}\label{thm:image}
For any map $f:A\to X$, the map $f^{\ast\infty}:A_X^{\ast\infty}\to X$ is an embedding that satisfies the universal property of the image inclusion of $f$.
\end{thm}

\subsection{Step three: establishing the smallness of the image}

Recall from \cref{defn:ess_small} that a type is said to be locally small if its identity types are equivalent to small types.

\begin{lem}
Consider a commuting square
\begin{equation*}
\begin{tikzcd}
A \arrow[r] \arrow[d] & B \arrow[d] \\
C \arrow[r] & D.
\end{tikzcd}
\end{equation*}
\begin{enumerate}
\item If the square is cartesian, $B$ and $C$ are essentially small, and $D$ is locally small, then $A$ is essentially small.
\item If the square is cocartesian, and $A$, $B$, and $C$ are essentially small, then $D$ is essentially small.
\end{enumerate}
\end{lem}

\begin{cor}
Suppose $f:A\to X$ and $g:B\to X$ are maps from essentially small types $A$ and $B$, respectively, to a locally small type $X$. Then $A\times_X B$ is again essentially small.
\end{cor}

\begin{lem}
Consider a type sequence
\begin{equation*}
\begin{tikzcd}
A_0 \arrow[r,"f_0"] & A_1 \arrow[r,"f_1"] & A_2 \arrow[r,"f_2"] & \cdots
\end{tikzcd}
\end{equation*}
where each $A_n$ is essentially small. Then its sequential colimit is again essentially small.
\end{lem}

\begin{thm}\label{thm:image_small}
For any map $f:A\to X$ from a small type $A$ into a locally small type $X$, the image $\im(f)$ is an essentially small type.
\end{thm}

Recall that in set theory, the replacement axiom asserts that for any family of sets $\{X_i\}_{i\in I}$ indexed by a set $I$, there is a set $X[I]$ consisting of precisely those sets $x$ for which there exists an $i\in I$ such that $x\in X_i$. In other words: the image of a set-indexed family of sets is again a set. Without the replacement axiom, $X[I]$ would be a class. In the following corollary we establish a type-theoretic analogue of the replacement axiom: the image of a family of small types indexed by a small type is again (essentially) small.

\begin{cor}\label{cor:im_small}
For any small type family $B:A\to\UU$, where $A$ is small, the image $\im(B)$ is essentially small. We call $\im(B)$ the \define{univalent completion} of $B$.
\end{cor}

\endinput

\section{Set quotients}\label{sec:set-quotients}

\subsection{Sets in homotopy type theory}
\begin{defn}
A type $A$ is said to be a \define{set} if there is a term of type
\begin{equation*}
\isset(A)\defeq \prd{x,y:A}\isprop(\id{x}{y}).
\end{equation*}
\end{defn}

\begin{lem}\label{lem:prop_to_id}
Let $A$ be a type, and let $(R,\rho):\mathsf{rRel}(A)$ be a reflexive relation on $A$ such that $R(x,y)$ is a proposition for each $x,y:A$. Then any fiberwise map
\begin{equation*}
\prd{x,y:A}R(x,y)\to (\id{x}{y})
\end{equation*}
is a fiberwise equivalence. Consequently, if there is such a fiberwise map, then $A$ is a set.
\end{lem}

\begin{proof}
Let $f:\prd{x,y:A}R(x,y)\to(\id{x}{y})$.
Since $R$ is assumed to be reflexive, we also have a fiberwise transformation
\begin{equation*}
\ind{x{=}}(\rho(x)):\prd{y:A}(\id{x}{y})\to R(x,y).
\end{equation*}
Since each $R(x,y)$ is assumed to be a proposition, it therefore follows that each $R(x,y)$ is a retract of $\id{x}{y}$. We conclude by \autoref{cor:id_fundamental_retr} that for each $x,y:A$, the map $f(x,y):R(x,y)\to(\id{x}{y})$ must be an equivalence.

Now it also follows that $A$ is a set, since its identity types are (equivalent to) propositions.
\end{proof}

\begin{eg}
One can apply \cref{lem:prop_to_id} using the \define{observational equality} $\mathrm{Eq}_\N:\N\to (\N\to\UU)$ given by
\begin{align*}
\mathrm{Eq}_\N(0,0) & \defeq \unit & \mathrm{Eq}_\N(\mathsf{succ}(n),0) & \defeq \emptyt \\
\mathrm{Eq}_\N(0,\mathsf{succ}(m)) & \defeq \emptyt & \mathrm{Eq}_\N(\mathsf{succ}(n),\mathsf{succ}(m)) & \defeq \mathrm{Eq}_\N(n,m)
\end{align*}
to show that $\N$ is a set. A routine induction argument shows that $\mathrm{Eq}_\N$ implies identity.
\end{eg}

\subsection{Equivalence relations}

\begin{defn}\label{defn:eq_rel}
Let $R:A\to (A\to\prop)$ be a binary relation valued in the propositions. We say that $R$ is an \define{($0$-)equivalence relation}\index{equivalence relation|textbf}\index{0-equivalence relation|see {equivalence relation}} if $R$ comes equipped with
\begin{align*}
\rho & : \prd{x:A}R(x,x) \\
\sigma & : \prd{x,y:A} R(x,y)\to R(y,x) \\
\tau & : \prd{x,y,z:A} R(x,y)\to (R(y,z)\to R(x,z)).
\end{align*}
Given an equivalence relation $R:A\to (A\to\prop)$, the \define{equivalence class}\index{equivalence class|textbf} $[x]_R:A\to\prop$ of $x:A$ is defined to be
\begin{equation*}
[x]_R\defeq R(x).
\end{equation*}
\end{defn}

\begin{defn}
Let $R:A\to (A\to\prop)$ be a $0$-equivalence relation.
We define for any $x,y:A$ a map\index{class_eq@{$\mathsf{eq\usc{}class}$}|textbf}
\begin{equation*}
\mathsf{eq\usc{}class}:R(x,y)\to ([x]_R=[y]_R).
\end{equation*}
\end{defn}

\begin{proof}[Construction.]
Let $r:R(x,y)$. By function extensionality, the identity type $R(x)=R(y)$ is equivalent to the type
\begin{equation*}
\prd{z:A}R(x,z)=R(y,z).
\end{equation*}
Let $z:A$. By the univalence axiom, the type $R(x,z)=R(y,z)$ is equivalent to the type
\begin{equation*}
\eqv{R(x,z)}{R(y,z)}.
\end{equation*}
We have the map $\tau_{y,x,z}(\sigma(r)):R(x,z)\to R(y,z)$. Since this is a map between propositions, we only have to construct a map in the converse direction to show that it is an equivalence. The map in the converse direction is just $\tau_{x,y,z}(r):R(y,z)\to R(x,z)$.
\end{proof}

\begin{prp}\label{thm:equivalence_classes}
Let $R:A\to (A\to\prop)$ be a $0$-equivalence relation.
Then for any $x,y:A$ the map
\begin{equation*}
\mathsf{eq\usc{}class} : R(x,y)\to ([x]_R=[y]_R)
\end{equation*}
is an equivalence.
\end{prp}

\begin{proof}
By the 3-for-2 property of equivalences, it suffices to show that the map
\begin{equation*}
\lam{r}{z}\tau_{y,x,z}(\sigma(r)) : R(x,y)\to \prd{z:A} \eqv{R(x,z)}{R(y,z)}
\end{equation*}
is an equivalence. Since this is a map between propositions, it suffices to construct a map of type
\begin{equation*}
\Big(\prd{z:A} \eqv{R(x,z)}{R(y,z)}\Big)\to R(x,y).
\end{equation*}
This map is simply $\lam{f} \sigma_{y,x}(f_x(\rho(x)))$.
\end{proof}

\subsection{The universal property of set quotients}

\begin{defn}
Let $R:A\to (A\to \prop)$ be an equivalence relation\index{equivalence relation|textit}, for $A:\UU$, and consider a map $q:A\to B$ where the type $B$ is a set, for which we have
\begin{equation*}
\prd{x,y:A}R(x,y)\to q(x)=q(y).
\end{equation*}
We will define a map
\begin{equation*}
\quotientrestr:(B\to X) \to \Big(\sm{f:A\to X}\prd{x,y:A}R(x,y)\to (f(x)=f(y))\Big).
\end{equation*}
\end{defn}

\begin{proof}[Construction]
Let $h:B\to X$. Then we have $h\circ q : A\to X$, so it remains to show that
\begin{equation*}
\prd{x,y:A}R(x,y)\to (h(q(x))=h(q(y)))
\end{equation*}
Consider $x,y:A$ which are related by $R$. Then we have an identification $p:q(x)=q(y)$, so it follows that $\ap{h}{p}:h(q(x))=h(q(y))$.
\end{proof}

\begin{defn}
Let $R:A\to (A\to \prop)$ be an equivalence relation\index{equivalence relation|textit}, for $A:\UU$, and consider a map $q:A\to B$ satisfying
\begin{equation*}
\prd{x,y:A}R(x,y)\to q(x)=q(y),
\end{equation*}
where the type $B$ is a set. We say that the map $q:A\to B$ satisfies the universal property of the \define{set quotient}\index{set quotient}\index{universal property!of set quotients|textit} $A/R$ if for any set $X$ the map
\begin{equation*}
\quotientrestr : (B\to X) \to \Big(\sm{f:A\to X}\prd{x,y:A}R(x,y)\to (f(x)=f(y))\Big)
\end{equation*}
is an equivalence.
\end{defn}

\begin{lem}
Let $R:A\to (A\to \prop)$ be an equivalence relation\index{equivalence relation|textit}, for $A:\UU$, and consider a commuting triangle
\begin{equation*}
\begin{tikzcd}[column sep=tiny]
A \arrow[rr,"q"] \arrow[dr,swap,"R"] & & U \arrow[dl,"m"] \\
& \prop^A
\end{tikzcd}
\end{equation*}
with $H:R\htpy m\circ q$, where $m$ is an embedding. Then we have
\begin{equation*}
\prd{x,y:A}R(x,y)\to (q(x)=q(y)).
\end{equation*}
\end{lem}

\begin{prp}\label{thm:quotient_up}
Let $R:A\to (A\to \prop)$ be an equivalence relation\index{equivalence relation|textit}, for $A:\UU$, and consider a commuting triangle
\begin{equation*}
\begin{tikzcd}[column sep=tiny]
A \arrow[rr,"q"] \arrow[dr,swap,"R"] & & U \arrow[dl,"m"] \\
& \prop^A
\end{tikzcd}
\end{equation*}
with $H:R\htpy m\circ q$, where $m$ is an embedding. Then the following are equivalent:
\begin{enumerate}
\item The embedding $m:U\to \prop^A$ satisfies the universal property of the image of $R$.
\item The map $q:A\to U$ satisfies the universal property of the set quotient $A/R$.
\end{enumerate}
\end{prp}

\begin{proof}
Suppose $m:U\to \prop^A$ satisfies the universal property of the image of $R$. Then it follows by \cref{thm:surjective}\marginnote{Need lemma that states that universal property of image inclusion iff the map into it is surjective} that the map $q:A\to U$ is surjective. Our goal is to prove that $U$ satisfies the universal property of the set quotient $A/R$. \marginnote{Proof incomplete}
\end{proof}

\begin{rmk}
\cref{thm:quotient_up} suggests that we can define the quotient of an equivalence relation $R$ on a type $A$ as the image of a map. However, the type $\prop^A$ of which the quotient is a subtype is not a small type, even if $A$ is a small type.
Therefore it is not clear that the quotient $A/R$ is essentially small\index{essentially small}, as it should be. Luckily, our construction of the image of a map allows us to show that the image is indeed essentially small, using the fact that $\prop^A$ is locally small\index{locally small}.
\end{rmk}

\section{The Rezk completion}

We recall from \cite{AhrensKapulkinShulman} that a pre-category $\mathcal{A}$ consists of a type $A$ of objects, a $\set$-valued binary relation $\mathsf{hom}:A\to A\to\set$ of morphisms, equipped with identity morphisms
\begin{equation*}
\mathsf{id} : \prd{x:A} \mathsf{hom}(x,x)
\end{equation*}
and a composition operation
\begin{equation*}
\mathsf{comp} : \prd{x,y,z:A} \mathsf{hom}(y,z)\to \mathsf{hom}(x,y)\to \mathsf{hom}(x,z),
\end{equation*}
which is associative and satisfies the unit laws. The isomorphisms $x \cong y$ from $x$ to $y$ in a pre-category are defined in the expected way, and the pre-category $\mathcal{A}$ is said to be \define{Rezk-complete} if the canonical map
\begin{equation*}
\prd{x,y:A} (x=y)\to (x\cong y)
\end{equation*}
is an equivalence. Rezk-complete pre-categories are also called \define{categories}. By the univalence axiom, the pre-category $\mathbf{Set}$ of small sets is Rezk-complete.

A pre-category is said to be \define{(essentially) small} if its types of objects and morphisms are (essentially) small, and a pre-category is said to be \define{locally small} if its hom-types are essentially small. Thus, the category $\mathbf{Set}$ is locally small.

\begin{defn}
Let $\mathcal{A}$ be a pre-category. A functor $\mathcal{F}:\mathcal{A}\to\mathcal{B}$ into a (Rezk-complete) category $\mathcal{B}$ is said to be a \define{Rezk-completion} if the pre-composition map
\begin{equation*}
\mathsf{Functor}(\mathcal{B},\mathcal{C})\to\mathsf{Functor}(\mathcal{A},\mathcal{C})
\end{equation*}
is an equivalence, for every (Rezk-complete) category $\mathcal{C}$.
\end{defn}

\begin{thm}\label{cor:rezkcompletion}
The Rezk completion $\hat{A}$ of any small pre-category $A$ can be constructed in any
univalent universe that is closed under graph quotients,
and $\hat{A}$ is again a small category.
\end{thm}

\begin{proof}
In the first proof of Theorem 9.9.5 of \cite{hottbook}, the Rezk completion of
a precategory $A$ is constructed as the image of the action on objects of the
Yoneda embedding $\mathbf{y}:A\to\mathbf{Set}^{\op{A}}$.

The hom-set $\mathbf{Set}^{\op{A}}(F,G)$ is the type of natural transformations
from $F$ to $G$. It is clear from Definition 9.9.2 of \cite{hottbook}, that the
type $\mathbf{Set}^{\op{A}}(F,G)$ is in $\UU$ for any two presheaves $F$ and $G$
on $A$. In particular, the type $F\cong G$ of isomorphisms from $F$ to $G$
is small for any two presheaves on $A$.

Since $\mathbf{Set}$ is a category, it follows from Theorem 9.2.5 of
\cite{hottbook} that the presheaf pre-category $\mathbf{Set}^{\op{A}}$ is a category.
Since the type of isomorphisms between any two objects is
small, it follows that the type of objects of
$\mathbf{Set}^{\op{A}}$ is locally small.

Hence we can use \autoref{thm:modified-join} to construct the image of the
action on objects of the Yoneda-embedding. The image constructed in this way
is of course equivalent to the type $\hat{A}_0$ defined in the first proof of
Theorem 9.9.5. Hence the arguments presented in the rest of that proof apply
as well to our construction of the image. We therefore conclude that the Rezk completion
of any small precategory is a small category.
\end{proof}

\begin{eg}\label{eg:emspaces}
Any group $G$ can be seen as a pre-category $\mathcal{G}$ with a single object and the group $G$ as morphisms. The Rezk-completion of $\mathcal{G}$ is the Eilenberg-Mac Lane space $K(G,1)$. Thus we conclude that any univalent universe closed under pushouts contains the Eilenberg-Mac Lane spaces $K(G,1)$ for any small group $G$.
\end{eg}


\hypertarget{meshgen}{%
\section{Meshgen}\label{meshgen}}

The \texttt{meshgen} module is used to create \texttt{Mesh} objects
corresponding to a specified domain. It provides the \texttt{MeshGen}
class to perform the meshing, which are created with the following
arguments:

\begin{lstlisting}
MeshGen(domain, boundingbox)
\end{lstlisting}

Domains are specified by a scalar function that is positive in the
region to be meshed and locally smooth. For example, to mesh the unit
disk:

\begin{lstlisting}
var dom = fn (x) -(x[0]^2+x[1]^2-1)
\end{lstlisting}

A \texttt{MeshGen} object is then created and then used to build the
\texttt{Mesh} like this:

\begin{lstlisting}
var mg = MeshGen(dom, [-1..1:0.2, -1..1:0.2])
var m = mg.build()
\end{lstlisting}

A bounding box for the mesh must be specified as a \texttt{List} of
\texttt{Range} objects, one for each dimension. The increment on each
\texttt{Range} gives an approximate scale for the size of elements
generated.

To facilitate convenient creation of domains, a \texttt{Domain} class is
provided that provides set operations \texttt{union},
\texttt{intersection} and \texttt{difference}.

\texttt{MeshGen} accepts a number of optional arguments:

\begin{itemize}

\item
  \texttt{weight} A scalar weight function that controls mesh density.
\item
  \texttt{quiet} Set to \texttt{true} to suppress \texttt{MeshGen}
  output.
\item
  \texttt{method} a list of options that controls the method used.
\end{itemize}

Some method choices that are available include:

\begin{itemize}

\item
  \texttt{"FixedStepSize"} Use a fixed step size in optimization.
\item
  \texttt{"StartGrid"} Start from a regular grid of points (the
  default).
\item
  \texttt{"StartRandom"} Start from a randomly generated collection of
  points.
\end{itemize}

There are also a number of properties of a \texttt{MeshGen} object that
can be set prior to calling \texttt{build} to control the operation of
the mesh generation:

\begin{itemize}

\item
  \texttt{stepsize}, \texttt{steplimit} Stepsize used internally by the
  \texttt{Optimizer}
\item
  \texttt{fscale} an internal ``pressure''
\item
  \texttt{ttol} how far the vertices are allowed to move before
  retriangulation
\item
  \texttt{etol} energy tolerance for optimization problem
\end{itemize}

\texttt{MeshGen} picks default values that cover a reasonable range of
uses.

\hypertarget{domain}{%
\section{Domain}\label{domain}}

The \texttt{Domain} class is used to conveniently build a domain by
composing simpler elements.

Create a \texttt{Domain} from a scalar function that is positive in the
region of interest:

\begin{lstlisting}
var dom = Domain(fn (x) -(x[0]^2+x[1]^2-1))
\end{lstlisting}

You can pass it to \texttt{MeshGen} to specify the region to mesh:

\begin{lstlisting}
var mg = MeshGen(dom, [-1..1:0.2, -1..1:0.2])
\end{lstlisting}

You can combine \texttt{Domain} objects using set operations
\texttt{union}, \texttt{intersection} and \texttt{difference}:

\begin{lstlisting}
var a = CircularDomain(Matrix([-0.5,0]), 1)
var b = CircularDomain(Matrix([0.5,0]), 1)
var c = CircularDomain(Matrix([0,0]), 0.3)
var dom = a.union(b).difference(c)
\end{lstlisting}

\hypertarget{circulardomain}{%
\section{CircularDomain}\label{circulardomain}}

Conveniently constructs a \texttt{Domain} object correspondiong to a
disk. Requires the position of the center and a radius as arguments.

Create a domain corresponding to the unit disk:

\begin{lstlisting}
var c = CircularDomain([0,0], 1)
\end{lstlisting}

\hypertarget{halfspacedomain}{%
\section{HalfSpaceDomain}\label{halfspacedomain}}

Conveniently constructs a \texttt{Domain} object correspondiong to a
half space defined by a plane at \texttt{x0} and a normal \texttt{n}:

\begin{lstlisting}
var hs = HalfSpaceDomain(x0, n)
\end{lstlisting}

Note \texttt{n} is an ``outward'' normal, so points into the
\emph{excluded} region.

Half space corresponding to the allowed region \texttt{x\textless{}0}:

\begin{lstlisting}
var hs = HalfSpaceDomain(Matrix([0,0,0]), Matrix([1,0,0]))
\end{lstlisting}

Note that \texttt{HalfSpaceDomain}s cannot be meshed directly as they
correspond to an infinite region. They are useful, however, for
combining with other domains.

Create half a disk by cutting a \texttt{HalfSpaceDomain} from a
\texttt{CircularDomain}:

\begin{lstlisting}
var c = CircularDomain([0,0], 1)
var hs = HalfSpaceDomain(Matrix([0,0]), Matrix([-1,0]))
var dom = c.difference(hs)
var mg = MeshGen(dom, [-1..1:0.2, -1..1:0.2], quiet=false)
var m = mg.build()
\end{lstlisting}

\hypertarget{mshgndim}{%
\section{MshGnDim}\label{mshgndim}}

The \texttt{MeshGen} module currently supports 2 and 3 dimensional
meshes. Higher dimensional meshing will be available in a future
release; please contact the developer if you are interested in this
functionality.


\iffalse
This is a big-ish template that I've been using for years now.
A lot of the includes are redundant, so if you're reading this and have no idea why
I've included most things, don't worry - I don't either.
Also, compiling this (to PDF, for instance) relies on CSS files present on my
computer (coming from the packages). So it might not look as nice if you
do it on yours (especially if you don't have all the packages).

Ya'll've been warned.
\fi

\documentclass{article}

    \usepackage{tikz}
    \usetikzlibrary{%
        decorations.pathreplacing,%
        decorations.pathmorphing%
    }
    \usepackage{subcaption}
    \usepackage{todonotes}
    \usepackage{amsmath}
    \usepackage{amssymb}
    \usepackage{color}
    \usepackage{bm}
    % \usepackage[dvipsnames]{xcolor}
    \usepackage{graphicx,float}
    \usepackage{caption}
    \usepackage{float}
    \usepackage[hidelinks]{hyperref}
    \usepackage{enumitem}
    \usepackage[bottom]{footmisc}
    \usepackage{flexisym}
    \usepackage{cancel}
    \usepackage[braket]{qcircuit}
    \usepackage[margin=.8in, tmargin=.3in]{geometry}
    \usepackage{mathtools}  %
    \renewcommand{\baselinestretch}{1.2}
    \newcommand{\eps}{\epsilon}

    \newcommand{\vq}{\mathbf{q}}
    \newcommand{\vqdot}{\mathbf{\dot{q}}}
    \newcommand{\vqddot}{\mathbf{\ddot{q}}}
    \newcommand{\vx}{\mathbf{x}}
    \newcommand{\vy}{\mathbf{y}}
    \newcommand{\vystar}{\mathbf{y}^*}
    \newcommand{\vu}{\mathbf{u}}
    \newcommand{\vv}{\mathbf{v}}
    \newcommand{\vf}{\mathbf{f}}
    \newcommand{\vr}{\mathbf{r}}
    \newcommand{\vp}{\mathbf{p}}
    \newcommand{\vg}{\mathbf{G}}
    \newcommand{\bM}{\mathbf{M}}
    \newcommand{\bC}{\mathbf{C}}
    \renewcommand{\vec}{\bm}
    \newcommand{\dotvec}[1]{\bm{\dot{#1}}}
    \newcommand{\bS}{\mathbf{S}}
    \newcommand{\bJ}{\mathbf{J}}
    \newcommand{\vlam}{\pmb{\lambda}}
    \newcommand{\vxdot}{\mathbf{\dot{x}}}
    \newcommand{\argmin}{\operatornamewithlimits{arg\ min}}

    % \newcommand{\tens}{\otimes}

\newcommand{\tens}[1]{%
  \mathbin{\mathop{\otimes}\limits_{#1}}%
}

    \setlength{\parskip}{\baselineskip}
    % \newcommand{\l}{\left(}

    \usepackage{titlesec}
    \usepackage{physics}
    \usepackage{multicol}

    \setlength\parindent{0pt}
    \captionsetup{justification=centering}

    \title{Classical Mechanics: From Newton to Euler-Lagrange}
    \date{\today}
    \author{Traiko Dinev \textless traiko.dinev@gmail.com\textgreater}

\begin{document}
\begin{multicols}{2}[Classical Mechanics: From Newton to Euler-Lagrang]
\maketitle
\textit{NOTE: These notes are a summary of Classical Mechanics: A Theoretical Minimum. They also contain some of David Morrin. Some of the notebooks are exercises I did through my own research.}

\textit{NOTE: Note this "summary" is NOT a reproduction of the course materials nor is it copied from the corresponding courses. It was entirely written and typeset from scratch.}

\textit{License: Creative Commons public license; See README.md of repository}

Hey, it is me, the author of these notes. I used to think reading physics was pointless practically speaking, so I did it a bit sparingly. But then.. I actually needed it in my own research! So I'm citing my work here~\cite{dinev2020modeling}. Hopefully it encourages more academics to read physics at a moderate to advanced level, as you never know when you'll need it!

\section{Basics}
A position of a particle in space is $\bm{r} = (x, y, z)$, its velocity is $\bm{v} = \dot{\bm{r}} = \frac{d\bm{r}}{d t}$. By (almost) definition of derivatives, we have:

\begin{equation}
    \bm{r}(t) = \bm{r}_0 + \bm{v}_0 t
\end{equation}

if $\bm{v}(t) = \bm{v}_0$ is independent of time. Adding acceleration to this, by \textbf{Newton's second law}, $\bm{a} = \dot{\bm{v}} = \bm{F}/m$, we can arrive at the following differential equation:

\begin{equation}
    \dot{\bm{r}}_t = \bm{v}(t) = \bm{v}_0 + \bm{a}_0 t
\end{equation}

Integrating this, we obtain:

\begin{align}
    \bm{r}(t) &= \bm{r}_0 + \int_0^t \bm{v}(t) dt = \bm{r}_0 +  \int_0^t \bm{v}_0 + \bm{a}_0 t\ dt \\
    &= \bm{r}_0 + \bm{v}_0 t + \frac{\bm{a}_0}{2} t^2 = \bm{r}_0 +  \bm{v}_0 t + \frac{\bm{F}}{2m} t^2
\end{align}

assuming that the acceleration is not a function of time. Otherwise you'd integrate the above ODE for each timestep.

\section{Generalized Co-ordinates and Equations of Motion}
\textit{The following is from a robotics summary of mine}

    A system's dynamics are most generally described by a second-order non-linear equation:

    \begin{equation} \label{eq:dynamics}
        \ddot{\vec{q}} = \vec{f}(\vec{q}, \dot{\vec{q}}, \vec{u}, t)
    \end{equation}

    where $\dot{\vec{q}}$ and $\ddot{\vec{q}}$ are the first and second derivative of $\vec{q}(t)$ -- the state of the system and $\vec{u}(t)$ -- the control inputs at time $t$. $\vec{f}$ is a matrix describing the dynamics. \autoref{eq:dynamics} describes the time-evolution of the world's configuration. This depends on the state at time $t$ as well as the control inputs we apply. We derive the equations of motion by using Newton's second law of motion. For all robots without kinematic chains (loop mechanical connections), the above can be generally written as the following product:

    \begin{equation} \label{eq:dynamics_standard}
        \vec{M}(\vec{q}) \ddot{\vec{q}} + \vec{C}(\vec{q}, \dot{\vec{q}}) \dot{\vec{q}} =
            \tau_g(\vec{q}) + \vec{B}\vec{u}
    \end{equation}

    Here $\vec{M}$ is the inertia matrix, $\vec{C}$ is the Coriolis matrix, $\tau_g$ is the gravity vector and $\vec{B}$ maps control inputs $\vec{u}$ into forces. For many systems this simplifies even further:

    \begin{equation} \label{eq:affine}
        \ddot{\vec{q}} = \vec{f}_1(\vec{q}, \dotvec{q}, t) + \vec{f}_2(\vec{q}, \dotvec{q}, t)\ \vec{u}
    \end{equation}

    We call such systems \textbf{control affine}. Such systems have dynamics that are affine (linear plus offset) in $\vec{u}$.

\section{Lagrange}
So how do we obtain the equations of motion of a system? They all obey Newton's second law, but there is another way of stating that is more general. Define the Lagrangian of a system as the difference of its kinetic energy and potential energy:

\begin{equation*}
    \mathcal{L} = T - V
\end{equation*}

The fundamental law then states that for any system the motion from point A to point B is that which minimizes the integral of this difference along the path. In a sence it is a \textbf{stationary action principle}, that is the action (the integral) is stationary (not necessarily a minimum or maximum). Firstly, let's start with conservative forces (which as we'll see will make a certain term equal to 0). We write the integral of the lagrangian as:

\begin{equation*}
    S = \int_{t_A}^{t_B} \mathcal{L}\ dt
\end{equation*}

and we call $S$ the \textbf{action}. The principle states that the action is stationary, i.e.:

\begin{equation*}
    \delta S = 0
\end{equation*}

In generalized coordinates the system's kinetic energy is $\frac{1}{2} \dot{q}^T M \dot{q}$. For readability we will stick to the one-dimensional case, trusting that this extends (and it does) to any dimensions. For now, all we need to know is that the Lagrangian $\mathcal{L}$ is a function of time, generalized positions, and generalized velocities:

\begin{equation*}
    \mathcal{A} = \mathcal{A}(t, q(t), \dot{q}(t))
\end{equation*}

Firstly, we need to be clear that the path a system takes must start and end at a fixed point (otherwise this doesn't work), that is $q(t_A) = q_A, q(t_B) = q_B$ and the same for velocities. We want to find the minimum of the action, that is:

\begin{equation*}
    \min_{q(t)} S = \min \int_{t_A}^{t_B} \mathcal{L}\ dt
\end{equation*}

where we note two things. 1) we are minimizing w.r.t. a function and 2) -- we are minimizing an integral.
Consider a petrubation of the path the system takes (the thing under the min above):

\begin{align*}
    q'(t) &= q(t) + \epsilon \eta(t) \\
    \dot{q}'(t) &= \dot{q}(t) + \epsilon \eta'(t)
\end{align*}

where we need to have $\eta(t_A) = \eta(t_B) = 0$, because we've fixed the start and end points. At the minimum any variation must yield a greater action (flip for maximum), that is:

\begin{equation*}
    S[t, q(t), \dot{q}(t)] \leq S[t, q(t) + \epsilon \eta(t), \dot{q}(t) + \epsilon \eta'(t)]
\end{equation*}

which means that the minimum is at $\epsilon = 0$. Minimum means that the derivative needs to be $0$. First we define the thing that's at the minimum to be a function of $\epsilon$:

\begin{equation*}
    f(\epsilon) = S[t, q'(t), \dot{q}'(t)]
\end{equation*}

Now we can say that this is minimized at $\epsilon = 0$ (follows from the assumption that action is minimized). Then we calculate its total derivative:

\begin{equation}
    \frac{d f}{d \epsilon} = \frac{d}{d \epsilon}\int_{t_A}^{t_B} \mathcal{L}(t, q'(t), \dot{q}'(t))\ dt = \int_{t_A}^{t_B} \frac{d \mathcal{L}}{d \epsilon}\ dt
\end{equation}

The derivative of the Lagrangian w.r.t $\epsilon$ is:

\begin{equation*}
    \frac{d \mathcal{L}}{d \epsilon} = \frac{\partial \mathcal{L}}{\partial t} \frac{d t}{d \epsilon} + \frac{\partial \mathcal{L}}{\partial q'} \frac{dq'}{d \epsilon} + \frac{\partial \mathcal{L}}{\partial \dot{q}'} \frac{d \dot{q}'}{d \epsilon}
\end{equation*}

The first term is $0$ since $\epsilon$ has no effect on $t$. We can do a substitution of the pertrubation derivatives to obtain:

\begin{equation*}
    \frac{d \mathcal{L}}{d \epsilon} = \frac{\partial \mathcal{L}}{\partial q'} \eta(t) + \frac{\partial \mathcal{L}}{\partial \dot{q}'} \eta'(t)
\end{equation*}

note the slighly confusing notation, where $q'$ stands for variation in $q$ and $\eta'$ stands for $\eta$'s derivative. At least we consistenly use the inconsistency. Now the derivative of $f(\epsilon)$ needs to be $0$ at $\epsilon = 0$. This means we can remove all the pertrubations in the Lagrangian (set $q' = q, \dot{q}' = \dot{q}$) Meaning:

\begin{equation*}
    \left. \frac{d \mathcal{L}}{d \epsilon} \right\vert_{\epsilon = 0} = \int_{t_A}^{t_B} \left[ \frac{\partial \mathcal{L}}{\partial q} \eta(t) + \frac{\partial \mathcal{L}}{\partial \dot{q}} \eta'(t) \right] dt = 0
\end{equation*}

Next we need to integrate by parts. Remembering that if we have:

\begin{equation*}
    \frac{d\ \eta f}{dt} = \eta \frac{df}{dt} + f \frac{d\eta}{dt}
\end{equation*}

then we can integrate to get:

\begin{equation*}
    \int f \frac{d \eta}{dt} dt = \eta f - \int \eta \frac{df}{dt} dt
\end{equation*}

We apply integration by parts on the second term:

\begin{align*}
    & \int_{t_A}^{t_B} \left[ \frac{\partial \mathcal{L}}{\partial q} \eta(t) + \frac{\partial \mathcal{L}}{\partial \dot{q}} \eta'(t) \right] dt \\
    & = \int_{t_A}^{t_B} \frac{\partial \mathcal{L}}{\partial q} \eta(t)\ dt +
        \int_{t_A}^{t_B} \frac{\partial \mathcal{L}}{\partial \dot{q}} \eta'(t) dt \\
    & = \int_{t_A}^{t_B} \frac{\partial \mathcal{L}}{\partial q} \eta(t)\ dt +
        \left[ \frac{\partial \mathcal{L}}{\partial\dot{q}} \eta(t) \right]_{t_A}^{t_B}
        - \int_{t_A}^{t_B} \frac{d}{dt} \frac{\partial \mathcal{L}}{\partial \dot{q}} \eta(t) \ dt \\
    & = \int_{t_A}^{t_B} \frac{\partial \mathcal{L}}{\partial q} \eta(t)\ dt
    - \int_{t_A}^{t_B}  \frac{d}{dt} \frac{\partial \mathcal{L}}{\partial \dot{q}} \eta(t)\ dt \\
    & = \int_{t_A}^{t_B} \left[ \frac{\partial \mathcal{L}}{dq} \eta(t) - \frac{d}{dt} \frac{\partial \mathcal{L}}{\partial \dot{q}} \eta(t) \right] \ dt = \\
    & = \int_{t_A}^{t_B} \left[ \frac{\partial \mathcal{L}}{\partial q} - \frac{d}{dt} \frac{\partial \mathcal{L}}{\partial\dot{q}} \right] \eta(t) \ dt
\end{align*}

where we apply the bundary conditions $\eta(t_A) = \eta(t_B) = 0$. Now we have this remaining:

\begin{equation*}
    \int_{t_A}^{t_B} \left[ \frac{\partial \mathcal{L}}{\partial q} - \frac{d}{dt} \frac{\partial \mathcal{L}}{\partial \dot{q}} \right] \eta(t) \ dt = 0
\end{equation*}

where this is true at $\epsilon = 0$. Feynman has the best visual proof but the idea is that if it's true for any $\eta(t)$ I can make it any delta function (peak) $\eta(t_i) =  1$ for any $i$ I choose then the first thing must be 0. Meaning:

\begin{equation}
    \frac{\partial \mathcal{L}}{\partial q} - \frac{d}{dt} \frac{\partial \mathcal{L}}{\partial \dot{q}} = 0
\end{equation}

Normally this is written the other way around:

\begin{equation}
    \frac{d}{dt} \frac{\partial \mathcal{L}}{\partial \dot{q}} - \frac{\partial \mathcal{L}}{\partial q} = 0
\end{equation}

The conservative forces show as potential energies. Namely:

\begin{align*}
    \mathcal{L} &= T - V \\
    T &= \frac{1}{2} m \dot{q}^2 \\
    F_{pot} &= -\frac{d V}{dt}
\end{align*}

with a negative sign convention in physics. This is just Newton's second law for a particle:

\begin{align*}
    \mathcal{L} &= \frac{1}{2} m v^2 - V(x) \\
    \frac{d}{dt} m v&= -\frac{d V(x)}{dt} \\
    m a &= F
\end{align*}

Lastly, remember external (non-conservative) forces? They show up on the right:

\begin{equation}
    \frac{d}{dt} \frac{\partial \mathcal{L}}{\partial \dot{q}} - \frac{\partial \mathcal{L}}{\partial q} = F_{ext}
\end{equation}

\section{Example}

\begin{figure}[H]
    \centering
    \begin{tikzpicture}[
        media/.style={font={\footnotesize\sffamily}},
        interface/.style={
            postaction={draw,decorate,decoration={border,angle=-45,
                        amplitude=0.3cm,segment length=2mm}}},
        scale=1.2
        ]

        % Round rectangle
        % \fill[gray!10,rounded corners] (-4,-3) rectangle (4,0);
        % Interface
        \draw[blue,line width=.5pt,interface](-3,0)--(3,0);
        % Vertical dashed line
        \draw[dashed,gray](0,0)--(0,3);

        % Coordinates system
        % \draw(0,0.15)node[above]{$x$};
        \draw[<->,line width=1pt, shift={(-3cm, 0cm)}] (1,0) node[above]{$x$}-|(0,1) node[left]{$z$};

        \path (0,0)++(80:2.cm)node{$\theta$};
        \draw[->] (0,2.5) arc (90:30:.75cm);

        \path (0,0)++(-1.2, 1)node{$\phi$};
        \draw[->] (-.6,.55) arc (-120:-250:.5cm);

        \draw[gray,dashed](0,1)--(1,.7);
        \path (1.3, .6)node{$x,z$};

        \draw[line width=1pt](0,1cm)circle(.5cm);
        \draw[gray](0,1)--(1.5,2.5);

        \draw[gray, line width=1pt](1.1, 1.9)--(0.9,2.1);
        \draw[gray, line width=1pt](0.9,2.1)--(0.6,1.8);
        \draw[gray, line width=1pt](1.1,1.9)--(0.8,1.6);

        \draw[line width=1pt](1.5,2.5)circle(.1cm);
        \path(1.1, 2.6) node{$m_b$};

        \draw[<->,line width=.5pt,blue,dashed](0.2,0.7)--(1.8, 2.2);
        \path (1.25, 1.25) node{$l$};
    \end{tikzpicture}

    \caption{A Variable-Length Wheeled Inverted Pendulum (VL-WIP) model.}
    \label{fig:dynamics}
\end{figure}

We use the Lagrangian method to derive the dynamics of the system.
First, we define the position $x_b, z_b$ of the mass $m_b$ and its velocity $\dot{x}_b, \dot{z}_b$:
%
% \vskip 0.1in
\begin{align}
    x_b &= x + l\ \sin(\theta) \\
    z_b &= z + l\ \cos(\theta) \nonumber \\
    \dot{x}_b &= \dot{x} + \dot{l}\ \sin(\theta) +
        l\ cos(\theta)\ \dot{\theta} \nonumber \\
    \dot{z}_b &= \dot{z} + \dot{l}\ \cos(\theta) -
        l\ sin(\theta)\ \dot{\theta} \nonumber
\end{align}
% \vskip 0.1in
%
Lagrange's method states that for a system with total kinetic energy $T$ and potential energy $U$:
%
\vskip 0.1in
\begin{equation} \label{eq:lagrange}
    \frac{d}{dt} \frac{\partial \mathcal{L}}{\partial \vqdot_i} - \frac{\partial \mathcal{L}}{\partial \vq_i} = \vf_{ext},
\end{equation}
\vskip 0.1in
%
where $\mathcal{L} = T - U$ is the system's Lagrangian and $\vf_{ext}$ are external forces applied to the system. We now need to compute the system's kinetic and potential energy.

In general, every link will have a rotational and translational kinetic energy component. For the wheel we include a rotational kinetic energy term $I_w \dot{\phi}^2$ where $I_w = m_w R_w^2$ is the moment of inertia of the wheel. Since the point mass has a zero moment of inertia, it only has a translational kinetic energy $m_b \vv_b^T \vv_b$, where $\vv_b$ is the velocity of the point mass.

The only potential energy component is due to gravity $g$ acting on the wheel and the point mass. This leads to:
%
\begin{align} \label{eq:energy1}
    T &= \frac{1}{2}(I_w \ \dot{\phi}^2 + m_w \dot{x}^2 +
        m_w \dot{z}^2 + m_b \dot{x}_b^2 + m_b \dot{z}_b^2) \\
    U &= m_w g z + m_b g z_b \label{eq:energy2} \\
    \mathcal{L} &= T - U \label{eq:energy3}
\end{align}

\textit{Note} I haven't written it out here, but you will obtain a system of equations, which you can solve however you want (scipy has a package to check your working). Then you can obtain the mass matrix, the coriolis matrix and the gravity vector.

\section{Conservation Laws (Noether)}
\subsection{Generic Proof (Simplified)}
A symmetry is defined as $\delta S = 0$ for a certain variation in the path. As before, we define:

\begin{align*}
    q_i'(t) &= q_i(t) + \eta_i(q(t), t) \\
    \delta q_i &= q_i'(t) - q_i(t) = \eta_i(q(t), t)
\end{align*}

for all the generalized coordinates $q_i$. Then, since the action is invariant under the symmetry, we have:

\begin{equation*}
    \delta S = S[q'] - S[q] = 0
\end{equation*}

Noether's theorem states that the quantity (Q):

\begin{equation*}
    Q = \sum_i \frac{\partial \mathcal{L}}{\partial \dot{q}_i} \eta_i
\end{equation*}

Starting with the action vanishing, we have:

\begin{align*}
    &\delta S = \sum_i \int_{t_A}^{t_B} \delta \mathcal{L}\ dt
         = \int_{t_A}^{t_B} \frac{\partial \mathcal{L}}{\partial q_i} \delta q_i + \frac{\partial \mathcal{L}}{\partial \dot{q}_i} \delta \dot{q}_i\ dt = \\
    &= \sum_i \int_{t_A}^{t_B} \frac{\partial \mathcal{L}}{\partial q_i} \eta_i + \frac{\partial \mathcal{L}}{\partial \dot{q}_i} \eta'_i \ dt = \\
    &= \sum_i \int_{t_A}^{t_B} \left[ \frac{d}{dt} \frac{\partial \mathcal{L}}{\partial \dot{q}_i} - \frac{\partial \mathcal{L}}{\partial q_i} \right] \eta_i\ dt - \left. \frac{\partial \mathcal{L}}{\partial \dot{q}_i} \eta_i \right\vert_{t_A}^{t_B}
\end{align*}

where we used the same integration by parts trick as before. Now, if the system obeys the Newton-Euler equation of motion, then the integral vanishes and we have:

\begin{align*}
    \delta S = \sum_i \left. \frac{\partial \mathcal{L}}{\partial \dot{q}_i} \eta_i \right\vert_{t_A}^{t_B} = Q(t_B) - Q(t_A) = 0
\end{align*}

Since we did not specify or impose any constraints on $t_A$ and $t_B$, this is valid for any path. Thus:

\begin{align*}
    \frac{d}{dt} \sum_i \frac{\partial \mathcal{L}}{\partial \dot{q}_i} \eta_i = 0
\end{align*}

and $Q$ is conserved as its derivative is $0$ everywhere along the path.

\subsection{Back to Physics}
First we need a couple of definitions. A \textbf{cyclic coordinate} is one that does not occur in the Lagrangian, i.e.:

\begin{equation}
    \frac{\partial \mathcal{L}}{\partial q} = 0
\end{equation}

The \textbf{conjugate momemtum} is defined as:

\begin{equation}
    p = \frac{\partial \mathcal{L}}{\partial \dot{q}}
\end{equation}

\subsubsection{Translational Invariance Leads to Conservation of Linear Momentum}
Consider a translation transformation:

\begin{equation*}
    q_i' = q_i + \epsilon_i
\end{equation*}

Then if the Lagrangian is invariant under such a transformation, the quantinity:

\begin{equation*}
    Q = \sum_i \frac{\partial \mathcal{L}}{\partial \dot{q}_i} \epsilon_i
\end{equation*}

is conserved and so is the conjugate momentum, since $\epsilon$ is a constant.

\subsubsection{Total Linear Momentum Conservation}
Consider a vector potential that only depends on the relative distance between $N$ particles $\bm{x}_i$, where $i \in [1, N]$:

\begin{equation*}
    \mathcal{L} = \sum_i \left[ m_i \frac{1}{2} \dot{\bm{x}}_i^T \dot{\bm{x}}_i - \sum_{b \neq a} V(\bm{x}_a - \bm{x}_b) \right]
\end{equation*}

Then shifting the entire system by the same vector does not change the lagrangian and the action. The velocity change is $0$ and the potential change is also $0$ as:

\begin{align*}
    \bm{x}_i' &= \bm{x}_i + \bm{\epsilon} \\
    \bm{x}_a' - \bm{x}_b' &= (\bm{x}_a + \bm{\epsilon}) - (\bm{x}_b + \bm{\epsilon}) = \bm{x}_a - \bm{x}_b
\end{align*}

which means that:

\begin{equation*}
    Q = \sum_i \frac{\partial L}{\partial \dot{\bm{x}}_i} =
        \sum_i m_i \bm{\dot{x}}_i
\end{equation*}

is conserved. This is the sum linear momentum of the system.

\subsubsection{Rotational Invariance Leads to Angular Momentum Conservation}
TODO

\section{Time-Translational Invariance Leads to Energy Convervation}

\section{Harmonic Oscillator: Hamiltonians}

\section{Gibbs Liouville}

\section{Poisson Brackets}

\section{Electromagnetic Force}
\subsection{Lorentz Force Law}
\subsection{Vector Potential}
\subsection{Gauge Invariance}

\bibliographystyle{IEEEtran}
\bibliography{IEEEabrv,IEEEconf,bib}
\end{multicols}
\end{document}


\chapter*{Conclusions and perspectives}
\label{chap_conclusion_perspectives}
This work lies in between the two research domains of turbulence and image processing. The main objective was to explore a large spectrum of approaches to estimate small-scale turbulence from given measurements at large scales only. One contribution of the thesis is to review conventional methods. We have also adapted other models inspired from recent works in image processing and proposed new methods to our context.

The estimation of small-scale turbulence from measurements has been addressed via two problems described in chapter \ref{chap_problem_definition}. The first problem is to find a relationship between large and small scales of turbulence given training samples at all scales. The second is to measure and combine complementary information, which are from space and time in this case. DNS data of an isotropic turbulence and a channel flow are used to setup different numerical experiments. These data give access to all scales of turbulence, which are used as the reference to qualify different approaches. Low-resolution fields of large scales are virtually extracted from the reference ones to which the reconstructions are compared.

For the two problems, we have proposed two families of approaches. The first one is to learn an empirical relationship between large and small scales through simple regression models or through learning coupled representations of different scales using dictionary learning. The second group of methods is based on the fusion of information. Different assumptions are exploited to propose schemes to combine available measured data. Two fusion models are developed, which use either similarity of structures in the flows or probabilistic models to find compromise estimates given the measurements.

Chapter \ref{chap_linearregression} reviewed regression models, which find mapping functions between low-resolution and high-resolution fields. The function can be linear (via a set of coefficients) or nonlinear (using fixed kernel spaces). Model performances are usually sensitive to some hyper-parameters. This problem was addressed by optimizing a bias-variance trade-off using the cross-validation technique. Comparing reconstruction results, these models work better than standard spline interpolation. Nonlinear regressions also give more accurate reconstruction than linear ones.

Chapter \ref{chap_dictionarylearning} discussed \textit{dictionary learning}, a generalization of principal component analysis (called \textit{proper orthogonal decomposition} in turbulence studies) to learn redundant bases that better represent turbulent fields in a sparse manner. A couple of representations are learned and used to reconstruct the high-resolution fields from low-resolution measurements. In the case of direct subsampling that could happen in real experiments, the aliasing problem appears. We observe that the coupled dictionary learning method does not permit to de-alias. The model has not brought significant improvements compared to spatial interpolation. Once avoiding this problem by a prefiltering step, dictionary learning gives significant improvements compared to interpolation. In the range of scale between 0.5 and 1.5 the cutoff, it reduces the energy loss by about $ 60 \% $ and the total error by about $ 15 \% $.

Chapter \ref{chap_NLM} discussed a non-local means-based propagation model as the first fusion model. The model was based on the hypothesis of \textit{rapid distortion}, which assumes that small scales are advected by large ones. These small-scale information from the LTHS planes are propagated in time based on the similarity level of large scales in space. The model works very well at low subsampling ratios in space and time (corresponding to about $ 1\% $ of energy loss) where it significantly improves the reconstruction accuracy (by about $ 40\% $) compared to single interpolation in space. However, the model is not very robust when the energy losses are more severe. With large subsampling ratios in space (for instance $ \dimsh/\dimsl =6 \times 6 $), it suffers from severe losses and can not recover completely even at large-scale. With large ratios in time (for instance $ \dimth/\dimtl = 8 $) , similarities of large scales decay rapidly and make propagation models less accurate.

To further exploit all available information from measurements, a simple yet efficient fusion model was proposed in chapter \ref{chap_BayesianFusion}. The model estimates a high-resolution field that maximizes its \textit{posterior} probability given the measurements. A Bayesian framework is used and further simplifications lead to a linear fusion formula, a weighted sum of the two single interpolations from the two measured data. Weighted coefficients are covariance matrices of unknown small scales, which are learned from measurements. The model reconstructs high-resolution fields with significant improvement compared to single interpolations from either sources. The superiority of this model is emphasized in the case where energy losses are balanced in space and time. In such cases, improvements are observed both at large and small scales.

Chapter \ref{chap_comparisons} provided a more global view of all proposed methods. We synthesized reconstructions by all approaches for the same setups where measurements in space and time are subsampled in a balanced manner. Dictionary learning is omitted in this comparison because of its slightly different configuration of numerical experiments. Single models to solve the reconstruction problem can be predefined (interpolation) or learned adaptively from the training data (regression). Adaptive models reconstruct the fields more accurately. More significant improvements are observed when combining complementary measurements in space and time. NLM-based model gives very accurate reconstructions at low energy losses. At higher subsampling ratios, this model is not able to over-perform temporal interpolation. Bayesian fusion model is simpler yet more efficient, and gives the best reconstruction in all cases.

\subsection{Suggestions for future works}
The main purpose of this work was a first exploration of a set of methods for the reconstruction of fully resolved turbulent fields from low resolution measurements. Turbulent fields are highly disordered and much less regular than natural images, therefore the inverse problem of reconstruction is even more difficult in our case. This thesis has opened many new directions to develop tools for turbulence studies. Results have suggested also some new directions for signal/image processing studies to provide even more adapted tools. The present work gives rise to the following suggestions for future works.

\subsubsection*{Extending to the reconstruction of three-component velocity fields?}
The whole thesis has dealt with the reconstruction of streamwise velocity fields. All results can be reproduced for the other two components independently. However, all components are connected physically through Navier-Stokes equations. Further investigations are needed to exploit the information such as cross-correlation between components, vorticities and divergence of the velocity fields. One idea is to impose the prior of \textit{divergence-free}. Another idea would be to use vorticities as features instead of derivatives to learn coupled dictionaries.

\subsubsection*{Ensemble of models as generalizing Bayesian fusion model?}
Chapter \ref{chap_comparisons} has compared performances of all proposed models. It has shown that different models, though exploiting different sources of information or assumptions, give better results than single interpolations. Combining multi-sources of measurements through their interpolations gives more accurate reconstructions compared to one complex single model. This observation suggests to further ensemble different models and take advantage of each single model. Regressions, by learning an adaptive interpolator, could replace spline interpolations. NLM-based propagation models by exploiting the similarity information also appear as a good candidate to replace simple interpolation.

\subsubsection*{Highly nonlinear mapping function between large and small scales of turbulence?} This thesis has reviewed regression models, but they remain rather simplistic to describe this highly nonlinear relation between scales. This is suggested when observing advantages of kernel regressions in chapter \ref{chap_comparisons}. Other ideas, especially \textit{neural network} and \textit{deep learning} \citep{dong2014image}, could be studied. Such approaches permit much more complex and highly nonlinear mapping functions between input low-resolution and output high-resolution fields, not restricted to a fixed kernel space as the KRR model. However, such a model requires an extremely large amount of training samples. The patch-wise approach is potentially a good candidate to give access to more samples and to localize the information. Such models could be combined with sparse prior \citep{wang2015deep} discussed in chapter \ref{chap_dictionarylearning} and proven to further improve reconstruction accuracy.

\subsubsection*{Dictionary learning for other inverse problems in turbulence?} As a more efficient representation compared to PCA and predefined wavelets, dictionary learning has demonstrated its possibility to solve the inverse problem of reconstructing high-resolution velocities from low-resolution measurements. The improvement is significant when aliasing is handled carefully but inoperative in the case of direct subsampling. This approach is not limited to super-resolution only. Other inverse problems such as removing noise or estimating missing pixels could be studied. One particular idea is to apply dictionary learning to D time-resolved experimental data of ``\textit{Shake-The-Box}''-PIV \citep{schroder2015advances}. By following the particles, the method resolves till pixel size. However, velocity fields in a uniform grid is estimated by simple interpolations. This could be done with dictionary learning. Viewing the fields with many missing pixels as random sensing, the approach could learn the missing small scales from the position it ``\textit{sees}'' and propose better estimates than interpolation. Also, measurements noise could be separated from the true information in the same step.

\subsubsection*{Dictionary learning to deal with aliasing from direct subsampling?} Aliasing is a known problem when the sensing system directly subsamples the fields without the prefiltering step. The first attempt of dictionary learning to reconstruct the high-resolution fields from the subsampled measurements has failed potentially due to aliasing. Without its presence, the model gives clear improvements compared to single interpolation. The problem of removing aliasing could be addressed and solved using dictionary learning as well. Coupled dictionaries could be learned to represent the fields with and without aliasing. The model in chapter \ref{chap_dictionarylearning} would become a three-stage super-resolution model, with an intermediate step to remove aliasing.

\subsubsection*{A more complete fusion model, a Bayesian way?} The fusion model proposed in this work uses only a very simple weighted sum formula. The prior of the estimated field is omitted from the formula due to the assumption of a non-informative prior. Full covariance matrices to represent all sources of space-time correlations are also simplified to diagonal ones. Some work remains to be done to further exploit the information from measurements by building more complete covariance matrices. Also, a good prior could further improve further the reconstruction. The prior could carry physical properties of the flow, for example divergence free, energy spectra or even full Navier-Stokes equations. In case the full fields are given as training data, covariance matrices could also be learned adaptively. In such cases, the weights of the fusion model could be learned to take the properties of turbulence into account.

\subsubsection*{Combining fusion and dictionary learning?} Dictionary learning has shown to be a good representation of turbulent fields using a sparsity prior. This prior helps in solving the ill-posed problem of reconstructing high-resolution fields. Fusion models by combining multi-sources of measurements are simple but also very efficient to solve the problem. The idea of combining sparse prior and fusion could be studied, inspired by \citet{wei2015hyperspectral}. In chapter \ref{chap_BayesianFusion}, the optimization problem is:
\begin{equation}
	\z = \argmin_{\z} \left\lbrace \frac{1}{2} \Mdist{\z -\Interp_t\x}{\Sigma_{\h_t}} + \frac{1}{2}\Mdist{\z -\Interp_s\y}{\Sigma_{\h_s}} \right\rbrace
\end{equation}
By imposing the sparse prior, the problem could be rewriten as:
\begin{equation}
	\z = \dict \dictco \subjectto \{\dict,\dictco\} = \argmin_{\dict,\dictco} \left\lbrace \frac{1}{2} \Mdist{\dict \dictco -\Interp_t\x}{\Sigma_{\h_t}} + \frac{1}{2}\Mdist{\dict\dictco -\Interp_s\y}{\Sigma_{\h_s}} + \lambda \normone{\dictco} \right\rbrace
\end{equation}
or avoiding the interpolations, hence not bringing aliasing terms into the system, as:
\begin{equation}
		\z = \dict \dictco \subjectto \{\dict,\dictco\} = \argmin_{\dict,\dictco} \left\lbrace \frac{1}{2} \Mdist{\Sub_t\dict \dictco -\x}{\Sigma_{\h_t}} + \frac{1}{2}\Mdist{\Sub_s\dict\dictco -\y}{\Sigma_{\h_s}} + \lambda \normone{\dictco} \right\rbrace
\end{equation}
Solving the above highly non-convex problem could be addressed thanks to recent progresses in solving alternating optimization problems \citep{boyd2011distributed,parikh2014proximal}.

\subsubsection*{Increase resolution of PIV measurements using different setups?} Regression and interpolation are restricted to reconstruct high-resolution fields from low-resolution ones at the same scene, while it is not the case for coupled dictionaries learning. One very promising setup would be to combine high-resolution PIV with time-resolved PIV (Tr-PIV). High-resolution PIV can measure the flow at a small field-of-view but at very high spatial resolution. These measurements could be used to train the dictionaries. Then Tr-PIV measures the flow at a much larger field-of-view but at lower resolution. Using the trained dictionaries, one could estimate time-resolved velocity fields at large field-of-view and high spatial resolution. One should notice also the limitation of such an approach in de-aliasing to carefully design the sensing system. Another configuration could be to use two different PIV systems to measure HTLS and LTHS fields (see figure \ref{fig:space-time_measurements}) and apply fusion models to maximize the level of useful information.

\subsubsection*{Co-conception design of experiments?} The thesis has studied the performances of various models for different configurations at various subsampling ratios. This gives an idea of which method to choose for one particular setup. From a certain loss of energy due to subsamplings, this work suggests the maximum level of accuracy and scale one could expect to reconstruct. This information is very useful to design new challenging experiments in order to maximize the expected level of small scale content after post-processing. This thesis connects to the current research area of co-conception. The idea is to co-design the measurement system with respect to some pre-defined post-processing procedure.