A wide variety of physical phenomena are modeled in the space-time domain by partial differential equations. The purpose of this section is to review generalities about PDEs and suited strategies depending on the nature of the equations so that solutions can be derived.
\subsection{General concepts}
A system of partial differential equations can be written by means of a vector operator $\vect{\Gc}$ of independent and dependent variables $(x_1,...,x_N)$ and $(u_1,...,u_I)$:
\begin{equation}
  \label{eq:diff_operator}
  \vect{\Gc}\(x_1,...,x_N,u_1,...,u_I,\drond{u_1}{x_1},..., \drond{^Mu_I}{x_N^M}\) = \vect{0}
\end{equation}
The dimension of the system is given by the size $I$ of the array $\vect{\Uc}^T=[u_1,...,u_I] \in \Rbb^I$, referred to as the \textit{unknown vector}. The highest derivative of the unknown vector in the system defines the \textit{order of the system} $M$. In equation \eqref{eq:diff_operator} and in what follows, sans-serif symbols refer to matrices while calligraphic symbols stand for column arrays. Furthermore, the partial derivatives of a quantity $u$ with respect to a variable $x$ may be written $u_x$ when there is no ambiguity. Making use of index notation and the convention of implicit summation over repeated indices, a system of partial differential equations reads:
\begin{equation*}
   \sum_{k=1}^{N} \sum_{p=1}^{M}\Asf_{ij}^p \drond{^p\Uc_j}{x_k^p} + \Sc_i = 0
\end{equation*}
or equivalently, in matrix form:
\begin{equation}
  \label{eq:diff_system_matrix}
  \sum_{k=1}^{N}\sum_{p=1}^{M}\tens{\Asf}^p \drond{^p\vect{\Uc}}{x_k^p} + \vect{\Sc} =  \vect{0}
\end{equation}
Coefficients matrices $\tens{\Asf}^p$ and the vector $\vect{\Sc}$ may depend on independent variables and the unknown vector ($x_1,...,x_N,\vect{\Uc}$) leading to different types of partial differential systems. Namely, whether those terms are functions of the $x_k$ or not leads respectively to \textit{linear systems with variable coefficients} or to \textit{linear systems with constant coefficients}. The system remains \textit{linear} if $\vect{\Sc}$ depends linearly on $\vect{\Uc}$, and is \textit{semi-linear} if the relation is non-linear. Finally, if $\tens{\Asf}^p$ depends on the vector $\vect{\Uc}$ and its derivatives up to order $M-1$, the system is called \textit{quasi-linear}.

The \textit{Cauchy problem} consists in finding a solution $\vect{\Uc}$ of system \eqref{eq:diff_system_matrix} that satisfies a set of given prescribed values. Geometrically speaking, the solution of such a problem can be seen as the building of a hyper-surface of $\Rbb^{I+N}$, hence the term of \textit{integral surface} for $\vect{\Uc}$. Such a problem can be reduced to that of solving a first-order Cauchy problem by using suitable changes of variables \cite[p.54]{PDEs}, we will therefore focus on first-order PDEs.

\subsection{Notion of characteristics -- Hyperbolic problems}
The theorem of \textit{Cauchy--Kowalewski} locally ensures the existence of solutions of a Cauchy problem for partial differential systems and is based on the restrictive requirement of analytic coefficient matrices and initial data (see \cite[p.46]{PDEs}). The case of first-order systems however, only requires continuity and differentiability conditions and is based on the concept of \textit{characteristics}, which makes the development of a solution more flexible.

\subsubsection*{First-order quasi-linear equations}
To illustrate the aforementioned notions, we consider the first-order quasi-linear PDE with independent variables $x$ and $t$:
\begin{equation}
  \label{eq:1st_order_pde}
   a u_x + b u_t  = c
\end{equation}
where coefficients $a$ and $b$ are such that $a^2 + b^2 \neq 0$. Given values of $u$ are prescribed along a curve defined by means of a parameter $\eta$ in the $(x,t)$ plane as $\Cscr_0:(x(\eta),t(\eta))$, so that $u(x(\eta),t(\eta))$ draws a curve $\Cscr$ in the space $(x,t,u)$.
%In what follows, $\Cscr$ is referred to as the \textit{initial curve}.
We assume that $\Cscr_0$ is regular, namely $\ddroit{x}{\eta}^2 + \ddroit{t}{\eta}^2 \neq 0$, and that one of the derivatives, say $\ddroit{t}{\eta}$, does not vanish. Figure \ref{fig:initial_curve} shows an example of such an \textit{initial curve} $\Cscr$ prescribing values of $u$ along a parametrized curve of the $(x,t)$ plane.
%The value of $u$ on $\Cscr$ being used as a starting point for the solution of the Cauchy problem, $\Cscr$ is referred to as the \textit{initial curve}.
\begin{figure}[h]
  \centering
  \input{chapter2/pgfFigures/initialCurve}
  \caption{Example of initial curve $\Cscr$ in the $(x,t,u)$ space and its projection $\Cscr_0$ in the $(x,t)$ plane.}
  \label{fig:initial_curve}
\end{figure}
With data given along $\Cscr$, the Cauchy problem is equivalent to that of finding a surface $u(x,t)$ that contains the initial curve and satisfies \eqref{eq:1st_order_pde}.
Thus, one seeks the partial derivatives $u_x$ and $u_t$ of $u$ on $\Cscr$ in order to extend the given data into a strip in the neighborhood of the initial curve.
The total derivative of $u$ along $\Cscr$ being:
\begin{equation}
  \label{eq:dtot_u}
  \ddroit{u}{\eta}=u_x\ddroit{x}{\eta}+u_t\ddroit{t}{\eta}
\end{equation}
one gets, after multiplying the previous equation by $d\eta$, the following relation between the partial derivatives of $u$ on the initial curve:
\begin{equation*}
  u_t= \ddroit{u}{t} - u_x\ddroit{x}{t}
\end{equation*}
Then, equation \eqref{eq:1st_order_pde} can be rewritten as:
\begin{equation}
  \label{eq:normal_form_pde}
  (a - b\ddroit{x}{t})u_x = c - b\left.\ddroit{u}{t}\right|_{t\in\Cscr_0}
\end{equation}
The right-hand side of equation \eqref{eq:normal_form_pde} is known along $\Cscr$ so that the Cauchy problem admits a unique solution $u_x$ if and only if:
\begin{equation}
  \label{eq:non-characteristics}
  \ddroit{x}{t}\neq \frac{a}{b}
\end{equation}
An initial curve satisfying the condition \eqref{eq:non-characteristics} is a \textit{non-characteristic curve} and enables to uniquely determine a solution of the Cauchy problem. On the other hand, an initial curve defined such that $\ddroit{x}{t} = \frac{a}{b}$ is a \textit{characteristic curve} and yields infinitely many solutions \cite[p.65]{Courant}.

%A \textit{non-characteristic curve} is an initial curve satisfying the condition \eqref{eq:non-characteristics}, otherwise it is a \textit{characteristic curve}.

\subsubsection*{Geometrical representation of characteristic curves}
Consider a partial differential equation of the form \eqref{eq:1st_order_pde}, and prescribed values of $u$ along a characteristic curve $\Cscr$. Since one cannot find a unique solution of the Cauchy problem in this case, infinitely many integral surfaces $u^{(i)}(x,t)$ with normal vectors $\vect{n}^{(i)}=[u^{(i)}_x,u^{(i)}_t,-1]$ can intersect $\Cscr$. Those integral surfaces satisfy equation \eqref{eq:1st_order_pde} and hence, $\vect{n}^{(i)}\cdot \vect{w}=0$ where $\vect{w}=[a,b,c]$, so that the set of tangent planes to solutions $u^{(i)}(x,t)$ forms a fan whose axis is $\vect{w}$.
This situation is depicted in figure \ref{fig:plan_fan} for which an initial characteristic straight line $\Cscr$ is contained by integral surfaces satisfying a PDE of the form \eqref{eq:1st_order_pde}.
%Let $u(x,t)$ be a surface in the $(x,t,u)$ space which normal vector is $\vect{n}=[u_x,u_t,-1]$. This surface is an integral surface if it is a solution of equation \eqref{eq:1st_order_pde} and hence, if $\vect{n}\cdot \vect{w}=0$ with $\vect{w}=[a,b,c]$.
%This situation is depicted in figure \ref{fig:plan_fan} for which an initial characteristic straight line $\Cscr$ is contained by integral surfaces satisfying some pde of the form \eqref{eq:1st_order_pde}.
% Figure \ref{fig:plan_fan} shows a collection of integral surfaces for some partial differential equation \eqref{eq:1st_order_pde} with prescribed $u$ along a straight line $\Cscr$.
% Then, the set of tangent planes to those solutions $u^{(i)}(x,t)$ forms a fan of planes which axis is $\vect{w}$.
\begin{figure}[h!]
  \centering
  \input{chapter2/pgfFigures/plan_fan}
  \caption{Examples of integral surfaces passing through the same curve $\Cscr$ defined such that $t=\mathrm{constant}$ and $u=\mathrm{constant}$ along $\Cscr$.}
  \label{fig:plan_fan}
\end{figure}

$\newline$
\textit{Characteristic line elements}, tangent to all integral surfaces $u^{(i)}(x,t)$ are then defined as:
\begin{equation}
  \label{eq:monge_axis}
  \matrice{dx \\ dt \\ du} = \matrice{a \\ b \\c}
\end{equation}
Introduction of a parameter $\eta$ and integration of equation \eqref{eq:monge_axis} yield a one-parameter family of characteristic curves of the PDE:
\begin{equation*}
  x=x(\eta) \quad ; \quad t=t(\eta) \quad ; \quad u=u(\eta)
\end{equation*}
Hence, a characteristic curve is tangent at every point to all the integral surfaces, and an infinity of integral surfaces cross one characteristic curve. As a consequence, if the initial curve is a characteristic curve, infinitely many integral surfaces contain it so that the Cauchy problem cannot be solved.
However, the following statement holds \cite[p.63]{Courant}:
\begin{theorem}[Courant]
  \label{th:integral_surface_generated}
  Every surface $u(x,t)$ generated by a one-parameter family of characteristic curves is an integral surface.

  Conversely, every integral surface is generated by a one-parameter family of characteristic curves.
\end{theorem}
This theorem will be used in what follows to solve the Cauchy problem.

%\subsection{The method of characteristic}
\subsubsection*{First-order quasi-linear systems}
The concept of characteristic curves is now extended to first-order quasi-linear systems of dimension $I$.
Consider the following system written in matrix form:
\begin{equation}
  \label{eq:1st_order_quasi-linear_syst}
  \Absf^t\(x,t,\vect{\Uc}\) \: \vect{\Uc}_t + \Absf^x\(x,t,\vect{\Uc}\)\: \vect{\Uc}_x + \vect{\Sc} = \vect{0}
\end{equation}
Similarly to quasi-linear PDEs, given values of $\Ucb$ are prescribed along a regular curve $\Cscr_0:(x(\eta),t(\eta))$ defining an initial curve $\Ucb(x(\eta),t(\eta))$ of the $(x,t,\Ucb)$ space. The Cauchy problem consists in finding all the derivatives of $\Ucb(x,t)$ such that equation \eqref{eq:1st_order_quasi-linear_syst} is satisfied in the vicinity of $\Cscr$.
Making use of the total derivative of $\Ucb$ along the initial curve:
\begin{equation}
  \label{eq:dtot_Uvect}
  \ddroit{\Ucb}{\eta}=\Ucb_x\ddroit{x}{\eta}+\Ucb_t\ddroit{t}{\eta} \quad \rightarrow \quad \Ucb_t = \left.\ddroit{\Ucb}{t}\right|_{t\in\Cscr_0} - \Ucb_x  \ddroit{x}{t}
\end{equation}
system \eqref{eq:1st_order_quasi-linear_syst} can be rewritten:
\begin{equation}
  \label{eq:normal_form}
  \( \Absf^x - \lambda \Absf^t \) \vect{\Uc}_x + \Scb + \Absf^t\left.\ddroit{\Ucb}{t}\right|_{t\in\Cscr_0} = \vect{0}
\end{equation}
where:
\begin{equation}
  \label{eq:lambda_slope}
  \lambda=\ddroit{x}{t}
\end{equation}
With $\Scb$ and $\Absf^t\left.\ddroit{\Ucb}{t}\right|_{t\in\Cscr_0}$ known along $\Cscr_0$, the Cauchy problem admits a unique solution $\vect{\Uc}_x$ along $\Cscr$ if the determinant of the system does not vanish, that is:
\begin{equation}
  \label{eq:characteristic_determinant}
  D=\abs{\Absf^x - \lambda \Absf^t} \ne 0
\end{equation}
where D is called the \textit{characteristic determinant} of system \eqref{eq:1st_order_quasi-linear_syst}. If D does not have real roots along $\Cscr_0$, the problem is said \textit{elliptic} and the Cauchy problem can be solved. Indeed, in that case the knowledge of $\Ucb$ along the initial curve allows the computation of derivatives and hence, the building of an integral strip defined by $\Ucb,\Ucb_x,\Ucb_t$. If the characteristic determinant admits $I$ real roots on the other hand, system \eqref{eq:normal_form} can no longer be solved. Those eigenvalues come along with left and right eigenvectors respectively defined as:
\begin{equation}
  \label{eq:eigenvectors}
  \Lc^k_i  \Asf^x_{ij} = \lambda_k \Lc^k_i \Asf^t_{ij} \quad ; \quad \Asf^x_{ij}\Rc^k_j = \lambda_k \Asf^t_{ij}\Rc^k_j \qquad k=1,...,I
\end{equation}
\begin{remark}
  Note that eigenvectors can be stored as matrices $\Rbsf$ and $\Lbsf$ where $\Rsf_{ij}=\Rc^j_i$ and $\Lsf_{ij}=\Lc_j^i$.
\end{remark}

\begin{definition}
  \label{def:hyperbolic_system}
  A first-order system of $I$ partial differential equations is said \textbf{hyperbolic} if it admits real eigenvalues and $I$ independent eigenvectors \cite{Courant}.
For those problems, one can draw a set of one-parameter families of characteristic curves $\varphi^k$ in the $(x,t)$ plane by integrating the relations $\lambda_k=dx/dt$ ($1 \leq k \leq I$).
\end{definition}

\begin{example}
  \label{ex:charac1}
  Consider the first-order system with variable coefficients
\begin{equation*}
 \matrice{x &0 \\0 &-x} \drond{}{t} \matrice{\Uc_1 \\ \Uc_2} + \drond{}{x}\matrice{\Uc_1 \\ \Uc_2} = \matrice{0 \\0}
\end{equation*}
whose characteristic determinant \eqref{eq:characteristic_determinant} is:
\begin{equation*}
  (1-\lambda x)(1+\lambda x)=0
\end{equation*}
We thus have two solutions $\lambda_{1,2}=\pm 1/x$ leading, by integration of \eqref{eq:lambda_slope}, to two one-parameter families of characteristic curves:
\begin{equation*}
  t_1(x)=\frac{1}{2}x^2+c_1  \quad \text{and} \quad t_2(x)=-\frac{1}{2}x^2+c_2
\end{equation*}
Those curves are drawn in figure \ref{fig:exampleCharac}\subref{subfig:curve_lines} for several values of integration constants $c_1$ and $c_2$.
\end{example}
\begin{example}
  \label{ex:charac2}
  Consider now the first-order system with constant coefficients
\begin{equation*}
 \matrice{1 &0 \\0 &2} \drond{}{t} \matrice{\Uc_1 \\ \Uc_2} + \drond{}{x}\matrice{\Uc_1 \\ \Uc_2} = \matrice{0 \\0}
\end{equation*}
whose eigenvalues, according to equation \eqref{eq:characteristic_determinant} satisfy
\begin{equation*}
  (1 - \lambda )(1- 2\lambda)=0
\end{equation*}
Two real roots exist $\lambda_1=1 \: ; \: \lambda_2=1/2$, leading by integration of \eqref{eq:lambda_slope} to two one-parameter families of straight lines:
\begin{equation*}
  t_1(x)=x+c_1  \quad \text{and} \quad t_2(x)=2x+c_2
\end{equation*}
Unlike example \ref{ex:charac1}, coefficient matrices do not depend on independent variables, thus yielding characteristic straight lines in the $(x,t)$ plane (see \ref{fig:exampleCharac}\subref{subfig:straight_lines}).
\end{example}
\begin{figure}[h]
  \centering
  \subcaptionbox{Example \ref{ex:charac1}: $\lambda_{1,2}=\pm 1/x$\label{subfig:curve_lines}}{\input{chapter2/pgfFigures/exampleCharac}}
  \subcaptionbox{Example \ref{ex:charac2}: $\lambda_{1}=1 \:\text{and} \: \lambda_2=1/2$\label{subfig:straight_lines}}{\input{chapter2/pgfFigures/exampleCharac_const}}
  \caption{Family of characteristic curves corresponding to the eigenvalues of the first-order systems given in examples \ref{ex:charac1} and \ref{ex:charac2}.}
  \label{fig:exampleCharac}
\end{figure}

\subsection{The method of characteristics}
As theorem \ref{th:integral_surface_generated} states, an integral surface is generated by a one-parameter family of characteristic curves. Therefore the knowledge of those curves enables the building of the solution of the Cauchy problem. Indeed, the projection of the quasi-linear system \eqref{eq:1st_order_quasi-linear_syst} onto the \textit{left eigenbasis} or \textit{left characteristic basis} leads to:
\begin{equation*}
  \vect{\Lc}^k \( \Absf^t \vect{\Uc}_t + \Absf^x\vect{\Uc}_x \) + \vect{\Lc}^k \vect{\Sc}= \vect{0}
\end{equation*}
%Introduction of the definition of left eigenvectors \eqref{eq:eigenvectors} then yields:
where $\Lcb^k$ satisfies \eqref{eq:eigenvectors}, and hence:
\begin{equation*}
  \vect{\Lc}^k  \Absf^t \( \vect{\Uc}_t +\lambda_k \vect{\Uc}_x   \) + \vect{\Lc}^k \vect{\Sc}=\vect{0}
\end{equation*}
In this equation, the \textit{directional derivative} of $\vect{\Uc}$ along the $k$th characteristic curve $\varphi^k$ arises, namely:
\begin{equation*}
 \left.\ddroit{\Ucb}{t}\right|_{t\in\varphi^k} = \Ucb_t + \lambda_k \Ucb_x
\end{equation*}
Thus, along a characteristic curve a system of partial differential equations reduces to a system of \textit{Ordinary Differential Equations} (ODEs) composed of the following \textit{characteristic equations}:
\begin{equation}
  \label{eq:PDEs_ODEs}
  \vect{\Lc}^k  \(\Absf^t \ddroit{\Ucb}{t} + \Scb \)=\vect{0}
\end{equation}
Integration of equations \eqref{eq:PDEs_ODEs} yields a set of \textit{integral curves} from which the Cauchy problem can be solved.
%It then comes out that the Cauchy problem can be solved as the system \eqref{eq:PDEs_ODEs}.
Indeed, the solution at a point of the $(x,t)$ plane can be determined by tracing backward the characteristic curves to the initial curve and integrating ODEs \eqref{eq:PDEs_ODEs} along those paths according to the \textit{method of characteristics}.
Note that if $\Scb$ is zero, then $\Ucb$ is constant along characteristic curves.

To illustrate the method, let us consider again the quasi-linear system of example \ref{ex:charac1} for which the Cauchy problem is built by prescribing initial conditions along the $x$-axis. Note that "initial conditions" have now a physical meaning since they are defined at $t=0$, the Cauchy problem is then an \textit{Initial Value Problem (IVP)}. Through a point $(x^*,t^*)$ pass two characteristic curves, each belonging to a different one-parameter family. The solution at this point can be determined by integrating the ODE corresponding to the first (\textit{resp. second}) eigenvalue of the system between $(x^1,0)$ (\textit{resp. $(x^2,0)$}) and $(x^*,t^*)$. The singularity of hyperbolic problems can hence be circumvented by using the characteristic structure in order to determine a unique solution.
\begin{figure}[h]
  \centering
  \input{chapter2/pgfFigures/characMethod2x2}
  \caption{Domain of dependence of the solution at $(x^*,t^*)$ for the system of example \ref{ex:charac1}.}
  \label{fig:charac_method2x2}
\end{figure}
We see that only a segment of the initial curve has an influence on the solution at a given point. Namely, the intersections of the initial curve and characteristic curves with the highest and the lowest slopes define the \textit{domain of dependence} of the solution at this point (see figure \ref{fig:charac_method2x2}). This property of hyperbolic problems implies the existence of waves that propagate information at finite speeds corresponding to the eigenvalues of a quasi-linear form. The theory presented so far will be applied in what follows to solid mechanics.

%%% Local Variables:
%%% mode: latex
%%% ispell-local-dictionary: "american"
%%% TeX-master: "../mainManuscript"
%%% End:


\section{Sparse Coding}

\subsection*{Orthogonal Basis}
For $x$ and orthogonal $U = (u_1|..|u_{D}) \in \mathbb{R}^{DxD}$.
Encoding $\mathbf{z = U^\top x}$.
Decoding $\mathbf{x = U z = UU^\top x  = x }$.
Tresholding $ \mathbf{\hat{x}} = \mathbf{U\hat{z}}$, $\hat{z}_i = z_i$ if $ \lvert z_i \rvert > \epsilon$ else 0.
Reconstr. Error $\|\mathbf{x}-\mathbf{\hat{x}}\|^2 = \|\sum_{d\notin\sigma}\langle\mathbf{x},\mathbf{u}_d\rangle \mathbf{u_d} \|^2 =  \sum_{d\notin\sigma}\langle\mathbf{x},\mathbf{u}_d\rangle ^2$. Proof: $\|\sum x_i \|^2 = \sum \|x_i \|^2$ if $x_i$ orthogonal. Advantages: efficient inverse, energy/length preservation. \\
\textbf{Haar wavelets:} \\
scaling: $[1,1,1,1]$, mother wavelet: $[1,1,-1,-1]$, dilated: $[1,-1,0,0]$, translated: $[0,0,1,-1]$.

\textbf{Fourier vs Wavelets:}\\
\textbullet Fourier: good for periodic signals, global support, no time info/only frequencies themselves. \\
\textbullet Wavelets: good for localized signals like abrupt changes/irregularities, represents a signal in time and frequency domain.

\subsection*{Overcomplete Basis}
$\mathbf{U} \in \mathbb{R}^{D \times  L}$ for \# atoms $ = L > D = \mathsf{dim}\text{(data)}$. $x \in \mathbb{R}^D, z \in \mathbb{R}^L$ because $Uz=x$.
Encoding ill-posed problem $\rightarrow$ add constraint $\mathbf{z}^\star \in \argmin_\mathbf{z} \lVert \mathbf{z} \rVert_0$ s.t. $\mathbf{x} = \mathbf{Uz}$. NP-hard and non-convex \\ $\rightarrow$ approx with Basis pursuit i.e relax to 1-norm (convex) or with Matching Pursuit.

\textbf{Coherence} \\
$m(\mathbf{U}) = \max_{i,j:\, i \neq j} | \mathbf{u}_i^\top \mathbf{u}_j |$ \\
\textbf{Matching Pursuit (MP)}
a greedy approximation algorithm.
Objective: $\mathbf{z}^\star \in \argmin_{\mathbf{z}} \|\mathbf{x} - \mathbf{Uz} \|_2$, s.t. $\|\mathbf{z}\|_0 \leq K$
\begin{inparaenum}[\color{red}1.]
	\item init: $z \leftarrow 0, r \leftarrow x$
	\item while $\|\mathbf{z}\|_0 < K$ do
	\item select atom with max correlation $d^\star = \argmax_d |\langle \mathbf{u}_d, \mathbf{r} \rangle|$
	\item update coefficients: $z_{d^\star} \leftarrow z_{d^\star} + \langle \mathbf{u}_{d^\star}, \mathbf{r} \rangle$
	\item update residual: $\mathbf{r} \leftarrow \mathbf{r} - \langle \mathbf{u}_{d^\star}, \mathbf{r} \rangle \mathbf{u}_{d^\star}$.
\end{inparaenum}

\textbf{Compressive Sensing}: Compress data while gathering:
\begin{inparaitem}[\color{red}\textbullet]
  \item $\mathbf{x} \in \mathbb{R}^D$, $K$-sparse in o.n.b. $\mathbf{U}$. $\mathbf{y} \in \mathbb{R}^M$ corresponds to $M$ lin. combinations/measurements of signal; $\mathbf{y} = \mathbf{Wx} = \mathbf{WUz} = \mathbf{\theta z}$, $\theta \in \mathbb{R}^{M \times D}$
  \item Reconstruct $\mathbf{x} \in \mathbb{R}^D$ from $\mathbf{y}$; find $\mathbf{z}^\star \in \argmin_{\mathbf{z}}\|\mathbf{z}\|_0$, s.t. $\mathbf{y} = \mathbf{\theta z}$ (e.g. with MP). Given $\mathbf{z}$, reconstruct $\mathbf{x}$ via $\mathbf{x} = \mathbf{Uz}$
\end{inparaitem}
\\Sufficient conditions:
\begin{inparaitem}[\color{red}\textbullet]
  \item $\mathbf{W} = $ Gaussian random projection, i.e. $w_{ij}\sim\mathcal{N}(0, \frac{1}{D})$
  \item M $\geq cK log(\frac{D}{K})$, where $c$ is some constant
\end{inparaitem}

\subsection*{Dictionary Learning}
Adapt the dictionary to signal characteristics. Objective: $(\mathbf{U}^\star, \mathbf{Z}^\star) \in \argmin_\mathbf{U,Z} \| \mathbf{X} - \mathbf{U} \cdot \mathbf{Z} \|_F^2$ \\
\textbf{K-SVD (Iter Greedy Minimization):}
\begin{inparaenum}[\color{red} 1.]
  \item Coding step: $\mathbf{Z}^{t+1} \in \argmin_\mathbf{Z} \| \mathbf{X} - \mathbf{U}^t \mathbf{Z} \|_F^2 = \sum_{i=1}^N \|x_i-U^tz_i \|^2$ s.t $\|z_i\|_0 \leq K$. Use any pursuit algorithm.
  \item Dict update step: $\mathbf{U}^{t+1} \in \argmin_\mathbf{U} \| \mathbf{X} - \mathbf{UZ}^{t+1} \|_F^2$, s.t $\forall l\in [L]:\|\mathbf{u}_l\|_2 = 1$.
   idea: update one atom $u_l$ at a time. $\min_{u_l}\|\mathbf{X} - \mathbf{U}\mathbf{Z}\|_F^2 = \min_{u_l}\|X-(\sum_{e \neq l}u_ez_e^\top + u_lz_l^\top)\|_F^2= \min_{u_l}\|\mathbf{R}_l - \mathbf{u}_l(\mathbf{z}_l)^\top\|_F^2$ where $z_l$ is the $l$-th row of matrix $Z$ and $R_l$ is the residual due to atom $u_l$. Use SVD $\mathbf{R}_l = \sum_i \sigma_i \tilde{u}_i \tilde{v}_i^\top$ then $\mathbf{u}^*_l=\tilde{\mathbf{u}}_1$ and $\mathbf{z}^*_l=\tilde{\mathbf{v}}_1$ (use power iteration for efficiency).
\end{inparaenum}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% UMB-CS240-2016S: Programming in C
% Copyright 2016 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/UMB-CS240-2016S
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 1}

To check whether a given non-negative integer $n$ is a prime, it suffices to check if it is divisible by at least one of the elements in $\mathbb{S}_n =  \{ k_i \in \mathbb{P} | k_i < \sqrt{n} \}$.
If $\mathbb{S}_n$ is known, it takes $|\mathbb{S}_n|$ operations to check for primality of $n$.
This is much better than when $\mathbb{S}_n$ is not known in which case it takes $\mathbb{O}(n \log \log n)$ operations to construct it.

Suppose we can store $\mathbb{P} = \mathbb{S}_n$, as set of all \textit{known} prime numbers, in a file and access it when testing primality of another non-negative integer $m$.
This way, if $1 << m < n$, primality testing of $m$ would be trivial.
On the other hand, if $1 << n < m$, $\mathbb{S}_n$ would still be very useful in constructing $\mathbb{S}_m$ and required operations for prime-checking $m$ will be reduced by $c_1 n \log \log n + c_2$ operations where $c_1$ and $c_2$ are positive integers.
In this case, once $m$ is prime-checked, we can update the list of \textit{known} prime numbers to $\mathbb{S}_m$ for later use.

You are asked to write a program \texttt{fast-prime.c} that accepts a set of non-negative integer numbers as command line arguments and, using the above-mentioned algorithm, checks for their primality.

You may not include any header file other than \texttt{fast-prime.h}.
In addition, your \texttt{fast-prime.c} file should accompany a \texttt{Makefile} to facilitate building process and artifact removal.

Following illustrates the expected functionality of your program.

\newpage

\begin{terminal}
$ ls
fast-prime.c fast-prime.h Makefile
$ make
gcc -c -o fast-prime.o fast-prime.c
gcc -o fast-prime fast-prime.o -Werror -Wall -std=gnu99 -I.
$ ./fast-prime
error: missing command line argument
$ ./fast-prime 6
6     : not prime
$ ls
fast-prime fast-prime.c fast-prime.h fast-prime.o
Makefile primes.log
$ cat primes.log
2
3
5
$ ./fast-prime 25 37 39 29
25    : not prime
37    : prime
39    : not prime
29    : prime
$ ls
fast-prime fast-prime.c fast-prime.h fast-prime.o Makefile primes.log
$ cat primes.log
2
3
5
7
11
13
17
19
23
29
31
37
$ make clean
rm -rf fast-prime
rm -rf primes.log
find . -name '*.o' -delete
\end{terminal}