\documentclass[main.tex]{subfiles}

\begin{document}
	\section{Reciprocal lattice and scattering} \label{sec:scattering}
	\subsection{Theory}
	The reciprocal lattice is an incredibly useful construct, as it allows for an easy description of wave phenomena, where the main variable is the wave vector $ \V{k} $. The reciprocal lattice is a lattice (as defined in the previous section), but in reciprocal space. This again means that the free variable is not position, but the wave vector.

	The definition of a reciprocal lattice is all of the points $ \V{G} $, that satisfy
	\begin{equation}\label{eq:reciprocal_lattice}
		e^{i \V{G} \D \V{R}} = 1,
	\end{equation}
	for any lattice point in real space $ \V{R} $. Further, the primitive lattice vectors of the reciprocal lattice all satisfy the property \cite{kittel}
	\begin{equation}\label{eq:rec_prim_def}
		\V{a}_i \D \V{b}_j = 2\pi \delta_{ij},
	\end{equation}
	where $ \delta_{ij} $ is the familiar Kronecker delta. This property of the reciprocal lattice vectors is what makes $ \V{G} $ a lattice in reciprocal space. The real space lattice points have the coordinates $ \V{R} = n_1 \V{a}_1 + n_2 \V{a}_2 + n_3 \V{a}_3 $, and say an arbitrary point in the reciprocal space is constructed as
	\begin{equation}\label{eq:rec_lat_vec}
		\V{G} = m_1 \V{b}_1 + m_2 \V{b}_2 + m_3 \V{b}_3,
	\end{equation}
	then plugging these into Eq. \eqref{eq:reciprocal_lattice} we get
	\begin{equation}
		e^{i (n_1 \V{a}_1 + n_2 \V{a}_2 + n_3 \V{a}_3) \D (m_1 \V{b}_1 + m_2 \V{b}_2 + m_3 \V{b}_3)} = e^{2\pi i (n_1 m_1 + n_2 m_2 + n_3 m_3)},
	\end{equation}
	where the second equality follows from the defining property of the primitive lattice vectors in reciprocal space. For $ \V{G} $ to be part of the reciprocal lattice, for any choice of integer $ n_i $, then $ m_i $ need also be integers. As such the reciprocal lattice really is a lattice, as defined in section \ref{sec:lattice_theory}, but in the reciprocal space. Usually (in the case of lattice planes and scattering) we express the 3 coefficients for the reciprocal lattice vectors using Miller indices ($ hkl $), where $ h=m_1, k=m_2, l=m_3 $.

	Now, to actually construct the reciprocal primitive lattice vectors, the following formulas are used:
	\begin{equation}\label{eq:rec_prim_vec}
		\V{b}_1 = \frac{2\pi \, \V{a}_2 \times \V{a}_3}{\V{a}_1 \D (\V{a}_2 \times \V{a}_3)}, \quad \V{b}_2 = \frac{2\pi \, \V{a}_3 \times \V{a}_1}{\V{a}_2 \D (\V{a}_3 \times \V{a}_1)}, \quad \V{b}_3 = \frac{2\pi \, \V{a}_1 \times \V{a}_2}{\V{a}_3 \D (\V{a}_1 \times \V{a}_2)},
	\end{equation}
	where the cross product in the numerator ensures that $ \V{a}_i $ and $ \V{b}_j $ are orthogonal to each other for $ i \neq j $, and the denominator serves as a sort of normalization, such that the dot product equals 2$ \pi $. Further, by dimensional analysis, $ \V{b}_i $ has dimensions $ \e{m}\inverse $, as expected for wave vectors.

	The reciprocal lattice can also be understood in terms of families of lattice planes. First, a lattice plane is defined as any plane that contains at least 3 lattice points (this also guarantees that an infinite set of lattice points is contained by the plane, by virtue of the non-uniqueness of the primitive lattice vectors). A family of lattice planes is an infinite series of parallel planes, equally spaced, such that all points of the lattice is contained in the planes, and that all planes contain lattice points.

	First consider the series of planes containing the points $ \V{r}_m $, defined by
	\begin{equation}
		\V{G} \D \V{r}_m = 2\pi m,
	\end{equation}
	for some integer $ m $. This form ensures that all lattice points are contained in the planes. The minimum distance between planes can then be calculated from
	\begin{equation}
		\V{G} \D (\V{r}_2 - \V{r}_1) = 2\pi,
	\end{equation}
	as the minimum distance will be when $ (\V{r}_2-\V{r}_1) $ and $ \V{G} $ are parallel, yielding
	\begin{equation}
		d = \frac{2\pi}{|\V{G}|}.
	\end{equation}
	Now any reciprocal lattice vector will give yield a set of equidistant, parallel planes, but not all of these sets will be families of lattice planes. The family of lattice planes in the direction $ \U{G} $ will have have some distance $ d = 2\pi /|\V{G}| $, where $ \V{G} $ is a reciprocal lattice vector. An infinite set of reciprocal lattice vectors share this direction, but increasing the magnitude of the reciprocal lattice vector $ \V{G} $ will decrease the distance between the set of planes, and at some point there will be planes that do not contain any lattice points.

	As such, there is some minimum magnitude of the reciprocal lattice vector that defines a given family of lattice planes. This means that for a family of lattice planes, the distance is $ d = 2\pi /| \V{G}_{min}| $.

	\subsubsection{Scattering}
	In a scattering experiment, an incoming collection of waves (be it electrons, neutrons, x-ray photons or something completely different) with wave vector $ \V{k} $ is incident upon a crystal. Some of these will be scattered into states with wave vector $ \V{k}' $ whilst others will pass on through.

	The following derivation of the scattering criteria and intensities follow that of \cite{simon}.

	To find the general conditions for scattering, we start with Fermi's Golden Rule, which is a measure of the transition rate between states. It is given as
	\begin{equation}\label{eq:Scat_fermi}
		\Gamma(\V{k}', \V{k}) = \frac{2\pi}{\hbar}|\braket{\V{k}' | V | \V{k}}|^2 \delta(E_{\V{k}'}-E_{\V{k}}),
	\end{equation}
	where the matrix element is
	\begin{equation}\label{eq:Scat_mat_el_initial}
		\braket{\V{k}' | V | \V{k}} = \infint \frac{e^{-i \V{k}'\D\V{r}}}{\sqrt{L^3}} V(\V{r}) \frac{e^{i \V{k}\D\V{r}}}{\sqrt{L^3}} \ud \V{r} = \frac{1}{L^3} \infint e^{-i(\V{k}'-\V{k}) \D \V{r}} \, V(\V{r}) \ud \V{r},
	\end{equation}
	which is just the Fourier transform of the potential! Further, we assume the potential is periodic in the unit cell, allowing us to use the same trick as with the Nearly Free Electron Model: Since $ V(\V{r} + \V{R}) =  V(\V{r}) $ for any lattice point $ \V{R} $, we can define $ \V{r} = \V{R} + \V{x} $ where $ \V{x} $ is a position within the unit cell, and then split up the integral into an infinite sum over lattice points:
	\begin{equation}\label{eq:Scat_mat_el}
		\braket{\V{k}' | V | \V{k}} = \frac{1}{L^3}\infint e^{-i(\V{k}'-\V{k})\D(\V{R}+\V{x})} V(\V{R}+ \V{x}) \ud \V{x} = \frac{1}{L^3} \sum_{\V{R}} e^{-i(\V{k}'-\V{k}) \D \V{R}}\int_{\substack{\text{unit-} \\\text{cell}}} e^{-i (\V{k}'-\V{k}) \D \V{x}} V(\V{x}) \ud \V{x}.
	\end{equation}
	Now, as in the one dimensional case, this sum of complex exponentials has two possible outcomes. If $ \V{k}'-\V{k} $ is a reciprocal lattice vector, all the terms are unity and the sum adds up to the total number of unit cells in the crystal. If $ \V{k}'-\V{k} $ is not a reciprocal lattice vector, then the terms will just oscillate, like roots of unity, summing to 0. This condition is called the Laue condition:
	\begin{equation}\label{eq:Scat_Laue}
		\V{k}' - \V{k} =  \V{G}.
	\end{equation}
	Furthermore, when the scattered wave leaves the crystal, it has to have the same magnitude of the wave vector as the incoming wave, as required from the delta function in Fermi's Golden Rule:
	\begin{equation}\label{eq:Scat_E_cons}
		|\V{k}'| = |\V{k}|.
	\end{equation}
	Together these conditions are statements of conservation of crystal momentum and energy respectively. With the conditions met we still need to find the actual scattering amplitudes. This is where the integral in Eq. \eqref{eq:Scat_mat_el} comes in. It turns out that the intensity of the scattered wave vector is proportional to the absolute square of this integral, called the \textit{structure factor} \cite{simon}:
	\begin{equation}
		I \propto |S(\V{G})|^2, \quad S(\V{G}) = \int_{\substack{\text{unit-} \\\text{cell}}} e^{-i \V{G} \D \V{x}} V(\V{x}) \ud \V{x}.
	\end{equation}
	This is as far as is workable without specifying the form of the potential. For now we will assume we are working with neutron scattering. Since neutrons are not charged, they only scatter from the atomic cores by nuclear forces, and not from electrons. For this reason we model the potential of each atom in the unit cell as a delta function with some appropriate potential strength associated:
	\begin{equation}\label{eq:Scat_neutron_pot}
		V(\V{x}) \sum_{\text{atoms} \,j} f_j \delta(\V{x}-\V{x}_j),
	\end{equation}
	where the potential strength is called the \textit{form factor}. With this potential, the structure factor becomes
	\begin{equation}\label{eq:Scat_struct}
		S(\V{G}) = \sum_{\text{atoms} \,j} f_j e^{i \V{G} \D \V{x}_j}.
	\end{equation}

	For the purposes of the program, we will further restrict the available lattice to a simple cubic with a basis. This will allow us to illustrate the core ideas of scattering whilst keeping any clutter to a minimum.

	Another thing this allows us to show is systemic absences. For a cubic lattice with a basis, the structure factor becomes
	\begin{equation}\label{eq:Scat_struct_ortho}
		S(\V{G}) = S_{hkl} = \sum_{\text{atoms}\, j} f_j e^{i (h\V{b}_1 + k\V{b}_2 + l\V{b}_3) \D \V{x}_j} = \sum_{\text{atoms}\, j} f_j e^{2\pi i (hx_j + ky_j + lz_j)},
	\end{equation}
	where the coordinates of $ \V{x}_j $ are in units of the lattice constant $ a $. For a bcc lattice, which corresponds to a simple cubic lattice, with two identical atoms at $ (0,0,0) $ and $ (1/2, 1/2, 1/2) $ (again in units of the lattice constant), the structure factor becomes
	\begin{equation}
		S_{hkl} = f\ (1 + e^{2\pi i (h/2 + k/2 + l/2)}) = f\ (1 + e^{\pi i (h+k+l)}) = f \ (1+ (-1)^{h+k+l}),
	\end{equation}
	meaning that for there to be any scattering for a bcc lattice, $ h+k+l $ must be even. This is what is known as a systemic absence. There is also a systemic absence for fcc lattices, which can be thought of as a simple cubic lattice, with identical atoms at $ (0,0,0), (1/2, 1/2, 0), (1/2, 0, 1/2) $ and $ (0, 1/2, 1/2) $:
	\begin{equation}
		S_{hkl} = f\ (1 + e^{\pi i (h+k)} + e^{\pi i (k+l)} + e^{\pi i (h+l)}).
	\end{equation}
	Here all of $ h+k, k+l $ and $ h+l $ must be even, corresponding to $ h,k,l $ all being either even or odd. As such there are systemic absences in both bcc and fcc lattices, but not simple cubic lattices, where all combinations of $ h,k $ and $ l $ can lead to scattering.

	\subsection{Implementation}
	The scattering program creates one figure with two sets of axes. One interactive set with the physical setup, including the desired crystal structure, incoming probe beam, detector screen and detected scattering events. The second shows only a top down view of the detector screen with the detected scattering events.

	For the first set of axes, the scattering program builds upon the lattice plotting program. It plots the desired lattice (simple cubic with a basis), calculates scattering for this crystal given a list of form factors and an incoming wave vector, and displays the results on a simulated detection screen. The wave vectors point of impact is kept fixed, but the magnitude and angle of incidence can be changed by the user.

	On the second set of axes we plot a top down view of the detection plane, along with scattering events and the associated Miller indices. To make the program more user friendly, we (by default) define the incoming wave vector in units of $ 2\pi/a $, such that the user does not need to always include these factors.

	Calculating the actual scattering is done by first creating the reciprocal lattice with Eq. \eqref{eq:rec_prim_vec}, and next creating an array of reciprocal lattice vectors with indices $ h,k,l \in \{-5, -4, \dots, 5\} $ (excluding $ h=k=l=0 $ as this just results in a "scattered" wave vector equal to the incident wave vector). This of course does not constitute the whole possible range reciprocal lattice vectors, but a line has to be drawn somewhere. This interval includes $ 11^3 - 1 = 1330$ different reciprocal lattice vectors, which should be plenty to get an understanding of scattering.

	This array of reciprocal lattice vectors is then used to calculate the "scattered" wave vectors by Eq. \eqref{eq:Scat_Laue}. These do not necessarily meet the other criteria of energy conservation, though, and there are further criteria to consider.

	The first additional criteria is the direction of the scattered wave vector. The scattered wave vector has to point in the direction of the detection screen, otherwise it will not physically "hit" the screen. In the program the detection screen is placed parallel to the $ xy $-plane, with $ z=z_0, z_0 > 0 $. As such any scattered wave vector will need to have $ k'_z > 0 $ to be detected.

	Second is any systemic absences. The reason we need to check for this is the fact that we show the Miller indices on the second set of axes. So if we do not filter out any scattering wave vectors, whose intensity is zero, we will get a spot on the detection plane with zero intensity, but the Miller indices still plotted. The systemic absence is calculated by taking the absolute square of  Eq. \eqref{eq:Scat_struct} and checking whether or not it is equal to 0. If so we exclude the wave vector from the scattered events.

	These three criteria (conservation of energy, proper direction, and a lack of systemic absence) are all calculated by the program, and if a scattered wave vector does not fulfil all of them, it is discarded.

	Left are only a handful, if any, of scattered wave vectors. For each of these, the impact point of the scattered wave vector on the detection plane has to be calculated. This is done by calculating the intersection between a line and a plane. The line is defined by the point of impact $ \V{p}_0 $ of the incident wave vector along with the scattered wave vector $ \V{k}' $. The plane is defined as mentioned above, with $ z=z_0 $. In this case the intersection happens when
	\begin{equation}
		p_{0,z} + t\D k'_z = z_0,
	\end{equation}
	for some value of $ t $. This value of $ t $ can then be used to calculate the point of intersection $ \V{p} $ as
	\begin{equation}
		\V{p} = \V{p}_0 + t \V{k}'.
	\end{equation}
	These points are then plotted in the first set of axes along with the detection plane and the incoming wave vector (scaled so its length is equal to the wavelength), and on the second set of axes along with the associated Miller indices.

	Two additional features for the scattering program are also available. The first is the "show all" feature. This plots all outgoing wave-vectors and their associated lines (starting at the impact point for the incident beam, and ending at the intersection between the outgoing wave vector and the detection plane).

	The second is a "highlighting" feature. This feature works by taking a set of Miller indices and highlighting the scattering associated with said set (if scattering occurs for this reciprocal lattice vector). It plots the relevant scattering event in a different colour, and plots the associated family of planes.

	\begin{wrapfigure}{r}{3cm}
		\includegraphics[width=3cm]{figures/plane_distance.pdf}
		\caption{The distance between to adjacent planes is $ d $, and the vertical distance $ D $ is $ d/\cos \theta$}
		\label{fig:plane_distance}
	\end{wrapfigure}

	These planes are created in one of two ways. If the plane is \textit{not} perpendicular to the $ xy $-plane (corresponding to a normal vector $ \V{n} $ with a $ z $-component different from 0) the equation of the plane is used with the origin as the starting point $ \V{r}_0 $. For the normal vector, the program uses the displacement vector $ \V{d} = 2\pi \U{G} / |\V{G}| $:
	\begin{equation}
		0 = \V{d} \D (\V{r}-\V{r}_0) = \V{n} \D \V{r}, \quad \Leftrightarrow \quad z = -\frac{d_x x + d_y y}{d_z}.
	\end{equation}
	The program then calculates the $ z $-component of the plane from a given array of $ x $ and $ y $-values. This, of course, only creates one plane. Each subsequent plane is created by displacing the original plane vertically by an amount $ d/\cos \theta = d^2/d_z$ (see figure \ref{fig:plane_distance}). This process is repeated in the positive direction until the smallest $ z $-value of the uppermost plane is outside of the plot box, and likewise in the negative direction, yielding a full set of parallel planes, spaced by $ \V{d} $ (at least within the plot box).

	However, if the plane \textit{is} perpendicular to the $ xy $-plane, this method does not work. Here the program creates a plane from the span of two vectors perpendicular to the displacement vector. Since the plane is perpendicular to the $ xy $-plane, one such vector is $ \U{z} $, the unit vector in the $ z $-direction. Then the second vector can just be taken as the cross product between $ \V{d} $ and $ \U{z} $. Again the origin is taken as the starting point:
	\begin{equation}
		\V{r} = \V{r}_0 + s \U{z} + t (\U{z} \times \V{d}).
	\end{equation}
	To get any subsequent planes, an integer multiple of the displacement vector $ \V{d} $ is added to the starting plane.

	\begin{wrapfigure}{r}{2in}
		\includegraphics[width=2in]{figures/lattice_planes_1.pdf}
		\caption{A bcc lattice showing the (001) family of lattice planes.}
		\label{fig:lattice_planes}
	\end{wrapfigure}
	For both of these methods the planes need to be limited so they are only plotted within the plot box. This is done by replacing the $ z $-component of any point outside the plot box with \texttt{NaN}, which will cause Matplotlib to not plot the associated point. This is especially pertinent in the second case, where not only the $ z $-component may be outside the plot box, but also the $ x $ and $ y $-components may be. Because of this $ \U{z}/4 $ and $ (\U{z} \times \U{G})/4 $ are used to increase the resolution of each plane, along with a large range of values for $ s $ and $ t $ (in this case taken to be integers, though the same effect could be achieved by using a more densely packed interval for $ s $ and $ t $ and keeping the original vectors).

	Further, a second program is added alongside the scattering program. This second program just allows the user to plot any crystal, along with any family of lattice planes, defined by a set of Miller indices supplied by the user.

	\subsection{Examples}
	Say we just want to plot the (001) family of lattice planes for a bcc lattice, then we write the following and get figure \ref{fig:lattice_planes}.
\begin{lstlisting}
Reciprocal(lattice_name="bcc",
		   indices=(0,0,1))
\end{lstlisting}
	As another example, consider the case of simulating neutron scattering on a cubic lattice with a four atom basis (one at lattice points, the others at the centres of the faces of the unit cell). We set the relative form factors as 1 for the atoms on the lattice points and 0.5 for the others. We set $ \V{k}_{in} = (0, 0, -1.5)$ in units of $ 2\pi/a $. Lastly we highlight the scattering corresponding to the Miller indices (112). This is all done by the following line, and produces figure \ref{fig:scattering_no_systemic}.
\begin{lstlisting}
Scattering(k_in=np.array([0, 0, -1.5]),
		   basis=np.array([[0, 0, 0],
						   [0.5, 0.5, 0],
						   [0.5, 0, 0.5],
						   [0, 0.5, 0.5]]),
		   highlight=(1,1,2),
		   form_factor=np.array([1, 0.5, 0.5, 0.5]))
\end{lstlisting}
	But what if we want to plot scattering on regular fcc lattice, with the same wave vector as before? Then we just set the scattering length for all atoms to 1. This gives figure \ref{fig:scattering_systemic}, where we have also set identical colours for the four atoms to emphasize that they are now identical.
\begin{lstlisting}
Scattering(k_in=np.array([0, 0, -1.5]),
		   basis=np.array([[0, 0, 0],
						   [0.5, 0.5, 0],
						   [0.5, 0, 0.5],
						   [0, 0.5, 0.5]]),
		   form_factor=np.array([1, 0.5, 0.5, 0.5]),
		   colors=['xkcd:cement']*4)
\end{lstlisting}
	Note on this figure the absence of any scattering. This is an example of a systemic absence for the fcc lattice, because not all indices in each of the 5 sets are even (or odd). The highlighted scattering event in figure \ref{fig:scattering_no_systemic} has 2 odd indices and 1 even index.

	\begin{figure}
		\centering
		\includegraphics[width=\linewidth]{figures/scattering_no_systemic.pdf}
		\caption{Scattering on a cubic lattice with a four atom basis (one at lattice points and the others on the faces of the unit cell), with form factors of 1 for the lattice point atoms, and 0.5 for the others. The choice of incoming wave vector gives rise to 5 scattering events, corresponding to 5 different sets of miller indices.}
		\label{fig:scattering_no_systemic}
	\end{figure}

	\begin{figure}
		\centering
		\includegraphics[width=\linewidth]{figures/scattering_systemic.pdf}
		\caption{The same scattering setup as in figure \ref{fig:scattering_no_systemic}, but with equal form factors for the four atoms (effectively an fcc lattice). Note the absence of scattering events due to systemic absence, since for each of the sets of Miller indices either $ h+k $, $ k+l $ or $ h+l $ is odd.}
		\label{fig:scattering_systemic}
	\end{figure}

\end{document}


\chapter{Network theory}

  \section{Definitions}

    \subsection{Graph}

      A graph $G$ is a pair of sets $(V, E)$ such that $E \subseteq [V]^2$. $V$ or $V(G)$ is a set of arbitrary objects called \emph{vertices} (singular is \emph{vertex}) or \emph{nodes}, while $E$ or $E(G)$ is a set of vertex pairs, called \emph{edges} or \emph{arcs} on occasion; elements of $E$ are two-element subsets of $V$\cite{Diestel2012}.

      There are two graph types in respect to edge type: \emph{undirected} and \emph{directed}, pictured on figure \ref{fig:graphs_orientation_undirected} and \ref{fig:graphs_orientation_directed} respectively. In the former, edges are ordered pairs of vertices, i.e. $e = (u, v) = (v, u)$. In the latter they are unordered or just sets of two vertices, that is $e = (u, v) \neq (v, u) = d$.
      \begin{figure}[H]
        \centering
          \begin{subfigure}[b]{0.27\textwidth}
            \includegraphics[width=\textwidth]{chapters/02_problem_definition/graph_undirected}
            \caption{An undirected graph.}
            \label{fig:graphs_orientation_undirected}
          \end{subfigure}
          \qquad\qquad\qquad
          \begin{subfigure}[b]{0.27\textwidth}
            \includegraphics[width=\textwidth]{chapters/02_problem_definition/graph_directed}
            \caption{A directed graph.}
            \label{fig:graphs_orientation_directed}
          \end{subfigure}
        \caption{Types of graphs in respect to orientation.}
        \label{fig:graphs_orientation}
      \end{figure}

      Then we have \emph{simple graphs}, where there are no edges from a vertex to itself (called loops) and there are no more than one edge between any two distinct vertices (i.e. edges form sets). On the other side we have \emph{multigraphs} (or \emph{pseudographs}), that may both have loops (not everyone agrees with that) and multiple edges (also called parallel edges) between different vertices (thus they form a multiset).
      \begin{figure}[H]
        \centering
          \begin{subfigure}[b]{0.25\textwidth}
            \includegraphics[width=\textwidth]{chapters/02_problem_definition/graph_simple}
            \caption{A simple graph.}
            \label{fig:graphs_simple}
          \end{subfigure}
          \qquad\qquad\qquad
          \begin{subfigure}[b]{0.25\textwidth}
            \includegraphics[width=\textwidth]{chapters/02_problem_definition/graph_multi}
            \caption{A multigraph.}
            \label{fig:graphs_multi}
          \end{subfigure}
        \caption{Types of graphs in respect to edge distinctness.}
        \label{fig:graphs_types}
      \end{figure}

      If $(u, v)$ is an edge in an undirected graph then we call $u$ a \emph{neighbour} of $v$ and vice versa. Then we call the number of such neighbours a \emph{degree} of a vertex. In directed graphs however, if $u \rightarrow v$ is a directed edge, then we call $u$ the \emph{predecessor} of $v$, which in turn we call a \emph{successor} of $u$. The \emph{in-degree} of a vertex is the number of predecessors and the \emph{out-degree} of a node is the number of its successors. It is important to note that given $u$ and $v$ and if $u \rightarrow v$ and $v \rightarrow u$ then $u \leftrightarrow v$.

      A graph $H = (U, D)$ is a \emph{subgraph} of $G = (V, E)$ if $U \subseteq V$ and $D \subseteq E$.

    \subsection{Paths}

      \subsubsection{Path}

        A \emph{path} is a sequence of edges where each successive edge share a vertex and all other edges have no vertices in common.

      \subsubsection{Cycle}

        A \emph{cycle} is a path which starts and ends at the same vertex and has at least one edge.

      \begin{figure}[H]
        \centering
        \begin{minipage}[b]{0.43\textwidth}
          \centering
          \includegraphics[width=\textwidth]{chapters/02_problem_definition/path}
          \captionof{figure}{A path.}
          \label{fig:path}
        \end{minipage}
        \qquad\qquad
        \begin{minipage}[b]{0.3\textwidth}
          \centering
          \includegraphics[width=\textwidth]{chapters/02_problem_definition/cycle}
          \captionof{figure}{A cycle.}
          \label{fig:cycle}
        \end{minipage}
      \end{figure}

      \subsubsection{Tree}

        A \emph{tree} is a special graph that is a connected acyclic graph. A tree is also a minimal connected graph, meaning that removing any of the edges will make the graph disconnected.

      \subsubsection{Spanning tree}

        A \emph{spanning tree} of graph $G$ is a subgraph that is a tree and contains all vertices of $G$. Obviously, no spanning trees exist for disconnected graphs.

      \begin{figure}[H]
        \centering
        \begin{minipage}[b]{0.35\textwidth}
          \centering
          \includegraphics[width=\textwidth]{chapters/02_problem_definition/tree}
          \captionof{figure}{A tree.}
          \label{fig:tree}
        \end{minipage}
        \qquad
        \begin{minipage}[b]{0.55\textwidth}
          \centering
          \includegraphics[width=0.7\textwidth]{chapters/02_problem_definition/tree_spanning}
          \captionof{figure}{Spanning tree (bold edges) of grid graph.}
          \label{fig:tree_spanning}
        \end{minipage}
      \end{figure}

    \subsection{Weighted graph}

      A \emph{weighted graph} is a graph that associates a label---called \emph{weight}---with every edge in the graph. Weights of edges are usually represented with real numbers. The \emph{weight of a path} in such graph is the sum of the edge weights of the edges forming a path. A non-edge has sometimes a special weight assigned with a value of infinity. Weight can also represent \emph{costs}, then words are used instead of numeric weight. A graph is always assumed to be unweighted unless otherwise has been stated.

    \subsection{Graph isomorphism}

      An \emph{isomorphism} of graphs $G$ and $H$ is a bijection between the vertices of $G$ and $H$
      \begin{equation}
        f: V(G) \rightarrow V(H)\mbox{,}
      \end{equation}
      such that an edge exists between any two vertices $u$ and $v$ of $G$ if and only if an edge exists between $f(u)$ and $f(v)$ in graph $H$. A correspondence of this kind is commonly called \textquote{edge-preserving bijection,} where isomorphisms are \textquote{structure-preserving bijections} in general notion.

      The above definition assumes that graphs are undirected and unweighted. However, a different definitions of isomorphism may be applied to other types of graphs, by adding the requirements to preserve the corresponding additional elements of structure like link directions, weights, \emph{etc.}, with the following exception in respect to graph labelling. If labels in graph are uniquely taken from the integer range $1,\ldots,n$, where $n$ is the number of the vertices of the graph, two labeled graphs will be isomorphic if the corresponding underlying unlabelled graphs are isomorphic.
      \begin{table}[H]
        \centering
        \begin{minipage}[h]{0.3\textwidth}
          \centering
          \includegraphics[width=0.55\textwidth]{chapters/02_problem_definition/isomorphism_1}
        \end{minipage}
        \quad
        \begin{minipage}[h]{0.3\textwidth}
          \centering
          \includegraphics[width=\textwidth]{chapters/02_problem_definition/isomorphism_2}
        \end{minipage}
        \qquad
        \begin{minipage}[h]{0.3\textwidth}
          \begin{tabularx}{\textwidth}{|C{1}|} \hline
          \rowcolor[gray]{0.75} $f: V(G) \rightarrow V(H)$\\\hline
            $f(1) = a$ \\\hline
            $f(2) = b$ \\\hline
            $f(3) = c$ \\\hline
            $f(4) = d$ \\\hline
            $f(5) = e$ \\\hline
            $f(6) = f$ \\\hline
            $f(7) = g$ \\\hline
            $f(8) = h$ \\\hline
          \end{tabularx}
        \end{minipage}
        \captionof{figure}{Graphs $G$ (left), $H$ (middle) and an isomorphism between them (right).}
      \end{table}

      The concept of \emph{graph isomorphism} makes it possible to characterise graph properties essential to the structure of the graphs themselves from properties associated with graph representations like graph drawings or data structures for graphs and labelings, \emph{etc.} For example, if a graph has exactly one cycle, then all graphs in its isomorphism class\footnote{A collection of graphs isomorphic to each other.} also have exactly one cycle. On the other hand, in the common case when the vertices of a graph are (represented by) the integers $1, 2, \ldots, N$, then the expression
      \begin{equation}
        \sum_{v \in V(G)} v\cdot\mbox{deg} v\mbox{,}
      \end{equation}
      may be different for two isomorphic graphs. But that was mentioned as an exception to the general definition.

    \subsection{Graph properties}

      In order to focus on the abstract structure of graphs, we define \emph{graph properties} as maintained under all possible \emph{isomorphisms} of a graph. However, there is a distinction referred to this term; specifically, a \emph{property} is usually referred to descriptive characterisations of graphs (i.e. it is a class of graphs), while \emph{invariant} is used for properties expressed quantitatively (i.e. it is a function from graphs to some other set, like $\mathbb{N}$). To avoid naming collisions, from now on I will mention only properties and invariants in the former sense.

      \subsubsection{Graph invariants}

        \paragraph{Order}

          The \emph{order} of a graph is the number of vertices in a graph and is denoted by $|V(G)|$ or $|G|$.

        \paragraph{Size}

          The \emph{size} of a graph is the number of its edges\cite{Harris2000} in a graph and is denoted by $|E(G)|$ or $||G||$. In an undirected graph, there are $0 \leq E \leq \binom{V}{2}$ and in directed graph: $0 \leq E \leq V(V-1)$.

        \paragraph{Eccentricity}

          The \emph{eccentricity} $\epsilon(v)$ of vertex $v$ is the greatest geodesic distance between $v$ and any other vertex.

        \paragraph{Radius}

          The \emph{radius} $r$ of a graph is the minimum eccentricity of any vertex in the graph.

        \paragraph{Diameter}

          The longest of the shortest path lengths is the \emph{diameter} $d$ of a graph. It is the maximum eccentricity of any vertex in the graph. In order to find it, one first need to find the shortest path between each pair of vertices; then the greatest length of any of these paths would be the diameter of the graph.

      \subsubsection{Graph properties}

        \paragraph{Connected}

          Let $G = (V, E)$ be a graph. A non-empty graph $G$ is \emph{connected} if any two of its vertices are linked by a path in $G$, i.e.
          \begin{equation}
            \forall_{i,j \in \mathbb{N} \cap i,j < |G|} \forall_{v_i \in V} \exists_{v_j \in V} (v_i, v_j) \in E\mbox{.}
          \end{equation}
          In other words, we call a graph \emph{connected} if there is a path from any vertex to any other vertex.

          A maximal connected subgraph of $G$ is a \emph{component} of $G$. The empty graph has no components since connected graphs are non-empty.

          A graph that is \emph{disconnected} consists of several \emph{connected components}. Two vertices are considered to be in the same connected component if there is a path between them.

          $G$ is called $k$\emph{-connected} (for $k \in \mathbb{N}$) if $k < |G|$ and $G - X$ is connected for every $X \subseteq V$ with $k > |X|$. Every non-empty graph is $0$-connected and $1$-connected graphs are precisely non-trivial connected graphs. The greatest integer $k$ such that $G$ is $k$-connected is the \emph{connectivity} $\kappa(G)$ of $G$. Hence graph is disconnected if and only if $\kappa(G) = 0$. The simplest 2-connected graphs are the cycles; all the others can be inductively constructed by adding paths to cycles.

        \paragraph{Cyclic}

          Graph is cyclic when it consists of a single cycle.

        \paragraph{Acyclic}

          A graph is called acyclic if there are no subgraphs that are cycles.

        \paragraph{Bipartite}

          We call a graph a \emph{bipartite} or a \emph{bigraph} if a graph has vertices that can be divided into two disjoint sets $U$ and $V$ such that the edges only exist between independent vertices in $U$ and $V$ and never between vertices of the same set\cite{Diestel2012}. Equivalent definition is that a bipartite graph is a graph that does not contain any cycles with odd lengths\cite{AsratianDenleyHaggkvist1998}. An example of bipartite graph is presented on figure \ref{fig:cs_network} in section \ref{sec:cs}.

          The two sets $U$ and $V$ may be thought of as a colouring of the graph with two colours: if one colours all nodes in $U$ blue, and all nodes in $V$ green, each edge has endpoints of differing colours, as is required in the graph colouring problem\cite{AsratianDenleyHaggkvist1998,Scheinerman2012}. In contrast, such a colouring is impossible in the case of a non-bipartite graph, such as a triangle: after one node is coloured blue and another green, the third vertex of the triangle is connected to vertices of both colours, preventing it from being assigned either colour.

          A bigraph with partition sets $U$ and $V$ with $E$ denoting edges of the graph is often written as $G=(U,V,E)$. If such graph is disconnected, it may have more than one bipartition;\cite{ChartrandZhang2008}. In this case, the $(U,V,E)$ notation is helpful in specifying one particular bipartition that may be of importance in an application.

          Bipartite graphs very often arise naturally when dealing with real-world examples. For instance, a graph of movie actors and movies, with an edge between an actor and a movie if the actor has starred in that movie. It is a classic example of an affiliation network, which is a type of bigraph used in analysis of social networks\cite{WassermanFaust1994}.

  \section{Graphs representation}

    Graphs are usually visualised as \emph{embeddings}. An embedding of a graph $G(V, E)$ is representation of $G$ on $\Sigma$ plane in which all $v \in V$ are associated to points on $\Sigma$ and each of $e \in E$ are arcs or straight line segments between points corresponding to vertices at both ends of $e$. If no two arcs ever intersect, the embedding of a graph is called \emph{planar}, otherwise it is called a \emph{non-planar}. Because the same graph can have many embeddings that are different, it is important not to confuse the particular embedding with a graph itself. In particular, planar graphs can have non-planar embeddings too.

    There are many other ways of graph visualisation, but apart from the above and below, they are not useful for analysis of the data that is a subject of this thesis.

    Graphs can be explicitly represented by either \emph{adjacency matrices} or \emph{adjacency lists}.

    The adjacency matrix of a graph $G$ is a $|V| \times |V|$ matrix of indicator variables. Each entry in this matrix indicates whether a particular edge between indexes is or is not in graph $G$:
    \begin{equation}
      A[i, j] = [(i, j) \in E] \mbox{.}
    \end{equation}
    For undirected graph, similar to the edges, the adjacency matrix is always symmetric: $A[i, j] = A[j, i]$. In simple graphs, diagonal entries in $A$ are all zeros since loops are not allowed. The most valuable property of adjacency matrix is that we can decide in $\Theta(1)$ time whether two arbitrary vertices are connected by an edge just by looking at the appropriate elements of the matrix. We also can scan for a list of all vertex neighbours by looking at the corresponding row (or column), which will require $\Theta(V)$ time. The main disadvantage is that this matrix will always take $\Theta(V^2)$ space regardless of how many edges the graph really have. This makes adjacency matrices efficient for \emph{dense} graphs.
    \begin{table}[H]
        \centering
        \begin{minipage}[b]{0.3\textwidth}
          \centering
          \includegraphics[width=0.8\textwidth]{chapters/02_problem_definition/adjacency}
          \captionof{figure}{Graph $G$.}
        \end{minipage}
        \quad
        \begin{minipage}[b]{0.33\textwidth}
          $$\begin{pmatrix}
            0 & 1 & 0 & 0 & 1 \\
            1 & 0 & 1 & 0 & 0 \\
            0 & 1 & 0 & 1 & 1 \\
            0 & 0 & 1 & 0 & 0 \\
            1 & 0 & 1 & 0 & 0
          \end{pmatrix}$$
          \captionof{figure}{Adjacency matrix.}
        \end{minipage}
        \qquad
        \begin{minipage}[b]{0.27\textwidth}
          \begin{tabularx}{\textwidth}{L{1}}
            $1 \to 2 \to 5$ \\
            $2 \to 1 \to 3$ \\
            $3 \to 2 \to 4 \to 5$ \\
            $4 \to 3$ \\
            $5 \to 1 \to 3$
          \end{tabularx}
          \caption{Adjacency list.}
        \end{minipage}
      \end{table}

    On the other hand, adjacency lists are good when dealing with \emph{sparse} graphs (graphs with relatively small number of edges). Adjacency list is an array of linked lists (one per vertex). For undirected graphs each edge $(u, v)$ is stored twice: once in $u\mbox{'s}$ neighbourhood and once in $v\mbox{'s}$ neighbour list. For directed graph each edge is stored only once. Either way, adjacency list will take $O(V+E)$ space; listing all the neighbours of $v$ vertex takes $O(1+\mbox{deg}\,v)$ time (by scanning the neighbour list). We can determine whether $(u, v)$ is an edge by scanning neighbours of $u$ in $O(1+\mbox{deg}\,u)$ time.


\subsection{Lindelöf space}

In a Lindelöf space all open covers have countable subcovers.

This is weaker than compactness, which requires that every open cover has a finite subcover.


\section{Recurrent neural networks}
\label{sec:RNNs}

The nonlinear filtering algorithm that we choose to explore is the recurrent neural network (RNN). RNN's have advanced the state-of-the-art in many sequence and time-series processing tasks \cite{Greff2017}. (Examples include speech and language modelling \cite{LeCun2015,Goodfellow2016}; protein structure prediction \cite{Sonderby2014}; and diagnosis prediction from intensive care unit time series \cite{Lipton2015}).\footnotemark{} Relevant for SWR detection, RNN's can process multi-channel time-series, and calculating a new output sample has a relatively low cost\footnote{In contrast with very deep fully connected or convolutional neural networks.}, which is beneficial for real-time operation.

\footnotetext{For the reader with an interest in neuroscience: RNNs have recently been used in various computational studies. Examples include \cite{Cueva2018} and \cite{Banino2018}, where an RNN was trained on a navigational task, after which the entries of the internal state vector of the RNN exhibited grid-cell-like activation patterns; \cite{Song2017}, who combined reinforcement learning theory with RNNs to model reward seeking and value-based computations; \cite{Li2017}, where RNNs were used to quantify and classify behaviour of the \emph{C. Elegans} model organism; and \cite{Guclu2017}, who modelled the fMRI hemodynamic response using an RNN. For an overview of artifical neural networks in general as models in neuroscience, see \cite{Kriegeskorte2015}.}

A recurrent neural network is a nonlinear dynamical system, driven by an input signal $\z_t \in \reals^C$. In the case of SWR detection, $\z_t$ is the multichannel LFP recording, with $C$ the number of channels. The RNN maintains an internal state vector $\h_t \in \reals^M$, which is updated at each discrete time step $t$ as:
%
\begin{equation}
\label{eq:RNN_f}
\h_t = f(\h\prev, \z_t).
\end{equation}
%
$f$ is a nonlinear function, parametrised by the coefficients of several affine transformations\footnote{Linear transformations (rotate, shear, scale), plus translations.} (see \cref{sec:GRU_eqs} for the complete update equations). The state vector $\h_t$ allows the RNN to maintain an efficient and task-relevant memory of past input samples \cite{LeCun2015}.\footnotemark{} We can read out the state vector of the RNN to obtain an output time series $n_t \in \reals$, with
%
\begin{equation}
\label{eq:RNN_g}
n_t = g(\h_t),
\end{equation}
%
where $g$ is again a parametrized nonlinear function.

\footnotetext{The vector $\h_t$ is also called the `hidden state', and its $M$ entries are sometimes called `hidden units', `neurons', or `memory cells'.}

By tuning the coefficients of the affine transformations in $f$ and $g$ in an offline training phase, we can change the behaviour of the RNN so that its output $n_t$ for a given input signal $\z_t$ approaches a desired output $y_t$. RNN's are thus also a data-driven algorithm. (See \cref{sec:RNN-optim} for a description of this training phase, which involves the so called 'backpropagation through time' method). In the case of SWR detection, we can design a training signal $y_t$ that is for examples $1$ during SWR events in $\z_t$, and $0$ otherwise (see \cref{sec:data-driven-algorithms}). If we avoid overfitting to the training data, we can use the trained RNN to detect SWR's in unseen signals $\z_t$. This is, in short, the proposed method of SWR detection in this chapter.


\section*{Regression}
\subsection*{Linear Regression}
Error: $\hat{R}(w) = \sum_{i=1}^n (y_i - w^Tx_i)^2 = ||Xw-y||^2_2$\\
$w^* = \underset{w}{\operatorname{argmin}} \sum_{i=1}^n (y_i - w^Tx_i)^2$\\
Closed form: $w^*=(X^T X)^{-1} X^T y$\\
$\nabla_w \hat{R}(w) = -2 \sum_{i=1}^n (y_i-w^T x_i) \cdot x_i = 2X^T (Xw-y)$

\subsection*{Convex / Jensen's inequality}
$\text{g(x) is convex} \Leftrightarrow x_1,x_2 \in \mathbb{R}, \lambda \in [0,1]: g''(x) > 0$\\
$g(\lambda x_1 + (1-\lambda) x_2) \leq \lambda g(x_1) + (1-\lambda) g(x_2)$

\subsection*{Gradient Descent}
1. Start arbitrary $w_o \in \mathbb{R}$\\
2. For $t = 1,2,...$ do $w_{t+1} = w_t - \eta_t \nabla \hat{R}(w_t)$

\subsection*{Expected Error (True Risk)}
Assumption: data set generated iid: $R(w) =$\\
$\int P(x,y) (y-w^Tx)^2 \partial x \partial y = \mathbb{E}_{x,y}[(y-w^Tx)^2]$\\
$\hat{R}_D(w) = \frac{1}{|D|}\sum_{(x,y)\in D} {(y-w^Tx)^2}$ (estim. error)

\subsection*{Gaussian/Normal Distribution}
$\sigma =$ standard deviation, $\sigma^2 =$ var., $\mu =$ mean:\\
$f(x) = \frac{1}{\sqrt{2\pi\sigma^2}} exp(-\frac{(x-\mu)^2}{2\sigma^2})$

\subsection*{L2-reg: Ridge Regression}
Regularization: $\underset{w}{\operatorname{min}} \sum \limits_{i=1}^n (y_i - w^Tx_i)^2 + \lambda ||w||_2^2$\\
Closed form solution: $w^*=(X^T X + \lambda I)^{-1} X^T y$\\
$(X^T X + \lambda I)$ always invertible.\\
Gradient: $\nabla_w \hat{R}(w) = -2 \sum \limits_{i=1}^n (y_i-w^T x_i) \cdot x_i + 2 \lambda w$

%\subsection*{L1-regularized regression (the Lasso)}
%Regularization: $\underset{w}{\operatorname{min}} \sum \limits_{i=1}^n (y_i - w^Tx_i)^2 + \lambda ||w||_1$\\
%Encourages coefficients to be exactly 0.

\subsection*{Standardization}
Goal: each feature: $\mu = 0$, unit $\sigma^2$: $\tilde{x}_{i,j} = \frac{(x_{i,j}-\hat{\mu}_j)}{\hat{\sigma}_j}$\\
$\hat{\mu}_j = \frac{1}{n}\sum_{i=1}^n x_{i,j}$, $\hat{\sigma}_j^2 = \frac{1}{n}\sum_{i=1}^n {(x_{i,j}-\hat{\mu}_j)}^2$

%\subsection*{Multivariate Gaussian}
%$\sigma =$ covariance matrix, $\mu$ = mean\\
%$f(x) = \frac{1}{2\pi \sqrt{|\Sigma|}} e^{- \frac{1}{2} (x-\mu)^T \Sigma^{-1} (x-\mu)}$

%\subsection*{Regularization}
%The error term $L$ and the regularization $C$ with regularization parameter $\lambda$: $\min \limits_w L(w) + \lambda C(w)$\\
%L1-regularization for number of features \\
%L2-regularization for the length of $w$

%my idea
%\subsection*{Regularization}
%A lot of supervised learning problems can be written in this way: $\lambda$: $\min \limits_w \hat{R}(w) + \lambda C(w)$\\


The Self Organizing Map, SOM, introduced by Kohonen is a
non-supervised neural
learning algorithm. The map is composed of neighboring cells which are
in competition by means of mutual interactions and they adapt in order
to match characteristic patterns of the examples given during the
learning. The SOM is usually on a plane (2D).

The algorithm implements a nonlinear projection from a high
dimensional feature space to a lower dimension space, usually 2D. It
is able to find the correspondence between a set of structured data
and a network of much lower dimension while keeping the topological
relationships existing in the feature space. Thanks to this
topological organization, the final map presents clusters and their
relationships.

%\subsection{The algorithm}
Kohonen's SOM is usually represented as an array of cells where each
cell is, $i$, associated to a feature (or weight) vector  $\underline m_i = \left[m_{i1},m_{i2},\cdots,m_{in}\right]^T\in
\mathbb{R}^n$ (figure \ref{carte}).\\
\begin{figure}
\center
\includegraphics[width=0.45\textwidth]{carte.eps}
\itkcaption[Kohonen's Self Organizing Map]{Kohonen's Self Organizing Map}
\label{carte}
\end{figure}

A cell (or neuron) in the map is a good detector for a given input
vector $\underline x = \left[x_{1},x_{2},\cdots,x_{n}\right]^T\in
\mathbb{R}^n$ if the latter is {\em close} to the former. This
distance between vectors can be represented by the scalar product
$\underline{x}^T\cdot\underline{m_i}$, but for most of the cases other
distances can be used, as for instance the Euclidean one. The cell
having the weight vector closest to the input vector is called the
{\em winner}.

%\subsubsection{Learning}
The goal of the learning step is to get a map which is representative
of an input example set. It is an iterative procedure which consists
in passing each input example to the map, testing the response of each
neuron and modifying the map to get it closer to the examples.

\begin{algo}
SOM learning:
\begin{enumerate}
\item $t=0$.
\item Initialize the weight vectors of the map (randomly, for instance).
\item While $t<$ number of iterations, do:
\begin{enumerate}
\item $k=0$.
\item While $k<$ number of examples, do:
\begin{enumerate}
\item Find the vector $\underline{m}_i(t)$ which minimizes the distance
$d(\underline{x}_k,\underline{m}_i(t))$
\item For a neighborhood $N_c(t)$ around the winner cell, apply the transformation:
\begin{equation}
\underline{m}_i(t+1)=\underline{m}_i(t)+\beta(t)\left[\underline{x}_k(t)-\underline{m}_i(t)\right]
\label{khoupdate}
\end{equation}
\item $k=k+1$
\end{enumerate}
\item $t=t+1$.
\end{enumerate}

\end{enumerate}
\end{algo}

In \ref{khoupdate}, $\beta(t)$ is a decreasing function with the
geometrical distance to the winner cell. For instance:
\begin{equation}
\beta(t)=\beta_0(t)e^{-\frac{\parallel \underline{r}_i -  \underline{r}_c\parallel^2}{\sigma^2(t)}},
\end{equation}
with $\beta_0(t)$ and $\sigma(t)$ decreasing functions with time and
$\underline{r}$ the cell coordinates in the output -- map -- space.\\

Therefore the algorithm consists in getting the map closer to the
learning set. The use of a neighborhood around the winner cell allows
the organization of the map into areas which specialize in the
recognition of different patterns. This neighborhood also ensures that
cells which are topologically close are also close in terms of the
distance defined in the feature space.


\chapter{More group structure of the cube}\label{chap:structure}
In this chapter, we dive into more details about the group structure of Rubik's Cubes and explain how to find the group size of an arbitrarily large Rubik's Cube. While giving the definition of the semi-direct group, we illustrate the explicit structure of $G_3$ to build up for the next chapter.

\section{Size of the Rubik's Cube group}
\par In Chapter~\ref{chap:encryption}, we mentioned that there exist some illegal states of the Rubik's Cube $C_3$ and thus the group $G_3$ constructed by states of $C_3$ is a subgroup of $S_{48}$. We now want to show how to find the size of group $G_3$ and we expand the method to illustrate how to calculate size of group $G_n$ constructed by an arbitrary large cube $C_n$.
\par Recall Figure~\ref{fig:cube-type}, we showed that there are 8 corner cubes in $C_3$. The 8 corner cubes can freely exchange their locations and they can be arranged in $8!$ ways. When we fix locations of the 8 corner cubes, each of them can be arranged in 3 different orientations, giving $3^8$ possibilities for each ordering of the corner pieces. Similarly, there are 12 edge cubes that can freely exchange their locations and the can be arranged in $12!$ ways. When we fix locations of the 12 edge cubes, each of them can be arranged in 2 different orientations, giving $2^{12}$ possibilities for each permutation of the edge cubes. However, further reductions are required due to the structure of the Rubik's Cube.
\par The reason we need to further reduce the group size is explicitly explained in Chapter 11 in paper\cite{janet}. We will not repeat all the details presented there but give the essential part of the argument. For the 8 corner cubes of $C_3$, only 7 of them can be oriented independently, meaning the orientation of the 8th corner cube depends on the preceding seven. Hence, when we fix locations for all 8 corner cubes, they can have at most $3^7$ possible different orientations. Similarly, when we determine orientations for 11 of the 12 edge cubes, the orientation of the last edge cube will be fixed. It follows that there are $2^{11}$ possible orientations for each permutation of the edge cubes. So for now, the order of $C_3$ can be calculated as $8! \cdot 3^7 \cdot 12! \cdot 2^{11}$. In \cite{janet}, a labelling system on the cube was created, and the authors proved that any permutation of the edges must be an even permutation. Therefore we must reduce this number by half. Finally, we find the order of $C_3$ is $8! \cdot 3^7 \cdot 12! \cdot 2^{10} = 43,250,003,274,489,856,000$.
\par For a hint of the proof for above statements, recall that in Chapter~\ref{chap:encryption}, we defined each valid state as a shuffling result of a series of fundamental moves. By analyzing properties of those permutations, we can obtain a good understanding on how to determine if a state of $C_3$ is valid.
\par Now we can explain how to find group order of $G_n$ for a arbitrarily large cube $C_n$. The calculation for corner cubes will stay true for any cube $C_n$ but the calculation for edge cubes will change since cubes with different side lengths have different number of edge cubes and for larger cubes, there will be two types of edge cubes. We also need to introduce a couple more different small cubes with single visible colored piece. We can use $C_7$ to illustrate all types of small cubes. Consider Figure~\ref{fig:7-center-corner} as one face of the cube. The small cubes colored in blue are the corner cubes and those colored in yellow are the fixed center cubes. Those small cubes behave the same as they are in $C_3$.
\begin{figure}[ht]
    \centering
    \begin{minipage}{0.45\textwidth}
        \centering
        \includegraphics[width=4cm]{figures/structure/7_grid_center_corner.png}
        \caption{Center and corner cubes}\label{fig:7-center-corner}
    \end{minipage}
    \begin{minipage}{0.45\textwidth}
        \centering
        \includegraphics[width=4cm]{figures/structure/7_grid_edge.png}
        \caption{Edge cubes}\label{fig:7-edge}
    \end{minipage}
\end{figure}
In Figure~\ref{fig:7-edge}, we are showing the two types of edge cubes. The ones colored in green are called \textit{wing edges} and the ones colored in grey are called \textit{middle edges}. Clearly, when we shuffle the cube, the middle edge will only permute with other middle edges and the wing edges will permute with other wing edges. The middles edges in $C_7$ generate exactly the same amount of permutations and orientations as they are in $C_3$. We can further split the wing edges into two sets as shown in Figure~\ref{fig:7-wing-edge}. The yellow wing edges will only permute with other yellow wing edges and the blue ones will permute with other blue ones. For an arbitrary large cube $C_n$, we can calculate the number of sets of wing edges it has by finding $\lfloor \frac{n - 2}{2} \rfloor$ and each set of wing edges has $2 \cdot 12 = 24$ small cubes. Due to the inner structure of Rubik's Cubes, different from the middles edges, each one of the wing edges has only one possible orientation when it is at fixed locations.
\begin{figure}[ht]
    \centering
    \begin{minipage}{0.45\textwidth}
        \centering
        \includegraphics[width=4cm]{figures/structure/7_grid_wing_edge.png}
        \caption{Wing edges}\label{fig:7-wing-edge}
    \end{minipage}
    \begin{minipage}{0.45\textwidth}
        \centering
        \includegraphics[width=4cm]{figures/structure/7_grid_plus_center.png}
        \caption{$+$ centers}\label{fig:7-plus-center}
    \end{minipage}
\end{figure}
\par Besides the fixed centers, there are three more types of small cubes with only one visible piece we want to introduce. The ones displayed in Figure~\ref{fig:7-plus-center} are called the ``\textit{$+$ centers}'' since they form the $+$ shape. Similar to the wing edges, they can be further split into two sets and those pieces only permute with same colored ones. Secondly, we want to show the ``\textit{$\times$ centers}''. As displayed in Figure~\ref{fig:7-x-center}, those pieces form the $\times$ shape and they can be split to two sets as well.
\begin{figure}[ht]
    \centering
    \begin{minipage}{0.45\textwidth}
        \centering
        \includegraphics[width=4cm]{figures/structure/7_grid_x_center.png}
        \caption{$\times$ centers}\label{fig:7-x-center}
    \end{minipage}
    \begin{minipage}{0.45\textwidth}
        \centering
        \includegraphics[width=4cm]{figures/structure/7_grid_oblique_center.png}
        \caption{Oblique centers}\label{fig:7-oblique-center}
    \end{minipage}
\end{figure}
The number of sets of $+$ centers and $\times$ centers of an arbitrarily large cube $C_n$ can be calculated by the equation defined above for wing edges, and each set of $+$ centers or $\times$ centers contains $4 \cdot 6 = 24$ small cubes. Finally, the pieces that are not covered by the $+$ shape and the $\times$ shape are denoted the oblique centers. They are displayed in Figure~\ref{fig:7-oblique-center}. Though these three type of centers pieces are different, they permute in a similar fashion; they can change their locations freely but not their orientations.
\par We can further differentiate odd cubes and even cubes. Since $C_7$ is an odd cube, all types of small cubes exist within it. For an even cube, there will be no middle edges and ``$+$ centers'' since the fixed center piece does not exist. Therefore, there is no preferred orientation of an even cube, and some sequences of fundamental movements will be equivalent to rotating the entire cube in a three-dimensional space. Thus, the number of permutations is reduced by a factor of 24. This is because all 24 possible positions and orientations of the first corner are equivalent because of the lack of fixed centres. After addressing all types of small cubes and the difference between odd and even cubes, we now have all the tools we need to calculate group size of $G_n$ for an arbitrary cube $C_n$. In Table~\ref{tab:permutation}, we show details of this calculation.
\begin{table}[ht]
    \centering
    \begin{tabular}{|p{3cm}|p{11cm}|}
        \hline Type of cube & Number of orderings \\ \hline
        \hline Corner cubes & All cubes can arrange their corners in $8! \cdot 3^7$ different ways.  \\
        \hline Middle edges & All odd cubes can arrange their middles edges in $12! \cdot 2^{10}$ ways. Since even cube does not have middle edges, an arbitrary cube $C_n$ has $(12! \cdot 2^{10})^{n \mod 2}$ ways to order its middles edges. \\
        \hline Wing edges & An arbitrary cube $C_n$ has $(24!)^{\lfloor \frac{n - 2}{2} \rfloor}$ ways to order its wing edges. \\
        \hline Fixed centers & Only odd cubes have fixed centers and since they are fixed, they do not any anymore possible orderings to the calculation. \\
        \hline \pbox{3cm}{\vspace*{0.1cm} ``$+$ centers'' \\ ``$\times$ centers'' \\  Oblique centers \vspace*{0.1cm}} & \vspace*{-0.8cm} Since those center pieces permute in a similar fashion, we can put them together to calculate the number of orderings they generate. The details of this calculation can be found in paper\cite{size} and the result is given as $(\frac{24!}{(4!)^6})^{\lfloor (\frac{n - 2}{2})^2 \rfloor}$. \\ \hline
    \end{tabular}
    \caption{Number of permutation different type of small cubes generates}\label{tab:permutation}
\end{table}
Finally we can put everything together and the order of group $G_n$ for states of an arbitrary Rubik's Cube $C_n$ can be calculated by the following function:
$$f(n) = 8! \cdot 3^7 \cdot (12! \cdot 2^{10})^{n \mod 2} \cdot (24!)^{\lfloor \frac{n - 2}{2} \rfloor} \cdot (\frac{24!}{(4!)^6})^{\lfloor (\frac{n - 2}{2})^2 \rfloor} \cdot 24^{-((n + 1) \mod 2)}$$
In Table~\ref{tab:cube-size}, we show some calculations we obtained by using the equation above to demonstrate how fast the corresponding group size increases when the side length of a Rubik's Cube increase.
\begin{table}[ht]
    \centering
    \begin{tabular}{|c|c|}
        \hline Cube group & Group size \\
        \hline $G_3$ & $4.325 \times 10^{19}$ \\
        \hline $G_4$ & $7.401 \times 10^{45}$ \\
        \hline $G_5$ & $2.829 \times 10^{74}$ \\
        \hline $G_6$ & $1.572 \times 10^{116}$ \\
        \hline $G_7$ & $1.950 \times 10^{160}$ \\ \hline
    \end{tabular}
    \caption{Group size of different Rubik's Cubes}
    \label{tab:cube-size}
\end{table}
This quantifies our claim in Chapter~\ref{chap:encryption} that the key space explodes as the size of our cube increases. The function $f(n)$ gives us the size of an arbitrary $G_n$, but the exact algebraic structure is not known for $n > 3$.

\section{Semi-direct product of a group}
The group structure of $G_3$ is well understood and it can be expressed as a semi-direct product of other common groups such as the cyclic group. Before we show how to represent $C_3$, we need to review some group theory knowledge, and some definitions here will help in understanding the key exchange protocol in Chapter~\ref{chap:exchange} as well.
\begin{definition} A function $\phi$ from a group $G$ to a group $H$ is a \textbf{homomorphism} if $\phi$ preserves the group operation; that is if $\phi(ab) = \phi(a)\phi(b)$ for all $a, b \in G$.
\end{definition}
\begin{definition} An \textbf{isomorphism} is a homomorphism $\phi$ from a group $G$ to a group $H$ that is one-to-one and onto.
\end{definition}
\begin{definition} An \textbf{automorphism} is an isomorphism from a group $G$ onto itself. The set of automorphisms of a group $G$ is denoted by $Aut(G)$.
\end{definition}
\begin{definition} The \textbf{alternating group}, denoted $A_n$ is the group of all even permutations in $S_n$.
\end{definition}
\begin{definition} Let $G$ be a group and $H$ be a subgroup of $G$, the subgroup $H$ is a \textbf{normal subgroup} of $G$, denoted by $H \triangleleft G$, if $\forall g \in G$, $gH = Hg$.
\end{definition}
\begin{definition} Let $G$ and $H$ be two groups. Then the \textbf{direct product} of $G$ and $H$ is the group $G \times H$ under the operation $(g_1, h_1) \cdot (g_2, h_2) = (g_1g_2, h_1h_2)$ where $g_1,g_2 \in G$ and $h_1,h_2 \in H$.
\end{definition}
\begin{definition} Let $H_1$ and $H_2$ be two subgroups of $G$, then $G = H_1 \rtimes H_2$ is a \textbf{semi-direct product} if:
    \begin{enumerate}
        \item $G = H_1H_2$.
        \item $H_1 \cap H_2 = e$, where $e$ is the identity element of $G$.
        \item $H_1 \triangleleft G$.
    \end{enumerate}
\end{definition}
\par We can now describe the group structure of $G_3$. A complete proof can be found in \cite{janet}, but we will sketch the details here. Let us consider two subgroups of $G_3$. First the subgroup $H_1$ of cube orientations, which leave the position of every small cube fixed but can change their orientations. One can show this subgroup is a normal subgroup of $G$. It can be represented by moves that flip a few edges or twist corners. For example, the move ``$R\,U\,D\,B2\,U2\,B'\,U\,B\,U\,B2\,D'\,R'\,U'$ '' flips two edges. The structure of this group is $\mathbb{Z}_3^7 \times \mathbb{Z}_2^{11}$ since the group of rotations of each corner is a cyclic group of order $3$ and we mentioned that we can determine orientations of 7 among 8 corner cubes freely and the last one will be fixed. The argument for flipping edges is exactly the same.
\par In addition, we take the subgroup $H_2$ which permutes the positions of the small cubes but does not change their orientations. Similar to above, this group can be represented by a direct product of two subgroups, which are the group of permutations on the corners $S_8$ and the group of even permutations on the edges $A_{12}$ (because of the even parity). Hence the structure of $H_2$ is $S_8 \times A_{12}$. Notice that here we do not mind if $H_2$ is a normal subgroup or not.
\par Clearly $H_1 \cap H_2 = e$, where $e$ is the identity in $G_3$, since $H_1$ keeps all small cubes fixed but changes their orientations, whereas $H_2$ always moves the cubes. Since $H_1$ and $H_2$ account for all possible change on the cube $C_3$, $H_1H_2$ can form the entire group $G_3$. It follows that the cube group $G_3$ is isomorphic to the semi-direct product of these two groups:
$$G_3 = H_1 \rtimes H_2 \cong (\mathbb{Z}_3^7 \times \mathbb{Z}_2^{11}) \rtimes (S_8 \times A_{12})$$
By finding the group order of above semi-direct product, we will again retrieve the same result we found earlier in this chapter. For instance, the largest order of an element in $G_3$ is 1260 and one such element is ``$R\,U2\,D'\,B\,D'$ ''\cite{order}. However, we have not yet found a efficient method to generalize those results to other Rubik's Cube groups $G_n$.
\par In this chapter, we introduced methods to find group size of $G_n$ of an arbitrary large cube $C_n$. The explosion of the group sizes we observed suggests that larger cubes may make brute force attacks more computationally expensive. Although not known for larger cubes, we are able to give the isomorphism class for $G_3$. Since this additional structure is known, it could open up the door to opportunities to further reduce computation cost of the brute attacks on encryption based on $C_3$.


\documentclass[a4paper,10pt]{article}

\usepackage[utf8]{inputenc}
\usepackage{amsmath}

\title{Yet another derivation of the KLT update rule}
\author{}

\begin{document}
\maketitle

We want to optimize this function with respect to the displacement d,
\begin{equation}
 E(d) = \sum_{(x,y)\in W} (I(x,y) - J(x+d_x,y+d_y))^2 \ .
\end{equation}
We will do this iterativelly.

Assume that we have a previous guess $\tilde d$ of the displacement, and we only want to compute the update $\delta d$ such that
\begin{equation}
 d = \tilde d + \delta d \ .
\end{equation}
Using the notation
\begin{equation}
 \tilde J(x,y) = J(x + \tilde d_x, y + \tilde d_y) \ ,
\end{equation}
the functional to be optimized at each iteration is
\begin{equation}
 E(\delta d) = \sum_{(x,y)\in W} (I(x,y) - \tilde J(x + \delta d_x, y + \delta d_y))^2 \ .
\end{equation}
The first order approximation of $\tilde J$ with respect to  $\delta d$ is
\begin{equation}
\begin{split}
 \tilde J(x + \delta d_x, y + \delta d_y)
   & = \tilde J(x,y) + \partial_x \tilde J(x,y) \delta d_x + \partial_y \tilde J(x,y) \delta d_y
\\ & = \tilde J(x,y) + \nabla \tilde J(x,y)^T \delta d \ .
\end{split}
\end{equation}
Inserting this into the functional gives
\begin{equation}
 E(\delta d) \approx \sum_{(x,y)\in W} (I(x,y) - \tilde J(x,y) - \nabla \tilde J(x,y)^T \delta d)^2 \ .
\end{equation}
Deriving with respect to $\delta d$ and equalling to 0 gives
\begin{equation}
 0 =
 \sum_{(x,y)\in W} (I(x,y) - \tilde J(x,y)
          - \nabla \tilde J(x,y)^T \delta d)
        \nabla \tilde J(x,y) \ ,
\end{equation}
which is a 2-vector equallity.  Factorizing $\delta d$,
\begin{equation}
 \sum_{(x,y)\in W} \nabla \tilde J(x,y) \nabla \tilde J(x,y)^T \  \delta d
 = \sum_{(x,y)\in W} (I(x,y) - \tilde J(x,y)) \nabla \tilde J(x,y)
\end{equation}
which is a set of 2 linear equations on $\delta d$.

We define now the gradient matrix as
\begin{equation}
 Z = \sum_{(x,y)\in W} \nabla \tilde J(x,y) \nabla \tilde J(x,y)^T
= \sum_{(x,y)\in W}
  \begin{pmatrix}
   (\partial_x \tilde J)^2
&  \partial_x \tilde J \partial_y\tilde J
\\ \partial_x \tilde J \partial_y\tilde J
&  (\partial_y \tilde J)^2
  \end{pmatrix}
\end{equation}
and error vector as
\begin{equation}
 \mathbf e = \sum_{(x,y)\in W} (I(x,y) - \tilde J(x,y)) \nabla \tilde J(x,y) \ .
\end{equation}
The system to solve at each iteration is
\begin{equation}
 Z \delta d = \mathbf e \ .
\end{equation}

\section{Implementation Details}
\subsection{Smoothing the Images}
Before tracking images should be slightly smoothed.  There are two reasons for this:
\begin{enumerate}
 \item The linearization of $J$ only makes sense if $J$ is smooth.
 \item The tracking equations require to compute the gradient of $J$.  Given sampling of $J$ at a certain scale, there is no way to get its gradient at the same scale with an accurate precision.  We can however compute the gradient at a smoother scale by convolving $J$ with the derivatives of a Gaussian kernel.  This is
 \begin{equation}
  \bar J = G \star J  \quad \text{and} \quad \nabla\bar J = \nabla G \star J \ .
 \end{equation}
When doing this, we are computing the derivatives of the smooth version $\bar J$, not the derivatives of $J$.  Therefore, we have to use the smooth version $\bar J$ in all the equations instead of $J$, if we want the derivatives to be coherent with the image.
\end{enumerate}
Since we smooth $J$, it seems reasonable to also smooth $I$ so that they are at the same scale.

To sum up, given the two images $I$ and $J$, tracking has to be done over their smoothed versions
\begin{equation}
  \bar I = G \star I \ , \quad
  \bar J = G \star J  \quad \text{and} \quad
  \nabla\bar J = \nabla G \star J \ .
\end{equation}

\end{document}


\section{Extended Kalman Filter}
\label{ekf_section}

In estimation theory, the extended Kalman filter (EKF) is the nonlinear version of the Kalman filter which linearizes about an estimate of the current mean and covariance. In the case of well defined transition models, the EKF has been considered to estimate the state. In this system we have a state and an observer these are given by equation.
$ X=AX +BU + W_d $  and $ Y=CX + W_n $
In this $ W_d $ is disturbance and $ W_n $ is the noise by sensor.
So Kalman filter is used judges between data be disturbance and noise.

If $W_d >> W_n$ that means disturbance amount is higher than noise hence state estimation can rely more on sensor data for mapping than on previous state. And sensor is more reliable.

if $W_n >> W_d$ that means noise amount is higher than disturbance hence state estimation can rely more on previous plot data for mapping than on sensor. And state is more reliable than sensor.

So accordingly in paper we learned that non-linear system is defined as.

\begin{figure}[!htb]
\includegraphics[width=\textwidth]{./figures/EKF.jpg}
\caption{Kalman filter state estimation~\cite{ByBo2015} }
\end{figure}

\begin{algorithm}
\caption{Kalman filter algorithm}\label{alg-gd}
\begin{algorithmic}[1]
\STATE Kalman Filter \({(\mu_{t-1},\Sigma_{t-1},u_t,z_t)}\)
\STATE \( {\mu_{t}}=A_t \mu_{t-1} + B_t u_t \)
\STATE \( \Sigma_{t}=A_t \Sigma_{t-1} A_t^T +R_t\)
\STATE \( K_t = \Sigma_{t} C_t(C_t^T \Sigma_{t}C_t^T+Q_t)^{-1}\)
\STATE \( \mu_{t}=\mu_{t}+K_t(z_t-C_t \mu_t) \)
\STATE \(\Sigma_{t}=(I-K_t C_t)\Sigma_{t}\)
\STATE return \( \mu_t ,  \Sigma_t \)
\end{algorithmic}
\end{algorithm}

So now lets take non-linear state equation as follows

\begin{equation}
\overrightarrow{z}_{k+1}= \overrightarrow{f}_k (\overrightarrow{z}_k) + v_k
\end{equation}

\begin{equation}
\begin{pmatrix}
  \overrightarrow{x}_{k+1} \\
  \overrightarrow{v}_{k+1} \\
  \overrightarrow{a}_{k+1} \\
  \overrightarrow{\lambda}_{k+1} \\
 \end{pmatrix}
 =
 \begin{pmatrix}
  I_3 & \frac{T}{\lambda}I_3  & \frac{T^2}{2 \lambda} I_3 & 0\\
  0 & I_3 & T I_3 & 0 \\
  0 & 0 & I_3 & 0 \\
  0 & 0 & 0 & 1\\
 \end{pmatrix}
\begin{pmatrix}
\overrightarrow{x}_{k} \\
  \overrightarrow{v}_{k} \\
  \overrightarrow{a}_{k} \\
  \overrightarrow{\lambda}_{k} \\
 \end{pmatrix}
\end{equation}

where $x_k+1$ is the position without scale of the IMU/Camera and $v_{k+1}$ , $a_k+1$ are the velocity and acceleration of the IMU/Camera in metric unit [m]. $ν_k$ is the gaussian process noise.
Main reason behind extended kalman filter working is when ever transition takes places between matices resultant output remains Gaussian and have defined mean and co-variance so in out treatment we can only do those transformation where we dont loose condition of system being Gaussian.
For example, Linear curve when transformed is Gaussian but if the function is non linear it is resultant does not result into Gaussian so whole point of using Kalman filter is killed.
So in Extended Kalman filter we linearize the system.
Every vector in $z_k$ is resolved in the world frame W. Note that we do not
include the orientation information in the model nor use it as a measurement in order. to keep the algorithm simple and fast. On each acceleration measurement we do the conversion from the inertial to the world frame by using a zero order hold  of the unfiltered attitude measurement returned by the visual SLAM framework. As we work in a middle size environment with enough loops we assume negligible drift win the SLAM map and assume thus highly accurate attitude estimation from the visual SLAM framework. The model in its linearized form yields,

\begin{figure}[!htb]
\includegraphics[width=\textwidth]{./figures/NonLinear.jpg}
\caption{Non-Linear system~\cite{ByBo2015}}
\end{figure}

\begin{equation}
F_k=
 \begin{pmatrix}
  I_3 & \frac{T}{\lambda}I_3  & \frac{T^2}{2 \lambda} I_3& -\frac{T}{ \lambda^2} I_3 - \frac{T^2}{ 2 \lambda^2} I_3\\
  0 & I_3 & T I_3 & 0 \\
  0 & 0 & I_3 & 0 \\
  0 & 0 & 0 & 1\\
 \end{pmatrix}
\end{equation}

For fusion implementation we consider the measurements in different observation
vectors. For a multi rate filter, as it is in our case, the literature suggests two solutions. One would be using a (higher order) hold to synchronize the different measurements. Another is to weight the uncertainty of the measurement according to its temporal occurrence. The authors claim no certainty at all if no
measurement is available (i.e. the measurement noise variance is infinite). Thus the update equations simplify to improve results. A more complex weighting function (i.e. exponential decay in time) could also be applied, however, at the cost of speed. The measurement updates for the vision and the IMU yields (‘V’ and ‘I’ denotes Vision and IMU)

The innovation done by authors in for the vision part is,
\begin{equation}
K_{V,k}=P_k^- H_{V,k}^T (H_{V,k} P_k H_{V,k}^T +R_v)^{-1}
\end{equation}
\begin{equation}
\overrightarrow{z}_{k}=\overrightarrow{z}_{k}+K_{V,k}(\overrightarrow{x}_{SLAM}-H_{V,k} \overrightarrow{z}_{k})
\end{equation}
\begin{equation}
P_k=(I-K_{V,k} H_{V,k}) P_k^-
\end{equation}

The innovation for by authors in the IMU part is,
\begin{equation}
K_{I,k}=P_k^- H_{I,k}^T (H_{I,k} P_k H_{I,k}^T +R_I)^{-1}
\end{equation}
\begin{equation}
\overrightarrow{z}_{k}=\overrightarrow{z}_{k}+K_{I,k}(\overrightarrow{x}_{IMU}-H_{I,k} \overrightarrow{z}_{k})
\end{equation}
\begin{equation}
P_k=(I-K_{I,k} H_{I,k}) P_k^-
\end{equation}

The two matrices $R_I$ , $R_V$ are the noise covariance matrices for the vision and IMU measurement inputs $x_{SLAM}$ , $a_{IMU}$ which are resolved in the world frame W.
The vector $x_{SLAM}$ is the position without scale obtained from the vision algorithm(SLAM). The IMU measurement $a_{IMU}$ needs special attention, because significant errors arise in the conversion from the raw IMU output.
\begin{equation}
\overrightarrow{a}_{w}=R_{wc}R_{ca} (\overrightarrow{a}_a - \overrightarrow{b})-\overrightarrow{g}_w
\end{equation}


\documentclass[12pt]{article}
\usepackage[utf8]{inputenc}
\usepackage{float}
\usepackage{amsmath}

\usepackage[hmargin=3cm,vmargin=6.0cm]{geometry}
%\topmargin=0cm
\topmargin=-2cm
\addtolength{\textheight}{6.5cm}
\addtolength{\textwidth}{2.0cm}
%\setlength{\leftmargin}{-5cm}
\setlength{\oddsidemargin}{0.0cm}
\setlength{\evensidemargin}{0.0cm}

\begin{document}

\section*{Student Information }
%Write your full name and id number between the colon and newline
%Put one empty space character after colon and before newline
Full Name : Zeynep Özalp \\
Id Number : 2237691 \\

% Write your answers below the section tags
\section*{Answer 1}
\subsection*{a}
Let $R(x), G(x), B(x)$ be generating functions for red, green blue candies respectively. Actually, we can extend the terms of R(x) and B(x) to infinity but notice that we can not select 10 red candies since blue candies should be odd and the least odd number is 1. We can select at most 5 blue candies since there must be at least 4  red candies. Thus, generating functions are
$$R(x)=x^4+x^5+x^6+x^7+x^8+x^9$$
$$G(x)=1+x^2+x^4$$
$$B(x)=x+x^3+x^5$$
Then the ways to select 10 candies is the coefficient of $x^{10}$ in, let say $S(x)$, where $S(x)=R(x)G(x)B(x)$.
$$S(x)=(x^4+x^5+x^6+x^7+x^8+x^9)(1+x^2+x^4)(x+x^3+x^5)$$
$$S(x)=(x^4+x^5+x^6+x^7+x^8+x^9)(x+2x^3+3x^5+2x^7+x^9)$$
The coefficient of $x^{10}$ in $S(x)$ is 6.
\subsection*{b}
If we have 5 candies of each of them, we can not select more than 5 candies. So, we need to delete the terms with power more than five in generating functions of them. Therefore,
$$R(x)=x^4+x^5$$
$$G(x)=1+x^2+x^4$$
$$B(x)=x+x^3+x^5$$
$$S(x)=(x^4+x^5)(1+x^2+x^4)(x+x^3+x^5)$$
$$S(x)=(x^4+x^5+x^6+x^7+x^8+x^9)(x+x^3+x^5)$$
The coefficient of $x^{10}$ in $S(x)$ is 3.
\subsection*{c}
$$F(x)=x\dfrac{7x}{(1-2x)(1+3x)}$$
Let $F(x)=xG(x)$
$$G(x)=\dfrac{a}{1-2x}+\dfrac{b}{1+3x}=\dfrac{7x}{(1-2x)(1+3x)}$$
$a+3ax+b-2bx=7x$. When we solve, $a=7/5,\ b=-7/5$
$$G(x)=\frac{7}{5}\bigg( \dfrac{1}{1-2x}-\dfrac{1}{1+3x}\bigg)$$
Notice that
$$\dfrac{1}{1-2x}=\sum_{k=0}^{\infty}2^kx^k \longleftrightarrow <1,2,2^2,...>$$
$$\dfrac{1}{1+3x}=\sum_{k=0}^{\infty}(-3)^kx^k \longleftrightarrow <1,(-3),(-3)^2,...>$$
$$G(x)=\frac{7}{5}\sum_{k=0}^{\infty}(2^k-(-3)^k)x^k\longleftrightarrow \frac{7}{5}<0,2-(-3),2^2-(-3)^2,...>$$
$$F(x)=xG(x)=\frac{7}{5}\sum_{k=0}^{\infty}(2^k-(-3)^k)x^{k+1}$$
$$F(x)\longleftrightarrow \frac{7}{5}<0,0,1,(2-(-3)),(2^2-(-3)^2),...>$$
$$F(x)\longleftrightarrow <0,0,\frac{7}{5},\frac{7(2-(-3))}{5},\frac{7(2^2-(-3)^2)}{5},...>$$
\subsection*{d}
First, notice that $s_1=8s_0+10^0=8s_0+1=9$. So, $s_0=1$.
$$s_n=8s_{n-1}+10^{n-1}$$
$$s_nx^n=8s_{n-1}x^n+10^{n-1x^n}$$
Let $G(x)=\sum_{n=0}^{\infty}s_nx^n$. Since $s_0=1$, subtract 1 from $G(x)$ to start from $n=1$.
$$G(x)-1=\sum_{n=1}^{\infty}s_nx^n=\sum_{n=1}^{\infty}(8s_{n-1}x^n+10^{n-1}x^n)$$
$$G(x)-1=8\sum_{n=1}^{\infty}s_{n-1}x^n+\sum_{n=1}^{\infty}10^{n-1}x^n$$
$$G(x)-1=8x\sum_{n=1}^{\infty}s_{n-1}x^{n-1}+x\sum_{n=1}^{\infty}10^{n-1}x^{n-1}$$
$$G(x)-1=8x\sum_{n=0}^{\infty}s_{n}x^{n}+x\sum_{n=0}^{\infty}10^{n}x^{n}$$
$$G(x)-1=8xG(x)+\dfrac{x}{1-10x}$$
where I have used the Table 1 on page 542 to evaluate the second summation. Thus,
$$G(x)=\dfrac{1-9x}{(1-8x)(1-10x)}$$
$$G(x)=\dfrac{A}{1-8x}+\dfrac{B}{1-10x}$$
Solving this gives $A=1/2, B=1/2$.
$$G(x)=\frac{1}{2}\bigg(\dfrac{1}{1-8x}+\dfrac{1}{1-10x}\bigg)$$
Again, use Table 1 to convert these into sums.
$$G(x)=\frac{1}{2}\bigg(\sum_{n=0}^{\infty}8^nx^n+\sum_{n=0}^{\infty}10^nx^n\bigg)$$
$$G(x)=\sum_{n=0}^{\infty}\frac{1}{2}(8^n+10^n)x^n$$
Therefore,
$$s_n=\frac{1}{2}(8^n+10^n)$$

\section*{Answer 2}
For simplicity, I will use the term "A-set" to represent the set of all numbers that are divisible by a given integer in a given interval.
\subsection*{a}
Let $k=cm$ where c is an integer and $A_k=\{k_1,k_2k_3,..\}$. If $k|k_i$ where $k_i\in A_k$, then $k_i=bk$ where b is an integer.
$$k_i=b(cm)=bcm$$
So, all elements in $A_k$ are divisible by m. Thus, $A_m$ contains all the elements of $A_k$ since these elements are divisible by m. Then,
$$A_k\subseteq A_m$$
\subsection*{b}
If n is a composite number, n has a prime divisor less than or equal to $\sqrt{n}$, by theorem 2 on page 258. So the greatest prime divisor of n can be $\sqrt{n}$.\\
From part a, the A-set of a composite number is the subset of the A-sets of prime divisors of it. So, we do not need to consider A-sets of composite numbers because all numbers which can be divisible by a composite number is also divisible by a prime number. Since the greatest prime number which divides the greatest composite number, which is n, can be less than or equal to $\sqrt{n}$ and since all other composite numbers less than  or equal to n are divisible by some primes $p\leq \sqrt{n}$, the right-hand side of equation 1 is equal to the union of A-sets of prime numbers up to $\sqrt{n}$.
\subsection*{c}
Assume that n is divisible by m. Thus,
$$A_m=\{2m,3m,4m,...,(\frac{n}{m}m)\}$$
Clearly, there are $n/m-1$ elements in $A_m$. Now assume that n is not divisible by m. Let k is the number before n that is divisible by m. Thus,
$$A_m'=\{2m,3m,4m,...,(\frac{k}{m}m)\}$$
in the interval $(m,k]$.
Clearly, there are $k/m-1$ elements in $A_m'$. Since n is greater than k and is not divisible by m, $|A_m|=|A_m'|$. Thus, $|A_m|=k/m-1=\lfloor n/m \rfloor -1$.
\subsection*{d}
Since a and b are relatively prime, the intersection of $A_a$ and $A_b$ is the $lcm(a,b)=ab$ and its multiples. This set is the union of $A_{ab}$ and $\{ab\}$. So, the only number is $lcm(a,b)=ab$.
\subsection*{e}
Since all the elements in P is primes, $A_p$ is the multiples of those primes and the intersection of all A-sets is the A-set of the number, say k, such that
$$k=p_1p_2p_3...$$
where $p_i$'s are the elements of P. Therefore,
$$\bigg |\bigcap_{p \in P}A_p\bigg |=|A_k|+1$$
Add 1 because $k$ should be in the intersection of all A-sets of p's in P but  is not in the A-set of k.
\subsection*{f}
From part b,
$$|C_{45}|=\bigg|\bigcup_{primes\ p\leq \sqrt{45}}A_p\bigg|$$
$$|C_{45}|=|A_2\cup A_3\cup A_5|=|A_2|+|A_3|+|A_5|-|A_2\cap A_3|-|A_2\cap A_5|-|A_5\cap A_3|+|A_2\cup A_3\cup A_5|$$
\subsection*{g}
From part b, c, d and e: \\
$|A_2\cap A_3|=|A_6|+1=7$
$|A_2\cap A_5|=|A_{10}|+1=4$
$|A_5\cap A_3|=|A_{15}|+1=3$
$|A_2\cup A_3\cup A_5|=|A_{30}|+1=1$
$$|C_{45}|=|A_2|+|A_3|+|A_5|-|A_6|-|A_{10}|-|A_{15}|+|A_{30}|$$
$$|C_{45}|=21+14+8-7-4-3+1=30$$
Notice that $C_{45}$ is defined in the interval (1,45] so 1 is not in $C_{45}$. Thus the number of composite numbers up to 45 is $|C_{45}|+1=31$. Subtracting this from 45 will give the number of primes up to 45.
$45-31=14$

\section*{Answer 3}
\subsection*{a}
If $\ll$ is a transitive relation, then for pairs for all arbitrary (a,b), (c,d), (e,f) in $Z^2$, the following is true.
$$[((a,b)\ll (c,d)) \wedge ((c,d)\ll (e,f))]\rightarrow ((a,b)\ll (e,f))$$
If $(a,b)\ll (c,d)$ and $(c,d)\ll (e,f)$, this means
$$(a<c) \vee (a=c \wedge b\leq d)$$
$$(c<e) \vee (c=e \wedge d\leq f)$$
If $\ll$ is a transitive relation
$$(a<e) \vee (a=e \wedge b\leq f)$$
must hold.\\
1. Consider the first case: $(a<c)$ and $(c<e)$. Then, $a<e$ and relation holds for the first case.\\
2. Consider the second case: $(a=c \wedge b\leq d)$ and $(c=e \wedge d\leq f)$. Clearly $a=e$ and since $b\leq d \leq f$, this implies $b\leq f$.
Therefore, $\ll$ is a transitive relation.
\subsection*{b}
Equivalence relations are reflexive, symmetric, transitive relations.
\subsubsection*{Reflexivity}
For all functions f, k in R such that $x\geq k$, the following is true for any k.
$$f(x)=f(x)$$
$$f\ \alpha \ f$$
Thus, $\alpha$ is reflexive.
\subsubsection*{Symmetry}
If $f\ \alpha \ g$, for $x\geq k$, $f(x)=g(x)$. If $g\ \alpha \ f$, for $x\geq k$, $g(x)=f(x)$.
If $f\ \alpha \ g$ holds, so do $g\ \alpha \ f$ for the same arbitrary k and vice versa. Thus, $\alpha$ is symmetric.
\subsubsection*{Transitivity}
If $\alpha$ is a transitive relation, then for pairs f, g and h the following is true.
$$[(f\alpha g) \wedge (g\alpha h)]\rightarrow (f \alpha h)$$
If $f\alpha g$ and $g\alpha h$, this means
$$f(x)=g(x)$$ for every $x \geq k_1$.
$$g(x)= h(x)$$ for every $x \geq k_2$.\\
Since these tree functions are equal at points greater than the maximum of $k_1$ and $k_2$, clearly
$$f \alpha h$$ for every $x \geq max(k_1,k_2)$
Thus, $\alpha$ is transitive.

\end{document}


\documentclass[12pt]{amsart}
\usepackage{geometry} % see geometry.pdf on how to lay out the page. There's lots.
\geometry{a4paper} % or letter or a5paper or ... etc
\usepackage[T1]{fontenc}
\usepackage[latin9]{inputenc}
\usepackage{amsmath}
\usepackage{amsaddr}
\usepackage{amssymb}
\usepackage{hyperref}
\usepackage{dirtytalk}
\usepackage{float}
\usepackage{listings}
\usepackage{color}
 \title{Exploration of Patterns for use in Class}

\date{\today}

%%% BEGIN DOCUMENT
\begin{document}
\maketitle

\section{Fundamental Patterns}
\section{Definitions}
\begin{enumerate}
	\item Singleton: A set with one value.
	\item Set: A mathematical set. Ex. An undirected graph is a set of edge descriptions.
	\item Sequence: A mathematical sequence. Ex. A digraph is a sequence of edge descriptions.
\end{enumerate}
\section{Functor Pattern}
A  pattern that allows a generic function to apply a specific function  without changing or modifying the generic function.$\\$

\section{Fold Pattern:set or sequence to singleton}
For function $f$, set or sequence $S$, and singleton $a,\\$

\begin{align*}
 fold(f,S,a) &=
  \begin{cases}
   a & \text{if } S=\varnothing\\
   fold(f,S,f(x:x\in S,a)) & \text{otherwise}
  \end{cases}
\end{align*}
\section{Unfold Pattern:singleton to set or sequence}
For function $f$, and singleton $d\\$
\begin{align*}
unfold(f,d) &=
  \begin{cases}
   \varnothing & \text{if }f(d)=\varnothing\\
   f(d)\cup unfold(f,f(d)) & \text{otherwise}
  \end{cases}
\end{align*}

\section{Derived Patterns}
\subsection{Hylomorphism Pattern}
For functions $f$ and $g$, singleton $d$, and singleton $a,\\$
\begin{align*}
hylo(f,g,d,a) &= fold(f,unfold(g,d),a)
\end{align*}
\subsection{Map Pattern}
For set or sequence $A=\varnothing$, set or sequence $S$, and function $f$, where $f(s:s\in S, A)$ performs some transformation $t$ as well as $\{t(s)\}\cup A,\\$
\begin{align*}
map(f,S) &= fold(f,S,A)
\end{align*}
\subsection{Filter Pattern}
For set or sequence $A=\varnothing$, set or sequence $S$, and function $f$, where $f(s:s\in S, A)$ performs some comparison $c$ and $\{t(s)\}\cup A$ when $c(s)=true,\\$
\begin{align*}
filter(f,S) &= fold(f,S,A)
\end{align*}
\subsection{Monad Pattern}

\subsection{Sream Pattern}
unfold pattern derivative (Stream is the same as Lazy Evaluation)
\end{document}


\documentclass[11pt]{article}
\usepackage{graphicx}
\usepackage{fancyvrb}
\usepackage{url}
\title{RSA implementation - Homework \#3 \\ \bigskip \large CNS Course Sapienza}
\date{20th November 2020}
\author{Leonardo Razovic 1712242}
\begin{document}
\maketitle

\section{Introduction}
The \textit{Rivest–Shamir–Adleman} \textbf{(RSA)} cryptosystem revolutionized cryptography when it
emerged in 1977 as the first public-key encryption scheme; whereas classical, symmetric-key
encryption schemes use the same secret key to encrypt and decrypt messages, public-key encryption
(also called asymmetric encryption) uses two keys: one is your public key, which can be used by
anyone who wants to encrypt messages for you, and the other is your private key, which is required
in order to decrypt messages encrypted using the public key.
The RSA algorithm involves four steps: \textit{key generation, key distribution, encryption, and decryption}.
\subsection{Key generation}
The keys for the RSA algorithm are generated in the following way:\cite{rsa-wik}
\begin{itemize}
  \item In order to generate a key, you pick two large prime numbers $p$ and $q$. These numbers have to be picked at random, and in secret.
  \item You multiply them together to produce the modulus $n = p \cdot q$, which is public.
  \item Compute $\lambda(n)$, where $\lambda$ is Carmichael's totient function.\\
        Since $n = p \cdot q$, and $\lambda(n) = lcm(\lambda(p),\lambda(q))$, and since $p$ and $q$ are prime, $\lambda(p) = \phi(p) = p - 1$ and likewise $\lambda(q) = q - 1$.\\
        Hence $\lambda(n) = lcm(p - 1, q - 1)$.
  \item Pick an encryption exponent $e$ (which is also public) such that $1 < e < \lambda(n)$ and $gcd(e, \lambda(n)) = 1$.
        Usually, this value is either $3$ or $65537$. Because those numbers have a small number of $1$'s
        in their binary expansion, you can compute the exponentiation more efficiently.\\
        $(n, e)$ is the public key.
  \item There is a value $d$, the \textit{decryption exponent}, fairly easy to compute assuming that you know $p$ and $q$, such that $d$ is the modular multiplicative inverse of e modulo $\lambda(n)$.
\end{itemize}
\subsection{Key distribution}
Suppose that Bob wants to send information to Alice. Bob must know Alice's public key to encrypt the message and Alice must use her private key to decrypt
the message. To enable Bob to send his encrypted messages, Alice transmits her public key $(n, e)$
to Bob via a reliable, but not necessarily secret, route. Alice's private key $d$ is never distributed.
\subsection{Encryption}
Anyone can use the public key $(n, e)$  to encrypt a message $M$ into a ciphertext $C$ such that $C \equiv M^e (mod \ n)$
\subsection{Decryption}
Using $d$, you can decrypt the message like so: $M \equiv C^d (mod \ n)$
The security of RSA relies on that decryption operation being impossible without knowing the secret
exponent $d$, and that the secret exponent $d$ is very hard (practically impossible)
to compute from the public key $(n, e)$.
\section{Design}
To implement the protocol Rust 1.48.0 has been chosen, with the auxiliary help of several libraries to manage numbers with a high
amount of bit. The main libraries used are:
\begin{itemize}
  \item num-bigint: Big integer types for Rust, BigInt and BigUint.
  \item glass\_pumpkin: A cryptographically-secure, random number generator, useful for generating large prime numbers
  \item openssl: Provdes OpenSSL bindings for the Rust programming language, used to test the encryption/decryption using RSA and AES
\end{itemize}
\subsection{Generate $p$, $q$ and $n$}
The first step is to generate the values of p and q. These two values must be prime numbers, integers, and large enough to be considered safe.
To do this we have chosen to use \Verb"glass_pumpkin" that allows you to generate random prime numbers of a
desired bit length. The randomness comes from the operating system.
Since $p$ and $q$ are values of type \Verb"BigUint" and that this type implements \Verb"Mul" \cite{mul}
it is possible to claculate $n = p \cdot q$.
The length of $n$ in bits represents the length of the key, so if you generate $p$ and $q$ of $512$ bits, you get $n$ at 1024 bits.
The library for generating random prime numbers follows these steps:
\begin{itemize}
  \item Generate a random odd number of a given bit-length.
  \item Divide the candidate by the first 2048 prime numbers. (This helps to eliminate certain cases that pass Miller-Rabin but are not prime.)
  \item Test the candidate with Fermat's Theorem.
  \item Runs $log_2(bit\_length)$ + 5 Miller-Rabin tests with one of them using generator $2$.
  \item Run Lucas primality test.
\end{itemize}
\subsection{Generate $\lambda(n)$}
To compute $\lambda(n) = lcm(p - 1, q - 1)$ where $\lambda$ is Carmichael's totient function, it's possibile to use the \Verb"lcm()" function
provided by \Verb"num_bigint".
\subsection{Generate $e$}
To find $e$ we have two possibilities: use a precomputed value, like $3$ or $ 2^{16} + 1 = 65537$, or
pick a value such that $1 < e < \lambda(n)$ and $gcd(e, \lambda(n)) = 1$. In the code both of the options
are possibile, using the \Verb"gen_biguint_range()" function that generates a random \Verb"BigUint" within the given range, ($1$ and $\lambda(n)$ in our case)
and check if the greatest common divisor between $e$ and $\lambda(n)$ is equal to $1$.
\subsection{Perform Encryption and Decryption}
To perform the encryption of a given plaintext we can use the \Verb"modpow()" provided by \Verb"num-bigint" that returns
(plaintext \^{} exponent $e$) \% modulus $n$ in the encryption
and (ciphertet \^{} exponent $d$) \% modulus $n$ in the decryption phase.
\section{Evaluation}
All tests were performed using Rust 1.48.0 and OpenSSL \cite{openssl} 1.1.1h on Arch Linux running Linux 5.9.8, using an Intel i5 9600KF processor.
To evaluate the correctness and speed of my implementation I compared the results obtained with the results obtained by RSA via OpenSSL.
Each test was performed using Rust's test functions that have been run 20 times, the times reported are an average of the run times.
\begin{table}[h!]
  \centering
  \begin{tabular}{|l|l|l|}
    \hline
    \textit{}                    & \textit{My RSA} & \textit{OpenSSL RSA} \\ \hline
    \textit{Key Generation Time} & \textit{28105}  & \textit{20535}       \\ \hline
    Encryption Time              & 142             & 137                  \\ \hline
    Decryption Time              & 850             & 822                  \\ \hline
  \end{tabular}
  \caption{My implmentantion vs OpenSSL, times expressed in micros}
\end{table}
\\
To speed up the time for encryption and decryption, it is possible to use a smaller exponent $e$, such as $65537$
\begin{table}[h!]
  \centering
  \begin{tabular}{|l|l|l|}
    \hline
    \textit{}                    & \textit{My RSA} & \textit{OpenSSL RSA} \\ \hline
    \textit{Key Generation Time} & \textit{23517}  & \textit{17096}       \\ \hline
    Encryption Time              & 135             & 132                  \\ \hline
    Decryption Time              & 405             & 398                  \\ \hline
  \end{tabular}
  \caption{My implmentantion using a fixed exponent vs OpenSSL, times expressed in micros}
\end{table}
\\
Another test to evaluate the speed was done by comparing the times obtained using AES-128-CBC in OpenSSL,
here we can see very much the slowness of RSA towards AES.
\begin{table}[h!]
  \centering
  \begin{tabular}{|l|l|l|}
    \hline
    \textit{}                & \textit{My RSA} & \textit{OpenSSL AES} \\ \hline
    \textit{Generating Keys} & \textit{25107}  & \textit{0}           \\ \hline
    Encryption Time          & 712             & 650                  \\ \hline
    Decryption Time          & 704             & 638                  \\ \hline
  \end{tabular}
  \caption{My implmentantion vs OpenSSL AES-128-CBC, times expressed in micros}
\end{table}
\\
Using the Linux \Verb"perf" tool it was possible to create a flame graph which shows that most of the time is spent to
generate the keys, especially in checking if the randomly generated numbers are prime.\\
\begin{figure}[h]
  \includegraphics[scale=0.17]{flame.png}
  \caption{The flame graph generated by the complete execution of key generation, encryption, and decryption}
\end{figure}
\section{Conclusion}
In this homework it was possible to study and implement an unsafe, but valid,
version of RSA and compare it with real (and safe) implementations.
The results obtained show how slow an asymmetric scheme, such as RSA, can be, compared to a symmetric scheme, such as AES-128 in CBC mode.

\begin{thebibliography}{9}
  \bibitem{mul}
  Crate num\_bigint documentation.\\
  \url{https://docs.rs/num-bigint/0.3.1/num_bigint/struct.BigUint.html#impl-Mul\%3CBigUint\%3E}
  \\Accessed: 2020-19-11.

  \bibitem{openssl}
  OpenSSL Software Foundation.\\
  \url{https://www.openssl.org/index.html}
  \\ Accessed: 2020-19-11.

  \bibitem{rsa-wik}
  RSA (cryptosystem)\\
  \url{https://en.wikipedia.org/wiki/RSA_(cryptosystem)}
  \\Accessed: 2020-20-11.
\end{thebibliography}
\end{document}


\subsection{BN Matching}
The traditional method of implementing style transfer is using the mean square error (MSE) of the Gramian Matrix of the feature maps. This the the method that is used in the original Gatys style transfer and is also the basis in Ruder. But there are other such methods, one of which is BN Matching which was first implemented by Li \textit{et al.} \cite{Li:1}.\newline\newline
BN (batch normalization) Matching is based on matching the mean and standard deviation of the features. This type of styling uses 1st order statistics, unlike traditional style transfer which uses 2nd order statistics. The advantage of using 1st order statistics is that it requires less computation and therefore is faster. For that reason it is often utilized for commercial purposes. The downside is that it sacrifices quality for speed.\newline\newline
The original purpose of BN normalization is to reduce the issue of internal covariate shifting, which is a common problem with training deep neural networks. The normalization works by standardizing each feature into mini-batches and then it learns a common slope and bias for each mini-batch. The BN layer makes transformations on our model for each feature $j$ in the following manner.
\begin{equation}
\begin{aligned}
\label{eq:batch_normalization}
    \hat{x}_j&=\frac{x_j-{\mathbb{E}}[X_{.j}]}{\sqrt{Var[X_{.j}]}}\\
    y_j&=\gamma_j\hat{x}_j + \beta_j
\end{aligned}
\end{equation}
Here $x_j$ and $y_j$ are the input and output scalars of one neuron in our data, $X_{.j}$ is the $j^{th}$ column of the input, while $\gamma_j$ and $\beta_j$ are to be learned. Using this method we can calculate the batch normalization for each feature. The core idea of this is to align the distribution of our data.  With this we can use BN statistics to represent the style of our image and we can construct a loss function by matching the mean and standard deviation of the features.


%!TEX root = /home/renaud/Documents/EPL/tfe/latex/tfe.tex
\section{Extension to continuous time}

From a general viewpoint, the discrete process can be interpreted as an approximation of its continuous counterpart: whereas the state of the discrete-time random walker can only change at unit-time intervals, the continuous-time random walkers undergo a waiting time between each change of state which is itself a random variable. More precisely, the waiting time is a continuous memoryless random variable distributed exponentially. Obviously, the transition probabilities from one node to the other are the same for both discrete- and continuous-time processes, only the time at which the jump occurs may vary. The continuous-time process corresponding to~\eqref{eq:discreteMP} is governed by the following dynamics:
\begin{equation} \label{eq:continuousMP_general}
    	\dot{\b p} = \b p \diag\left\{\bs \lambda(\b q)\right\} \b Q^{-1} \b A - \b p \diag\left\{\bs \lambda(\b q)\right\} = -\b p \b L,
\end{equation}
where $\lambda_i(\b q)$ is the rate at which random walkers leave node $i$, and
\begin{equation}
	\b L = \diag\{\bs \lambda(\b q)\}[-\b Q^{-1} \b A + \b I].
\end{equation}
Two particular cases of this process are implemented by the stability software and are thus examined here, depending on the choice of $\bs \lambda(\b q)$: the so-called \textit{normalized Laplacian dynamics} and \textit{standard (combinatorial) Laplacian dynamics}. Their names come from the similarity that arises between $\b L$ and the normalized/standard Laplacian matrix. Each of those two dynamics represent best different physical processes. The former corresponds to the choice $\bs \lambda_{norm}(\b q) = \b 1$. Hence, the expected waiting time is $1$ at every node, and $\b L = -\b Q^{-1} \b A + \b I = -\b M + \b I$. The latter corresponds to $\bs \lambda_{combi}(\b q) = \b q/\langle \b q \rangle$. In that case, $\b L = (-\b A + \b Q)/\langle \b q \rangle$ and the average waiting time at node $i$ is $\langle \b q \rangle/q_i$. Hence, the expected waiting time at a given node is smaller (resp. larger) than $1$ if the total weight of the outgoing edges from that node is larger (resp. smaller) than the average total weight of the outgoing edges on the network. However, the expected waiting time over the whole network is $\langle \langle \b q \rangle/\b q \rangle = 1$.\label{expectedtimebetweenjumps} The corresponding governing equations are respectively
\begin{equation} \label{eq:continuousMP_norm}
	\dot{\b p} = \b p \b Q^{-1} \b A - \b p = \b p \b M - \b p
\end{equation}
for the normalized Laplacian and
\begin{equation} \label{eq:continuousMP_combi}
    	\dot{\b p} = \b p \frac{\b A}{\langle \b q \rangle} - \b p\frac{\b Q}{\langle \b q \rangle}
\end{equation}
for the combinatorial Laplacian.

The clustered autocovariance matrix for partition $\P$ at time $t$ is easily generalized to
\begin{equation}
	\b R(t;\P) = \b H_{\P}^{\t}(\b \Pi\b P(t) - \bs \pi^{\t}\bs \pi)\b H_{\P},
\end{equation}
where $\b P(t)$ is the the transition matrix of the process at time $t$: $\b P(t) = \Exp^{-t\b L}$. The continuous-time definition of the stability of a partition $\P$ follows almost straightforwardly:
\begin{equation}
	r(t;\P) = \trace \left[ \b R(t; \P) \right].
\end{equation}
Notice that it is not necessary to minimize over the time interval $[0,t]$: indeed, it can be shown that $\trace \left[ \b R(t;\P) \right]$ is monotonically decreasing with time. The interpretation in terms of a random walk is similar to the discrete case: let $P(\C,t)$ be the probability that a random walker is in community $\C$ at time $t$ if it was initially in $\C$, when the system is at stationarity. Discounting the probability of such an event to take place by chance at stationarity and summing over all communities of $\P$ leads to the definition of the stability of the partition $\P$:
\begin{equation} \label{eq:generalstability}
	r(t;\P) = \sum_{\C \in \P} P(\C,t) - P(\C,\infty).
\end{equation}
By ergodicity, the memory of the initial condition is lost at infinity and $P(\C,\infty)$ is thus equal to the probability that two independent walkers are in $\C$ at stationarity. Equation~\eqref{eq:generalstability} tells us that only the communities in which a random walker is likely to stay bring a positive contribution to stability, where \textit{likely to stay} means that the probability for a walker to be in its initial community at time $t$ is larger than the probability of that event occurring by chance at stationarity. The stability curve of the graph can now be expressed as a continuous function of $t$:
\begin{equation}
	r(t) = \max_{\P} r(t;\P).
\end{equation}