% !TeX spellcheck = en_GB
\section{Operational Weather Forecast Model}\label{sec:DIM:MEPS}
MetCoOp Ensemble Prediction System (MEPS) became operational at Met-Norway in November 2016 when the extreme Christmas storm occurred over Norway. Comparing model data with actual observations helps to validate %verify the agreement between
model predictions. % and ground-based measurements.
\\
MEPS is used as weather forecast at the Norwegian Meteorological Institute, the Swedish Meteorological and Hydrological Institute (SMHI) and the Finnish Meteorological Institute (FMI), \citep{muller_arome-metcoop:_2017, koltzow_metcoop_2017}.
It replaced Mèteo-France Applications of Research to Operations at MEsoscale (AROME)-MetCoOp, which was operational from March 2014 until November 2016. %, when it was replaced with an ensemble prediction system (EPS) based on AROME-MetCoOp.
%\textcolor{red}{Say more about the model itself! == AROME, Both models are based on...}
Both models are a branch of the Hirlam Aladin Regional Meso-scale Operational NWP In Europe (HARMONIE) AROME model, version 40h1.1. MEPS and AROME-MetCoOp therefore built on the bases of AROME-France a convective-scale model. The physical parametrisations are from the French mesoscale non-hydrostatic atmosphere model (Meso-NH). The model got operational in 2008 and has a horizontal resolution of \SI{2.5}{\km} \citep{seity_arome-france_2010}.

%Ensemble prediction has the goal to improve the forecast by ensemble averaging, to provide an indication of the reliability of the forecast, and a quantitative basis for probabilistic forecasting \citep{kalnay_atmospheric_2003}.
\subsection{Ensemble Prediction System}
The main difference between AROME-MetCoOP and MEPS is that AROME-MetCoOp only contains a deterministic prediction whereby MEPS has additionally nine individual perturbed ensemble member \citep{metcoop_wiki_description_2017}.
\\
\Cref{fig:MEPS:kalnay_ens} shows the schematic of an ensemble prediction system (EPS). An ensemble forecasting system requires the definition of the initial amplitude and the horizontal and vertical structure of the perturbation. In general, the initial perturbation is chosen to be close to the observations. The initial condition for the disturbance is within a circle (\Cref{fig:MEPS:kalnay_ens}) of the observation uncertainty. In an ensemble prediction system one of the members is the deterministic forecast (single control forecast) where the other forecast members start from a slight perturbed state of the deterministic forecast. For a short time, should the forecast members be close together, this is a few hours in mesoscale prediction (deterministic in \Cref{fig:MEPS:kalnay_ens}). After a certain period of time, the forecast of the different perturbed members are so large that they have to be considered as stochastic (\Cref{fig:MEPS:kalnay_ens}).
\\
Important is, that the observations, which are going to be used for the initialisation are within the spread of the individual ensemble member forecasts \citep{kalnay_atmospheric_2003}.
%%% image Kalnay Fig 6.4.1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[t!]
	\centering
	\includegraphics[width=0.49\textwidth]{./fig_MEPS/ensemble}
	\caption{Schematic of ensemble prediction. Circle, representing the uncertainty of the initial conditions. Individual lines, perturbed ensemble member from the initial conditions ending in solution space, grey area \citep{kalnay_atmospheric_2003}.}\label{fig:MEPS:kalnay_ens}
\end{figure}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\noindent

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%% MEPS %%%%%%%%%%%%%%
\subsection{MetCoOP Ensemble Prediction System}\label{sec:AROME}
In principle, MEPS is a short-term weather forecast consisting of ten ensemble members %forecast system
with \SI{66}{\hour} prediction time and a horizontal resolution of \SI{2.5}{\km} and 65 vertical levels. Hourly \SI{66}{\hour} forecast data is available at Met-Norway for the deterministic and the first perturbed ensemble member. \SI{54}{\hour} forecasts are stored for the three hourly values.
\\
The lower layer, near the ground is approximately \SI{12}{\km} height. With increasing height decreases the vertical resolution of \SIrange{25}{200}{\metre} in the lower \SI{3}{\km}. The model top is at located at approximately \SI{23}{\km}.
The initialisation of each member is performed at \SIlist{00;06;12;18}{\UTC} \citep{metcoop_wiki_description_2017}.
Forecast data saved for the deterministic and first ensemble member have a time resolution of one hour for the \SI{66}{\hour} forecast period. The other eight members have data stored every three hours for up to \SI{48}{\hour} forecast time.
% %\\
%%% image MEPS resolution %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\input{./images_tex/meps_Haukeli_map}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\noindent
\Cref{fig:meps:Norway} shows
the MEPS model domain and its elevation as it was operational for December 2016. It covers Scandinavian countries including open water such as the Atlantic Ocean, the North and the Baltic Seas.
A representation of the horizontal resolution zoomed for the Haukeliseter site is shown in \Cref{fig:meps:site}.
%Haukeliseter is surrounded by a complex terrain with mountains up to \SI{1500}{\metre} no the west and the north and the more open terrain to the south-east.
The topographical resolution of MEPS and its influence on local wind and precipitation will be discussed in \Cref{sec:res:oro_infl}.
To compare the measurements from the surface with the MEPS data, the closest model grid point to Haukeliseter, is used (\Cref{fig:meps_site}). The closest grid point to Haukeliseter is \ang{59.80}\,N, \ang{7.22}\,E at \SI{1041}{\metre} above sea level.
\\
The centre of the model is approximately at \ang{63.5}\,N, \ang{15}\,E.
The horizontal grid points are %projected on a
Lambert projected to receive the same area size of each grid cell.
%The outer, parent grid is the ECMWF-IFS model (European Centre for Medium-Range Weather Forecasts Integrated Forecasting System) with a horizontal resolution of \SI{9}{\km} \citep{homleid_verification_2016}. The ECMWF-IFS forecasts are used \SI{6}{\hour} prior to the actual cycle in MEPS.
The regional model MEPS receives initial and boundary conditions from the global ECMWF-IFS (European Centre for Medium-Range Weather Forecasts Integrated Forecasting System) before it can produce its own regional forecasts. In addition, to produce the forecast analysis the background model is initiated for upper-air and surface data assimilation \citep{muller_arome-metcoop:_2017}.
The horizontal resolution of the parent ECMWF grid is \SI{9}{\km}, has \num{137} model levels, and the model level top is at \SI{80}{\km}. The ECMWF-IFS forecasts are available \SI{5}{\hour} later than the model runs at Met-Norway. MEPS is updated each third hour using obsrvations received in real-time from the global observing system \citep{homleid_verification_2016}.
%used \SI{3}{\hour} prior to the actual cycle in MEPS with.
Since initial conditions such as observations have uncertainties as well as the model has internal variability, it has to reach a background climatology state (spin-up) before the output can be analysed. %has mistrust, and  the own climatology needs to be approached, a model has to stabilize before the simulations can be trusted.
%The initial conditions such as observations and the model itself have uncertainties.
\citet{warner_tutorial_1997} states, spin-up time varies depending on the amount and quality of the initial and boundary conditions. If only a few mesoscale initial condition are available then the model should be initialised well before the forecast time. This will allow the model to spin-up mesoscale structures that are responsive to large-scale and local forcing.
In MEPS, the spin-up time %varies depending on the quality of the initial and boundary conditions,
can be assumed to be \SI{6}{\hour} for precipitation \citep[personal communication,][]{Priv_Comm_Koltzow}.
\\
To model the snow in AROME-MetCoOp an one-layer atmosphere model scheme is implemented. The representation is covered by an adjustment of the three-class ice parametrization (ICE3) scheme (\Cref{sec:AROME:adjustment}).
This includes three variables such as: snow water equivalent (SWE), snow density, and snow albedo \citep{muller_arome-metcoop:_2017}.
How liquid-phase processes are separated from slow ice-phase processes are described in \Cref{sec:MesoNH}.
% \\
% As synoptic observations are included in the model the snow-depth predictions underlay a special performance. Observations of snow-depth are only available at \SIlist{06;18}{\UTC}, therefore the snow analysis is only performed twice daily \citep{muller_arome-metcoop:_2017, homleid_verification_2016}.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%% MESONH %%%%%%%%%%%%%%
\subsection{Meso-NH and the ICE3 Scheme} \label{sec:MesoNH}
The physical parametrisation within AROME is based on the French research communities' Meso-NH. The microphysical scheme in the Meso-NH atmospheric simulation system is based on the Kessler scheme for liquid processes whereas the ICE3 parametrisation scheme is for cold processes \citep{meteo_france_meso-nh_2009}. The purpose of the scheme is to model as correctly as possible the ice phase in the atmosphere. The three-class parametrisation scheme is coupled to a Kessler scheme for the warm processes \citep{pinty_mixed-phased_1998}. \cite{mccumber_comparison_1991} concluded from their case study of simulating two different types of tropical convection, that at least three different ice categories are necessary to cover most precipitation but that applications might be case specific.
%\\%
\par\medskip\noindent
According to the \cite{meteo_france_meso-nh_2009} documentation, the ice phase microphysical scheme includes:
\begin{itemize}
	\item [$\mathbf{i}$:] pristine ice phase
	\item [$\mathbf{s}$:] snowflake type from lightly rimed large ice crystals or dry clusters, and
	\item [$\mathbf{g}$:] heavily rimed crystals, such as graupel, frozen drops or hail.
\end{itemize}
Within the ICE3 scheme no distinction between hail and graupel exists and therefore the physical discrimination is in the growth mode of graupel and hail is neglected. \\
To achieve snow water content within MEPS the number intercept parameter ($N_0$, [\SI{}{\metre^{-3}\per\mm}]), slope parameter of exponential size distribution ($\lambda$, [\SI{}{\per\metre}]), mass diameter ($D$, [\SI{}{\mm}]) and  the particle size distribution [\SI{}{\metre^{-3}\per\mm}] of pristine ice ($n_\mathbf{i}$), snowflakes ($n_\mathbf{s}$), and rimed crystals ($n_\mathbf{g}$)  has to be determined.
According to \cite{caniaux_numerical_1994}, the particle size distribution in the ICE3 scheme follows the Marshall-Palmer distribution (\Cref{eq:num_dens}). The goal in ICE3 is to use a varying intercept parameter dependent on the ice category. The study of \citet{caniaux_numerical_1994} has shown that $N_0$ can be parametrised with:
\begin{align}
	N_0 & = C \lambda^x  \label{eq:model_N0}
	\\
	\log_{10}C & = -3.55x + 3.89  \nonumber
\end{align}
where $C$ and $x$ are constants depending on the ice category %and represent the relation between each other in
(\Cref{eq:model_N0}).
\\
The ice water content for primary ice, snowflakes, and rimed crystals is then assumed to be similar to \Cref{eq:SWC}, but the integration limits range from zero to infinity and mass (($m$, [\SI{}{\kg}])), and particle size distribution ($n(D)$) are dependent on the diameter of the hydrometeor particle. The mass of a single particle and PSD (\Cref{eq:mass_diameter,eq:PSD_MEPS}) are represented depending on the ice category (\Cref{tab:ice_parameter})
\begin{align}
	m(D) & = aD^b 	\label{eq:mass_diameter} \\
	n(D) & = N_0 g(D)	\label{eq:PSD_MEPS}
\end{align}
$a$, $b$ are the characterisations of the parameters according to their type (\Cref{tab:ice_parameter}) and $g(D)$ the generalised Gamma function:
\begin{align}
	g(D) = \frac{\alpha}{\Gamma(\nu)} \lambda^{\alpha \nu} D^{\alpha \nu -1} \exp\left( -(\lambda D)^\alpha \right)
\end{align}
with $\alpha$, $\nu$ the shape and tail dispersion parameters and $\Gamma(\nu)$ the gamma function.
%%% table ice parameters %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\input{./tables_tex/ice_category_scheme}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\\
After following the above equations including \Cref{eq:SWC} the exponential slope parameter of pristine ice, snow, and rimed crystals, $\lambda$ can be generated with $G(b)$, the gamma function:
\begin{align}
	\lambda & = \left( \frac{\text{SWC}}{aCG(b)}\right)^{\frac{1}{x-b}}
\end{align}
\\
For all hydrometeors the terminal fall velocity based on the diameter, $D$ is assumed.
\begin{align}
	V(D) = c D^d \left(\frac{\rho_{00}}{\rho_{dref}}\right)^0.4 . \label{eq:fall_velo_MEPS}
\end{align}
In \Cref{eq:fall_velo_MEPS} the last factor is the \citet{foote_terminal_1969} correction of the air density and $\rho_{00}$ being the air density at the reference pressure level $P_{00}$.
%
%
\\
\Cref{fig:ICE3_scheme} shows the summary of the microphysical processes for mixed phase clouds. The study focuses mostly on solid precipitation particles and therefore only the initiation and growth of pristine ice crystals $\mathbf{i}$, snowflakes $\mathbf{s}$, and rimed crystals $\mathbf{g}$ is presented.
%%% image ICE3 scheme  %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\input{./images_tex/meps_ice3_scheme}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\noindent
\\
Following \cite{pinty_mixed-phased_1998} and \Cref{fig:ICE3_scheme} it can be seen how AROME calculates ice growth. %
\begin{itemize}
	\item The ICE3 scheme starts with \textit{cold} - 'slow' processes for ice processes (right side in \Cref{fig:ICE3_scheme})
	\begin{itemize}
		\item homogeneous (HON) and heteorogeneous (HEN) nucleation
		\item vapour deposition of snow and graupel particles (DEP)
		\item aggregation (AGG) and auto conversion (AUT)
	\end{itemize}
	\item The second step is to initiate the \textit{warm} processes (left side in \Cref{fig:ICE3_scheme})
	\item Then including the
	\begin{itemize}
		\item aggregation and conversion-melting (CVM) for snowflakes and
		\item contact freezing of raindrops (CFR)
	\end{itemize}
	\item Followed by AGG and melting for graupel (MLT)
	\item And the melting from pristine ice and the Wegener-Bergeron-Findeisen (BER) effect
	\item finally integrate the sedimentation terms
\end{itemize}

%\cite{meteo_france_meso-nh_2009} documentation suggests starting the microphysics in the ICE3 scheme with 'slow' processes such as homogeneous and heterogeneous nucleation (HON, HEN), vapour deposition of snow and graupel particles (DEP), aggregation (AGG) and auto conversion (AUT), for ice processes right side in \Cref{fig:ICE3_scheme}. The second step is to initiate the warm processes left side in \Cref{fig:ICE3_scheme}. Then include the aggregation and conversion-melting (CVM) for snowflakes and contact freezing of raindrops (CFR). Add AGG and melting for graupel (MLT), and then the melting from pristine ice  and the Wegener-Bergeron-Findeisen (BER) effect and lastly the sedimentation terms.  \\

% are the primary ice crystals activated by either heterogeneous nucleation (HEN), when some ice nuclei are present or by homogeneous nucleation (HON), when the atmospheric temperature is below \SI{-35}{\celsius}. The growing process for these particles can be the Bergeron-Findeisen process (BER) or deposition of water vapour (DEP). \\
% Snow particles are initiated by auto conversion (AUT) of r$_i$ and grow by the riming of cloud droplets (RIM) or rain droplets (ACC), and by collection of small pristine crystals (AGG), indicated by the green lines in \Cref{fig:ICE3_scheme}. \\
% As indicated by the blue lines in \Cref{fig:ICE3_scheme} are graupel an effect of heavy riming and grow in the scheme if the riming aggregates are of larger diameter size than \SI{7}{\mm} \citep{meteo_france_meso-nh_2009}. As indicated can graupel also grow when larger colliding raindrops reshape snowflakes to graupel (ACC). Furthermore, is graupel growth affected by wet or dry (WET, DRY) accretion, when the surface temperature of graupel is larger than the environmental temperature. DRY graupel expansion happens as long as the surface temperature is less that the surrounding temperature, then collected drops will freeze. WET graupel growth appears, when a liquid film is on the surface of the graupel particle (surface temperature larger than surrounding) then the liquid condensate is shed away, and hail will be formed. In ICE3 is hail not specifically included it is mixed in to the graupel category and therefore a distinction between the two categories cannot be made.

%
%\newpage
\subsection{AROME-MetCoOp Adjustment}\label{sec:AROME:adjustment}
% refinement for AROME
Since the ICE3 scheme showed some weaknesses in AROME-MetCoOp for the boreal winter month, \cite{muller_arome-metcoop:_2017} introduced some modifications.
During cold conditions (\SI{2}{\metre} temperature between \SIlist{-5;-10}{\celsius}) the ICE3-scheme followed too low \SI{2}{\metre} temperature in AROME-MetCoOp. Furthermore, too much ice fog or low clouds were simulated for \SI{2}{\metre} temperature $\le$ \SI{-15}{\celsius}, all year long. After implementing the modifications such as separating fast liquid-phase processes from the slower ice-processes, as well as reducing sublimation speed of ice particles. Also, taking into account the difference of optical thickness between ice-phase clouds and water, mixed-clouds reduced the negative \SI{2}{\metre} temperature bias \citep{muller_arome-metcoop:_2017}. %the two meter temperature bias was reduced as well as an improvement of low-level clouds was shown.
A negative aspect of these adjustments was that the occurrence of fog increased.%, by an error in the surface scheme.

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% %%%%%%%%% SURFEX %%%%%%%%%%%%%%
% \section{SURFEX}
% \cite{masson_surfexv7.2_2013} \\
% SURFEX stands for 'surface externalisée' and is introduced into NWP models to ensure the consistent treatment of surface processes. It simulates the exchange of energy between four surface types and the atmosphere \citep{homleid_verification_2016}.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


%!TEX root = ../thesis.tex
%*******************************************************************************
%*********************************** First Chapter *****************************
%*******************************************************************************

\chapter{Viscous acoustic flow}  %Title of the First Chapter

\ifpdf
    \graphicspath{{Chapter1/Figs/Raster/}{Chapter1/Figs/PDF/}{Chapter1/Figs/}}
\else
    \graphicspath{{Chapter1/Figs/Vector/}{Chapter1/Figs/}}
\fi

In this chapter we start with the review of the fully compressible Navier--Stokes equations and aim to derive two limit cases: incompressible flow and acoustic flow equations from the power expansion of the initial problem by assuming that the flow is governed by some low characteristic numbers. We will see that in the case of low Mach number of the base and fluctuating flows the initial problem can be split into two sub-problems. Then, we discuss both the strong and the weak forms for each of two cases; finally, we consider possible boundary conditions and their implementation.

\section{Equations of motion}

The full compressible Navier--Stokes equations govern the motion of a fluid with the presence of viscous effects, heat conductivity, compressibility, external body forces and many others. In general, the equations in conservative form are given by \cite{LandauHydro}:

\begin{equation}
\label{eq:cNSPrimForm}
    \frac{\partial}{\partial t} \textbf{q} + \frac{\partial}{\partial x_k} \left( \textbf{f}^c_k(\textbf{q}) -  \textbf{f}^v_k(\textbf{q}, \nabla \cdot \textbf{q}) \right) = 0 \ \  \ in  \ \Omega
\end{equation}

Here we use $^c$ and $^v$ referring to convective and viscous parts of the equations. The vector of conservative variables $\textbf{q}$ and the fluxex $f^c(\textbf{q}), f^v(\textbf{q})$ are defined by

\begin{align}
\label{eq:cNSFluxes}
    \textbf{q} &= \begin{bmatrix}
           \rho \\
           \rho u_1 \\
           \rho u_2 \\
           \rho E
         \end{bmatrix},&
    f_k^c(\textbf{q}) &= \begin{bmatrix}
           \rho u_1 \\
           \rho u_1 u_k + P \delta_{1k} \\
           \rho u_2 u_k + P \delta_{2k}\\
           \rho u_k H
         \end{bmatrix},&
    f_k^v(\textbf{q}) &= \begin{bmatrix}
           0 \\
           \tau_{k1} \\
           \tau_{k2} \\
           \tau_{kj}u_j + \partial_{k} \kappa T
         \end{bmatrix}
  \end{align}

The viscous stress tensor $\tau_{ij}$ is proportional to the dynamic viscosity coefficient $\mu$ and equals to

\begin{equation}
    \tau_{ij} = \mu \left( \nabla_i u_j + \nabla_j u_i - \frac{2}{3} div(\textbf{u}) \  \delta_{ij} \right)
\end{equation}

Introduced variables $\rho, \textbf{u}, P, E$ and $H$ denote the density, velocity vector, pressure, total energy and enthalpy of the flow:

\begin{subequations}
    \begin{align}
        & P(\textbf{q}) = (\gamma - 1) \rho e, \\
        &\rho E = \rho e + \frac{\rho \textbf{u}^2}{2}, \\
        &H(\textbf{q}) = \frac{\rho E + P}{\rho}
    \end{align}
\end{subequations}

Static internal energy $e$ is related to the temperature $T$ of the flow by $e = c_v T$, where $c_v$ denotes the constant volume heat capacity. We can define the Prandtl number $Pr = \frac{\mu \gamma c_v}{\kappa}$, $\kappa$ is the thermal conductivity coefficient. For example, in the case of an ideal gas, a Prandtl number $Pr = 0.72$. We also need to introduce an equation of state, which relates the pressure, density and temperature, i.e. $\rho = \rho(P,T)$.

Usually, solving these equations is a challenge itself, including choosing an appropriate discretization and numerical methods, which is another noticeable research area. For example, discontinuous Galerkin finite element method \cite{hartmann2008} is a promising candidate to solve the full problem.

\section{Equations of motion in the low Mach number limit}

Now, let us introduce two parameters to describe the flow. Here we follow the results presented in \cite{Muller99lowmach} for the low Mach number asymptotics of the Navier--Stokes equations, and \cite{culick2006unsteady} for two-parameter expansion derivation.

First, consider that we have a steady fluid flow in a channel with rigid boundaries, and the mean inlet velocity is in order of $1-10$ m/s. Then the Mach number of the base mean flow can be characterized by a small parameter, $\mu = M^b$, since the velocity of the flow is much smaller than the speed of sound $c_s^b = \sqrt{\gamma P^{b}/\rho^{b}}$, such that the local Mach number is:

\begin{equation}
M = \frac{u}{\sqrt{\gamma P^{b}/\rho^{b}}} = \frac{u}{u^{b}} M^{b} \rightarrow 0 \ \ if \ \ M^{b} \rightarrow 0
\end{equation}

\noindent where $^b$ corresponds to the reference base dimensional values describing the flow. We choose the reference quantities such that the $u/u^{b}$ is in order of $\mathcal{O}(1)$.

Second, the flow is perturbed by a small boundary oscillations created by a piezo-element. These fluctuations are created at the boundary and propagate inside the channel. The inkjet printing device is designed to create a sequence of small droplets jetting through a tiny orifice, and these oscillations are the primal mechanism of droplets formation at the nozzle outlet boundary [REF]. Although the boundary displacement is negligibly small [REF], acoustic waves form inside the channel and represent an additional physical phenomenon to be studied. This type of fluctuating motion should be small in comparison to the mean quantities, and moreover be described by another parameter. Introducing a new small characteristic number, $\varepsilon$, we will use it to define the Mach number of the fluctuating flow.

These two small parameters, $\mu$ and $\varepsilon$, represent different physical phenomena. Consequently, the formal expansion of the governing equations will give us terms with different orders in $\mu, \varepsilon$: $\mathcal{O}(\mu^{k_1}\varepsilon^{k_2})$, where $k_1, k_2$ are some integer numbers (e.g., $k_2 = 0$ for a flow with no oscillations). This generally characterize the flow processes. For example, as will be shown later, setting $\varepsilon$ to zero and collecting terms of $\mathcal{O}(\mu)$ leads to the well-known momentum equation of an incompressible flow.

\subsection{Nondimensionalization}

To perform the following analysis of the low Mach number limit of the equations of motion, it is necessary to define the nondimensional variables. New equations will include dimensionless quantities such as the Reynolds number (if the flow is viscous), the Froude number (if gravity is included into the model), or, for instance, those related to the considered forces acting on the flow.

Dimensional reference quantities can be based on the far-field conditions, but in the case of the internal flow it seems more natural to refer to the steady flow state without mean flow: $\rho^b, T^b, c_s^b, \mu^b$ (reference density, temperature, speed of sound and viscosity) and a domain size $L$ as a reference length scale. The reference pressure $P^b$ used here is chosen as a function of density and speed of sound: $P^b = \rho^b (c_s^b)^2$. Hence a new set of variables becomes:

\begin{subequations}
\label{eq:NSderDimens}
\begin{align}
    \frac{\rho}{\rho^b} \rightarrow \rho, \ \ \frac{P}{P^b} \rightarrow P, \ \ \frac{T}{T^b} \rightarrow T, \ \ \frac{u}{c_s^b} \rightarrow u\\
    \frac{E}{P^b/\rho^b} \rightarrow E, \ \ \frac{H}{P^b/\rho^b} \rightarrow H, \\
    \frac{x}{L} \rightarrow x, \ \ \frac{t}{L/ c_s^b} \rightarrow t
\end{align}
\end{subequations}

The dimensionless parameters such as the Reynolds number appear in the equations at this stage, with $Re = \rho^b c_s^b L / \mu^b$. These values depend on the choice of the reference values.

While some approaches \cite{culick2006unsteady} are based on the speed of sound dimensionalization, also in some cases choosing a known velocity of the flow (for example, at the inlet) as a reference quantity helps obtaining slightly different and more demonstrative results, such as temporal multiple scales analysis \cite{Muller99lowmach}. However, this approach is not suitable if the inlet velocity is zero, whilst the speed of sound is never a small quantity. We will use speed of sound as a reference velocity for the compressible flow, and the non-zero inlet velocity in the incompressible case.

\subsection{Flow values expansion}

As discussed before, physical processes in the flow arise from two different sources: the mean flow characterized by the parameter $\mu$ and the fluctuating flow related to the second parameter $\varepsilon$. We can then write the flow state \textbf{q} as a sum of mean values $\bar{\mathbf{q}}$ and fluctuating values $\tilde{\mathbf{q}}$:

\begin{equation}
\label{eq:NSderMeanFluctExpansion}
\rho = \bar{\rho} + \tilde{\rho}, \ \ u = \bar{u} + \tilde{u}, \dots
\end{equation}

Refering back to the original idea of two parameters analysis, fluctuating values should be of the order in amplitude $\varepsilon$, while some mean values are in order of 1, not $\mu$. For instance, consider the no-mean flow configuration when $\mu = 0$, so the dimensional flow density becomes $\rho = \rho^b = \mathcal{O}(1)$.

We substitute (\ref{eq:NSderMeanFluctExpansion}) into the full system of compressible Navier--Stokes equations (\ref{eq:cNSPrimForm}) and gather terms which have the same order of magnitude (see Appendix A). In particular, holding only the leading terms in $\mu, \epsilon$, the split mass and momentum conservation equations become:

\begin{equation}
\label{eq:NSderMassSplit}
\left[ \partial_t  \bar{\rho} + \nabla_i (\bar{\rho} \bar{u}_i) \right] + \left[ \partial_t  \tilde{\rho} + \nabla_i (\bar{\rho} \tilde{u}_i) \right] + \mathcal{O}(\varepsilon^2, \varepsilon \mu, \dots) = 0
\end{equation}
%+ \left[ \nabla_i \left(\tilde{\rho} (\bar{u}_i + \tilde{u}_i) \right) \right]
\begin{equation}
\label{eq:NSderMomenSplit}
\left[ \partial_t  (\bar{\rho} \bar{u}_i) + \nabla_j (\bar{\rho} \bar{u}_i\bar{u}_j) + \nabla_i \bar{P} - \frac{1}{Re} \nabla_j \bar{\tau}_{ij} \right] + \left[ \partial_t (\bar{\rho} \tilde{u}_i) + \nabla_i \tilde{P} - \frac{1}{Re} \nabla_j \tilde{\tau}_{ij} \right] +  \mathcal{O}(\varepsilon^2, \varepsilon \mu, \dots) = 0
\end{equation}

The first bracket in both (\ref{eq:NSderMassSplit}) and (\ref{eq:NSderMomenSplit}) contains only terms related to the mean state $\bar{\mathbf{q}}$, and the second bracket gathers the fluctuating quantities $\tilde{\mathbf{q}}$. The rest of the equations represents the combination of the mean and fluctuating flow with terms containing both Mach numbers, thus describes the interactions between these two physical processes.

Direct consideration of the first bracket in (\ref{eq:NSderMomenSplit}) shows that all terms are proportional to some power of $\mu$: mean density $\bar{\rho}$ has the leading term of $\mathcal{O}(1)$ as shown before, the temporal derivative and the Reynolds number are proportional to $\mu$ since the speed of sound was chosen to nondimensionalize them (it can be clearly seen from (\ref{eq:NSderDimens}) that by choosing $u^b$ instead of $c_s^b$ as a reference velocity quantity results in the $u^b/c_s^b = \mathcal{O}(\mu)$ term appearing). Consequently, all terms in the first bracket of (\ref{eq:NSderMomenSplit}) are $\mathcal{O}(\mu^2)$ except the pressure gradient, which should be in order of $\mathcal{O}(1)$. To understand this seeming discrepancy, let us follow \cite{Muller99lowmach} and expand pressure in power series of $\mu$:

\begin{equation}
\bar{P} = \bar{P}_0 + \mu \bar{P}_1 + \mu^2 \bar{P}_2 + \mathcal{O}(\mu^3)
\end{equation}

Now, applying this expansion to the first bracket of (\ref{eq:NSderMomenSplit}) and gathering terms with the same power of the small parameter gives:

\begin{subequations}
\begin{flalign}
    &\mu^0: \ \ \nabla_i \bar{P}_0 = 0, \\&
    \mu^1: \ \ \nabla_i \bar{P}_1 = 0, \\&
    \mu^2: \ \ \partial_t  (\bar{\rho} \bar{u}_i) + \nabla_j (\bar{\rho} \bar{u}_i\bar{u}_j) + \nabla_i \bar{P_2} - \frac{1}{Re} \nabla_j \bar{\tau}_{ij} = 0
\end{flalign}
\end{subequations}

Performing similar analysis for the energy equation and considering the mean flow quantities behavior, it becomes clear that the zeroth and first order pressure terms are related to the total energy density and act like a thermodynamic pressure. The second order term $\bar{P}^2$ however plays a similar role as the pressure in the incompressible Navier--Stokes equation.

To finally derive the incompressible Navier--Stokes equation, assume that the variation of density and temperature of the flow is small in comparison to the no-flow state, $||\delta \bar{\rho}|| = ||\bar{\rho} - \rho^b|| \ll || \rho^b ||$, or, in other words, $\delta \bar{\rho} = \mathcal{O}(\mu)$. This leads us to the Boussinesq buoyancy equations \cite{Rehm}, which to the lowest order in $\mu$ results in:

\begin{subequations}
\begin{align}
        div(\bar{u}) = 0, \\
        \bar{\rho} \partial_t  (\bar{u}_i) + \bar{\rho} (\bar{u}_j \nabla_j)\bar{u}_i + \nabla_i \bar{P}_2 - \frac{1}{Re} \nabla_j \bar{\tau}_{ij} = 0
\end{align}
\end{subequations}

The second brackets in (\ref{eq:NSderMassSplit}) and (\ref{eq:NSderMomenSplit}) are the acoustic motion of the flow; terms are proportional to $\varepsilon$ and do not contain information about the mean flow. Since the first bracket terms were shown to be zero, we can conclude that the second bracket terms should also be zero, leading us to the equations of linear acoustics:

\begin{subequations}
\label{eq:acousticT}
\begin{align}
        \partial_t  \tilde{\rho} + \bar{\rho}  \nabla_i (\tilde{u}_i) = 0, \\
        \bar{\rho} \partial_t (\tilde{u}_i) + \nabla_i \tilde{P} - \frac{1}{Re} \nabla_j \tilde{\tau}_{ij} = 0
\end{align}
\end{subequations}

Necessary to note, that the problem can be separated in two smaller problems: incompressible mean flow and linear acoustics, only due to the fact that we assumed terms of $\mathcal{O}(\varepsilon^2, \varepsilon \mu, \dots)$ to be small and therefore we neglect the interaction between the mean flow and the fluctuating flow. Otherwise, mean state $\bar{\mathbf{q}}$ and fluctuating state $\tilde{\mathbf{q}}$ are coupled through the higher order terms which represent mass, momentum and energy transfer between them.

\clearpage
\section{Mean flow in the low Mach number limit}

Low Mach number expansion of the Navier--Stokes equations results in the mass and momentum conservation laws of incompressible base flow and classical linear acoustic problem. In this section we consider both steady and unsteady incompressible flow and briefly discuss the governing equations and weak formulation.

\subsection{Problem statement}

Incompressible Navier--Stokes equation can be considered as an equation of motion plus a constraint, that makes the flow divergenceless. The pressure variable is not directly related to the thermodynamic state, and adjusts in a way that the velocity constraint is satisfied. The governing equations are:

\begin{subequations}
\label{eq:IncNSeqFull}
    \begin{align}
    \label{eq:IncNSeq}
    \frac{\partial u_i}{\partial t} + (u_j \nabla_j) u_i + \nabla_i P - \frac{1}{Re} \Delta u_i = 0, \\
    \label{eq:div0cons}
    div(u) = 0
    \end{align}
\end{subequations}

This dimensionless form contains a characteristic Reynolds number $Re$, which defines the flow behaviour. Previously, speed of sound was used as a reference speed; however, in this case it is necessary to dimensionalize velocity by some known value related to the base flow itself, for example, inlet velocity, since there is no meaningful sound speed. Then, Reynolds number is defined as $Re = \frac{\rho U_{in} L}{\mu}$.

We consider four types of boundary conditions:

\begin{itemize}
    \item Inlet boundary

    Velocity is prescribed at the inlet boundary, but not pressure. This is Dirichlet boundary condition, namely:
    \begin{equation}
    u = U_{in} \ \ on \ \ \Gamma_{in}
    \end{equation}

    \item No slip boundary

    No slip boundary imposes no flow through the wall, and adhesion condition for viscous flows. Then, both normal and tangential velocity components are zero and the no slip Dirichlet boundary condition becomes:

    \begin{equation}
    u = 0 \ \ on \ \ \Gamma_{nsl}
    \end{equation}

    \item Slip boundary

    In some cases, for example if fluid is inviscid or a symmetry plane is modelled, slip wall boundary condition is required:

    \begin{equation}
    u_n = 0 \ \ on \ \ \Gamma_{nsl}
    \end{equation}

    Here $u_n$ is a normal component of the flow velocity. Slip boundary condition allows non-zero flow along the surface.

    \item Outlet boundary

    Outlet, or no-stress boundary is used to model outflow. Since pressure appears only in the gradient term, it is defined up to some constant. Let us set the outer pressure to zero, then the stress-free condition turns into

    \begin{equation}
    n_j (P\delta_{ij} - \frac{1}{Re} \nabla_j u_i) = P n_i - \frac{1}{Re} \frac{\partial u_i}{\partial n} = 0
    \end{equation}

\end{itemize}

Additionally, in the case of unsteady flow, initial condition $u(x,t=0)$ should be provided.

\subsection{Weak formulation}

Consider for now that the flow is steady, $\frac{\partial u_i}{\partial t} = 0$, and here we aim to construct a weak form of the reduced problem. Non-zero time derivative will be taken into account in the following section.

First, we multiply the incompressible Navier--Stokes equation (\ref{eq:IncNSeq}) by vector test function, $v_i$, and the divergence constraint (\ref{eq:div0cons}) by scalar function, $q$, and integrate both over the computational domain. Second, we integrate the highest order derivatives, $-\frac{1}{Re}\Delta u_i$ and $\nabla_i P$ by parts and collect all the integrals together. This leads to:

\begin{equation}
\label{eq:IncNSWeakForm}
\left<v_i (u_j \nabla_j) u_i - P div(v) + \frac{1}{Re}\nabla_j v_i \nabla_j u_i - q div(u)\right> + \left\{ vi \left( P n_i - \frac{1}{Re} \frac{\partial u_i}{\partial n} \right) \right\} = 0
\end{equation}

The boundary integral disappears completely due to the boundary conditions. Inlet and no slip Dirichlet boundaries result in $v = 0$ since the choice of test function is arbitrary. Slip boundary sets both $u_{\tau}, v_{\tau}$ to zero and the remaining term vanishes due to the symmetry (or, equivalently, no shear stress at the boundary). Outlet boundary condition is exactly the last boundary integral multiplyed by some function, such that the last remaining term is zero everywhere.

Summing up, the weak form of the incompressible Navier--Stokes equation with the given boundary types contains only a volumetric integral, with Dirichlet boundary conditions for inlet, slip and no slip boundaries.

\subsection{Newton method}

The problem (\ref{eq:IncNSeqFull}) is nonlinear, the convective term is proportional to $u^2$. Let us recall the weak form (\ref{eq:IncNSWeakForm}) and a steady residual form $\mathbb{R}(q)$ is:

\begin{equation}
\mathbb{R}(q) = \left<v_i (u_j \nabla_j) u_i - P div(v) + \frac{1}{Re}\nabla_j v_i \nabla_j u_i - q div(u)\right>
\end{equation}

It should be zero if $q = (u,P)$ satisfies the incompressible Navier--Stokes equation. Consider we have an initial guess of the flow state $q_0 = (u_0, P_0)$, which can, for instance, be chosen zero everywhere or satisfy the linear Stokes equation. We employ a Newton method to solve the nonlinear system with a known initial guess. To find the next iterate we linearize the residual form by taking the Frechet derivative with respect to state $q$. Linearization of the Dirichlet boundary conditions is straightforward, which finally gives:

\begin{equation}
\label{eq:IncNSFrechet}
\mathbb{R}(q^{n+1}) = \mathbb{R}(q^n) + \frac{\partial \mathbb{R}}{\partial q} \delta q + \mathcal{O}(\delta q^2)
\end{equation}

And the linearized weak problem becomes:

\begin{equation}
\left<v_i \left( (u_j \nabla_j) \delta u_i + (\delta  u_j \nabla_j) u_i \right) - \delta  P div(v) + \frac{1}{Re}\nabla_j v_i \nabla_j \delta  u_i - q div(\delta u)\right> = - \mathbb{R}(q^n)
\end{equation}

Then the initial nonlinear problem turns into a linear system on $\delta q$. Setting (\ref{eq:IncNSFrechet}) to zero and finding $\delta q$, we update the previous solution as $q^{n+1} = q^n + \delta q$ and calculate the $l_2$-norm of the updated residual, $\Vert \mathbb{R}(q^{n+1}) \Vert _2$. If the norm is less than the chosen tolerance, the process is stopped. Otherwise, we continue the convergence procedure.

A damped Newton method can be useful for some problems. It means, that the solution is updated by weighed step, $\alpha_w \delta q$, which is dynamically chosen to maintain better convergence rate. In present research, we used the undamped Newton algorithm.

\subsection{Unsteady mean flow}

Now, consider time-dependent Navier--Stokes equation, with a new residual form to be introduced:

\begin{equation}
    \mathbb{N}(q(t)) = \left< v_i \frac{\partial u_i}{\partial t} \right> + \mathbb{R}(q(t))
\end{equation}

There are several methods of solving the unsteady problem, for example, fractional step method \cite{DoneaFrac}. However, we want to employ the existing solution of the steady problem, and we aim to derive an implicit scheme for time integration. The idea is to perform the temporal discretization first, and then reduce new system to previous case. The Euler approach presents time derivative as

\begin{equation*}
    \frac{\partial u}{\partial t} \rightarrow \frac{u^{n+1} - u^n}{\Delta t}
\end{equation*}

Here we perform time discretization as $t^k = k\Delta t$, where $\Delta t$ is time step. The unsteady weak formulation then becomes:

\begin{equation}
\label{eq:IncNSweakUnst}
    \left< v_i \frac{u^{n+1}_i}{\Delta t} \right> + \mathbb{R}(q^{n+1}) = \left< v_i \frac{u^{n}_i}{\Delta t} \right>
\end{equation}

This is still a nonlinear problem, and we follow the same strategy as for the steady problem. Taking the Frechet derivative of (\ref{eq:IncNSweakUnst}), the iterative scheme is given by:

\begin{subequations}
\begin{align}
    \left< v_i \frac{\delta u_i}{\Delta t}  + v_i \left( (u_j \nabla_j) \delta u_i + (\delta  u_j \nabla_j) u_i \right) - \delta  P div(v) + \frac{1}{Re}\nabla_j v_i \nabla_j \delta  u_i - q div(\delta u)\right> = \\
    \label{eq:IncNSweakUnsRHS}
    - \mathbb{R}(q) + \left< v_i \frac{u^{n}_i - u_i}{\Delta t} \right>
\end{align}
\end{subequations}

Here linear system is solved for $\delta q$, $u^n$ corresponds to the known velocity at the previous time step, and $u_i$ is the velocity iterate updated as $u_i \rightarrow u_i + \delta u_i$ until the convergence criteria is met. As the right hand side (\ref{eq:IncNSweakUnsRHS}) goes to zero, a new solution is found for time $t^{n+1}$.

The advantage of this approach is only a little difference from the steady formulation. First, the time derivative term $v_i \frac{\delta u_i}{\Delta t}$ is added to the bilinear form, and second, the semi-time derivative $\frac{u^{n}_i - u_i}{\Delta t}$ appears on the linear form. Essentially, the solution process doesn't differ from that previously discussed.
\clearpage

\section{Acoustic flow in low Mach number limit}

In the previous sections we discussed the technique of splitting the initial set of compressible Navier--Stokes equations into two smaller problems, the incompressible mean flow and the acoustic flow, governed by two small parameters, $\mu$ and $\varepsilon$, respectively. Here we consider the acoustic, or the fluctuating flow created by an oscillating piezo-element boundary. We develop the general strong and weak formulation of the acoustic problem, types of boundary conditions relevant to the problem, to finally obtain the adjoint system and the stability sensitivity to the changes in shape.

\subsection{Problem statement}

We recall the result of the low Mach number expansion, and the linear acoustic equations (\ref{eq:acousticT}) will be the primal object of interest. Since the viscous source term in the energy equation (\ref{eq:cNSFluxes}) is proportional only to the squared amplitude of the velocity $\mathcal{O}(u^2)$, this allows us to assume that the energy production due to the viscosity is small thus making our flow isentropic for the purpose of relating $\tilde{P}$ to $\tilde{\rho}$. Note that we retain viscosity in the momentum equation, where it appears at first order in $u$. The isentropic flow property allows us to eliminate the pressure term in the momentum equation which in dimensional case is related to density through the speed of sound, $c_s$: $\nabla \tilde{P} = c_s^2 \nabla \tilde{\rho}$. Summing up, the remaining dimensional continuity and momentum equations are:

\begin{subequations}
\label{eq:linNS}
\begin{align}
        \partial_t \tilde{\rho} + \rho_0 div(\tilde{u}) = 0, \\
        \rho_0 \partial_t \tilde{u}_i + c_s^2 \nabla_i \tilde{\rho} - \mu \left(\Delta \tilde{u}_i + \frac{1}{3} \nabla_i div(\tilde{u}) \right) = 0.
\end{align}
\end{subequations}

For the purpose of the stability analysis of the acoustic problem, instead of solving the unsteady problem we move to the frequency domain assuming solutions in form of $\tilde{\mathbf{q}}(x,t) = \hat{\mathbf{q}}(x) e^{st}$, such that $\partial_t (\rho', u') \rightarrow s (\hat{\rho}, \hat{u})$, where $\hat{\rho}, \hat{u}$ are  complex  amplitudes  and $s$ is a complex  frequency  of  a  mode. New dimensionless variables become: $\tilde{\rho}/\rho_0 \rightarrow \hat{\rho}, \tilde{u}_i / c_s \rightarrow \hat{u}_i , \mu/ (\rho_0 c_s L) \rightarrow \nu \equiv Re^{-1}, s L/c_s \rightarrow s $. We  look  for  a  normal mode solution, so the  eigenvalue problem can finally be written as:

\begin{subequations}
\label{eq:strViscAc}
\begin{align}
        -div(\hat{u}) = s \hat{\rho}, \\
        \nabla_i \hat{\rho} - \nu \left(\Delta \hat{u}_i + \frac{1}{3} \nabla_i div(\hat{u}) \right) = - s \hat{u}_i .
\end{align}
\end{subequations}

The sign of the first equation becomes important for the symmetry of the weak form. The momentum equation can be rewritten in the form of the Newton's law, by introducing a new variable for the stress tensor $\hat{\sigma}_{ij} = - \hat{\rho} \delta_{ij} + \nu \left( \nabla_j \hat{u}_i + \nabla_i \hat{u}_j - 2/3 \delta_{ij} div(\hat{u}) \right)$. The divergence of the stress tensor is an acting force, thus

\begin{equation}
s \hat{u}_i = \nabla_j \hat{\sigma}_{ij}
\end{equation}

Now the acceleration term on the left hand side equals to the force term on the right hand side. We will use the stress tensor notation while formulating the weak problem and the boundary conditions.

\subsection{Weak formulation}

Now, the weak form of the given dimensionless eigenvalue problem is introduced. Multiplying the first scalar equation by a scalar test function, $w$, and the momentum equation by a vector test function, $v_i$, and integrating over the domain we have:

\begin{subequations}
\begin{align}
        \left< -div(\hat{u}) w \right> =\left< s \hat{\rho} w \right>, \\
        \left< - v_i \nabla_j \hat{\sigma}_{ij} \right>= \left<- s \hat{u}_i v_i\right>
\end{align}
\end{subequations}

After summation and integration by part the highest order derivatives once we will have several boundary integrals, namely:

\begin{equation}
\label{eq:AcousGenerWeak}
    \left< - div(\hat{u}) w  + \hat{\sigma}_{ij} \nabla_j v_i \right>  - \left\{ v_i n_j \hat{\sigma}_{ij} \right\} = s \left< \hat{\rho} w - \hat{u}_i v_i \right>.
\end{equation}

The volumetric terms $\left<\dots\right>$  and the boundary terms $\left\{\dots\right\}$ of the weak form are not yet symmetric to the swap of the test functions to trial functions. We won't symmetrize the boundary integrals at this point, but after the boundary conditions are considered. Regarding the volume integrals, from the symmetry of the stress tensor it is clear that:

\begin{subequations}
\begin{align}
        &\left<\hat{\sigma}_{ij} \nabla_j v_i \right> = \left<\frac{1}{2}\hat{\sigma}_{ij} (\nabla_j v_i + \nabla_j v_i) \right>  = \\
        &\left<- \hat{\rho} div(v) +  \frac{\nu}{2} \left( \nabla_j \hat{u}_i + \nabla_i \hat{u}_j - \frac{2}{3} \delta_{ij} div(\hat{u}) \right) (\nabla_j v_i + \nabla_i v_j) \right> = \\
        \label{eq:DirWeakSymm}
        &\left<- \hat{\rho} div(v) +  \frac{\nu}{2} \left( \nabla_j \hat{u}_i + \nabla_i \hat{u}_j - \frac{2}{3} \delta_{ij} div(\hat{u}) \right) (\nabla_j v_i + \nabla_i v_j - \frac{2}{3} \delta_{ij} div(v) ) \right>
\end{align}
\end{subequations}

Substituting (\ref{eq:DirWeakSymm}) back to the (\ref{eq:AcousGenerWeak}), one can see, that by rearranging terms in the volume integral it is possible to maintain symmetry between the test and trial functions.

The weak formulation can be alternatively written in matrix form:

\begin{equation}
    \mathbf{A}\mathbf{q}_k = s_k \mathbf{B}\mathbf{q}_k
\end{equation}

Here \textbf{A},\textbf{B} are the matrix representation of the left and right  hand  side  differential  operators in (\ref{eq:strViscAc}),  $\mathbf{q}_k$ is the state vector $(\hat{\rho}, \hat{u})$, i.e. a natural mode of the system, and $s_k$ is  the $k$-th eigenvalue.

\subsection{Boundary conditions}

Since the acoustics is decoupled from the mean flow, we will call inlet and outlet boundaries $free$ boundaries, while the $slip$ and $no slip$ boundaries still have the same meaning as for the mean flow problem. Then, boundary conditions for two state variables - density and velocity perturbations $\rho$ and $u$, can be written as:

\begin{subequations}
\begin{align}
    \hat{\sigma}_{ij} n_j  = 0 \ \ on  \ \ \Gamma_{free}, \\
    \hat{u}_i = 0 \ \ on \ \ \Gamma_{nsl},\\
    \hat{u}_i n_i = 0 \ \ on \ \ \Gamma_{sl}
\end{align}
\end{subequations}

The free boundary condition reads as zero stress on the inlet and outlet boundaries, and in the limit of zero viscosity $\nu \rightarrow 0$ it acts like a pressure (density) node.

Additionally, the presence of a slip boundary imposes the symmetry of the system. Then, every variable implies to be either an even or an odd function. In case of the normal velocity, $u_n$, it is odd as a result of the $u_i n_i = 0$ boundary condition. Then it is clearly seen (e.g. from the continuity equation), that two other variables, $\rho$ and $u_{\tau}$ must be even with respect to the normal symmetry. This gives us an additional boundary condition $\frac{\partial}{\partial n}u_{\tau}= 0$ on the symmetry plane.

%Let us make a small remark regarding a vector's components normal and tangential to a boundary. Consider we have a general expression for a scalar product of a vector $a_i$ and a normal directional derivative of a vector $\frac{\partial b_i}{\partial n}$. It can be shown, that:

%\begin{subequations}
%\begin{align}
%    a_i \frac{\partial b_i}{\partial n} = \left(a_n n_i + a_{\tau} \tau_i \right) n_j \nabla_j \left( b_n n_i + b_{\tau} \tau_i \right) = \\
%    \label{eq:scalProdDirDer}
%    \left(a_n \frac{\partial b_n}{\partial n} + a_{\tau} \frac{\partial b_{\tau}}{\partial n} \right) + \left(a_n b_{\tau} - a_{\tau} b_n \right) n_j n_i \nabla_j \tau_i
%\end{align}
%\end{subequations}

%Here we denote $n_i$ as $i-$th component of the surface normal vector and ${\tau}_i$ stands for a vector tangent to surface; $a_n, b_n$ are (scalar) projections of two vectors on $n$, and $a_{\tau}, b_{\tau}$ are (scalar) tangent projections. Note, that in (\ref{eq:scalProdDirDer}) there is a term which represents surface curvature; it appears because of the changing direction of surface normal along the boundary. While applying boundary conditions to the weak form, one should be careful not to forget the presence of the surface curvature. However, in many cases both $a_{\tau}$ and $b_{\tau}$ are zero, so the curvature term vanishes.

Now, we can apply these boundary conditions to the weak formulation by introducing different boundary terms. Moreover, it becomes clear how to make the full weak form symmetric and still consistent with the strong form.

\begin{itemize}
  \item Free boundary, $\Gamma_{free}$

  First, free boundary models an open end; thus, a zero force boundary condition is considered. This yields:

  %\begin{equation}
  %\left\{ v_i n_j \hat{\sigma}_{ij} \right\}_{\Gamma_{free}} = \left\{ v_n n_i n_j \hat{\sigma}_{ij} + v_{\tau} \tau_i n_j \hat{\sigma}_{ij} \right\}_{\Gamma_{free}}
  %\end{equation}
  \begin{equation}
  \left\{ v_i n_j \hat{\sigma}_{ij} \right\}_{\Gamma_{free}} = 0
  \end{equation}
  %The first term on the right hand side vanished due to the force boundary condition, and the second term after substitution of $\hat{\sigma}_{ij}$ becomes:

  %\begin{equation}
  %\left\{ - \nu v_{\tau} \tau_i \frac{\partial u_i}{\partial n} \right\}_{\Gamma_{free}}
  %\end{equation}

  %The $u_{\tau} = 0$ is a Dirichlet boundary condition, and if one apply the usual finite element Dirichlet boundary condition on the tangential velocity, the corresponding test function $v_{\tau}$ can be (or is automatically) chosen to be zero, thus the boundary term vanished completely. However, in some cases tangential direction is not constant (e.g. if the inlet boundary is curved), therefore it becomes less straightforward how to impose the $u_{\tau}$ value.

  A possible way to enforce the zero force boundary condition  is to impose it weakly by introducing a penalty boundary term \cite{Nitsche1971}, \cite{freund1995weakly}. We add a symmetric term consistent with the primal formulation, such that the boundary integral over the free surface becomes

  %\begin{equation}
  %\left\{ - \nu  v_{\tau} \tau_i \frac{\partial u_i}{\partial n} \right\}_{\Gamma_{free}} = \left\{ - \nu \left( v_{\tau} \tau_i \frac{\partial u_i}{\partial n}  + u_{\tau} \tau_i \frac{\partial v_i}{\partial n} \right) \right\}_{\Gamma_{free}} + \beta \left\{  u_{\tau}  v_{\tau} \right\}_{\Gamma_{free}}
  %\end{equation}

  \begin{equation}
  \frac{\beta}{h} \left\{ \sigma_{ij} n_j , \sigma_{ik}^\dagger n_k \right\}_{\Gamma_{free}}
  \end{equation}

  Here $\beta$ is a constant coefficient, and $h$ is the triangulation cells size; the ''daggered'' stress tensor takes test functions as variables: $\sigma_{ik}^\dagger = \sigma_{ik} (w, v)$. The updated form of the boundary term is consistent with the initial problem, since $\sigma_{ij} n_j$ vanishes on ${\Gamma_{free}}$.

  \item No slip boundary, $\Gamma_{nsl}$

  Second, the no slip boundary represents a rigid wall with no flow through it, so the velocity is zero at $\Gamma_{nsl}$ and no restriction on density is applied. Even if the no slip boundary is curved, i.e. if the normal and tangential directions depend on the position on the boundary, the Dirichlet boundary condition can be easily applied and thus the test function can be set to zero on $\Gamma_{nsl}$. Note, that the boundary term in (\ref{eq:AcousGenerWeak}) is linear on $v_i$ thus it vanishes.

  \item Slip boundary, $\Gamma_{sl}$

  Third, the slip or symmetry boundary implies that there is no normal flow, but the tangential velocity is not specified. The slip boundary in viscous flows usually appears as a plane of symmetry, which is a line in 2D or a plane surface in 3D, thus the Dirichlet boundary condition should be generally easy to implement. In other words, the normal velocity should vanish $u_n = 0$ as well as the normal test function can be set to zero $v_n = 0$. Assuming the surface normal being constant along the slip boundary, the surface integral becomes:

  \begin{equation}
  \begin{aligned}
  \left\{ v_i\sigma_{ij} n_j \right\}_{\Gamma_{sl}} =   \left\{ v_n n_i \sigma_{ij} n_j + v_{\tau} \tau_i \sigma_{ij} n_j \right\}_{\Gamma_{sl}} = \\
  \left\{ v_{\tau} \nu \left( \frac{\partial u_{\tau}}{\partial n} + \frac{\partial u_n}{\partial \tau} \right)  \right\}_{\Gamma_{sl}} = 0
  \end{aligned}
  \end{equation}

  The first term vanishes due to the symmetry of the symmetry of the slip plane, and the second term is zero since the normal velocity component is constant in the tangent direction.

  \item Robin (compliant) boundary

  Finally, the last boundary type to be presented is a mixed Robin boundary, or a compliant boundary. Previous boundaries were defined by either Dirichlet or Neumann type of boundary conditions. Furthermore, we can assume a mixed case or equivalently a boundary condition which relates the components of the state vector $\hat{\mathbf{q}}$ to each other; or, a value to its gradient. The velocity Robin boundary condition generally appears in the following form:

  \begin{equation*}
     \frac{\partial u}{\partial n} = \frac{1}{\varepsilon} (u - u_0) + f \ \ on \ \ \Gamma_{Rob}
  \end{equation*}

  As the parameter $\varepsilon$ approaches two limit cases, $\varepsilon \to 0$ or $\varepsilon \to \infty$, the Robin boundary condition turns into a inhomogeneous Dirichlet or Neumann boundary condition, respectively. The quantities of $u_0$ and $f$ then play role of prescribed velocity or velocity gradient at the boundary. Boundary condition becomes homogeneous when $u_0$ or $f$ are zero.

  Following \cite{JuntunenS09}, it is possible to extend the Nitsche approach for imposing Dirichlet boundary condition \cite{Nitsche1971} to broader range of boundary types with $\epsilon \ge 0$. The final weak formulation is consistent, symmetric and successfully compared to the traditional techniques. We refer to \cite{JuntunenS09} for the detailed derivation of the general method, and the following section will consider the particular case of the Robin condition.
\end{itemize}

\subsection{Compliant boundary}

We consider the following problem: the domain now acts as a solid structure which boundaries are not fixed and can displace, reacting to the flow. This is a fluid-structure interaction (FSI) problem, and the modeled geometry and thus the computational domain may change as a result of boundary displacement. When the displacement is big enough to actually affect the mean flow, computational meshes and structure position should be moved. A number of methods describing the motion of fluid particles and mesh nodes exists, for instance the Arbitrary-Lagrange-Eulerian (ALE) approach \cite{Donea1982}. Moving boundary condition prescribes equality of the flow and solid surface quantities, namely kinematic and dynamic conditions. First, as soon as the surface is compliant (in contrast to permeable), the continuity of displacements and velocities is required; second, the stresses in the fluid and the body should be the same \cite{Donea2004}. These conditions couple the fluid-structure system.

Moreover, one can consider a FSI problem when boundaries are compliant, but surface deformation is tiny in comparison to the domain's size. This is the case of the inkjet printer chamber, whose height is in order of 100 $\mu$m and the boundary displacement is less than 100 nm. Then the relative displacement is lower than $10^{-3}$ and modelling this phenomena as a full FSI problem with the mesh movement is excessive. However, the flow can differ from those obtained with rigid slip or no slip boundary, especially for the oscillating acoustic quantities. This leads us to the idea of treating the compliant boundary in a more suitable way: with no actual domain and mesh deformation but taking into account the nature of the solid-fluid interaction on this boundary type.

The Robin boundary can describe a boundary which act like a moving membrane, reacting to the perturbations in the flow. The acoustic compliant boundary condition links the flow density (or pressure) and velocity at a certain surface point and a complex boundary impedance $Z$ is a function which relates the normal velocity component to pressure \cite{Myers1980}:

\begin{equation}
\label{eq:InitImpedAcoustic}
u_i n_i = - \frac{P}{Z}
\end{equation}

Furthermore, we should extend this definition for the viscous acoustic problem. Recalling the idea of the impedance concept, \enquote{impedance denotes originally the ratio between a force amplitude and a velocity amplitude} \cite{rienstra}. Now, a force is represented not only by pressure, but also by viscous stress tensor. Then, the impedance-based relation becomes:

\begin{equation}
\label{eq:InitImped}
Z u_i = \sigma_{ij} n_j  =   (-P \delta_{ij} + \tau_{ij}) n_j
\end{equation}

We would neither restrict the velocity to be zero in the tangential direction, nor allow only normal displacements of the compliant boundary. Boundary impedance describes the physical properties of the boundary, such as stiffness, elasticity, thickness and others. Usually, the impedance value is known for harmonic motion with a given frequency, making the boundary condition frequency dependent: $Z = Z(s)$. As discussed before, both acoustic pressure and velocity are considered to be proportional to $(\tilde{P}, \tilde{u}) \sim e^{st}$ thus they can become out of phase due to the complex impedance boundary condition.

To apply the compliant boundary condition to viscous acoustic flow, we need to rearrange some terms in (\ref{eq:InitImped}). First, performing low Mach number expansion of the impedance expression and nondimensionalizing it, we obtain an equation of the velocity and density fluctuations coupled through the dimensionless acoustic impedance, $\hat{Z} = \frac{Z}{\rho^b c_s^b}$. Second, the pressure fluctuation is substituted with density. Summing up:

\begin{equation}
\label{eq:BCCompl}
\hat{Z} \hat{u}_i= \hat{\sigma}_{ij} n_j  \ \ on \ \ \Gamma_{com}
\end{equation}

Now, we can derive boundary integrals of the weak problem on the compliant boundary. The boundary relation (\ref{eq:BCCompl}) can now be inserted into the weak form.

\begin{equation}
\left\{ v_i n_j \hat{\sigma}_{ij} \right\}_{\Gamma_{com}} = \left\{ Z v_i u_i \right\}_{\Gamma_{com}}
\end{equation}

The boundary integral is symmetric; for the same reason as we did in the free boundary case, additional penalty term should be added to impose compliant boundary condition. Consequently, the compliant boundary integral becomes:

\begin{equation}
\label{eq:ComplBoundInt}
\left\{\hat{Z} v_i u_i \right\}_{\Gamma_{com}} + \frac{\beta}{h} \left\{ Z u_i - \hat{\sigma}_{ij} n_j , Z v_i - \hat{\sigma}_{ij}^{\dagger} n_j \right\}_{\Gamma_{com}}
\end{equation}

The initial acoustic problem is an eigenvalue problem, and previously the boundary terms were independent of $s$. Both terms in (\ref{eq:ComplBoundInt}) is proportional to $\hat{Z}(s)$, so the impedance needs to be moved to the right hand side of the weak form. But then the problem becomes nonlinear on $s$, which requires to introduce some complication to the calculation routine. While this is not preferable, we assume $\hat{Z}$ to be fixed and only rearrange the term directly dependent on $s$. We solve the simplified eigenvalue problem iteratively, substituting the complex frequency into the known impedance function until the convergence criteria is reached. The algorithm for the direct and adjoint problems will be discussed separately.

Finally, two limit cases should be discussed. Compliant boundary condition is in close relation to both free and no slip conditions, which is clearly seen from (\ref{eq:BCCompl}). As $\hat{Z} \to 0$, the compliant boundary turns into the free surface, i.e. $n_j \hat{\sigma}_{ij} \to 0$. The boundary integral (\ref{eq:ComplBoundInt}) approaches the free boundary term as well. If the impedance amplitude is high, $\hat{Z} \to \infty$, we can expect no slip behavior, and the terms proportional to $\hat{Z}$ in (\ref{eq:ComplBoundInt}) dominate, weakly imposing Dirichlet boundary condition on $\Gamma_{com}$ by penalty formulation. The concept of one universal parameter, $\hat{Z}$ in this case, which describes each of the existing boundary types with only one quantity, is quite convenient, since the development of a numerical solver becomes more generalized in comparison to boundaries with different physics behind.


\section{Deep Learning: Advanced}
\subsection{Autoencoders}
\subsubsection{Variational Autoencoders}
\subsection{General Adversarial Networks}
\subsection{Encoder-Decoder Models}
Encoder-decoder models allow for many to many RNN sequence learning. The idea
for them is two take a source sequence, encode it to a vectorized output, and
pass the hidden state over to the decoder (and possible the encodings themselves)
to generate output. These models come up in machine tranlsations, allowing
models to translate a source language into a target language.
\subsubsection{Encoders}
% The encoder, or \emph{inference network}, recieves an input sequence
% $\arrow{x} = \left[x_1, \cdots, x_n \right]$ and encodes the sequence through
% recurrent layers.
\subsubsection{Decoders}
\subsection{Attention Models}
Attention models were discussed by Bahdanau, and later Luong. They are a means
to alter the representation of a set of encodings to pay attention to certain
sequence elements more so than others. In an encoder-decoder model with out
attention, the decoder relies exclusivly on the hidden state to hold all
information. Attention, by contrast, uses the hidden state to create a context
vector of the encodings.
\subsubsection{Context Vector}
Attention mechanisms rely on one simple concept: producing context vectors.
Hence, given a set of encodeings $h = \left[h_i, \cdots, h_j, \cdots h_n \right]$
from the encoder, and the current hidden state $s_{i-1}$, we compute a score
for each encoding $h_j$, denoted $score(s_{i-1}, h_j)$. These scores are discussed
in later sections. With a score computed for each encoding, we apply a softmax
across all scores:
\begin{equation}
  a(s_{i-1}, h_j) = \frac{\exp(score(s_{i-1}, h_j))}{\sum_{j'} \exp(score(s_{i-1}, h_{j'}))}
\end{equation}
These softmax proabilities are attention weights, and our context vector $c_i$
is the weighted sum of the encodings given these weights:
\begin{equation}
  c_i = \sum_{j'} a(s_{i-1}, h_{j'}) \cdot h_{j'}
\end{equation}
These context vectors are appended to the embedding of the token set into the decoder.
\subsubsection{Concat (Bahdanau) Attention}
The concat attention scoring mechanism has two learnable weights, $v_a$ and $W_a$
and is defined as:
\begin{equation}
  score(s_{i-1}, h_j) = v_a^T \cdot \tanh \left( W_a \cdot \left[ s_{i-1} ; h_j \right] \right)
\end{equation}
\subsubsection{Luong Attention Mechanisms}
Luong defined several global attention mechanisms, defined here:
\begin{equation}
  score(s_{i-1}, h_j) =
    \begin{cases}
      s_{i-1}^T \cdot W_a \cdot h_j & general \\
      s_{i-1}^T \cdot h_j & dot
    \end{cases}
\end{equation}
Luong also explored local attention weights, with monotonic alignment and
predictive alignment. The difference in local attention is that the
context vector is derived on source hidden states within the window
$\left[p_t - D , \, p_t + D\right]$ for some $D$ and decoder time step $t$ (i.e. $i-1$).
In monotonic alignment, aligned position $p_t=t$.
In predicitve alignment, the aligned position is:
\begin{equation}
  p_t = S \cdot \sigma \left( v_p^T \tanh \left( W_p s_{i-1} \right) \right)
\end{equation}
where $v_p$ and $W_p$ are learnable parameters,
$S$ is the source sentence length. In addition, the weights are gaussian centered
around $p_t$:
\begin{equation}
  a(s_{i-1}, h_j) = \frac{\exp(score(s_{i-1}, h_j))}{\sum_{j'} \exp(score(s_{i-1}, h_{j'}))} \cdot \exp\left(-\frac{(s-p_t)^2}{2\left(\frac{D}{2}\right)^2}\right)
\end{equation}
Here, $s$ is an integer within the window centered at $p_t$.

\subsection{Word Embeddings}
Word Embeddings seek to create a dense vector representation of real values
to represent a single word in a euclidean space. Word vectors are weight matrices
where each row corresponds to a word, acessed through a numerical token representing
the word itself. However, several models have been developed to train these
word embedding weights to encode lexical meaning.
\subsubsection{N-Gram Models}
A simple apporach is to take a context of size $n$ words that appear before our
target word, and attempt to train a model to predict a word $w_i$ that appears
in a training corpus. We effectively average the input word embeddings, apply
a linear layer to project the data to the vocab size, and maximize the probability
of predicitng the target word (i.e. apply a softmax of the output vector).
\begin{equation}
  \max \, p(w_{i} \,|\, w_{i-n}, \dots, w_{i-1}) = \max \frac{\exp f(w_{i-n}, \dots, w_{i-1})_i}{\sum_{j=1}^{|V|}\exp f(w_{i-n}, \dots, w_{i-1})_j }
\end{equation}
Where our model is:
\begin{equation}
  f(w_{i-n}, \dots, w_{i-1}) = W_d \cdot \left( \frac{1}{n}\sum_{j=1}^n W_e[w_{i-j}] \right) + b_d \in \mathbb{R}^{|V|}
\end{equation}
Here, $|V|$ is the size of the vocabulary, $W_d\in\mathbb{R}^{d\times|V|}$ is
a projection matrix used for training along with bias $b_d$,
and $W_e\in\mathbb{R}^{|V|\times d}$ are the word embeddings we seek to train.
$d$ is the embedding dimension. \\
Now we can write the negative log likelihood as:
\begin{equation}
    \min -\log p(w_{i} \,|\, w_{i-n}, \dots, w_{i-1}) = - f(w_{i-n}, \dots, w_{i-1})_i + \log \sum_{j=1}^{|V|} \exp f(w_{i-n}, \dots, w_{i-1})_j
\end{equation}
\subsubsection{Continuous Bag-of-Words Models (CBOW)}
The continuous bag of words model, or CBOW, seeks to predict a center word
given the surrounding context words. For instance, given a context size of $n$,
for a target center word $w_i$, we are given context words
$w_{i-n}, \dots w_{i-1}, w_{i+1}, \dots w_{i+n}$ and attempt to predict $w_i$.
Hence:
\begin{equation}
  \max \, p(w_{i} \,|\, w_{i-n}, \dots, w_{i-1}, w_{i+1}, \dots, w_{i+n}) = \max \frac{\exp f(w_{i-n}, \dots, w_{i-1}, w_{i+1}, \dots, w_{i+n})_i}{\sum_{j=1}^{|V|}\exp f(w_{i-n}, \dots, w_{i-1}, w_{i+1}, \dots, w_{i+n})_j }
\end{equation}
Where our model is:
\begin{equation}
  f(w_{i-n}, \dots, w_{i-1}, w_{i+1}, \dots, w_{i+n}) = W_d \cdot \left( \frac{1}{2*n}\sum_{j=-n,j\not=0}^n W_e[w_{i+j}] \right) + b_d \in \mathbb{R}^{|V|}
\end{equation}
Here, $|V|$ is the size of the vocabulary, $W_d\in\mathbb{R}^{d\times|V|}$ is
a projection matrix used for training along with bias $b_d$,
and $W_e\in\mathbb{R}^{|V|\times d}$ are the word embeddings we seek to train.
$d$ is the embedding dimension. \\
Now we can write the negative log likelihood as:
\begin{equation}
  \begin{split}
    \min -\log p(w_{i} \,|\, w_{i-n}, \dots, w_{i-1}, w_{i+1}, \dots, w_{i+n}) =& - f(w_{i-n}, \dots, w_{i-1}, w_{i+1}, \dots, w_{i+n})_i \\
    &+ \log \sum_{j=1}^{|V|} \exp f(w_{i-n}, \dots, w_{i-1}, w_{i+1}, \dots, w_{i+n})_j
  \end{split}
\end{equation}
\subsubsection{Skip-Gram Model}
\subsubsection{Negative Sampling}


\documentclass[american]{beamer}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\setcounter{secnumdepth}{3}
\setcounter{tocdepth}{3}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{babel}
\usepackage{amsmath}

\usetheme{Berlin}
\usecolortheme{whale}

\title{Random Walk with Varying Transition Probabilities Applied on Tennis Modelling}
\subtitle{Mathsport International 8 - 2021}
\institute[Tomáš Kouřim (tom@skourim.com)]{\inst{} Faculty of Nuclear Sciences and Physical Engineering, CTU Prague \and
    \inst{} Institute of Information Theory and Automation, CAS CR Prague}
\date{25.6.2021}
\author[\selectlanguage{czech}%
Tomáš Kouřim\selectlanguage{czech}%
]{Tomáš~Kouřim \and }
\newcommand{\nologo}{\setbeamertemplate{logo}{}} % command to set the logo to nothing
\setbeamertemplate{itemize item}[circle]
\setbeamertemplate{itemize subitem}[square]
\setbeamercovered{transparent}
%\logo{\includegraphics[height=1.3cm]{../CMSIM_Logo.pdf}}

\begin{document}
    \maketitle

    \section{Motivation}\label{sec:motivation}

    \begin{frame}{Random walk}
        \begin{definition}
            A man starts from a point $O$ and walks $l$ yards in a straight line;
            he then turns through any angle whatever and walks another $l$
            yards in a second straight line.
            He repeats this process $n$ times.
            I require the probability that after these $n$ stretches he is at
            a distance between $r$ and $r+\delta r$ from his starting point, $O$.

            {\footnotesize{}\medskip{}\emph{[Karl Pearson: The problem of the random walk.(1905)]}}

            \vspace{10mm}
            \begin{itemize}
                \item[]<2-> \large{Where is the \emph{drunken sailor}?}
            \end{itemize}
        \end{definition}
    \end{frame}

    \begin{frame}{Motivation - Random process with varying probability}
        \includegraphics[width=1\textwidth]{../../simulations/probability_25_steps_type_success_punished_two_lambdas}
    \end{frame}

    \begin{frame}{Motivation}
        \begin{itemize}
            \item Failure of a machine
            \begin{itemize}
                \item repair after failure
                \item preventive maintenance
            \end{itemize}
            \item<2-> Occurrence of a disease
            \begin{itemize}
                \item cure of the disease
                \item prevention (i.e. lifestyle change)
            \end{itemize}
            \item<3-> Development of sports match
            \begin{itemize}
                \item goal scored, point achieved
                \item period won
            \end{itemize}
        \end{itemize}
    \end{frame}

    \begin{frame}{Motivation - Random process with varying probability}
        \includegraphics[width=1\textwidth]{../../simulations/probability_dots_25_steps_type_success_punished_two_lambdas}
    \end{frame}

    \section{Model description}\label{sec:model-description}
    \begin{frame}{Random walk with varying probabilities}
        \begin{itemize}
            \item<1-> Random walk with memory based the on standard Bernoulli random walk
            \item<2-> Given starting probability $p_0$
            \item<3-> $X_t\in\{-1,1\}$ with $X_t \sim {\textrm{Bernoulli} }(p_{t-1})$
            \item<4-> Memory coefficient $\lambda\in(0,\,1)$ affecting the development of probabilities $p_{t}$ as
            \onslide<4->\begin{flalign*}
                            X_{t} & =1\rightarrow p_{t}=\lambda p_{t-1} &
                            X_{t} & =-1\rightarrow p_{t}=1-\lambda(1-p_{t-1}) &
            \end{flalign*}
            \vspace{-5mm}
            \begin{itemize}
                \item[-->]<5-> ``Success punishing''
            \end{itemize}
            \onslide<6->\begin{flalign*}
                            X_{t}&=1\rightarrow p_{t}=1-\lambda(1-p_{t-1})&
                            X_{t}&=-1\rightarrow p_{t}=\lambda p_{t-1}&
            \end{flalign*}
            \vspace{-5mm}
            \begin{itemize}
                \item[-->]<6-> ``Success rewarding''
            \end{itemize}
        \end{itemize}
    \end{frame}

    \begin{frame}{Example - RW development}
        \includegraphics[width=1\textwidth]{../../simulations/single_walk_1000_steps_type_success_punished}
    \end{frame}

    \begin{frame}{Example - RW development}
        \includegraphics[width=1\textwidth]{../../simulations/single_walk_1000_steps_type_success_punished_p0_0.55}
    \end{frame}

    \begin{frame}{Walk steps properties}

        \begin{flalign*}
            EX_{t} & =(2\lambda-1)^{t-1}(2p_{0}-1)&
        \end{flalign*}
        \vspace{-5mm}
        \begin{flalign*}
            \lim_{t\to+\infty}EX_{t}&=0&
        \end{flalign*}
        \onslide<2->
        \begin{flalign*}
            Var\,X_{t}&=1-(2\lambda-1)^{2(t-1)}(2p_{0}-1)^{2}&
        \end{flalign*}
        \vspace{-5mm}
        \begin{flalign*}
            \lim_{t\to+\infty}Var\,X_{t}&=1&
        \end{flalign*}

    \end{frame}

    \begin{frame}{Example - RW steps}
        \includegraphics[width=1\textwidth]{../../simulations/e_step_1000_walks_50_steps_type_success_punished}
    \end{frame}

    \begin{frame}{Walk probabilities properties}
        \begin{flalign*}
            EP_{t} & =(2\lambda-1)^{t}p_{0}+\frac{1-(2\lambda-1)^{t}}{2} &
        \end{flalign*}
        \vspace{-5mm}
        \begin{flalign*}
            \lim_{t\to+\infty}EP_{t} & =\frac{1}{2} &
        \end{flalign*}
        \onslide<2->
        \begin{flalign*}
            Var\,P_{t} & =(3\lambda^{2}-2\lambda)^{t}p_{0}^{2}+\sum_{i=0}^{t-1}K(i;p_{0},\lambda)(3\lambda^{2}-2\lambda)^{t-1-i}-k(t;p_{o},\lambda)^{2} &
        \end{flalign*}
        \vspace{-5mm}
        \begin{flalign*}
            \lim_{t\to+\infty}Var\,P_{t} & =\frac{\frac{1}{2}(1-\lambda^{2})}{-3\lambda^{2}+2\lambda+1}-\frac{1}{4} &
        \end{flalign*}
    \end{frame}

    \begin{frame}{Example - RW probabilities}
        \includegraphics[width=1\textwidth]{../../simulations/e_probability_1000_walks_50_steps_type_success_punished}
    \end{frame}

    \begin{frame}{Walk position properties}
        \begin{flalign*}
            ES_{t} & =S_{0}+(2p_{0}-1)\frac{1-(2\lambda-1)^{t}}{2(1-\lambda)} &
        \end{flalign*}
        \vspace{-5mm}
        \begin{flalign*}
            \lim_{t\to+\infty}ES_{t} & =S_{0}+\frac{(2p_{0}-1)}{2(1-\lambda)} &
        \end{flalign*}
        \onslide<2->
        \begin{flalign*}
            Var\,S_{t} & =t+4\sum_{i=0}^{t-1}\sigma(i;p_{0},0,\lambda)-a(t;p_{0},\lambda) &
        \end{flalign*}
        \vspace{-5mm}
        \begin{flalign*}
            \lim_{t\to+\infty}Var\,S_{t} & =c_{1}(p_{0},\lambda)t+c_{2}(p_{0},\lambda) &
        \end{flalign*}
    \end{frame}

    \begin{frame}{Example - RW position}
        \includegraphics[width=1\textwidth]{../../simulations/e_position_1000_walks_50_steps_type_success_punished}
    \end{frame}

    \begin{frame}{Success rewarding model}
        \onslide<2->
        \begin{flalign*}
            EX_{t} & =2p_{0}-1 &
        \end{flalign*}
        \vspace{-8mm}
        \begin{flalign*}
            Var\,X_{t} & =4p_{0}(1-p_{0}) &
        \end{flalign*}
        \onslide<1->
        \begin{flalign*}
            EP_{t} & =p_{0} &
        \end{flalign*}
        \vspace{-8mm}
        \onslide<2->
        \begin{flalign*}
            Var\,P_{t} & =(2\lambda-\lambda^{2})^{t}p_{0}^{2}+p_{0}(1-\lambda)^{2}\sum_{i=0}^{t-1}(2\lambda-\lambda^{2})^{i}-p_{0}^{2} &
        \end{flalign*}
        \begin{flalign*}
            ES_{t} & =S_{0}+t(2p_{0}-1) &
        \end{flalign*}
        \vspace{-8mm}
        \begin{flalign*}
            Var\,S_{t} & =4p_{0}(1-p_{0})t^{2}+a(p_{0},\lambda)t-a(p_{0},\lambda)\frac{1-(2\lambda-\lambda^{2})^{t}}{(1-\lambda)^{2}} &
        \end{flalign*}
    \end{frame}

    \begin{frame}{More complex models}
        \begin{itemize}
            \item Two memory coefficients $\lambda$ each affecting one direction of the walk
            \item Again two variants -- success punishing and success rewarding
            \onslide<2->\begin{flalign*}
                            X_{t-1} & =1\rightarrow p_{t}=\lambda_{0} p_{t-1} &
                            X_{t-1} & =-1\rightarrow p_{t}=1-\lambda_{1}(1-p_{t-1}) &
            \end{flalign*}
            \vspace{-5mm}
            \begin{itemize}
                \item[-->]<2-> ``Two-parameter success punishing model''
            \end{itemize}
            \onslide<3->\begin{flalign*}
                            X_{t-1}&=1\rightarrow p_{t}=1-\lambda_{0}(1-p_{t-1})&
                            X_{t-1}&=-1\rightarrow p_{t}=\lambda_{1} p_{t-1}&
            \end{flalign*}
            \vspace{-5mm}
            \begin{itemize}
                \item[-->]<3-> ``Two-parameter success rewarding model''
            \end{itemize}
            \item<4-> $M$ steps, $\lambda(t)$, $n$-dimensional walk
        \end{itemize}
    \end{frame}

    \section{Model application}\label{sec:model-application}

    \begin{frame}{Model fitting}
        \begin{enumerate}
            \item Find $p_{0}$
            \item<2-> Find $\lambda$ parameter(s)
            \item<3-> Choose optimal model type
            \begin{itemize}
                \item <4-> Akaike Information Criterion (AIC) \& numerical optimization
            \end{itemize}
        \end{enumerate}
    \end{frame}

    \begin{frame}{Real life application}
        \begin{itemize}
            \item Results of tennis matches from February till May 2021 together with real life odds (both \emph{pre-match} and \emph{in-play}) provided by a bookmaker
	        \item Data divided into training (February - April) and testing (May) datasets
            \item $p_{0}$ estimated using the first set winning odds provided by the bookmaker
            \item ``Single lambda success rewarding'' model selected as best fit using AIC and training data
        \end{itemize}
    \end{frame}

    \begin{frame}{Real life application}
        \begin{itemize}
            \item Simulated in-play betting on set winner
            \begin{itemize}
                \item<2-> Bet if $p\geq {1.2}\frac{1}{odds}$
                \item<3-> Three betting strategies tested
                \item<4-> ROI 98-148\% within just one month of betting
                \item<5-> Only 65 bets placed
            \end{itemize}
        \end{itemize}
    \end{frame}

    \begin{frame}{Example - RW position}
        \includegraphics[width=1\textwidth]{../../simulations/account_balance_development}
    \end{frame}

    \begin{frame}{Summary}
        \begin{itemize}
            \item A specific model of a random walk with memory
            \item Model properties derived
            \item Possible applications in a set of real life scenarios
            \item Initial results show big potential of the model
        \end{itemize}
    \end{frame}

    \begin{frame}[plain]
        \begin{quote}
            \begin{center}
                \huge{Thank you.}
            \end{center}
            \vspace{10mm}
            \begin{center}
                \large{tom@skourim.com}
            \end{center}
        \end{quote}
    \end{frame}

\end{document}

paper/presentation/Predicting Tennis Outcomes Using Random Walks.tex


%!TEX root = main.tex

\section{Syntax}

\subsection*{Identifiers}

$$
\begin{array}{@{}lrcl@{\qquad}l}
\text{(identifier)} & \x{id} &::=&
  \dots \\
\end{array}
$$

\subsection*{Types}

$$
\begin{array}{@{}lrcl@{\qquad}l}
\text{(type)} & \x{typ} &::=&
  \x{id}~\x{typ-args}
    & \text{constructor} \\&&|&
  \KK{actor}^? \K\{~\x{typ-fields}~\K\}
    & \text{object} \\&&|&
  \KK{var}^? \x{typ}~\K[~\K]
    & \text{array} \\&&|&
  \x{typ}~\K?
    & \text{option} \\&&|&
 \KK{class}^?  \x{typ-params} \x{typ} \Kto \x{typ}
    & \text{function} \\&&|&
  \KK{async}~\x{typ}
    & \text{future} \\&&|&
  \K(~( (\x{id}~\K:)^? \x{typ})\K,^\ast~\K)
    & \text{tuple} \\
[1ex]
\text{(type fields)} & \x{typ-fields} &::=&
  \KK{var}^? \x{id}~\K:~\x{typ}
    & \text{field} \\&&|&
  \x{id}~\x{typ-params}~\x{exp-params}~\K:~\x{typ}
    & \text{function field} \\&&|&
  \x{typ-fields}~\K;~\x{typ-fields}
    & \text{composition} \\
[1ex]
\text{(type parameters)} & \x{typ-params} &::=&
  (\Klt~(\x{id}~(\Ksub \x{typ})^?)\K,^\ast~\Kgt)^? \\
[1ex]
\text{(type arguments)} & \x{typ-args} &::=&
  (\Klt~\x{typ}\K,^\ast~\Kgt)^? \\
\end{array}
$$

\subsubsection*{Desugaring}

$$
\begin{array}{@{}lrcl@{}}
\text{(type)} &
\K( \dots \K, \x{id} \K: \x{typ} \K, \dots \K) &:=&
  \K( \dots \K, \x{typ} \K, \dots \K) \\
[1ex]
\text{(type field)} &
\x{id} \x{typ-params} \x{exp-params} \K: \x{typ} &:=&
  \x{id} \K: \x{typ-params} \x{exp-params} \Kto \x{typ} \\
[1ex]
\text{(type parameters)} &
\Klt \dots \K, \x{id} \K, \dots \Kgt &:=&
  \Klt \dots \K, \x{id} \Ksub \K{Any} \K, \dots \Kgt \\&
\epsilon &:=&
  \Klt \Kgt \\
\text{(type arguments)} &
\epsilon &:=&
  \Klt \Kgt \\
\end{array}
$$

\subsection*{Expressions}

$$
\begin{array}{@{}lrcl@{\qquad}l}
\text{(literal)} & \x{lit} &::=&
  \x{nat}
    & \text{natural number} \\&&|&
  \x{float}
    & \text{floating-point number} \\&&|&
  \x{char}
    & \text{unicode character} \\&&|&
  \x{text}
    & \text{unicode text string} \\
[2ex]
\text{(unary operators)} & \x{unop} &:=&
  \K{+} ~|~ \K\textendash ~|~ \K{\^{}} \\
[1ex]
%\text{(count operators)} & \x{cntop} &:=&
%  \K{++} ~|~ \K{\textendash\,\textendash} \\
%[1ex]
\text{(binary operators)} & \x{binop} &:=&
  \K{+} ~|~ \K\textendash ~|~ \K{*} ~|~ \K{/} ~|~ \K{\%} \\&&|&
  \K{\&} ~|~ \;\K\textbar\; ~|~ \K{\^{}} ~|~ \Klt\Klt ~|~ \Kgt\Kgt ~|~ \K{+}\Kgt\Kgt ~|~ \Klt\Klt\Kgt ~|~ \Klt\Kgt\Kgt \\
[1ex]
\text{(relational operators)} & \x{relop} &:=&
  \K= ~|~ \K{!=} ~|~ \Klt ~|~ \Kgt ~|~ {\Klt}{\K=} ~|~ {\Kgt}{\K=} \\
\end{array}
$$

$$
\begin{array}{@{}lrcl@{\qquad}l}
\text{(expression)} & \x{exp} &::=&
  \x{lit}
    & \text{literal} \\&&|&
  \x{id}
    & \text{variable} \\&&|&
  \x{unop}~\x{exp}
    & \text{unary operation} \\&&|&
  \x{exp}~\x{binop}~\x{exp}
    & \text{binary operation} \\&&|&
  \x{exp}~\x{relop}~\x{exp}
    & \text{relational operation} \\&&|&
  \x{exp}~\K?
    & \text{option} \\&&|&
  \K(~\x{exp}\K,^\ast~\K)
    & \text{tuple} \\&&|&
  \x{exp}~\K.~\x{nat}
    & \text{tuple projection} \\&&|&
  \KK{actor}^? \K\{~\x{exp-fields}~\K\}
    & \text{object} \\&&|&
  \x{exp}~\K.~\x{id}
    & \text{object projection} \\&&|&
  \x{exp}~\K{:=}~\K{exp}
    & \text{assignment} \\&&|&
%  \x{cntop}~\x{exp}
%    & \text{count update} \\&&|&
  \x{unop}{\!\K=}~\x{exp}
    & \text{unary update} \\&&|&
  \x{exp}~\x{binop}{\!\K=}~\x{exp}
    & \text{binary update} \\&&|&
  \K[~\void{\KK{var}^?} \x{exp}\K,^\ast~\K]
    & \text{array} \\&&|&
  \x{exp}~\K[~\x{exp}~\K]
    & \text{array indexing} \\&&|&
  \x{exp}~\x{typ-args}~\x{exp}
    & \text{function call} \\&&|&
  \KK{not}~\x{exp}
    & \text{negation} \\&&|&
  \x{exp}~\KK{and}~\x{exp}
    & \text{conjunction} \\&&|&
  \x{exp}~\KK{or}~\x{exp}
    & \text{disjunction} \\&&|&
  \KK{if}~\x{exp}~\x{exp}~(\KK{else}~\x{exp})^?
    & \text{conditional} \\&&|&
  \KK{switch}~\x{exp}~\K\{~(\KK{case}~\x{pat}~\x{exp})\K;^\ast~\K\}
    & \text{pattern match} \\&&|&
  \KK{while}~\x{exp}~\x{exp}
    & \text{guarded loop} \\&&|&
  \KK{loop}~\x{exp}~(\KK{while}~\x{exp})^?
    & \text{unconditional loop} \\&&|&
  \KK{for}~\K(~\x{pat}~\KK{in}~\x{exp}~\K)~\x{exp}
    & \text{iteration} \\&&|&
  \KK{label}~\x{id}~(\K:~\x{typ})^?~\x{exp}
    & \text{label} \\&&|&
  \KK{break}~\x{id}~\x{exp}^?
    & \text{break} \\&&|&
  \KK{continue}~\x{id}
    & \text{continue} \\&&|&
  \KK{return}~\x{exp}^?
    & \text{return} \\&&|&
  \KK{async}~\x{exp}
    & \text{future} \\&&|&
  \KK{await}~\x{exp}
    & \text{future synchronisation} \\&&|&
  \KK{assert}~\x{exp}
    & \text{assertion} \\&&|&
  \x{exp}~\KK{is}~\x{exp}
    & \text{instance check} \\&&|&
  \x{exp}~\K:~\x{typ}
    & \text{type annotatione} \\&&|&
  \x{exp}~\K;~\x{exp}
    & \text{sequence} \\&&|&
  \K\{~\x{dec}\K;^\ast~\K\}
    & \text{block} \\&&|&
  \x{dec}
    & \text{declaration} \\
[1ex]
\text{(expression fields)} & \x{exp-fields} &::=&
  \KK{private}^? \KK{var}^? \x{id}~(\K:~\x{typ})^?~\K=~\x{exp}
    & \text{field} \\&&|&
  \KK{private}^? \x{id}~\x{typ-params}~\x{exp-params}~(\K:~\x{typ})^?~\K=^?~\x{exp}
    & \text{function field} \\&&|&
  \x{exp-fields}~\K;~\x{exp-fields}
    & \text{composition} \\
[1ex]
\text{(expression parameters)} & \x{exp-params} &::=&
  \K(~(\x{id}~\K:~\x{typ})\K,^\ast~\K) \\
\end{array}
$$

\subsubsection*{Desugaring}

$$
\begin{array}{@{}lrcl@{}}
\text{(expression)} &
\K\{ \K\} &:=&
  \K\{ \K{()} \K\} \\&
\KK{if} \x{exp}_1 \x{exp}_2 &:=&
  \KK{if} \x{exp}_1 \x{exp}_2 \K{()} \\&
\KK{loop} \x{exp} &:=&
  \KK{loop} \x{exp}_1 \KK{while} \; \K{true} \\&
\KK{label} \x{id} \x{exp} &:=&
  \KK{label} \x{id} \K: \x{typ} \x{exp} \\&
\KK{label} \x{id} \KK{while} \x{exp}_1 \x{exp}_2 &:=&
  \KK{label} \x{id} \KK{while} \x{exp}_1 (\KK{label} \x{id}_{\K{continue}} \x{exp}_2) \\&
\KK{label} \x{id} \KK{loop} \x{exp}_1 \KK{while} \x{exp}_2 &:=&
  \KK{label} \x{id} \KK{loop} \, (\KK{label} \x{id}_{\K{continue}} \x{exp}_1) \, \KK{while} \x{exp}_2 \\&
\KK{label} \x{id} \KK{for} \x{id} \KK{in} \x{exp}_1 \x{exp}_2 &:=&
  \KK{label} \x{id} \KK{for} \x{id} \KK{in} \x{exp}_1 (\KK{label} \x{id}_{\K{continue}} \x{exp}_2) \\&
\KK{break} \x{id} &:=&
  \KK{break} \x{id} \K{()} \\&
\KK{continue} \x{id} &:=&
  \KK{break} \x{id}_{\K{continue}} \K{()} \\&
\KK{return} &:=&
  \KK{return} \, \K{()} \\
\end{array}
$$
$$
\begin{array}{@{}rcl@{}}
\text{(expression field)} \hfill \\
\KK{private}^? \KK{var}^? \x{id} \K: \x{typ} \K= \x{exp} &:=&
  \KK{private}^? \KK{var}^? \x{id} \K= \x{exp} \K: \x{typ} \\
\KK{private}^? \x{id} \x{typ-params} \x{exp-params} \, (\K: \x{typ})^? \K= \x{exp} &:=&
  \KK{private}^? \x{id} \K= \, \KK{func} \x{id} \x{typ-params} \x{exp-params} \, (\K: \x{typ})^? \K= \x{exp} \\
\end{array}
$$

\subsection*{Patterns}

$$
\begin{array}{@{}lrcl@{\qquad}l}
\text{(pattern)} & \x{pat} &::=&
  \K\textunderscore
    & \text{wildcard} \\&&|&
  \x{id}
    & \text{variable} \\&&|&
  \x{lit}
    & \text{literal} \\&&|&
  \x{unop}~\x{lit}
    & \text{modified literal} \\&&|&
  \x{pat}~\K?
    & \text{option} \\&&|&
  \K(~\x{pat}\K,^\ast~\K)
    & \text{tuple} \\&&|&
  \x{pat}~\K:~\x{typ}
    & \text{type annotation} \\
\end{array}
$$

\subsection*{Declarations}

$$
\begin{array}{@{}lrcl@{\qquad}l}
\text{(declarations)} & \x{dec} &::=&
  \x{exp}
    & \text{expression} \\&&|&
  \KK{let}~\x{pat}~\K=~\x{exp}
    & \text{immutable} \\&&|&
  \KK{var}~\x{id}~(\K:~\x{typ})^?~\K=~\x{exp}
    & \text{mutable} \\&&|&
  \KK{async}^?\, \KK{func}~\x{id}^? \x{typ-params}~\x{exp-params}~(\K:~\x{typ})^?~\K=^?~\x{exp}
    & \text{function} \\&&|&
  \KK{actor}^?\, \KK{class}~\x{id}^? \x{typ-params}~\x{exp-params}~\K=^?~\K\{~\x{exp-fields}~\K\}
    & \text{class} \\&&|&
  \KK{type}~\x{id}~\x{typ-params}~\K=~\x{typ}
    & \text{type} \\%&&|&
%  \x{dec}~\K;~\x{dec}
%    & \text{sequencing} \\
\end{array}
$$

\subsubsection*{Desugaring}

$$
\begin{array}{@{}rcl@{}}
\text{(declaration)} \hfill \\
\KK{let} \x{pat} \K: \x{typ} \K= \x{exp} &:=&
  \KK{let} \x{pat} \K= \x{exp} \K: \x{typ} \\
\KK{var} \x{id} \K: \x{typ} \K= \x{exp} &:=&
  \KK{var} \x{id} \K= \x{exp} \K: \x{typ} \\
\KK{func} \x{id}^? \x{typ-params} \x{exp-params} \K= \x{exp} &:=&
  \KK{func} \x{id}^? \x{typ-params} \x{exp-params} \K: \, \K{()} \, \K= \x{exp} \\
\KK{async} \, \KK{func} \x{id}^? \x{typ-params} \x{exp-params} (\K: \x{typ})^? \K= \x{exp} &:=&
\KK{func} \x{id}^? \x{typ-params} \x{exp-params} \, (\K: \, \KK{async} \x{typ})^? \K= \, \KK{async} \x{exp} \\
\end{array}
$$

\subsection*{Programs}

$$
\begin{array}{@{}lrcl@{\qquad}l}
\text{(program)} & \x{prog} &::=&
  \x{dec}\K;^\ast
\end{array}
$$


\section{Introduction}
%\cite{SurynekFSB16}
% Problem definition
% The \textit{Multi-Agent Path Finding} (MAPF) problem is defined by a graph $G = (V,E)$ and a set of $k$ agents $\{r_1 ... r_k \}$. Each agent $r_i$ has a start location $s_i \in V$ and a goal location $g_i \in V$. At each timestep the agents take an action that can \textit{move} them to an adjacent location or \textit{wait} in its current location. A \textit{path} for an agent $r_i$ is a sequence of actions that take $r_i$ from $s_i$ to $g_i$. The objective is to find a \textit{path} for each agent such that agents...

\emph{Multi-agent pathfinding (MAPF)} is the problem of finding $k$ conflict-free paths for a group of $k$ agents, each of which has the goal of reaching a goal state from a certain start location. Formally, given:

 Solving MAPF optimally for each one of the cost functions is intractable. Indeed, it is NP-complete when considering MAPF over 4-connected grids \cite{YuL13}.