\documentstyle[11pt,reduce]{article}
\title{A \REDUCE{} package for Symmetry}
\date{}
\author{Karin Gatermann\\
        Konrad-Zuse-Zentrum f\"ur Informationstechnik Berlin\\
        Takustrasse\ 7\\
        D--14195 Berlin--Dahlem\\
        Federal Republic of Germany\\
	E--mail: gatermann@zib.de}
\begin{document}
\maketitle
\index{SYMMETRY package}

This short note describes a package of \REDUCE{} procedures
that compute symmetry-adapted bases and block diagonal forms
of matrices which have the symmetry of a group.
The SYMMETRY package is the implementation
of the theory of linear representations
for small finite groups such as the dihedral groups.

\section{Introduction}

The exploitation of symmetry is a very important principle in mathematics,
physics and engineering sciences.
The aim of the SYMMETRY package is to give an easy access to the
underlying theory of linear representations for small groups. For
example the
dihedral groups $D_3,D_4,D_5,D_6$ are included.
For an introduction to the theory see {\sc Serre} \cite{Se77} or
{\sc Stiefel} and {\sc F\"assler} \cite{StFae79}.
For a given orthogonal (or unitarian) linear representation
\[
\vartheta : G\longrightarrow GL(K^n), \qquad K=R,C.
\]
the character $\psi\rightarrow K$, the
canonical decomposition or the bases of the isotypic
components are computed. A matrix $A$ having the symmetry of a linear
representation,e.g.
\[
\vartheta_t A = A \vartheta_t \quad \forall \, t\in G,
\]
is transformed to block diagonal form by a coordinate
transformation.
The dependence of the algorithm on the
field of real or complex numbers is controled by the switch {\tt complex}.
An example for this is given in the testfile {\em symmetry.tst}.

As the algorithm needs information concerning the irreducible representations
this information is stored for some groups (see the operators in Section 3).
It is assumed that only orthogonal (unitar) representations are given.

The package is loaded by

{\tt load symmetry;}

\section{Operators for linear representations}

First the data structure for a linear representation has to be explained.
{\em representation} is a list consisting of the group identifier and
equations which assign matrices to the generators of the group.

{\bf Example:}
\begin{verbatim}
   rr:=mat((0,1,0,0),
           (0,0,1,0),
           (0,0,0,1),
           (1,0,0,0));

   sp:=mat((0,1,0,0),
           (1,0,0,0),
           (0,0,0,1),
           (0,0,1,0));

   representation:={D4,rD4=rr,sD4=sp};
\end{verbatim}

For orthogonal (unitarian) representations the following operators
are available.

{\tt canonicaldecomposition(representation);}

returns an equation giving the canonical decomposition of the linear
representation.

{\tt character(representation);}

computes the character of the linear representation. The result is a list
of the group identifier and of lists consisting of a
list of group elements in one equivalence class and a real or complex number.

{\tt symmetrybasis(representation,nr);}

computes the basis of the isotypic component corresponding to the irreducible
representation of type nr. If the nr-th irreducible representation is
multidimensional, the basis is symmetry adapted. The output is a matrix.

{\tt symmetrybasispart(representation,nr);}

is similar as {\tt symmetrybasis}, but for multidimensional
irreducible representations only the first part of the
symmetry adapted basis is computed.

{\tt allsymmetrybases(representation);}

is similar as {\tt symmetrybasis} and {\tt symmetrybasispart},
but the bases of all
isotypic components are computed and thus a
complete coordinate transformation is returned.

{\tt diagonalize(matrix,representation);}

returns the block diagonal form of matrix which has the symmetry
of the given linear representation. Otherwise an error message occurs.

{\tt on complex;}

Of course the property of irreducibility depends on the field $K$ of
real or complex numbers. This is why the algorithm depends on $K$.
The type of computation is set by the switch {\em complex}.

\section{Display Operators}

In this section the operators are described which give access to the
stored information for a group.
First the operators for the abstract groups are given.
Then it is described how to get the irreducible representations
for a group.

{\tt availablegroups();}

returns the list of all groups for which the information such as
irreducible representations is stored. In the following {\tt group}
is always one of these group identifiers.

{\tt printgroup(group);}

returns the list of all group elements;

{\tt generators(group);}

returns a list of group elements which generates the group. For the
definition of a linear representation matrices for these generators
have to be defined.

{\tt charactertable(group);}

returns a list of the characters corresponding to the irreducible
representations of this group.

{\tt charactern(group,nr);}

returns the character corresponding to the nr-th irreducible representation
of this group as a list (see also {\tt character}).

{\tt irreduciblereptable(group);}

returns the list of irreducible representations of the group.

{\tt irreduciblerepnr(group,nr);}

returns an irreducible representation of the group. The output
is a list of the group identifier and equations
assigning the representation matrices to group elements.

\section{Storing a new group}

If the user wants to do computations for a group for which
information is not predefined,
the package SYMMETRY offers the possibility to supply information
for this group.

For this the following data structures are used.

{\bf elemlist} = list of identifiers.

{\bf relationlist}  = list of equations with identifiers and
operators $@$ and $**$.

{\bf grouptable} = matrix with the (1,1)-entry grouptable.

{\bf filename} = "myfilename.new".

\vspace{2cm}
The following operators have to be used in this order.

{\tt setgenerators(group,elemlist,relationlist);}

{\bf Example:}
\begin{verbatim}
   setgenerators(K4,{s1K4,s2K4},
     {s1K4^2=id,s2K4^2=id,s1K4@s2K4=s2K4@s1K4});
\end{verbatim}

{\bf setelements(group,relationlist);}

The group elements except the neutral element
are given as product of the defined
generators. The neutral element is always called {\tt id}.

{\bf Example:}
\begin{verbatim}
   setelements(K4,
        {s1K4=s1K4,s2K4=s2K4,rK4=s1K4@s2K4});
\end{verbatim}

{\bf setgrouptable(group,grouptable);}

installs the group table.

{\bf Example:}
\begin{verbatim}
   tab:=
    mat((grouptable,     id,    s1K4, s2K4, rK4),
        (id        ,     id,    s1K4, s2K4, rK4),
        (s1K4      ,    s1K4,     id,  rK4,s2K4),
        (s2K4      ,    s2K4,    rK4,   id,s1K4),
        (rK4       ,     rK4,   s2K4, s1K4,  id));

   setgrouptable(K4,tab);
\end{verbatim}

{\bf Rsetrepresentation(representation,type);}

is used to define the real irreducible representations of the group.
The variable {\tt type} is either {\em realtype} or {\em complextype}
which indicates the type of the real irreducible representation.

{\bf Example:}
\begin{verbatim}
   eins:=mat((1));
   mineins:=mat((-1));
   rep3:={K4,s1K4=eins,s2K4=mineins};
   Rsetrepresentation(rep3,realtype);
\end{verbatim}

{\bf Csetrepresentation(representation);}

This defines the complex irreducible representations.

{\bf setavailable(group);}

terminates the installation of the group203. It checks some properties of the
irreducible representations and makes the group available for the
operators in Sections 2 and 3.

{\bf storegroup(group,filename);}

writes the information concerning the group to the file with name
{\em filename}.

{\bf loadgroups(filename);}

loads a user defined group from the file {\em filename} into
the system.

\begin{thebibliography}{5}

\bibitem{JaKer81} G.\ James, A.\ Kerber: {\it Representation Theory
of the Symmetric Group.} Addison, Wesley (1981).

\bibitem{LuFal88} W.\ Ludwig, C.\ Falter: {\it Symmetries in Physics.}
Springer, Berlin, Heidelberg, New York (1988).

\bibitem{Se77} J.--P.\ Serre, {\it Linear Representations of Finite
Groups}. Springer, New~York (1977).

\bibitem{StFae79} E.\  Stiefel, A.\  F{\"a}ssler, {\it Gruppentheoretische
Methoden und ihre Anwendung}. Teubner, Stuttgart (1979).
(English translation to appear by Birkh\"auser (1992)).

\end{thebibliography}
\end{document}


%!TEX root = ./thesis.tex
\graphicspath{{./Figs/LiteratureReview/}}
% Acronym
\nomtypeA{LES}{Large Eddy Simulations}
% Neutral ABL
\nomtypeR{$S_\epsilon$}{Source term in turbulent dissipation rate equation}{}
\nomtypeR{$S_k$}{Source term in turbulent kinetic energy equation}{}

\chapter{Literature Review}
\section{Neutral Atmospheric Boundary Layer modelling}
\subsection{Horizontally homogeneous boundary layer}
The important task before modelling flows in ABL is obtaining equilibrium ABL, i.e. zero stream-wise gradients of all variables. For neutral atmospheric boundary layer, \textcite{Richards1993} proposed the appropriate boundary conditions of mean wind speed and turbulence quantities for the standard $k-\epsilon$ model. These profiles were derived assuming constant shear stress with height and applied for surface layer of ABL. These were used to model ABL surface layer as horizontally homogeneous turbulent surface layer (HHTSL). However, HHTSL was hard to achieved mostly due to the ground boundary conditions \cite{Yang2009}, which manifested in the decay of velocity profile due to a spike in the turbulent kinetic energy profile close to the ground. The consistency between wall boundary conditions, turbulence model with associated constants and also numerical schemes was shown to help to achieve HHTSL \cite{Jonathon2012, Parente2011, Yan2016}. Under HHTSL, the governing equations can be simplified as:
\begin{equation} \label{eq:Richards1993_eqns}
\begin{aligned}
\nu_t \pdv{u(z)}{z} = \frac{\tau_w}{\rho} = u_*^2 \\
\pdv{}{z} \left(\nu_t \pdv{u(k)}{z}\right) + S_k = 0\\
\pdv{}{z} \left(\frac{\nu_t}{\sigma_{\epsilon}} \pdv{\epsilon}{z}\right) (C_{1\epsilon} - C_{2\epsilon})\frac{\epsilon^2}{k} + S_\epsilon = 0
\end{aligned}
\end{equation}

The inlet boundary conditions proposed by \textcite{Richards1993}, widely used in CFD study of atmospheric flow are:
\begin{equation} \label{eq:Richards1993_inlet}
\begin{aligned}
u(z) &= \frac{u_{*}}{\kappa }\ln \frac{z+z_0}{z_0}\\
k &= \frac{u_{*}^2}{\sqrt{C_\mu}}\\
\epsilon &= \frac{u_{*}^3}{\kappa (z + z_0)}
\end{aligned}
\end{equation}

These profiles are assured a solution of Equation~(\ref{eq:Richards1993_eqns}), if the model constant, turbulent Prandtl number of the dissipation rate $\sigma_{\epsilon}$, is modified as:
\begin{equation} \label{eq:Richards1993_model_constrain}
\sigma_{\epsilon} = \frac{\kappa^2 }{(C_{\epsilon2}-C_{\epsilon1}) \sqrt C_\mu}
\end{equation}

Instead of altering model constants, \textcite{Pontiggia2009} derived the $\epsilon$ equation $z$-dependent source term from solution of Equation~(\ref{eq:Richards1993_inlet}):
\begin{equation} \label{eq:epsSourcePont}
S_\epsilon= \frac{\rho u_*^4}{(z+z_0)^2} \left[ \frac{(C_{\epsilon 2}-C_{\epsilon 1})\sqrt{C_\mu}}{\kappa^2} - \frac{1}{\sigma_{\epsilon}} \right] - \mu \frac{\rho u_*^3}{ 2 \kappa(z+z_0)^3}
\end{equation}
Under turbulence case, molecular viscosity is negligible, therefore the second term is usually ignored.

Constant inlet turbulence kinetic energy proposed by \textcite{Richards1993} is subjected to many arguments. Since velocity field is limited affected by turbulence kinetic energy but the concentration field because if enhancing dispersion effect of turbulence. As noted by \textcite{Parente2011}, decreasing $k$ with height was shown in many wind tunnel test. \textcite{Yang2009} proposed new profile of $k$ and $\epsilon$ for standard $k-\epsilon$ model. $k$, $\epsilon$ are the non-linear function of height as:
\begin{equation}
\begin{aligned}
k &=\frac{u_*^2}{C_\mu^{1/2}}\sqrt{C_1 \ln\left(\frac{z+z_0}{z_0}\right)+C_2}\\
\epsilon &=\frac{u_*^3}{\kappa (z+z_0)}\sqrt{C_1 \ln\left(\frac{z+z_0}{z_0}\right)+C_2}\\
\end{aligned}
\end{equation}
$C_1=-0.17$ and $C_2=1.62$ are constants fitted from their wind tunnel experiments. They also proposed modified standard model constants in Table~\ref{tab:kEpsModifiedYang2009}.
\def\rowWidth{0.06}
\begin{table}[htbp]
	\caption{The modified $k-\epsilon$ model constants by \textcite{Yang2009}} \label{tab:kEpsModifiedYang2009}
	\centering
	\begin{tabular}{p{\rowWidth\textwidth}p{\rowWidth\textwidth}p{\rowWidth\textwidth}p{\rowWidth\textwidth}p{\rowWidth\textwidth}}
		\toprule
		$C_{1\epsilon}$	& $C_{2\epsilon}$ & $C_{\mu}$ & $\sigma_k$ & $\sigma_\epsilon$ \\
		\midrule
		1.5 & 1.92 	& 0.028 & 1.67 & 2.51\\
		\bottomrule
	\end{tabular}
\end{table}

\textcite{Parente2011} presented an elaborate procedure to ensure the consistency for arbitrary inlet profile of turbulent kinetic energy $k$. Instead of altering model constants as \textcite{Yang2009}, the effect of non-constant $k$ on momentum and $\epsilon$ equation can be characterised by deriving equation for $C_\mu$:
\begin{equation}
C_\mu(z) = \frac{u_*^4}{k(z)^2}
\end{equation}
Source terms are added to $k$ and $\epsilon$ transport equations to ensure equilibrium condition:
\begin{equation}
\begin{aligned}
S_k &= \frac{\rho u_* \kappa}{\sigma_k} \pdv{}{z}\left((z+z_0) \pdv{k}{z}\right)\\
S_\epsilon &= \frac{\rho u_*^4}{(z+z_0)^2} \left[ \frac{(C_{\epsilon 2}-C_{\epsilon 1})\sqrt{C_\mu}}{\kappa^2} - \frac{1}{\sigma_{\epsilon}} \right]\\
\end{aligned}
\end{equation}

\textcite{Richards2011} revisited the problem of modelling the HHTSL by deriving the inlet profiles directly from the conservation and equilibrium equations. This allows various inlet profiles can be specified by varying the turbulence models constants. For standard $k-\epsilon$ models, the inlet profiles of velocity and turbulence properties are the same as Equation~(\ref{eq:Richards1993_inlet}). However they suggested to change the von Karman constant according to model constants as:
\begin{equation} \label{eq:vonKarman_constrain}
\kappa_{k-\epsilon} = \sqrt{(C_{\epsilon2}-C_{\epsilon1}) \sigma_{\epsilon} \sqrt C_\mu}
\end{equation}
Using the standard $k-\epsilon$ model constants (Table \ref{tab:kEpsCons}), we can yield $\kappa_{k-\epsilon} = 0.433$.

\textcite{Hargreaves2007} had shown that zero gradient velocity at the top boundary resulted in a decay of velocity downstream, due to the extraction energy at wall due to wall shear stress. A driving shear stress, zero flux of turbulent kinetic energy and a flux of dissipation rate $\epsilon$ are to be imposed at the upper boundary:
\begin{equation}\label{eq:RichardsTopBCs}
\begin{aligned}
\frac{\dd u}{\dd z}&= \frac{u_*}{\kappa z}\\
\frac{\mu_t}{\sigma_{\epsilon}} \frac{d \epsilon}{dz} &= -\frac{\rho u_{*}^4}{\sigma_{\epsilon} z}\\
\end{aligned}
\end{equation}

For $k-\omega$ models, the specific dissipation $\omega$ is solved instead of dissipation rate $\epsilon$. Profiles of $U$ and $k$ are the same, except the new effective von Karman constant is $\kappa_{k-\omega}=0.408$. Profiles for $\omega$ has expression of:
\begin{equation} \label{eq:hhtslOmeProfile}
\omega = \frac{u_{*}}{C_{\mu}^{1/2} \kappa_{k-\epsilon} z}
\end{equation}

Similarly to the $k-\epsilon$ turbulence models, a flux of $\omega$ should be imposed at top boundary:
\begin{equation}
{\mu_t} \frac{d \omega}{dz} = -\frac{\rho u_{*}^2}{C_{\mu}^{1/2} z}
\end{equation}

In present of obstacles, \textcite{Richards2011} had shown that eddy viscosity models like $k-\epsilon$ or $k-\omega$ resulted the over-prediction of stagnation pressures, while Reynolds stress model (RSM) \cite{Gibson1978} was significantly reduced this issue.

\subsection{Boundary conditions}
At the outlet boundary, the flow is assumed fully developed and unidirectional. All flow variables are supposed to be constant in this boundary. The placement of this boundary, therefore, is very important. As the placement is so close to the source, significant errors can be made due to the propagation of the source to other boundaries. Otherwise, if the placement is so far, the computational time will increase dramatically.

The top and side of the computational domain are external boundaries representing the far fields of flow. If a constant pressure is applied in these boundaries, this may alter the inlet wind profile in case prescribed pressure is not matched with the boundary velocity \cite{Luketa-Hanlin2007}. The zero gradient boundary condition, which set normal velocity to zero and all others variables are set equal to the inner values, or symmetry condition can be used at the top and side boundaries to reserve the wind profile and eliminate the effect of changing the inlet profiles.

At the wall boundary, two models usually applied for turbulence properties are Low Reynolds number (LRN) turbulence model \cite{Jones1972} and high Re number (HRN) with wall function \cite{Launder1974}. HRN models are usually less accurate, and also sensitive to the mesh resolution close to the wall. Adaptive wall functions were developed to overcome the restriction HRN, which is the first point above the wall to lie in the logarithmic layer \cite{Kalitzin2005}. \textcite{Backar2017} proposed a hybrid approach, so called numerical wall function, where wall adjacent cells are divided into sub-grid and governing equations are solved with appropriate boundary conditions in this sub-grid.

\section{Stratified Atmospheric Boundary Layer modelling}
Either the Reynolds Averaged Navier–Stokes (RANS) equations or Large Eddy Simulations (LES) \cite{Moeng1984,Saiki2000} are used for stratified atmospheric turbulence modelling. RANS turbulence models are still widely used in practical approach to overcome boundary conditions sensitivity and computational intensive of the LES.

Thermal stratification results from heat flux of the ground have significant effects to the buoyancy and ABL turbulence. For standard $k-\epsilon$ model, the source term that accounts for gravity effects in $\epsilon$ equation can be written as \cite{Alinot2005}:
\begin{equation}
S_{\epsilon b} =  C_{\epsilon 1}(1-C_{\epsilon 3}) \frac{\epsilon}{k} G_b
\end{equation}
$G_b$ is turbulent kinetic energy production source term due to buoyancy:
\begin{equation}
G_b = \beta g_i \frac{\mu_t}{\sigma_T}  \left( \pdv{T}{x_i} - \frac{g_i}{c_p} \right)
\end{equation}

For stable stratified ABL, turbulent kinetic energy $k$ and dissipation rate $\epsilon$ can be derived from Monin-Obukhov similarity theory profiles and solving the $k-\epsilon$ equation \cite{Luketa-Hanlin2007}. For the height $z \le 0.1h_{ABL}$:
\begin{equation} \label{eq:MO-turl-stab-surface}
\begin{aligned}
k &= 6 u_{*}^2\\
\epsilon &= \frac{u_{*}^3}{\kappa z}\left( 1.24 +4.3 \frac{z}{L_{MO}} \right)
\end{aligned}
\end{equation}

For the height $z>0.1h_{ABL}$:
\begin{equation} \label{eq:MO-turl-stab-above}
\begin{aligned}
k &= 6 u_{*}^2 \left( 1-\frac{z}{h_{ABL}} \right)^{1.75}\\
\epsilon &= \frac{u_{*}^3}{\kappa z}\left( 1.24 +4.3 \frac{z}{L_{MO}} \right) \left( 1-0.85\frac{z}{h_{ABL}} \right)^{1.5}
\end{aligned}
\end{equation}

Under unstable ABL, the heat flux from the ground and height of ABL play an important role in increasing the turbulence in the air flow. This vertical flow can be characterised using convective velocity scale Equation~(\ref{eq:convective-scaling}). Turbulent kinetic energy $k$ and dissipation rate $\epsilon$ under unstable ABL can be defined in Equation~(\ref{eq:MO-turb-unstab-surface}) for $z \le 0.1h_{ABL}$ and Equation~(\ref{eq:MO-turb-unstab-above}) for $z > 0.1h_{ABL}$.
\begin{equation} \label{eq:MO-turb-unstab-surface}
\begin{aligned}
k &= 0.36 w_{*}^2 + 0.85 u_{*}^2 \left( 1-3\frac{z}{h_{ABL}} \right)^{2/3}\\
\epsilon &= \frac{u_{*}^3}{\kappa z} \left( 1 + 0.5 \abs{\frac{z}{L_{MO}}} ^{2/3} \right)^{1.5}
\end{aligned}
\end{equation}

\begin{equation} \label{eq:MO-turb-unstab-above}
\begin{aligned}
k &=w_{*}^2 \left[ 0.36 + 0.9\left( \frac{z}{h_{ABL}} \right)^{2/3}\left( 1-0.8\frac{z}{h_{ABL}} \right)^{2} \right]\\
\epsilon &= \frac{w_{*}^3}{h_{ABL}} \left( 0.8-0.3\frac{z}{h_{ABL}} \right)
\end{aligned}
\end{equation}

In order to simulate atmospheric stratification effects, \textcite{Alinot2005} changed model constants (Table~\ref{tab:kEpsConsAlinot2005}) to achieve a better agreement with atmospheric profile from Monin-Obukhov theory.
\begin{table}[htbp]
	\caption{The standard $k-\epsilon$ model constants proposed by \textcite{Alinot2005}} \label{tab:kEpsConsAlinot2005}
	\centering
	\begin{tabular}{ccccc}
		\toprule
		$C_{1\epsilon}$	& $C_{2\epsilon}$ & $C_{\mu}$ & $\sigma_k$ & $\sigma_\epsilon$ \\
		\midrule
		1.176 & 1.92 & 0.0333 & 1 & 3.4 ($L_{MO} > 0$)\\
		&  &  & & -4.4 ($L_{MO} < 0$)\\
		\bottomrule
	\end{tabular}
\end{table}

Effects of atmospheric stratification on dense gas dispersion CFD simulations was addressed by \textcite{Pontiggia2009a}. The consistency between Monin-Obhukov profiles with $k-\epsilon$ model is obtained by addition of z-dependent source term $S_\epsilon$ to the $\epsilon$ transport equation. Under neutral atmospheric stability:
\begin{equation}
S_\epsilon(z) = \frac{\rho u_*^4}{z^2} \left[ \frac{(C_{\epsilon 2}-C_{\epsilon 1}) \sqrt{C_\mu}}{\kappa^2} - \frac{1}{\sigma_{\epsilon}}\right] -\mu \frac{u_*^3}{2\kappa z^3}
\end{equation}
Under stable condition:
\begin{equation} \label{eq:stablePontiggiaEpsSource}
S_\epsilon(z) = \frac{\rho u_*^4}{z^2} \left[ \frac{(C_{\epsilon 2}-C_{\epsilon 1}) \sqrt{C_\mu}}{\kappa^2} \Phi_\epsilon^2 \sqrt \frac{\Phi_\epsilon}{\Phi_m} - \frac{1}{\sigma_\epsilon}\left( \frac{2}{\Phi_m} - \frac{1}{\Phi_m^2} + \frac{T_*}{\kappa T} \right)\right] -\mu \frac{u_*^3}{2\kappa z^3}
\end{equation}

In case of cold dense gas dispersion, heat exchange of gas cloud and the ground surface is a significant heat transfer process. Forced heat convection model was used to find this heat flux \cite{Nielsen1999}:
\begin{equation}
q_s=h_{f} (T_s - T_f)
\end{equation}
$h_f$ is the local heat transfer coefficient, $T_f$ is the fluid temperature.
\textcite{Kovalets2006} used mixed coefficients between force heat convection and natural convection.

Several studies used CFD commercial software such as FLUENT; as well as open-source software such as OpenFOAM to simulate ABL layers. \textcite{Hargreaves2007} have shown that applying at fixed variable fluxes at top boundary is difficult in general CFD software and usually replaced by zero gradient (no shear stress), however this will result a decaying boundary layer. They also highlighted different approaches of modelling near wall region can lead to significantly unexpected results especially when using general CFD software. Wall function applied for atmospheric flow is necessarily modified as the standard wall function is based on experiments of sand grain roughed pipe \cite{Blocken2007} and wind engineering roughness length $z_0$ is far from similar to this kind of roughness. \textcite{Pieterse2013} used STAR-CCM+ commercial code to simulate thermally stratified ABL with the standard $k-\epsilon$ and SST $k-\omega$ turbulence model.

\textcite{Flores2013a} used Detached Eddy Simulation (DES) technique, which incorporates RANS models in near wall region and LES model in the rest, to simulate atmospheric wind circulation in open pit. The simulation takes in to account effects of buoyancy, stratification and complex geometry. A quasi-compressible approximation (treating density as explicit variable) was applied to incorporate stratification effect. \textcite{Riddle2004a} simulated neutrally stable atmospheric boundary layer with RSM turbulence model.

\section{CFD simulation of dense gas dispersion}
Value of turbulent Schmidt number $Sc_t$ are shown to have significant effect on dense gas dispersion concentration. Using wind tunnel data, \textcite{Mokhtarzadeh-Dehghan2012} reported that $Sc_t$ was related with flow thermal stability, which is increased from 0.5 to 2.3 for the flows with Richardson number from 0.1 to 16.

Several studies in literature of atmospheric dense gas dispersion using CFD approach adopted commercial software such as ANSYS fluent \cite{Gavelli2008,Zhang2015,Ikealumba2016,Meroney2012,Labovsky2011}, ANSYS CFX \cite{Sklavounos2004}, FLACS \cite{Hanna2004,Hansen2010a,Schleder2015} as well as open-source software such as OpenFOAM \cite{Mack2013, Fiates2016,Fiates2016a}, FDS \cite{Mouilleau2009,Ryder2004}. \textcite{Hansen2010a} validated FLACS in all tests in model evaluation database of LNG vapour dispersion. These include both wind tunnel test as well as field tests.

Several turbulence models were tested for dense gas dispersion problem. \textcite{Mack2013} used standard $k-\epsilon$ model with OpenFOAM solver \bera{reactingFoam}, to simulate wind tunnel test case \bera{DAT632}, which is the gravity driven flow of heavy gas in slope terrain. Different treatments of buoyancy term in $\epsilon$ equation are investigated. It was shown that standard $k-\epsilon$ is able to predict turbulence damping due to vertical negative density gradient. \textcite{Gavelli2008} applied RSM model to account for directional effect of Reynold stress field (using standard $k-\epsilon$ model as initial guess of turbulence) to simulate \bera{Falcon1} field test. \textcite{Gant2014} simulated \ce{C02} field test with realizable $k-\epsilon$ model. Different RANS models were also tested for dense gas flow over obstacle \cite{Sklavounos2004}. \textcite{Tauseef2011} used realizable $k-\epsilon$ for modelling two Thorney Island test cases.

\section{Concluding remarks}
From the investigation of literature, modifications of general CFD are required to successfully simulate the ABL turbulence. These was done intensively in commercial proprietary software. But little works were done in open-source code like OpenFOAM. The modification of general code should be done to successfully apply OpenFOAM in simulating ABL flows.

Ensuring accurate description of the ABL is an important task in any ABL flow study. This can be done by simulating the horizontally homogeneous ABL flow prior of dispersion study. Either the Reynolds Averaged Navier–Stokes (RANS) equations or Large Eddy Simulations (LES) are used for atmospheric turbulence modelling. RANS turbulence models are still widely used in practical approach to overcome boundary conditions sensitivity and computational intensive of the LES. Neutral and thermal stratified ABL should be taken into account to simulate ABL turbulence.

Turbulent Schmidt number $Sc_t$ had significant effect on ABL dense gas dispersion concentration \cite{Mokhtarzadeh-Dehghan2012}. Therefore, the solver should be able to take $Sc_t$ as input parameter. In case of LNG vapour dispersion, buoyancy effect and ground heat transfer are two important factors. LNG vapour density changes with its cloud temperature, therefore it behaves as dense gas in low temperature but as buoyant gas at higher temperature. The solver should take into account buoyancy effect in this situation.


\section{Theoretical Background}

\subsection{Concept Embedding Analysis}\label{sec:conceptanalysis}

To understand the process flow of an algorithm, it is of great value
to have access to interpretable intermediate outputs.
% This is not available for DNNs: Their intermediate output of layers,
% the latent spaces, are high-dimensional, entangled, and non-semantic,
% meaning semantic information is only indirectly embedded.
% The latent spaces of DNNs---the output spaces of sets of neurons or of
% complete layers---usually are far from interpretable.
% Instead, they are high-dimensional, entangled, and non-semantic, since
% features are automatically extracted from correlations in the data.
% However, information about semantic concepts may still be embedded in
% the latent space of a DNN.
The goal of concept embedding analysis is to answer \emph{whether},
\emph{how well}, \emph{how}, and with what
\emph{contribution to the reasoning}
information about semantic concepts is embedded into the latent spaces
(intermediate outputs) of DNNs, and to provide the result in an
explainable way.
Focus currently lies on finding embeddings in either the complete
output of a layer (image-level concepts), or single pixels of an
activation map of a convolutional DNN (concept segmentation).
%
%%% whether
To answer the \emph{whether}, one can try to find a decoder for the
information about the concept of interest, the \emph{concept embedding}.
This means, one is looking for a classifier on the latent space that
can predict the presence of the concept.
% An example are the mini-1-hidden-layer neural networks used in
% \cite{fuchs_neural_2018}.
% how well
The performance of the classifier provides a measure of \emph{how well}
the concept is embedded.
%
%%% how
For an explainable answer of \emph{how} a concept is embedded, the
decoder should be easily interpretable.
One constraint to this is introduced by the rich vector space
structure of the space of semantic concepts respectively word vector spaces
\cite{mikolov_linguistic_2013}:
The decoder map from latent to semantic space should preserve at least
a similarity measure.
For example, the encodings of \enquote{cat} and \enquote{dog} should
be quite similar, whereas that of a \enquote{car} should be relatively
distant from the two.
The methods in literature can essentially be grouped by their choice
of distance measure $\langle -,-\rangle$ used on the latent vector space.
A concept embedding classifier $E_c$ predicting the presence of concept $c$
in the latent space $L$ then is of the form
% \begin{gather*}
%   \SwapAboveDisplaySkip
$E_c(v)=\langle v_c,v\rangle > t_c$
% \quad\text{for~}
for $v\in L$,
% \end{gather*}
where $v_c\in L$ is the concept vector of the embedding,
and $t_c\in \R$.% is a threshold for binarizing.

Automated concept explanations \cite{ghorbani_towards_2019} uses
$L_2$ distance as similarity measure. They discover concepts in an
unsupervised fashion by k-means clustering of the latent space
representations of input samples. The concept vectors of the
discovered concepts are the cluster centers.
% In \cite{yeh_completeness-aware_2020}, an improvement of the clustering
% by further semantic constraints was suggested.
% The proposed approach in \cite{gu_semantics_2019} also uses
% clustering, but by cosine distance. Their method is supervised in that
% they try to find a normalized concept vector for a concept given by
% positive samples. An interesting finding was that the chosen
% semantic concepts usually have a clearly dominant cluster in the
% latent space.
% For regularization, they binarize the entries in the latent space vectors to cluster.
%
In TCAV~\cite{kim_interpretability_2018} it is claimed that the
mapping from semantic to latent space should be linear for best
interpretability. To achieve this, they suggest to use linear
classifiers as concept embeddings. This means they try to find a
separation hyperplane between the latent space representations of
positive and negative samples of the concept.
A normal vector of the hyperplane then is their concept vector,
and the distance to the hyperplane
% measured by the scalar product with the concept vector
is used as distance measure.
As method to obtain the embedding they use support vector machines (SVMs).
TCAV further investigated the contribution of concepts to given output
classes by sensitivity analysis.
A very similar approach to TCAV, only instead relying on logistic
regression, is followed by Net2Vec~\cite{fong_net2vec_2018}.
% They directly built upon Network Dissection~\cite{bau_network_2017},
% which tries to associate concepts to single filters in convolutional
% DNNs (the unit vectors in the latent space of an activation map pixel).
% Just as Network Dissection, Net2Vec
As a regularization, they add a filter-specific cut-off before the
concept embedding analysis to remove noisy small activations.
% They apply a ReLU to the activation map of each filter with fixed
% filter-specific threshold.
% Other than TCAV, they constrain the hyperplane to run
% through zero, \idest assume $t_c=0$, which may lead to worse embedding
% performance.
The advantage of Net2Vec over the SVMs in TCAV is that they can
more easily be used in a convolutional setting: They used a
1$\times$1-convolution to do a prediction of the concept for each
activation map pixel, providing a segmentation of the concept.
This was extended by \cite{schwalbe_concept_2020}, who suggested to
allow larger convolution windows
% , essentially doing a simplified concept detection.
to ensure that the receptive field of the window can cover
the complete concept. This avoids a focus on local patterns.
% instead of \forexample the concept shape.
% They also investigated some improvements on the optimization
% technique.
%
A measure that can be applied to concept vectors of the same layer
regardless of the analysis method, is that of
\emph{completeness} suggested in \cite{yeh_completeness-aware_2020}.
They try to measure, how much of the information relevant to the final
output of the DNN is covered by a chosen set of concepts vectors.
% Their idea for measurement is as follows: If the layer output is
% reduced to the linear sub-space spanned by the concept vectors, the
% overall performance of the DNN should not drop significantly if the
% set of concepts is complete.
% This reduction was approximated by adding a bottleneck layer right
% after the considered layer, where the unit vectors correspond to the
% concepts.
% The projection is defined by the concept vectors, the connection to
% the succeeding layer is learned.
They also suggested a metric to compare the attribution of each
concept to the completeness score of a set of concepts.

\subsection{Inductive Logic Programming}\label{sec:ilp}

Inductive Logic Programming (ILP)~\cite{muggleton1991inductive} is a
machine learning technique that builds a logic theory over positive
and negative examples ($E^+$, $E^-$). The examples consist of symbolic
background knowledge (BK) in the form of first-order logic predicates,
\forexample \ilprule{contains(Example, Part), isa(Part, nose)}. Here the
upper case symbols are variables and the lower case symbol is a
constant. The given BK describes that example \ilprule{Example}
contains a part \ilprule{Part} which is a nose. Based on the examples,
a logic theory can be learned. The hypothesis language of this theory
consists of logic Horn clauses that contain predicates from the BK. We
write the Horn clauses as implication rules,
\forexample
\ilprule{
  \begin{align*}
    \text{face(Example) :- }& \text{contains(Example, Part), isa(Part, nose)}\;.
  \end{align*}
}
For this work we obey the syntactic rules of the Prolog
programming language. The \ilprule{:-} denotes the logic implication
($\leftarrow$). We call the part before the implication the
\emph{head} of a rule and the part after it the \emph{body} or
\emph{preconditions} of a rule.

We use the framework Aleph~\cite{srinivasan2001aleph}
for this work since it is a flexible and adaptive general purpose ILP
toolbox. Aleph's built in algorithm attempts to induce a logic theory
from the given BK to cover as many positive examples $E^+$ as possible
while avoiding covering the negative examples $E^-$. The general
algorithm of Aleph can be summarized as
follows~\cite{srinivasan2001aleph}:

\begin{enumerate}
\item As long as positive examples exist, select one. Otherwise halt.
\item Construct the most-specific clause that entails the selected example and is within the language constraints.
\item Find a more general clause which is a subset of the current literals in the clause.
\item Remove examples covered by the current clause.
\item Repeat from step 1.
\end{enumerate}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "concept_embeddings_and_ilp"
%%% End:


\section{Problem Definition}
\label{sec:definition}

Loosely speaking, the high-level objective in transient-detection is to locate intrinsicly(?)-varying objects in each recently shot image of the sky.
More formally and in the concept of this project, this objective can be defined as finding the mapping
\begin{equation}
  \label{eq:def1}
  (I_t,I_s) \longrightarrow S_t
\end{equation}
where $I_s$ is the recently captured image, namely the \emph{science image}. $I_t$ is the \emph{template} or \emph{reference} iamge, and $S_t$ is the set of \emph{true transients} to be detected.

However, this problem has traditionally been broken down into two subprolems, namely, \emph{image differencing}\footnote{or interchangeably, \emph{difference imaging}} and \emph{smart-thresholding}; the latter being the process of finding a threshold above which all the pixels in the diff image, $I_d$, will be seen as candidate transient sources:
\begin{equation}
  \label{eq:def2}
  (I_t,I_s) \xrightarrow{diff} I_d \xrightarrow{threshold} S_t
\end{equation}

\begin{figure}[h]
  \centering
  \includegraphics[width=.8\textwidth]{material/diagram}
  \caption{Coarse illustration of information flow through the Alert Production (AP) pipeline. Paths in blue and green illustrate the ``traditional'' and ``modern'' approaches respectively.}
  \label{fig:diagram}
\end{figure}

The \emph{thresholding} step has traditionally been implemented as a simple $5\sigma$-thresholding, but other auxilliary approaches may wrap this stage to make it \emph{smart}er -- figure~\ref{fig:diagram}

\section{Learning-based Approaches}
\label{sec:learning}
The above process is prone to false positives (contamination) and false negatives (misses). Learning-based approaches come into play to mitigate this...

In the context built in the previous section, a learning based approach can be implemented in two broad ways:
\begin{itemize}
\item starting off $I_d$ -- a.k.a traditional real/bogus classification.
\item starting off $(I_t,I_s)$ -- a.k.a end-to-end, TransiNet-style.
\end{itemize}

\footnote{The term 'detection' in the field of computer vision can be translated to localization+classification in the astrophysics' terminology}

Pros and cons of the two approaches come in the following sections. However, the current decision is to implement and test the two in parallel. Therefore we shall attempt to bring the two implementations close to eachother -- ideally unified.

\subsection{Basic Real/Bogus Classifier}

\begin{figure}[h]
  \centering
  \includegraphics[width=.6\textwidth]{material/rb-classifier}
  \includegraphics[width=.6\textwidth]{material/rb-classifier-mod}
  \caption{Classical Real/bogus classifier (top) and the modified 3D classifier (bottom)}
  \label{fig:rbdiagram}
\end{figure}

Input-output definition of the problem:
\begin{itemize}
  \item input: cutouts from diff image ($I_d$) with the potential transient at the center
  \item output: reliability score for the potential transient
\end{itemize}

Although the output of a classifier is categorical (i.e. a class label), we will be using the output before the last Softmax~\footnote{TBD} layer, which can be intepretted as a probability.

\subsubsection{Drawbacks}
\begin{itemize}
\item has no access to the temporal~\footnote{throughout this document the term ``temporal'' is used to represent the transition from a template/reference image to a science image. It has to do, but should not be confused with the ``real'' time axis!} behaviour of the source. This is by definition outsourced to the upstream image differencing module and hence there is no ``learned intelligence'' in processing the ``transient-ness'' of the candidate.
\item any ``miss'' (false negative) in the upstream modules is inherited and irrecoverable.
\item can only handle single object per cutout -- when there are multiple true transients, it \emph{has to} ignore the others.
\end{itemize}

The first two drawbacks are a result of having separated the temporal processing (i.e. image differencing) from the final classification, whereas the last one has to do with how the problem and solution are defined based on cutouts.

% \paragraph{The case of ambiguous negatives}
% As of now it is not quite clear what a ``negative'' is for training; in the broadest implementation it can be cutout that is not positive. But since the inputs to this module are the outputs of imdiff, one could only use ``bogus candidates'' as negatives, which is closer to what happens in practice, and may give better results.

\subsubsection{Benefits}
\label{sec:rb_benefits}

Although as discussed this is just a sub-optimal learning-based approach to this problem, it still has the advantage of \emph{interpretability}; the generated output might be too contamined with false alarms and (at the same time) missed true transients, but the process is fully transparent to the downstream users of the alerts and they have the possibility to develop their own post-processing blocks per need.

\subsubsection{What will be learned?}
Since the temporal behaviour of the candidates fed to this module is assumed, it is foreseeable that the neural network will eventually learn a concept close to ``PSF-ness'', true to its title; real/bogus classification.

\subsubsection{Modified 3D Real/Bogus Classifier}
In this implementation the classifier has access to the template, science image pairs along with the diff image. This allows the network to make use of the temporal behavior of the source too.
Note however that still the difference image has a key role in the process, as the images are cropped around the sources detected in the difference image.

\subsection{End-to-End Simultaneous Localization and Classification (~TransiNet)}

\begin{figure}[h]
  \centering
  \includegraphics[width=.8\textwidth]{material/transinet-teaser}
  \caption{End-to-end image differencing, localization and classification. The output has to be clean and complete \emph{by definition} }
  \label{fig:transinet-teaser}
\end{figure}

\begin{itemize}
  \item input: a pair of science,template images: $(I_t,I_s)$
  \item output: $S_t$, set of true transients
  \item intermediate output: a ``score image'' where scores are assigned per-pixel
\end{itemize}

Based on TransiNet~\cite{transinet}. Does full detection (localization + classification) simultaneously -- along with any necessary implicit steps.
Since the temporal and spatial behaviors are considered at the same time, the definition of the task of this module would be to find all the time-variable real astronomical sources -- corresponding to the high-level definition of the problem in \ref{sec:definition}.

\subsection{The score}
As the task definition embraces a recognition according to spatial \emph{and} temporal features simultaneously, the score needs to be defined carefully to capture both aspects.

There are multiple available options:
\begin{itemize}
\item simple binary classification -- the resulting score may not be quite informative
\item regression -- more difficult to define
\item multi-class classification -- difficult data annotation
\end{itemize}

If we go with regression... there are bogus detections, variables, ``appearing transients''
\begin{equation}
  s=\left| \frac{A_2-A_1}{A_2+A_1} \right|
\end{equation}

% \section{Notes from the template}

% \subsection{How to handle LSST standard references?}

% The papers should cite standard LSST references\footnote{See \url{https://github.com/lsst-pst/LSSTreferences}},
% where appropriate. For the usage, please see below.  These examples all use the ADS handle, unless they are
% project docs then the use the project handle like LSE-17.

% All are on the lsst-texmf which you can get from \url{http://lsst-texmf.lsst.io}

% \subsubsection{LSST System and Science}

% The LSST system (brief overview of telescope, camera and data management subsystems),
% science drivers and science forecasts are described in:

% \begin{itemize}
% \item LSST Science Requirements Document: \cite{LPM-17}.
% \item LSST overview paper: \cite{2008arXiv0805.2366I}.
% \item LSST Science Book: \cite{abell2009lsst}.
% \end{itemize}
% %------------------------------------------------------------------------------

% \subsubsection{Simulations}

% The LSST simulations are described in a series of papers. Use of the LSST simulations should cite the LSST simulations overview paper \cite{2014SPIE.9150E..14C} and the specific simulation tools used:

% \begin{itemize}
% \item LSST Catalogs (CatSim): \cite{2014SPIE.9150E..14C}
% \item Feature-Based Scheduler: \cite{2018arXiv181004815N}
% \item Operations Simulator (OpSim): Scheduler \cite{2016SPIE.9910E..13D}, SOCS \cite{2016SPIE.9911E..25R}
% \item Metrics Analysis Framework (MAF): \cite{2014SPIE.9149E..0BJ}
% \item Image simulations (Phosim): \cite{2015ApJS..218...14P}
% \item Sky brightness model: \cite{2016SPIE.9910E..1AY}
% \item LSST Performance for NEO (or moving object) discovery: \cite{2018Icar..303..181J}
% \end{itemize}
% %------------------------------------------------------------------------------

% \subsubsection{Data Management}

% LSST data management system and the data products are described in:

% \begin{itemize}
%   \item The LSST Data Management System: \cite{2015arXiv151207914J}
%   \item Data Products Definition Document: \cite{LSE-163}
% \end{itemize}
%  %------------------------------------------------------------------------------

% \subsubsection{Camera}

% \begin{itemize}
%    \item Design and development of the LSST camera: \cite{2010SPIE.7735E..0JK}
% \end{itemize}
% %------------------------------------------------------------------------------

% \subsubsection{Telescope and Site}

% \begin{itemize}
%    \item Telescope and site overview and status in 2014:  \cite{2014SPIE.9145E..1AG}
% \end{itemize}
% %------------------------------------------------------------------------------

% \subsubsection{System Engineering}

% \begin{itemize}
%    \item LSST systems engineering: \cite{2014SPIE.9150E..0MC}
%    \item System verification and validation: \cite{2014SPIE.9150E..0NS}
% \end{itemize}
% %


\documentclass[12pt,a4paper]{article}
\usepackage{algorithm, algpseudocode, amsmath, amssymb, csquotes, empheq, geometry, graphicx, hyperref, listings, multirow, physics, siunitx, subcaption, upgreek}
\usepackage[section]{placeins}
\usepackage[justification=centering]{caption}

\title{Computational Physics\\Problem Set 4}
\author{Saleh Shamloo Ahmadi\\Student Number: 98100872}
\date{October 22, 2021}

\hypersetup{colorlinks=true, urlcolor=cyan}
\newcommand{\percfig}{../fig/percolation}
\newcommand{\rwfig}{../fig/random-walk}

\begin{document}
	\maketitle
	\section{Percolation}
	Note: we simulate \emph{site percolation}.
	\subsection{Correlation Length}
	The correlation length $\xi$ is a measure of the radius of the maximum closed (non-infinite) cluster;
	This shows the maximum length of interactions before phase transition occures and the lattice either becomes
	homogenous or clusters are broken off.

	In our analysis, we use the maximum radius of gyration of closed clusters to measure the correlation length
	(any similar measure is valid).

	The correlation length diverges (or peaks, for a finite lattice) at the critical probability $p_c$ of forming sites
	in the lattice; This is because at higher probabilities, percolation happens frequently and there are less clusters
	and at lower probabilites, clusters break off more at smaller length scales.

	\newgeometry{left=0in, right=0in}
	\begin{figure}
		\centering
		\begin{subfigure}{0.45\linewidth}
			\centering
			\includegraphics[width=\linewidth]{\percfig/gyration-full-10}
		\end{subfigure}
		\begin{subfigure}{0.45\linewidth}
			\centering
			\includegraphics[width=\linewidth]{\percfig/gyration-full-20}
		\end{subfigure}
		\begin{subfigure}{0.45\linewidth}
			\centering
			\includegraphics[width=\linewidth]{\percfig/gyration-full-40}
		\end{subfigure}
		\begin{subfigure}{0.45\linewidth}
			\centering
			\includegraphics[width=\linewidth]{\percfig/gyration-full-80}
		\end{subfigure}
		\begin{subfigure}{0.45\linewidth}
			\centering
			\includegraphics[width=\linewidth]{\percfig/gyration-full-160}
		\end{subfigure}
		\begin{subfigure}{0.45\linewidth}
			\centering
			\includegraphics[width=\linewidth]{\percfig/gyration-full-320}
		\end{subfigure}
	\end{figure}
	\newgeometry{top=0.1in, bottom=0.1in, left=0in, right=0in}
	\thispagestyle{empty}
	\begin{figure}
		\centering
		\begin{subfigure}{0.45\linewidth}
			\centering
			\includegraphics[width=\linewidth]{\percfig/gyration-zoom-10}
		\end{subfigure}
		\begin{subfigure}{0.45\linewidth}
			\centering
			\includegraphics[width=\linewidth]{\percfig/gyration-zoom-20}
		\end{subfigure}
		\begin{subfigure}{0.45\linewidth}
			\centering
			\includegraphics[width=\linewidth]{\percfig/gyration-zoom-40}
		\end{subfigure}
		\begin{subfigure}{0.45\linewidth}
			\centering
			\includegraphics[width=\linewidth]{\percfig/gyration-zoom-80}
		\end{subfigure}
	\end{figure}
	\begin{figure}
		\centering
		\includegraphics[width=0.9\linewidth]{\percfig/gyration-zoom-160}
	\end{figure}
	\restoregeometry
	\begin{figure}
		\centering
		\includegraphics[width=\linewidth]{\percfig/gyration-zoom-320}
	\end{figure}
	Close to the critical probability $p_c$, the following relation holds:
	\begin{equation}
		\xi \sim \abs{p - p_c}^{-\nu}
	\end{equation}
	$\nu$ is called the \emph{critical exponent} for the correlation length $\xi$. In finite lattices, because
	the correlation length is bounded by the lattice length, this relation breaks down at probabilites very
	close to $p_c$. We can find the valuse of $\nu$ with two methods: direct calculation from finite lattices
	or extrapolation.

	In direct calculation, we can use linear regression for the relation
	\begin{equation}
		\log{\xi} = -\nu\log(p - p_c) + C.
	\end{equation}
	This is not especially accurate, because it is based on limited lattices that do not accurately represent unbounded
	trends; The edges of these lattices cut off emerging clusters.
	\begin{figure}
		\centering
		\includegraphics[width=\linewidth]{\percfig/critical-exp}
	\end{figure}
	\begin{figure}
		\centering
		\includegraphics[width=\linewidth]{\percfig/extrapolation}
	\end{figure}

	The extrapolation method is as follows. in finite lattices, the maximum correlation length is proportional to the
	lattice size, since the correlation length at $p_c$ is unbounded for infinite lattices, and is only bounded by the
	edges of the lattice in finite lattices. So, we can approximate the behavior of an infinite lattice by the relation
	\begin{equation}
		\abs{p_c(\infty) - p_c(L)}^{-\nu} = L,
	\end{equation}
	where $p_c(\infty)$ is the infinite lattice's critical probablity and $L$ is the size of a finite lattice.
	Extrapolating $p_c(\infty)$ and $\nu$ with a non-linear curve fit, we get
	\begin{equation}
		p_c(\infty) = 0.5925 ,\qquad \nu = 1.346.
	\end{equation}
	It can be proved that the exact value of $\nu$ is $4/3$ and larges simulations have shown that $p_c(\infty)$ for
	site percolation is $0.5927$, so the accuracy of the extrapolated values are quite good, considering the limited
	sample size.

	\subsection{Fractal Dimension}
	The percolation clusters exhibit fractal self-similar behavior. Using a simple depth-frist search algorithm,
	we can generate clusters in a lattice and calculate their size and radius of gyration to find their
	fractal dimension; If $s$ is the size of the cluster and $d_f$ is the fractal dimension
	\begin{equation}
		s \sim \xi^{d_f},
	\end{equation}
	so, by using linear regression for the relation
	\begin{equation}
		\log{s} = d_f\log{\xi} + C
	\end{equation}
	we can calculate the fractal dimension $d_f$.
	\newgeometry{top=0.5in, bottom=1.5in, left=0in, right=0in}
	\begin{figure}
		\centering
		\begin{subfigure}{0.45\linewidth}
			\centering
			\includegraphics[width=\linewidth]{\percfig/fractal-vis-50}
		\end{subfigure}
		\begin{subfigure}{0.45\linewidth}
			\centering
			\includegraphics[width=\linewidth]{\percfig/fractal-vis-65}
		\end{subfigure}
	\end{figure}
	\restoregeometry
	\newgeometry{top=1in, bottom=1.5in}
	\begin{figure}[htb!]
		\centering
		\includegraphics[width=\linewidth]{\percfig/fractal-vis-55}
	\end{figure}
	\restoregeometry
	\newgeometry{top=0.3in, bottom=0.3in, left=0in, right=0in}
	\thispagestyle{empty}
	\begin{figure}[htb!]
		\centering
		\includegraphics[width=0.8\linewidth]{\percfig/fractal-vis-59}
	\end{figure}
	\begin{figure}[htb!]
		\centering
		\includegraphics[width=0.8\linewidth]{\percfig/fractal-50}
	\end{figure}
	\begin{figure}[htb!]
		\centering
		\includegraphics[width=0.9\linewidth]{\percfig/fractal-55}
	\end{figure}
	\thispagestyle{empty}
	\begin{figure}[htb!]
		\centering
		\includegraphics[width=0.9\linewidth]{\percfig/fractal-59}
	\end{figure}
	\restoregeometry
	\section{1D Random Walk}
	\subsection{Variance}
	We can use the recurrence relation
	\begin{equation}
		x(t) = x(t-\tau) + al,
	\end{equation}
	where $a$ is $+1$ with probability $p$ and $-1$ with probability $q$, to find the variance.
	The calculation is as follows:
	\begin{align}
		\expval{x^2(t)} &= \expval{x^2(t-\tau)} + 2l\expval{ax(t-\tau)} + \expval{a^2}l^2 \\
		&= \expval{x^2(t-\tau)} + l^2 + 2l(p - q)\expval{x(t-\tau)} \\
		&= \expval{x^2(t-\tau)} + l^2 + 2l^2\qty(\frac{t}{\tau}-1)(p - q)^2
	\end{align}
	(Note that $a^2$ is always $1$, and since $a$ is dependant from $x$, $\expval{ax} = \expval{a}\expval{x}$.
	Also, I used $\expval{x(t)} = \frac{l}{\tau}(p-q)t$, which has already been proven in the lecture notes) \\
	Repeating the recurrence relation until it reaches $x(0) = 0$
	\begin{align}
		\expval{x^2(t)} &= \frac{tl^2}{\tau} + \frac{2l^2}{\tau}(p - q)^2 \sum_{n=1}^{t/\tau - 1} n\tau \\
		&= \frac{tl^2}{\tau}\qty[1 + \qty(\frac{t}{\tau} - 1)(p - q)^2].
	\end{align}
	Substituting into the variance,
	\begin{align}
		\sigma^2(t) &= \expval{x^2(t)} - \expval{x(t)}^2 \\
		&= \frac{tl^2}{\tau}\qty[1 + \qty(\frac{t}{\tau} - 1)(p - q)^2] - \frac{t^2l^2}{\tau^2}(p-q)^2 \\
		&= \frac{tl^2}{\tau}\qty[1 - (p - q)^2].
	\end{align}
	But
	\begin{equation}
		p+q = 1 \implies (p+q)^2 = 1 \implies p^2 + q^2 = 1 - 2pq,
	\end{equation}
	and using this to simplify the expression
	\begin{equation}
		\sigma^2(t) = \frac{tl^2}{\tau}\qty[1 - (p^2 + q^2) + 2pq] = \frac{tl^2}{\tau}\qty[1 - 1 + 2pq + 2pq]
	\end{equation}
	\begin{empheq}[box=\fbox]{equation}
		\sigma^2(t) = \frac{4l^2}{\tau}pqt
	\end{empheq}
	\subsection{Simulation}
	\begin{figure}
		\centering
		\includegraphics[width=\linewidth]{\rwfig/randomwalk-50}
	\end{figure}
	\begin{figure}
		\centering
		\includegraphics[width=\linewidth]{\rwfig/randomwalk-60}
	\end{figure}
	\begin{figure}
		\centering
		\includegraphics[width=\linewidth]{\rwfig/randomwalk-70}
	\end{figure}
	\begin{figure}
		\centering
		\includegraphics[width=\linewidth]{\rwfig/randomwalk-80}
	\end{figure}
	\FloatBarrier
	\subsection{Trapping (Absorbing) Boundary}
	In the following figures, $L$ is the length of the \enquote{path} i.e. the number of cells in the 1D grid.
	\begin{figure}[htb!]
		\centering
		\includegraphics[width=\linewidth]{\rwfig/randomwalk-trap-sim}
		\caption{Averaged over 100000 walks}
	\end{figure}
	\begin{figure}[htb!]
		\centering
		\includegraphics[width=\linewidth]{\rwfig/randomwalk-trap-exp}
		\caption{Calculated up to $t=100L$ where $t$ is the number of steps}
	\end{figure}
	\begin{figure}[htb!]
		\centering
		\includegraphics[width=\linewidth]{\rwfig/randomwalk-trap}
		\caption{As you can see, the expected lifetime is nearly equal to the average lifetime in the simulations}
	\end{figure}
\end{document}


\chapter{Results}\label{chap3}
\thispagestyle{plain}

As mentioned in section \ref{chap2}, three machine learning algorithms have been trained, and the resulting models used to predict glaciers ice thickness. The same procedure has been applied to all the algorithms used, in order to compare the results between the different models.
All the algorithms have been trained and tested using the data available in the GlaThiDa data-set in the Alpine region. Only glaciers present in the GlaThiDa database belonging to region 11 of the RGI (Central Europe) have been used in the following analysis.

After training each algorithm the resulting model has been used to predict the ice volume of all the glaciers in the Alps. This is useful to compare the results of these models to those computed by \citet{Farinotti2019}.

\section{Results from Linear Regression}\label{linear}
The first model to be analyzed is the one obtained from training the linear regression algorithm. In order to assess how well a machine learning model makes predictions, a sub-sample of the data-set is usually left out from the training process (see section \ref{training}). With the left out data the score from Eq. \ref{eq:score} is computed by letting the model predict the values not used for training. This is done to make sure the model is not over-fitted. Complex machine learning algorithm in fact might be very good at predicting data inside the training sample but might perform much worse on data outside this sample (over-fitting).
This is usually not the case with linear regression, hence in principle there is no need to only use a sub-sample of the data to train the model. The $R^2$ coefficient (the score) could be computed using the whole data-set to train the model. The linear regression algorithm however, has been trained using the same procedure used for the other algorithms, to be able to better compare the models with each other, and try to find similarities and differences.

\subsection{Score and volume spread}\label{lr-score}

\begin{figure}[!tp]
	\centering
	\includegraphics[width=1.\textwidth]{figures/LR_score.pdf}
	\caption{Linear Regression: Volume and score values of the model after training it with 20 different sub-sample of data. On the x-axis each number identifies a different sample of data. On the left the total volume of the alpine glaciers present in the GlaThiDa is shown. On the right the $R^2$ coefficient for values left out of the training sample.}
	\label{fig:lr-score}
\end{figure}

Figure \ref{fig:lr-score} shows the volume and score for 20 different sub-samples used to train the linear regression algorithm. Each number on the x-axis represents a different sample used for training: of all the alpine glacier observations in the GlaThiDa, 75\% of them have been used to train the model and the remaining 25\% to compute the score.

The volume shown on the left side of Fig. \ref{fig:lr-score} has been computed on the full sample of alpine glaciers with ice thickness observations. Each value shown in the figure is obtained with a different model, derived from training it with the sub-samples. The mean volume for the 20 different sub-samples is $1.60\times 10^{10}$ $m^3$ with a relative standard deviation of 0.74\%. Sample 14 predicts the lowest of all volumes.

\begin{figure}[!tp]
	\centering
	\includegraphics[width=1.\textwidth]{figures/LR_thick_map.pdf}
	\caption{Linear Regression: Ice thickness distribution for two glaciers on top of the terrain slope angle. On the left, one of the glaciers used to train the model. On the right, one of the glaciers outside those used to train the model.}
	\label{fig:lr-map}
\end{figure}

It is worth noting that the model predicts ice thicknesses below zero for certain points. These have however been rounded to zero as a thickness below zero makes no physical sense.

The average score is 0.402 with a high 25.1\% relative standard deviation. Clearly this is due to the score dropping for sub-samples 5, 14 and 19. For most sub-samples the score is in fact above 0.4, but for those three it drops even below 0.2. Sub-sample 14 then leads to the lowest volume and a low score compared to the average.
The linear regression model trained with all the data in the data-set has score of 0.32 which  is the lowest among the three models used.

\begin{figure}[!tp]
	\centering
	\includegraphics[width=1.\textwidth]{figures/LR_thick_diff_map.pdf}
	\caption{Linear Regression: Maps of two glaciers showing the glaciers surface and the difference between the modeled ice thickness and the  observed one.}
	\label{fig:lr-diff-map}
\end{figure}

Fig. \ref{fig:lr-map} and \ref{fig:lr-diff-map} show respectively the ice thickness distribution, and the difference in ice thickness between the observations and the model, for the linear regression model trained with all the available data. Looking at the ice thickness distribution the model seems to predict a plausible distribution with higher thickness values in the center and lower ones at the borders and at the tongues of the glaciers.

The ice thickness difference from the observations reveals, for both glaciers, only few points to be accurately predicted by the model. Differences of up to 100m from the observations are shown for both glaciers. For glacier RGI60-11.00746, Gepatschferner, the the model seems to do a better job at predicting the upper part of the glacier compared to the middle-bottom part. Particularly in this part the model overestimates the ice thickness in the center of the glacier while underestimating it for points lying closer to the glacier border. For glacier RGI60-11.02773 very few points seem to be predicted accurately. In particular, for the middle-bottom part of the glacier, and the central glacier flow-line, the model almost always underestimates the ice thickness with values going from 25m to up to 100m lower than the observed ice thickness. In the top part of the glacier the highest positive differences are registered.

After training the linear regression model with all the available observations for the alpine glaciers, their total estimated volume of ice  is $1.40 \times 10^{11}m^3$, which is 9.8\% larger than the volume predicted by \citet{Farinotti2019}.

\subsection{Features Importance}

It is interesting to look into which of the variables used to train the model influence the outcome of the prediction the most.

A very easy way to do so for a linear regression model, is just to analyze the weights $\bm{\beta}$ (see Eq. \ref{eq:linear}), which the model assigned to each feature in order to make predictions. These are shown in Table \ref{tb:lr-coef}.
The higher the absolute values of the coefficients, the more the variable will influence the outcome of the model. Looking at these coefficients, the slope angle is the feature which is more relevant to the outcome of the model, followed by the linear mass balance. In particular the slope influences the prediction almost 3 times more than the altitude and over 2.5 times more than the distance from the border. This means that a doubling of the slope angle of the glacier surface, will result in an ice thickness change 3 times larger than the change predicted by a doubling of the altitude. It is worth noting that the slope angle has a negative sign, meaning that lower slope angles correspond to higher ice thicknesses in accordance with the literature about glaciers ice flow \cite[P. 298]{cuffey2010physics}.

\begin{table}
	\centering
	\caption{Linear Regressions: Coefficients for the linear regression model: higher absolute values mean that the feature have a higher influence on the model predictions}
	\begin{tabular}{|c|c|c|c|}
		\hline
		Slope Angle&Mass Balance&Distance From Border&Altitude \\
		\hline
		-19.7&12.7&7.7&-6.6 \\
		\hline
	\end{tabular}
	\label{tb:lr-coef}
\end{table}

The results form the coefficients analysis of the linear model, could be enough to have a complete picture about the variables influencing the output the most. An analysis showing the tools used for the other models is shown anyway, in order to have comparable results between the models, and to understand how these tools compare with the method of analyzing the magnitude of the linear model coefficients.

Figure \ref{fig:lr-heatmap} shows the coefficients deriving from the permutation analysis explained in section \ref{permutation}. This looks at changes in the score of the model, when the feature values for a single feature are shuffled randomly. If the score doesn't change, compared to the one achieved by the model with the correct order of the features values, it would mean that the feature is less relevant in making predictions. The higher the permutation coefficient value, the more relevant is the feature in making predictions. Figure \ref{fig:lr-heatmap} shows that using this method the slope angle is the feature which most influences the outcome of the prediction, which is in accordance with the values of the model coefficients in table \ref{tb:lr-coef}. Also for the other features the permutation shows the same result already found with the model coefficients. Interestingly for sub-sample 5, 14 and 19, there is a significant drop in the permutation coefficient for all of the features. These were also the sub samples with the lowest score, registered in the score analysis for the linear regression model.

\begin{figure}[!tp]
	\centering
	\includegraphics[width=1.\textwidth]{figures/LR_heatmap.pdf}
	\caption{Linear Regression permutation coefficients heatmap: each row in the figure represents a sub-sample of the data used for training the algorithm. Each column represents the permutation coefficient value for the considered feature.}
	\label{fig:lr-heatmap}
\end{figure}

Finally Fig. \ref{fig:lr-pdp} shows the change in thickness value for changes in values of the features. Features are scaled according to Eq. \ref{eq:scale}, hence the values on the x-axis of the figure do not represent the original input values, but the scaled ones used to train the model. Having scaled features helps reading this chart as features could have very different ranges and scales (for example altitudes ranging between  $2000m$ and almost $5000m$, compared to slope angles ranging from $0$ to $\frac{\pi}{2}$). Each line represents ice thickness changes when all other variables are kept constant. This provides for a visual comparison between the influence which each feature has on the prediction. For the linear regression case each line is a of course a straight line, but this will not be the case for the other models. Looking at the chart the same conclusions as for both the linear coefficient magnitude, and the permutation importance can be drawn.

\begin{figure}[!tp]
	\centering
	\includegraphics[width=1.\textwidth]{figures/LR_pdp.pdf}
	\caption{Linear Regression partial dependence plots: representation of the change in Ice Thickness compared to the change in the value of the features. Features values are scaled according to Eq. \ref{eq:scale}}
	\label{fig:lr-pdp}
\end{figure}

\section{Random Forest Regression}\label{rfr}
The same process of using different sub-samples to train the algorithm and test its performance used in the linear regression case, has been used for the random forest algorithm. This is particularly important for random forests because they are very prone to over-fitting. Especially the parameters determining the maximum tree depth, and the one determining the minimum number of samples required to split the tree, are those which had to be carefully adjusted in order to maintain a good balance between the model performances, and its inability to generalize for samples not used to train it. The maximum tree depth parameter has been tuned in order for all the nodes to be expanded, until all the samples at the node would have the same label, or until the minimum sample necessary for the tree to split would be reached. The minimum sample required for the tree to split has been set to 5\% of the number of samples present in the training data-set.

\subsection{Score and volume spread}\label{rfr-score}

\begin{figure}[!tp]
	\centering
	\includegraphics[width=1.\textwidth]{figures/RFR_score.pdf}
	\caption{Random Forest Regression: Volume and score values of the model after training it with 20 different sub-sample of data. On the x-axis each number identifies a different sample of data. On the left the total volume of the alpine glaciers present in the GlaThiDa. On the right the $R^2$ coefficient for values left out of the training sample.}
	\label{fig:rfr-score}
\end{figure}

Figure \ref{fig:rfr-score} shows the volume and score for 20 different sub-samples used to the random forest regression.

It must be reminded that as the name of the algorithm suggests, part of its training process is random, due to the random processes used to create the different trees of the forest (see section \ref{random-forest}). This means that training the model again could produce slightly different results then the ones shown in Fig. \ref{fig:rfr-score}.

The score shown seems to be dependent on the sample used for training. This could mean that the model has been over-fitted, but it's also a common characteristic of random forest algorithms \citep{RandomForest2018}. For some samples the score is around 0.6 but it gets as low as almost 0 for sample 12. As for the linear regression case low scores are registered for samples 5, 14 and 19 but also for samples 8, and 12.
The model trained with all the data available from GlaThiDa reached a score 0.57, which is the highest of all the models tested.

\begin{figure}[!tp]
	\centering
	\includegraphics[width=1.\textwidth]{figures/RFR_thick_map.pdf}
	\caption{Random Forest Regression: Ice thickness distribution for two glaciers on top of the terrain slope angle. On the left, one of the glaciers used to train the model. On the right, one of the glaciers outside those used to train the model.}
	\label{fig:rfr-map}
\end{figure}

\begin{figure}[!tp]
	\centering
	\includegraphics[width=1.\textwidth]{figures/RFR_thick_diff_map.pdf}
	\caption{Random Forest Regression: Maps of two glaciers showing the glaciers surface and the difference between the modeled ice thickness and the observed one.}
	\label{fig:rfr-diff-map}
\end{figure}

Contrary to the score values, the volume spread is not that high. The model seems to fail to correctly predict specific ice thickness, and its score is influenced by the sample chosen for training. This does not however seem to influence the total volume predicted for the glaciers for which this volume has been computed.

In Fig. \ref{fig:rfr-map} the ice thickness distribution predicted by the model trained using all the available observations for two different glaciers is shown. The ice thickness distribution is distributed in a rather discontinuous way, with large areas of the glaciers having relatively high but constant ice thickness values. It is also notable that the extreme values for ice thickness in both glaciers are the same. All this is to be expected for the random forest regression, as the values predicted at each leaf by the model are the mean of the ice thickness values for the samples which reached that leaf. This implies that abrupt changes in ice thickness are to be expected and that the model is not able to predict values higher than the maximum ice thickness value found in the training data-set.

Fig. \ref{fig:rfr-diff-map} shows the difference in ice thickness between the model prediction and the observations. For both glaciers differences up to 100m are shown. For glacier RGI60-11.00746 ice thicknesses values higher than those observed are visible in the middle-bottom part of the glacier towards its center and lower ones towards the border. This is similar to what already shown for the linear regression model. Higher predictions are very pronounced also in the top part of the glacier. For glacier RGI60-11.02773 the model predictions still show high differences with the observations, but the largest differences  seem to be less frequent compared to RGI60-11.02773. For this glacier the model predicts lower values than observed more frequently than higher ones. In general it seems to do a better job predicting glacier ice thicknesses for the glacier on the right.

The total volume for the alpine glaciers computed with the random forest regression model trained with all the available observations is $9.96 \times 10^{10}m^3$, which is 22.1\% lower than the volume predicted by \citet{Farinotti2019}.

\subsection{Features Importance}\label{rfr-features}

As for the linear regression model, random forest regression allows for a direct way of computing how influential each feature is at making predictions. This is computed as the total decrease in node variance, weighted by the proportion of samples reaching the node, averaged over all trees in the ensemble (see section \ref{random-forest}). This is a very convenient way of determining the importance of the features, but it has been found biased in some cases \citep{RandomFBias2007}. For this reason performing a permutation importance test on the model, can help to confirm the results obtained from the variance reduction method.

Figure \ref{fig:rfr-heatmap} shows the coefficients deriving from the permutation analysis on the left, and the feature importance deriving from the variance reduction on the right. Both are calculated using the models trained with the different training samples already used for the score analysis. The feature importance heatmap on the right predicts the slope angle to be the most important of the features. In fact it computes it to be on average 1.6 times more important than the mass balance in making predictions. Altitude and distance from the border seem to be of almost no relevance in making predictions for both the methods used. However the permutation importance points at the mass balance as the most important feature. The difference in the coefficients of the mass balance and the slope angle computed with the permutation importance analysis, isn't as pronounced as the difference for the same coefficients calculated with the variance reduction method. The values for the permutation importance of mass balance and slope angle are in fact very similar.

\begin{figure}[!tp]
	\centering
	\includegraphics[width=1.\textwidth]{figures/RFR_heatmap.pdf}
	\caption{Random Forest Regression feature importance heatmaps: On the left the permutation coefficient; on the right the feature importance from the variance reduction method. Each row in the figure represents a sub-sample of the data used to train the algorithm. Each column represents respectively the permutation coefficient (left) or the variance reduction (right) value for the considered feature.}
	\label{fig:rfr-heatmap}
\end{figure}

Fig. \ref{fig:rfr-pdp} shows the change in thickness value for changes in values of the features. Once again, feature values are scaled according to Eq. \ref{eq:scale}. Immediately one sees that the change in ice thickness is not linearly dependent from the change in feature value. All the lines don't look continuous and present clear steps marking the change in ice thickness. This is partly due to the possible abrupt change in feature value, but it is mostly a natural effect of the sharp splits happening in the node of a decision tree.  The mass balance has a change in its slope going from increasing to decreasing around 0 and then increasing again around 1.8. The slope line has a sudden pick and changes from decreasing to increasing just below 0. The altitude also is slightly non monotonic between -1 and 0 while the border distance is always monotone.
\begin{figure}[!tp]
	\centering
	\includegraphics[width=1.\textwidth]{figures/RFR_pdp.pdf}
	\caption{Random Forest Regression partial dependence plots: representation of the change in Ice Thickness compared to the change in the value of the features. Features values are scaled according to Eq. \ref{eq:scale}}
	\label{fig:rfr-pdp}
\end{figure}

Fig. \ref{fig:rfr-pdp} also is in accordance with the feature importance analysis showed in the heatmaps of Fig. \ref{fig:rfr-heatmap} as the altitude and border distance seem to lead to very small changes in the ice thickness prediction. Looking at the partial dependence plots, it also seems like the slope leads to a change in ice thickness only up until the scaled value reaches 0 (around $18^{\circ}$ angle). Noticeably this value of $18^{\circ}$ is in accordance with what reported by \citet{cuffey2010physics}[P. 296] which states that the approximation in Eq. \ref{eq:slope} is valid for small slope angles and that slope angles in glaciers rarely exceed $20^{\circ}$. In the data used for training the models in fact, slope angles larger than $18^{\circ}$ are outside the inter-quartile range. After this value the ice thickness prediction made by the random forest regression doesn't change. The maximum range of ice thickness change achieved by the slope angle is $40m$. For the mass balance there seem to be a change in ice thickness prediction throughout the whole range of its values up until scaled values just below 3. The steepest changes happen for values below 0. Its range in changes in ice thickness is almost $60m$ which is higher than the changes attributed to the slope angle.

\section{Support Vector Regression}\label{svr}
Support Vector Regression is the last model for which the results are presented here. The training process for this algorithm is notably slower compared to the linear regression and the random forest. However it is the model which reached the highest average score between the ones used.

\subsection{Score and volume spread}\label{svr-score}

\begin{figure}[!tp]
	\centering
	\includegraphics[width=1.\textwidth]{figures/SVR_score.pdf}
	\caption{Support Vector Regression: Volume and score values of the model after training it with 20 different sub-sample of data. On the x-axis each number identifies a different sample of data. On the left: the total volume of the alpine glaciers present in the GlaThiDa. On the right: the $R^2$ coefficient for values left out of the training sample.}
	\label{fig:svr-score}
\end{figure}

Fig. \ref{svr-score} shows on the left the volume spread for the alpine glaciers with ice thickness observations, and on the right the score of the model as defined in \ref{eq:score}. Each of the different values represents a model trained with a different sub-sample of data, the same sub-samples already used for the linear regression and random forest regression analysis.

The score shows a very similar pattern compared to the one shown in the linear regression model (see Fig. \ref{fig:lr-score}). Samples 5, 14 and 19 present a drop in the score achieved, exactly as already shown for the linear regression case. The average score however is higher reaching 0.55 on average and it is around 0.6 for most of the sub-samples used to train the model.
The value of the score for the model trained with all the data available from GlaThiDa is 0.45, which means that less than 50\% of the ice thickness variance is explained by the model.

The volume spread on the left of the figure shows a dispersion around 1.15\% and an average total volume of $1.53 \times 10^{10}m^3$. This is the lowest of all the models considered so far.

Looking at Fig. \ref{fig:svr-map}, the ice thickness distribution for the glacier RGI60-11.01450, Aletsch glacier, immediately looks strange. The ice thickness for most of the glacier seem to be in the order of 75$m$ with much higher values only reached in the very bottom part of it. It seems that the model is actually predicting a constant ice thickness for most of the glacier. This is of course not a desirable result as ice thicknesses are presumed to change throughout the surface of the glacier. The same behavior is not visible in the left glacier which shows different ice thicknesses uniformly distributed all over the glacier surface. Notice that the left glacier is one used to train the model, while the right one isn't.

Fig \ref{fig:svr-diff-map} shows differences between the ice thickness predicted by the trained support vector regression and the observed one. Both glaciers show differences up to 100m similar to the linear regression and random forest cases. Despite these large difference values, the support vector regression seems to be the model achieving the highest accuracy of the ones used, if comparing the three charts (Fig. \ref{fig:lr-diff-map} and \ref{fig:rfr-diff-map}). Gepatschferner, RGI60-11.00746, shows again a higher predicted ice thickness for the middle-bottom part of the glacier towards its center, but a bit less pronounced than the other two models. In this area, toward the borders, the model predicts in general lower ice thicknesses than the observed values, but with differences not as pronounced as the ones showed for the other two cases. As in te case of the other two models, in the top part of this glacier many predictions higher than the observed ice thickness are present especially in the figure bottom-right part. For glacier RGI60-11.02773 the support vector regression makes accurate predictions for the top part of the glacier and less accurate, with many predictions underestimating the observed ice thickness, in the bottom part of it.

The total volume of ice for the alpine glaciers predicted by the support vector machine is $9.58 \times 10^{10}m^3$, which is the lowest of all the models and 25.1\% lower than the volume predicted by \citet{Farinotti2019} with their ensemble model.

\begin{figure}[!tp]
	\centering
	\includegraphics[width=1.\textwidth]{figures/SVR_thick_map.pdf}
	\caption{Support Vector Regression: Ice thickness distribution for two glaciers on top of the terrain slope angle. On the left one of the glaciers used to train the model. On the right one of the glaciers outside those used to train the model.}
	\label{fig:svr-map}
\end{figure}

\begin{figure}[!tp]
	\centering
	\includegraphics[width=1.\textwidth]{figures/SVR_thick_diff_map.pdf}
	\caption{Support Vector Regression: Maps of two glaciers showing the glaciers surface and the difference between the modeled ice thickness and the observed one.}
	\label{fig:svr-diff-map}
\end{figure}

\subsection{Features Importance}\label{svr-features}
Unlike linear and random forest regression, support vector regression doesn't have a simple way of computing the influence each feature has in making a prediction when using a non-linear kernel to transform the features. In order to analyze which features are the most important, one needs indirect methods such as the permutation importance.

\begin{figure}[!tp]
	\centering
	\includegraphics[width=0.8\textwidth]{figures/SVR_heatmap.pdf}
	\caption{Support Vector Regression feature importance heatmap: each row in the figure represents a sub-sample of the data used to train the algorithm. Each column represents the permutation coefficient value for the considered feature.}
	\label{fig:svr-heatmap}
\end{figure}

In Figure \ref{fig:svr-heatmap} the coefficients for the permutation importance (section \ref{permutation}) for all the models trained with the 20 different sub-samples of the data are shown. Once again the sub-samples with the lowest score registered (see Fig. \ref{fig:svr-score}), sub-samples 5, 14 and 19 are in general the ones displaying a lower value of permutation importance, showing a dark blue line throughout the whole heatmap. Apart from those sub-samples, the linear mass balance above the point is the feature with the highest permutation importance even though values vary from 0.65 to 0.39. The second highest permutation importance values are those of the slope angle. In general then the most important features for the support vector regression model are the same as those of the random forest regression model. However the permutation coefficients of altitude and distance from the border are also comparable to those of the slope angle and they are also very close to each other, with the altitude seeming to play a bigger role compare to the border distance. All the features in general seem to play a role in the model final prediction.

To complete the analysis of how the features influence the prediction of the support vector regression, Fig. \ref{fig:svr-pdp} shows the change in thickness value depending on changes in feature value. Again, the non-linear relation between the two is evident in this chart.  None of the curve is monotone and for each feature the ice thickness seems to increase or decrease depending on the value of the specific feature. Mass balance and slope angle drive the largest ice thickness changes looking at the range of those changes reached by those features curves. For feature values between 2 and 1 however the rate of change in ice thickness is similar between the altitude and slope angle, meaning that both these features compete in determining the ice thickness for those range of values. The same thing happens for values between around -0.5 and around 1.8 for the border distance and the linear mass balance, where again the two seem to compete in determining the predicted ice thickness.
\begin{figure}[!tp]
	\centering
	\includegraphics[width=1.\textwidth]{figures/SVR_pdp.pdf}
	\caption{Support Vector Regression partial dependence plots: representation of the change in Ice Thickness compared to the change in the value of the features. Features values are scaled according to Eq. \ref{eq:scale}}
	\label{fig:svr-pdp}
\end{figure}

This also seems to be in accordance with the analysis obtained with the permutation importance coefficients (see Fig. \ref{fig:svr-heatmap}). All features in fact seem to have some relevance in predicting the ice thickness at least for some of their value ranges. Mass balance and slope angle however seem to be more relevant throughout all their value ranges and also lead to a larger change in ice thickness, when looking at the minimum and maximum ice thickness change reached by their extreme values.

% ==== SECTION 1 ===============================================================
%\section{Some Important Things to Know}\label{3sec:1}
%
%It is important to differentiate between \emph{facts} and \emph{interpretations}
%and between \emph{your contributions} and \emph{those of others}! Facts and
%your contributions are part of this chapter. Interpretations and contributions
%of others should be rather part of the chapter Discussion.
%
%
%\subsection{Experimental Parts in the Chapter Results}
%Experimental details should only appear in the chapter Results to the
%extent necessary to ensure comprehension. Painstaking descriptions of
%instruments, analysis procedures, field conditions, etc., should be part of the
%chapter Methodology. However, absolutely necessary is information that shows
%the \emph{reliability} of the findings and the \emph{robustness} of an
%innovation (e.g., a new analysis technique).
%
%
%\subsection{Numerical Results or so-called Data}
%Results are more than simply numbers! They are tiny but unique
%\emph{messages}. Data should not only be made ``visible'' (e.g., in
%figures, such as in Fig.~\ref{fig:1}) but should also be made
%\emph{articulate}.
%
%
%\subsection{Order of Presentation}
%Use chapters and (sub)sections to separate, e.g., various topics,
%questions, and problems, or to separate measurements from calculations.
%Arrange information in a consistent way, e.g., from simple to complex, from
%small to large (or vice versa in terms of scales: e.g., from
%synoptic-scale to micro-scale), from \emph{most important} (central) to
%\emph{least important} (peripheral). Arrange the material in order to maximize
%impact rather than sticking to a strict chronological order. Try to tell a story
%that consists of a beginning, followed by a gradual unfolding, and a
%``happy end''.
%
%
%\subsection{Cross-References}
%You can always refer to other parts of your thesis like in the following
%example: See chapter \ref{chap2} or section \ref{1sec:3} or
%Fig.~\ref{fig:1} or Table~\ref{tab:1} or equation~(\ref{2equ:1}).
%
%
%\section{Figure}\label{3sec:2}
%Figure \ref{fig:1} shows an example for an EPS figure with two panels. The
%topography of the Wipp Valley and Inn Valley is shown in Fig.~\ref{fig:1a}.
%Figure~\ref{fig:1b} shows the time series of potential temperature at two
%stations. In order to refer to a certain range of figure panels write, e.g.,
%Fig.~\ref{fig:1a}--\subref{fig:1b}.
%
%
%\begin{figure}[!tp]
%\centering
%\figuretopcapfalse
%\subfigure[][]{
%  \label{fig:1a}
%  \includegraphics[width=0.48\textwidth]{figure_topo.eps}
%}
%\subfigure[][]{
%  \label{fig:1b}
%  \includegraphics[width=0.47\textwidth]{figure_theta.eps}
%}
%\caption{(\subref{fig:1a}) Topographic map of the target area: Gray shaded
%elevations contours with increments of 200~m starting at 400~m~MSL and a white
%elevation contour line at 1600~m MSL. (\subref{fig:1b}) Time series of potential
%temperature (K) at Ellboegen (solid line) and Sattelberg (dashed line) from 00
%UTC 06 November to 00 UTC 07 November 1999. Labels in (\subref{fig:1a}) mark the
%location of Innsbruck (IBK), Ellboegen (ELB), and Sattelberg
%(SAB).}\label{fig:1}
%\end{figure}
%
%
%This template uses the \texttt{subfigure} environment with the option
%\texttt{FIGTOPCAP} to place the subfigure labels (\subref{fig:1a}) and
%(\subref{fig:1b}) at the top of the figure. However, since we want to have the
%caption at the bottom of figure, use \verb|\figuretopcapfalse|
%before the first \verb|\subfigure| command within the \texttt{figure}
%environment, otherwise the figure number produced by \verb|\ref| is wrong.
%
%
%\section{Table}
%Table \ref{tab:1} is an example for a table that consists of several
%rows and columns. Here, the \texttt{tabular} environment is used inside the
%\texttt{table} environment.
%
%
%\begin{table}[!htb]
%\centering
%\begin{footnotesize}
%\renewcommand{\tabcolsep}{4pt}
%\begin{tabular}{l p{3.5cm} l l l}
%\hline
%\hline
%Parameter & Code & Description & Units & Code\\
%\hline
%$g$ & \verb|$g$| & acceleration due to gravity&
%  m~s$^{-2}$ & \verb|m~s$^{-2}$| \\
%$T_d$ & \verb|$T_d$| & dew point temperature & $^{\circ}$C &
%  \verb|$^{\circ}$C| \\
%$\mathbf{v} \cdot \nabla T$ & \verb|$\mathbf{v} \cdot| \verb| \nabla T$| &
%  temperature advection & K~s$^{-1}$ & \verb|K~s$^{-1}$| \\
%$\vec{v} \cdot \nabla T$ & \verb|$\vec{v} \cdot| \verb| \nabla T$| &
%  temperature advection & K~s$^{-1}$ & \verb|K~s$^{-1}$| \\
%$\frac{\partial p}{\partial t}$ & \verb|$\frac{\partial p}|
%  \verb| {\partial t}$| & local pressure tendency & Pa~s$^{-1}$ &
%  \verb|Pa~s$^{-1}$| \\
%$p_0 \cos (kx - \omega t)$ & \verb|$p_0 \cos (kx| \verb| - \omega t)$| &
% wave expression & Pa & Pa \\
%\hline
%\end{tabular}
%\end{footnotesize}
%\caption{Some meteorological and mathematical parameters and expressions.}
%\label{tab:1}
%\end{table}
%
%
%\section{Figure and Table Captions}
%
%Figure and table captions \emph{must} contain all necessary information to
%understand the \emph{content} of the figure and table, without the need of
%additional text. Only in case of very complicated figures or tables, the caption
%may end with a remark such as ``See text for further explanation''. The
%\emph{interpretation} of the table or figure is not part of the caption, but
%should be given in the main text. In order to avoid repetitions, the phrase
%``As in Fig. xx, but for \dots'' is often used. Necessary information provided
%in the caption is a description of
%\begin{itemize}
%\item shown parameters together with units,
%\item date and time,
%\item contour intervals,
%\item location,
%\item line styles and markers,
%\item and others.
%\end{itemize}
%A list of figures and a list of tables at the beginning of the thesis (before
%chapter 1) is optional.
%
%
%
%\section{Title}
%
%The title of your science thesis should be kept as short as possible. It should
%represent an extremely compact summary of the thesis. The title should provide a
%clear and complete description of the topic and should contain many keywords
%(``what?'', ``how?'' and possibly ``why?''). The main title should not contain
%more than 10 words. An optional subtitle may be used if necessary (all together
%not more than 25 words). Important words and terms should be placed at the
%beginning of the title. Avoid unspecific expressions such as
%\begin{itemize}
%\item[] Investigation of ...
%\item[] Experiments on ...
%\item[] Results of ...
%\item[] Attempts to ...
%\end{itemize}
%Rather use expressions such as
%\begin{itemize}
%\item[] Influence of ... on ...
%\item[] Generation of ... with ...
%\item[] Dependence of ... upon ...
%\item[] Optimization of ... upon ...
%\end{itemize}
%Avoid technical abbreviations or acronyms and special symbols such as IR
%for infrared or $\theta$ for potential temperature.
%
%
%\section{Abbreviations and Symbols}
%Abbreviations (e.g., ECMWF) and symbols (e.g., $\vec{v}_g$) have to be
%defined, i.e. explained, at the place where the \emph{first} appear in the text.
%For example: The model was initialized with the operational analysis of the
%European Centre for Medium-Range Weather Forecasts (ECMWF). From the ECMWF
%fields of geopotential height we derive the geostrophic wind vector
%$\vec{v}_g$. The change of $\vec{v}_g$ with pressure $p$ reveals the thermal
%wind equation.
%
%A list of abbreviations and a list of symbols at the beginning of the thesis
%(before chapter 1) is optional.
%
%
%\section{Parameters and Units}
%
%Use italic letters for scalar quantities (e.g., $R$ and $g$), bold upright
%letters or arrows for vector quantities (e.g., $\mathbf{v}_g$ or $\vec{v}_g$)
%and sans-serif letters for tensors (e.g., $\mathsf{T}$). All this parameters
%should be written in mathematical mode (\verb|$ ... $|). Units should be
%written with normal upright letters (e.g., m~s$^{-1}$). Use spacing between
%numbers and individual units (e.g., $\phi=100$~W~m$^{-2}$). Less or no spacing
%is used between numbers and units in case of percent and degrees (e.g.,
%10\% and 5$^{\circ}$C). See also Table~\ref{tab:1} for further examples.
%
%
%\section{Footnotes}
%Do not use footnotes in an extensive way. Footnotes distract the reader from the
%main body of the document. Do not use footnotes for referring to literature,
%rather use the author-year citation system together with a bibliography (list
%of references) at the end of the thesis.