\chapter{Implementation}

In this chapter the technologies used during the project are explained. An overview of how the design laid down in chapter 3 was implemented is presented.
Any challenges that were faced during the implementation are also discussed and contrasted with alternative approaches.

\section{Technologies}
Following an investigation of the field and the available technologies it was decided that the system would be implemented in Python.
Python is a dynamically typed language which has support for multiple different programming paradigms.
Python is widely used today both in industry and the scientific community.
Libraries such as NumPy, SciPy, matplotlib and pandas are widely used in the fields of data-analysis and scientific computing.
This has proven that Python is an adequate substitute for languages such as C, C++, Java, R, Matlab, etc\cite{PythonR}.

Python also favors readability and minimalism compared to low level programming languages.
Despite being a relatively high level programming language, Python can also interface with C and C++ using the Cython library.
This is particularly useful when making code optimisations, such as vectorising code loops.

\subsection{Gensim}
Gensim is a Python toolkit for topic modeling and semantic analysis of textual data.
Gensim favours an online approach to training the models.
This is achieved using Python generator expressions, which can result in only one document from the corpus residing in main memory at any one time.
This small memory footprint and incremental approach allows Gensim to scale in relation to the size of the corpus.
Gensim interfaces very well with NumPy and SciPy libraries.
As well as this, it has been optimised to use the Cython library in order to speed up execution.

By using the Gensim toolkit the system was able to gain access to the following tools:
\begin{itemize}
    \item build a dictionary of features from the corpus
    \item convert the corpus to a bow representation
    \item apply tf-idf transformations to bow models
    \item train LDA on bow models
    \item convert bow models to NumPy arrays
    \item train Miklov's Word2Vec model
    \item easily load and save different representations of the corpus
    \item use Gensim's native indexing for document similarities
\end{itemize}

As shown above Gensim is quite extensive in it's functionality and thus allowed the easy manipulation of corpus.
Without access to Gensim it would have taken a considerable amount of time to implement all of this functionality.

\subsection{Annoy}
Annoy is a C++ library with Python bindings for approximate-Nearest-Neighbour search.
Annoy was created by Spotify to generate recommendations.
Within Spotify they use it to find similar users and similar songs.
Spotify's system contains millions of tracks and has approximately 60 million active users.
This shows that Annoy is a tried and tested approximate-Nearest-Neighbour library which has scaled to large data-sets.

Annoy does approximate-Nearest-Neighbour queries by building an index using locality sensitive hashing (LSH) and random projections.
LSH is a technique used to reduce the dimensionality of high dimensional data by mapping the data into hash bins.
LSH tries to map similar items into the same bin and is very different to cryptographic functions which try to reduce the possibilities of collisions\cite{slaney2008locality}.
Random projections are a type of dimensionality reduction techniques which work by projecting the data into a random subspace of lower dimension\cite{Dasgupta2000}.

Radim {\v R}eh{\r u}{\v r}ek, the creator of Gensim investigated the performance of different nearest neighbor libraries in November 2013 \cite{radimKnnInvestigation}.
In the final investigation the libraries were reduced down to just Annoy and FLANN.
During that investigation, Radim found that the query time for FLANN was approximately four times quicker than Annoy.
Annoy on the other hand had almost perfect accuracy (When compared to a non-approximate version of k-NN) compared to FLANN.

Annoy was selected for the k-NN investigation as it:
\begin{itemize}
    \item Was found to be the most accurate approximate-Nearest-Neighbour library
    \item has an easy to understand API
    \item Will be integrated into Gensim once the Boost library has been dropped as a requirement
    \item Has been shown to scale to very large data-sets
\end{itemize}

As shown above Annoy was chosen largely due to it's simplicity and the ease of integration with the Gensim toolkit.

\section{System Implementation}
In this section an outline of the system that was built, as prescribed in Chapter 3, is presented.
Where possible UML has been used to describe the relations between the different components of the system.

\subsection{System}
\begin{figure}[h]
    \centering
        \includegraphics[width=0.7\textwidth]{Figures/FYPClassDiagram.png}
    \caption{System class diagram}
    \label{fig:UMLClass}
\end{figure}

Figure~\ref{fig:UMLClass} presents an overview of the system as a UML class diagram.
The main class in the system is the Corpus class.
The Corpus class is an abstract class used to represent a bag-of-words corpus.
The docs attribute is a list of document names from the corpus.
The dictionary attribute is a Gensim Dictionary object which represents the words in the corpus and their respective term frequencies.

The Corpus classes static function get\_docs is an interesting function used to generate a list of corpus files.
The function receives a path to a system directory, a dictionary of file distributions in sub-directories and a maximum number of files.
This function then generates a list of files from each of the sub-directory while maintaining the specified distributions.
The function then returns this flattened list of file paths.
This function is useful as it ensures that the same proportion of files will be sampled from each arXiv defined topic area, no matter the size of the corpus currently being tested.

The transformation attribute is used to represent transformation that have been applied to the bag-of-words corpus.
An interesting feature of Gensim is that you can stack corpus transformations on top of each other.
This for example could be used to apply a tf-idf transformation followed by an Latent-Semantic-Indexing transformation.
This however posed a small problem for the system.

When creating the above system, code readability and code reuse were huge factors that were considered.
In an earlier version of the project, if the corpus had a transformation applied, the transformation was only executed when iterating through the corpus.
However if a corpus did not have any transformations applied to it, this required an nested if-statement, which reduced code readability.

An alternative to this was to create an identity transformation which inherited from Gensim's TransformationABC class.
This Identity\_transformation is basically an identity function which returns the input document vector.
This approach was based on monad transformers from Haskell where the inner-most monad of stacked monad-transformers is either the identity monad or an IO action.
This eliminated the unnecessary conditional statement and increased the code readability by applying functional programming paradigms.

The Paper\_Corpus class is a wrapper around Gensim's Text\_Corpus class.
Gensim favours document streaming and is agnostic as to how the corpus is stored.
The corpus could be stored in one file or distributed across the web.
The Text\_Corpus is an abstract class which allows the user to use Python's generator expressions to lazily load documents from the corpus and do any necessary pre-processing to the text.
As the arXiv corpus being investigated is organised as a collection of .txt files in multiple directories, this approach allowed the system to load documents one at a time and do some pre-processing to the text.
The pre-processing conducted includes; reading the document into memory, translating the text to lower-case, splitting the word-tokens on whitespace characters, and filtering out words that appear only once in the document.

Both the LDACorpus and the KNNCorpus files use a bag-of-words representation of the corpus and thus inherit from the Corpus class.
W2VCorpus on the other hand trains it's model using the plaintext corpus.
As stated above when implementing the system, code reuse and readability were two of the main factors considered.
When it came to implementing the Word2Vec model a choice was made to not inherit from the Corpus class even though there was some shared functionality.
This decision was made largely due to the difference in the semantics of the Word2Vec model and the models which used a bag-of-words representation.
This of course means that the DRY principle, from software-engineering, is violated but the shared functionality between the two classes is minor and separation ultimately increases the overall readability of the code.

The LDACorpus and the KNNCorpus are relatively small classes which inherit the majority of their functionality from the Corpus class.
The attributes for the LDA class and the k-NN expose the API for the underlying models imported from the Gensim and Annoy libraries.
Exposing the API allowed easier testing and tweaking of the algorithms being investigated.

\subsection{Code Harnesses}
Two code harnesses are used in the research project. The first harness is used to generate the LDA, k-NN and Word2Vec corpora and to log the time taken to generate each corpus.
The second harness is used to run similarity queries on each of the corpus files, log the time taken to run the similarity queries, log the results of each of the queries and then output the results in JSON.

\subsubsection{Generation Harness}
The generation harness is useful for the creation of corpora of varying size.
From the command line the script receives a list of corpus sizes to create, a corpus source directory and the algorithm to apply to the corpus.
An optional directory can also be passed as an argument to the script.
This optional directory is an output directory to store the generate corpus and model files.
If this directory is not specified a directory is created where the script is run and all files generated are stored in it instead.

The generation harness iterates through the list of corpus sizes and creates a corpus of that size.
For all of the algorithm apart from Word2Vec, a dictionary must be created and the corpus must be converted to a bag-of-words representation.
Word2Vec effectively skips this stage as it builds it's dictionary and model in the training stage.
For each corpus created, the time take to build the corpus is logged to a log file.
After the corpus and dictionary files have been created, the models is trained.
As Word2Vec differs from LDA and k-NN it trains using the original textual corpus.
LDA and k-NN on the other hand must use the dictionaries and bag-of-words representations created previously to train their models.
The times taken to train each of the models is also logged to a file.

\subsubsection{Similarity Harness}
The similarity harness automates the process of using the models, which were generated by the generation harness, to query for similar documents.
The command line arguments for the similarity harness are very similar to the generation harness.
The script receives a list of corpus sizes, a directory of query files, a directory containing the previously generated models and the algorithm being tested.

For each of the sizes passed to the script, the list of query files is processed into the appropriate data representation.
For LDA and k-NN, the dictionaries created previously are loaded and the files from the query directory are converted to a bag-of-words representation.
For Word2Vec the process is slightly different because we need to find the word vector representations of the query documents before we can find similar documents.
This is achieved by loading the query documents and then adding them to the Word2Vec model.
This allows us to get a vector representation of the document which can be used to find similar words and similar documents.

The similarity queries are then run on each of the documents from the query corpus.
The times taken to run the queries are logged to a file for later analysis.
The similar document results are also stored and later parsed into readable JSON.
The similarity results are parsed into JSON in order to make them more portable.
This was done as there are some interesting Javascript libraries which could be used to visualise the results.

\section{Conclusion}
In this chapter the implementation of the system designed in chapter three has been described.
Any challenges encountered have been stated and their solutions explained.
A complete system for building models of a corpus and for querying the generated models has been built.
This consists of components to:
\begin{itemize}
    \item process a large number of PDF documents into plain text
    \item sample a distribution of documents from the corpus
    \item convert documents into a bag-of-words representation
    \item build models of the corpus
    \item time the execution for building and querying the models
    \item query the generated models using a collection of plain text documents
    \item parse and log similarity results
    \item and to visualise the data generated
\end{itemize}
This system allows the easy analysis of machine learning algorithms on a textual corpus.
The system was originally intended for the arXiv corpus and the three machine learning algorithms stated previously. However, the system could be easily extended to use another text corpus or different algorithms.

In the following chapter, the results generated from the above system will be presented and analysed.


%!TEX root=../main.tex
\chapter{Numerical Implementation} \label{chap:numerics}
\minitoc

This chapter is devoted to the numerical implementation of the dynamic gradient damage model outlined in \cref{def:dynagraddama}. Semi-spatial discretization is performed in \cref{sec:spatial} based on the classical $C^0$ finite element method. The coupled two-field evolution problem is then discretized in the time domain based on the general $\beta$-Newmark scheme, see \cref{sec:timedis}. Some implementation issues are indicated in \cref{sec:implementation}.

In this contribution the spatial and temporal discretization is as usual decoupled and will be discussed below separately. Space-time finite element methods, see \cite{HughesHulbert:1988}, exploits fully the variational nature of the formulation \eqref{eq:vi} and can be considered as a possible improvement in the future. The elastic energy density split \eqref{eq:elasticTC} is adopted to take into account tension-compression asymmetry. The two damage constitutive laws \eqref{eq:at2} and \eqref{eq:at1} will be used. In this work the irreversibility condition and the variational inequality \eqref{eq:vi} will be effectively implemented, which amounts to solve numerically the weak wave equation \eqref{eq:weakform} and the minimality principle for damage \eqref{eq:crackmin} at every time step.

\section{Spatial Discretization} \label{sec:spatial}
The spatial discretization described here constitutes a dynamic extension of the work of \cite{BourdinFrancfortMarigo:2008,PhamAmorMarigoMaurini:2011} for the regularized fracture model \eqref{eq:Pualpha} and the quasi-static gradient damage model outlined in \cref{def:qsgraddama}. A discussion on the large-displacement situations is indicated at the end. The thematic subjects covered here are thus summarized in \cref{tab:summsd}.
\begin{table}[htbp]
\centering
\caption{Thematic subjects covered in this section} \label{tab:summsd}
\begin{tabular}{ccccc} \toprule
& Going dynamical & $\alpha\leftrightarrow\phi$ & $\nabla\alpha\to\Gamma$ & Experimental validation \\ \midrule
Theoretics & & & & \\
Numerics & \rightthumbsup & & & \\ \bottomrule
\end{tabular}
\end{table}

A semi spatial discretization is performed from a mesh $\Omega_h$ of the original domain $\Omega$. It should be ideally unstructured and uniform in mesh sizes otherwise some directions may be preferred when cracks propagate, see \cite{Negri:1999,PhamAmorMarigoMaurini:2011} and references therein for a discussion on the ideal mesh properties in order to achieve a better modeling of fracture via gradient-damage approaches. In particular, the typical element size $h$ of the mesh should be preferably smaller with respect to the internal length $\ell$ in order to calculate correctly the damage band profile, the dissipation energy \eqref{eq:surface} and the material response inside the crack process zone of order $\mathcal{O}(\ell)$. It leads often to a computationally demanding calculation which requires massive parallelization.

The displacement $\vec{u}_t$ and the damage field $\alpha_t$ will be both discretized with classical $C^0$ isoparametric finite elements based on $\Omega_h$. For two-dimensional plane problems, an arbitrary element possesses at every node 3 nodal degrees of freedom corresponding to 2 components of the displacement and 1 scalar value of the damage, see for example \cref{fig:q4}.
\begin{figure}[htbp]
\centering
\includegraphics[width=0.25\textwidth]{q4.pdf}
\caption{Local interpolation of $(\vec{u},\alpha)$ based on nodal values inside the element} \label{fig:q4}
\end{figure}
The symbols $\uvec$ and $\dvec$ are used to denote the current global displacement and damage nodal vectors. Inside a given physical element $\Omega_\me\in\Omega_h$, their local nodal vectors $\uvec^\me$ and $\dvec^\me$ achieve an interpolation of the displacement and damage fields as well as their derivatives
\begin{equation} \label{eq:interpolation}
\begin{aligned}
& \vec{u}_t(\vec{x})=\vec{N}(\vec{x})\uvec^\me\quad\text{and}\quad\eps(\vec{u}_t)(\vec{x})=\vec{B}(\vec{x})\uvec^\me\, , \\
& \alpha_t(\vec{x})=\vec{N}_\alpha(\vec{x})\dvec^\me\quad\text{and}\quad\nabla\alpha_t(\vec{x})=\vec{B}_\alpha(\vec{x})\dvec^\me
\end{aligned}
\end{equation}
where $\vec{N}$'s and $\vec{B}$'s are respectively the interpolation and differentiation matrices. The Voigt notation is assumed for the strain tensor $\eps(\vec{u}_t)$ and the damage gradient $\nabla\alpha_t$, which are written as column vectors. \emph{Linear} interpolation is used due to its low computational cost in a dynamic calculation. According to \cite{BourdinFrancfortMarigo:2008}, linear elements for the displacement and damage fields perform equally well compared to higher-order elements in terms of $\Gamma$-convergence of the damage dissipation energy. Finally this P1-P1 finite element discretization is not forbidden according to \cite{SimoneAskesPeerlingsSluys:2003}.

\paragraph{Displacement problem}
Plugging \eqref{eq:interpolation} into the weak elastic-damage dynamic wave equation \eqref{eq:weakform} and adopting the same interpolation for the testing function $\vec{w}_t$, we obtain the following spatially discretized dynamic evolution equation
\begin{equation} \label{eq:waveeqsdis}
\vec{M}\ddot{\uvec}=\vec{F}_\mathrm{ext}-\vec{F}_\mathrm{int}(\uvec,\dvec)
\end{equation}
with $\vec{M}$ the consistent mass matrix, $\vec{F}_\mathrm{ext}$ the external force vector corresponding to the potential \eqref{eq:power} and $\vec{F}_\mathrm{int}$ the internal force vector. The mass matrix is assembled from the elementary mass matrices which reads inside $\Omega_\mathrm{e}$
\[
\vec{M}=\int_{\Omega_\mathrm{e}}\rho\vec{N}^\mT\vec{N}\dx.
\]
For explicit dynamic situations, this matrix is lumped $\vec{M}\mapsto \overline{\vec{M}}$ using the classical row-summing techniques to improve computational efficiency and spectral properties, see \cite{Hughes:1987}. The global internal force vector is constructed from its elementary counterparts which are given by
\begin{equation} \label{eq:Fint}
\begin{aligned}
& \vec{F}_\mathrm{int}^\me=\int_{\Omega_\me}\vec{B}^\mT\sig\bigl(\vec{B}\uvec^\me,\vec{N}_\alpha\dvec^\me\bigr)\dx \\
&= \int_{\Omega_\me}\vec{B}^\mT\bigl(\mathsf{a}(\vec{N}_\alpha\dvec^\me)\sig_0^+(\vec{B}\uvec^\me)+\sig_0^-(\vec{B}\uvec^\me)\bigr)\dx\,,
\end{aligned}
\end{equation}
where the stress expression \eqref{eq:stressposneg} that takes into account tension-compression asymmetry is used. In explicit dynamics a residual stiffness $k_\mathrm{res}$ is not needed in the stiffness degradation function since no matrix inversion is needed, contrary to the implicit cases \cite{PhamAmorMarigoMaurini:2011,SchlueterWillenbuecherKuhnMueller:2014}.

For simplex finite elements (triangular and tetrahedral elements) the Jacobian of the transformation (between the reference and the physical elements) as well as the differentiation matrix $\vec{B}$ are constant, thus \eqref{eq:Fint} can be integrated exactly using an effective stress
\begin{equation} \label{eq:FintSim}
\vec{F}_\mathrm{int}^\me=\abs{\Omega_\me}\vec{B}^\mT\sig_\mathrm{eff}=\abs{\Omega_\me}\vec{B}^\mT(\mathsf{a}_\mathrm{eff}\sig_0^++\sig_0^-)\,,
\end{equation}
where $\abs{\Omega_\me}$ denotes the measure (area of volume) of $\Omega_\me$ and $\mathsf{a}_\mathrm{eff}$ stands for the effective stiffness degradation
\begin{equation} \label{eq:aeff}
\mathsf{a}_\mathrm{eff}=\frac{1}{\abs{\Omega_\me}}\int_{\Omega_\me}\mathsf{a}(\vec{N}_\alpha\dvec)\dx.
\end{equation}
Explicit expression of $\mathsf{a}_\mathrm{eff}$ can be readily computed and are given as follows.
\begin{itemize}
\item Triangular elements
\[
\mathsf{a}_\mathrm{eff}=\frac{1}{3}\left(\left(1-\frac{\alpha_1}{2}-\frac{\alpha_2}{2}\right)^2+\left(1-\frac{\alpha_2}{2}-\frac{\alpha_3}{2}\right)^2+\left(1-\frac{\alpha_1}{2}-\frac{\alpha_3}{2}\right)^2\right)
\]

\item Tetrahedral elements
\[
\mathsf{a}_\mathrm{eff}=1+\frac{1}{10}\sum_{i=1}^4\sum_{j=1}^4\alpha_i\alpha_j-\frac{1}{2}\sum_{i=1}^4\alpha_i
\]
\end{itemize}
The use of the effective stress $\sig_\mathrm{eff}$ improves computational efficiency and may also be served as an output of the current element.

For tensor product finite elements, a loop on the Gauss points is necessary and our experience suggests that 4 Gauss points for quadrilateral elements and 8 for hexagonal elements are sufficient. An effective stress can no longer be defined and the stress evaluated at these Gauss points will be used as the output variable.

\begin{remark}[Extension to large-displacements]
Numerical implications of the large-displacement extension proposed in \cref{sec:extensions} are discussed here. It is based on the explicit dynamics situations, where at the beginning of every time step the current configuration $\vphi_t(\Omega)$ is known and is updated from the last iteration. The Eulerian elastic-damage dynamic wave equation \eqref{eq:waveeq} will be naturally solved in the current deformed mesh obtained by constantly updating the mesh coordinates $\vec{x}_t=\vec{x}+\vec{u}_t(\vec{x})\in\Omega_t$.

This implies that the interpolation of the displacement and the damage \eqref{eq:interpolation}, as well as the calculation of the global internal force vector \eqref{eq:Fint} should be performed in the deformed element $\vphi_t(\Omega_\mathrm{e})$. If the interpolation $\vec{N}$'s can always be transformed to a reference element, the differentiation matrices $\vec{B}$'s should be evaluated with respect to the current deformed element geometry and hence require being constantly updated at the beginning of every time step.

The Hencky logarithmic strain $\vec{h}_t$ defined in \eqref{eq:logstrain} replaces the use of the linearized strain in the calculation of the stress tensor in \eqref{eq:Fint}. Although the polar decomposition is indeed unnecessary by remarking that $\log\vec{V}_t=\frac{1}{2}\log\vec{B}_t$, spectral decomposition of the left Cauchy-Green tensor $\vec{B}_t=\vec{F}_t\vec{F}_t^\mT$ is unavoidable.
\end{remark}

\paragraph{Damage problem}
Plugging the interpolation expression \eqref{eq:interpolation} into the energy minimization principle \eqref{eq:crackmin} and using the elastic energy density split \eqref{eq:elasticTC} leads to the following spatially discretized variational inequality
\begin{equation} \label{eq:crackdis}
q_{\uvec}(\dvec)\leq q_{\uvec}(\underline{\vec{\beta}})\text{ for all $\underline{\vec{\beta}}$ that $0\leq\dvec\leq\underline{\vec{\beta}}\leq 1$}
\end{equation}
which involves the following discrete functional which reads inside each element
\begin{equation} \label{eq:qdiscrete}
q_{\uvec}^\mathrm{e}(\dvec)=\int_{\Omega_\mathrm{e}}\bigl(\mathsf{a}(\vec{N}_\alpha\dvec^\me)\psi_0^+(\vec{B}\uvec^\me)+w(\vec{N}_\alpha\dvec^\me)+w_1\eta^2\dvec^{\me\mT}\vec{B}_\alpha^\mT\vec{B}_\alpha\dvec^\me\bigr)\dx.
\end{equation}
For sophisticated damage constitutive laws such as that used in \cite{LorentzGodard:2011}, it is a nonlinear convex functional where the Hessian matrix depends on the damage state during numerical solving. Fortunately, using the damage constitutive laws \eqref{eq:at2} and \eqref{eq:at1}, this functional $q_{\uvec}(\dvec)$ is quadratic with respect to the damage vector $\dvec$ and is given by
\begin{equation} \label{eq:Hb}
q_{\uvec}(\dvec)=\frac{1}{2}\dvec^\mT\vec{H}(\uvec)\dvec-\vec{b}(\uvec)^\mT\dvec.
\end{equation}
The Hessian matrix $\vec{H}$ and the second member vector $\vec{b}$ depend solely on the current deformation state $\uvec$ and hence are constant during the solving process of the damage problem. Their exact forms depend on the damage constitutive law used:
\begin{itemize}
\item For the \eqref{eq:at1} law for instance, they can be assembled from the elementary matrix and vector given by
\begin{align*}
\vec{H}^\me &= \int_{\Omega_\me}\left(2\psi_0^+(\vec{B}\uvec^\me)\vec{N}_\alpha^\mT\vec{N}_\alpha+2w_1\eta^2 \vec{B}_\alpha^\mT\vec{B}_\alpha\right)\dx, \\
\vec{b}^\me &= \int_{\Omega_\me}\left(2\psi_0^+\bigl(\vec{B}\uvec^\me\bigr)-w_1\right)\vec{N}_\alpha\dx.
\end{align*}

\item For the \eqref{eq:at2} model, they are given by
\begin{align*}
\vec{H}^\me &= \int_{\Omega_\me}\left(2\left(\psi_0^+\bigl(\vec{B}\uvec\bigr)+w_1\right)\vec{N}_\alpha^\mT \vec{N}_\alpha+2w_1\eta^2\vec{B}_\alpha^\mT \vec{B}_\alpha\right), \\
\vec{b}^\me &= \int_{\Omega_\me}2\psi_0^+\bigl(\vec{B}\uvec\bigr)\vec{N}_\alpha.
\end{align*}
\end{itemize}
These quantities can be evaluated exactly for simplex finite elements. For quadrilateral and hexahedral elements, an approximated using a numerical integration with respectively 4 and 8 Gauss points will be used.

\begin{remark}[Extension to large-displacements]
According to \eqref{eq:crackmin}, the damage minimality condition is naturally formulated in the initial reference configuration. From a physical point of view, this corresponds to the fact that the damage dissipation energy is destined to measure the length or the area of cracks defined in the reference configuration. Technically the concerned energetic functional can be written in the deformed mesh (at the expense of additional unpleasant $\vec{F}_t$ and $J_t$ terms), however it is not necessary \cite{PieroLancioniMarch:2007}.

From a implementation point of view, it implies that the interpolation and differentiation matrices $\vec{N}$'s and $\vec{B}$'s that corresponds to the initial configuration should be stored and will be used throughout the calculation.
\end{remark}

\section{Temporal Discretization} \label{sec:timedis}
This section described a temporal discretization of the dynamic gradient damage model. The thematic subjects covered here are thus summarized in \cref{tab:summtd}.
\begin{table}[htbp]
\centering
\caption{Thematic subjects covered in this section} \label{tab:summtd}
\begin{tabular}{ccccc} \toprule
& Going dynamical & $\alpha\leftrightarrow\phi$ & $\nabla\alpha\to\Gamma$ & Experimental validation \\ \midrule
Theoretics & & & & \\
Numerics & \rightthumbsup & & & \\ \bottomrule
\end{tabular}
\end{table}

Consider an arbitrary discretization $(t^n)$ of the time interval $I=[0,T]$ where the superscript $n$ denotes a quantity evaluated at the $n$-th time step. We will mainly focus on the time stepping procedures bringing the current known states $(\uvec^n,\dot{\uvec}^n,\ddot{\uvec}^n,\dvec^n)$ to the next time step $(\uvec^{n+1},\dot{\uvec}^{n+1},\ddot{\uvec}^{n+1},\dvec^{n+1})$. In the time-continuous model the weak elastic-damage dynamic wave equation \eqref{eq:weakform} and the damage minimality condition \eqref{eq:crackmin} are coupled in the first-order stability principle \eqref{eq:vi}. After temporal discretization $\uvec$ and $\dvec$ evaluated at the last time step $t=t^n$ and the current time step $t=t^{n+1}$ are in general involved in an implicit fashion. However, we observe that the energy minimization principle \eqref{eq:crackmin} for damage is not a genuine time evolution problem since time dependence is only introduced via the irreversibility condition. After a temporal discretization \eqref{eq:crackdis} at time $t=t^{n+1}$ reads
\begin{equation} \label{eq:crackstdis}
q_{\,\uvec^{n+1}}(\dvec^{n+1})\leq q_{\,\uvec^{n+1}}(\underline{\vec{\beta}}) \text{ for all $\underline{\vec{\beta}}$ that $0\leq\dvec^n\leq\underline{\vec{\beta}}\leq 1$}
\end{equation}
where the Hessian matrix and the second member vector in \eqref{eq:Hb} are evaluated at $\uvec^{n+1}$. The equation \eqref{eq:crackstdis} can be interpreted as a numerical minimization problem of the quadratic functional $q$ under the irreversible constraint that the current sought damage state $\dvec^{n+1}$ is pointwise within the bound $[\dvec^n,1]$
\begin{equation} \label{eq:cracknummin}
\dvec^{n+1}=\operatorname{arg min}q_{\,\uvec^{n+1}}(\cdot)\text{ subject to the constraints $0\leq\dvec^n\leq\dvec^{n+1}\leq 1$}.
\end{equation}
The next damage state $\dvec^{n+1}$ can thus be accurately calculated as long as the next deformation state $\uvec^{n+1}$ is known.

It remains then to specify the temporal discretization method used for the $\vec{u}$-problem. In this work we adopt the classical second-order accurate Newmark-$\beta$ integrator, which assumes the following time-stepping procedure
\begin{align}
\dot{\uvec}^{n+1} &= \dot{\uvec}^n+\frac{\Delta t}{2}\left(\ddot{\uvec}^n+\ddot{\uvec}^{n+1}\right)\,, \label{eq:un2} \\
\uvec^{n+1} &= \uvec^n+\Delta t\dot{\uvec}^n+\frac{1-2\beta}{2}\Delta t^2\ddot{\uvec}^n+\beta\Delta t^2\ddot{\uvec}^{n+1}. \label{eq:un1}
\end{align}
The implicit method $0<\beta\leq\frac{1}{2}$ may be suitable for intermediate situations between a quasi-static and an explicit dynamic calculation. In the present work it will be used to numerically investigate the quasi-static limit of the dynamic model, see \cref{sec:antiplane}. The explicit method $\beta=0$ with a lumped mass matrix $\overline{\vec{M}}$ should be preferred mainly in terms of computational efficiency for applications where the loading speed or the crack propagation speed is comparable to the material sound speed, since in general the time scale involved is typically of order $\mathcal{O}(\SI{1}{ms})\ll\mathcal{O}(\SI{1}{s})$. The explicit method is conditionally stable. The constraint prescribed on the current time increment $\Delta t$ is often determined by the CFL condition $\Delta t<\Delta t_\mathrm{CFL}=\min(h/c)$ where $h$ is the mesh size, $c$ is the material sound speed and the smallest value is chosen among all elements. This is not a very inconvenient feature since in the presence of high geometrical and material nonlinearities even unconditionally stable implicit schemes need a small time increment comparable to $\Delta t_\mathrm{CFL}$. In the calculation of the material sound speed, the current damage state $\alpha_t$ as well as the tension-compression split formulation \eqref{eq:elasticTC} is taken into account. Thus a totally damaged element under tension does not penalize the total computational time.

The spatially discretized elastic-damage equation reads
\begin{equation} \label{eq:waveeqsdistdis}
\vec{M}\ddot{\uvec}^{n+1}=\vec{F}_\mathrm{ext}^{n+1}-\vec{F}_\mathrm{int}(\uvec^{n+1},\dvec^{n+1}).
\end{equation}
It can be seen that the determination of the the new acceleration $\ddot{\uvec}^{n+1}$ requires the knowledge of the new deformation state $\uvec^{n+1}$ which itself determines the new damage field at time $t=t^{n+1}$ via \eqref{eq:cracknummin}. For the implicit Newmark method $\beta\neq 0$, \eqref{eq:un1} can thus be regarded as a nonlinear equation in $\uvec^{n+1}$, where nonlinearity results from the irreversibility condition when minimizing the total energy \eqref{eq:cracknummin}. To decouple the $(\uvec^{n+1},\dvec^{n+1})$ problem, we use a staggered time-stepping procedure as used in \cite{Bourdin:2011,BordenVerhooselScottHughesLandis:2012,HofackerMiehe:2012} among others. The idea is to update the acceleration $\ddot{\uvec}^{n+1}$ while fixing the damage state at its \emph{previous} known value $\dvec^n$. Hence \eqref{eq:waveeqsdistdis} becomes
\begin{equation} \label{eq:waveeqsdistdis2}
\vec{M}\ddot{\uvec}^{n+1}\approx\vec{F}_\mathrm{ext}^{n+1}-\vec{F}_\mathrm{int}(\uvec^{n+1},\dvec^n).
\end{equation}
When a relatively small time-step is used, it is expected that the damage increment $\dvec^{n+1}-\dvec^n$ is bounded and the staggered time-discrete model will converge to the continuous one, cf. \cite{LarsenOrtnerSuli:2010}. Introducing the displacement prediction at time $t=t^{n+1}$
\[
\tilde{\uvec}^{n+1}=\uvec^n+\Delta t\dot{\uvec}^n+\frac{1-2\beta}{2}\Delta t^2\ddot{\uvec}^n\,,
\]
from \eqref{eq:un1} we obtain the equation for $\uvec^{n+1}$
\begin{equation} \label{eq:getuvecnp1}
\vec{M}\frac{\uvec^{n+1}-\tilde{\uvec}^{n+1}}{\Delta t^2}=\beta\bigl(\vec{F}_\mathrm{ext}^{n+1}-\vec{F}_\mathrm{int}(\uvec^{n+1},\dvec^n)\bigr).
\end{equation}
In general due to the use of tension-compression asymmetry models \eqref{eq:elasticTC}, it is a nonlinear equation which requires several Newton iterations and the evaluation of the current tangent stiffness. The time-stepping procedure for the dynamic gradient damage model based on the implicit Newmark-$\beta$ method in a prediction-correction form is summarized in \cref{algo:implicit}.
\begin{algorithm}[htbp]
\caption{Staggered time-stepping procedure for the dynamic gradient damage model based on the implicit Newmark-$\beta$ method} \label{algo:implicit}
\begin{algorithmic}[1]\linespread{1.2}\selectfont\normalsize
\For{every successive time step $n\geq 0$}
  \State Prediction $\tilde{\uvec}^{n+1}=\uvec^n+\Delta t\dot{\uvec}^n+\frac{1-2\beta}{2}\Delta t^2\ddot{\uvec}^n$.
  \State Prediction $\tilde{\underline{\vec{v}}}^{n+1}=\dot{\uvec}^n+\frac{\Delta t}{2}\ddot{\uvec}^n$.
  \State Solve $\uvec^{n+1}$ via \eqref{eq:getuvecnp1}.
  \State Update $\ddot{\uvec}^{n+1}=(\uvec^{n+1}-\tilde{\uvec}^{n+1})/(\beta\Delta t^2)$.
  \State Update $\dot{\uvec}^{n+1}=\tilde{\underline{\vec{v}}}^{n+1}+\frac{\Delta t}{2}\ddot{\uvec}^{n+1}$.
  \State Solve $\dvec^{n+1}$ via \eqref{eq:cracknummin}.
\EndFor
\end{algorithmic}
\end{algorithm}

\begin{remark}
After temporal discretization, the elastic-damage dynamic wave equation \eqref{eq:weakform} and the damage minimality condition \eqref{eq:crackmin} can also be solved in a monolithic fashion inside a nonlinear Newton solver described for example in \cite{BordenVerhooselScottHughesLandis:2012,SchlueterWillenbuecherKuhnMueller:2014}. The irreversible condition contained in \eqref{eq:cracknummin} should be included in the monolithic solver to ensure that the damage variable is subject to a bound constraint during solving. Future work could be devoted to a detailed analysis of these schemes in terms of computational efficiency.
\end{remark}

\begin{remark}
In the staggered time-stepping procedure, the approximation $\ddot{\uvec}^{n+1}\approx\ddot{\uvec}^{n}$ simplifies the coupled solving of the displacement and the damage. However, we admit that the $(\uvec^{n+1},\dvec^{n+1})$ couple obtained at the end of the current time step do not necessarily satisfy the weak elastic-damage dynamic wave equation \eqref{eq:weakform} and the damage minimality condition \eqref{eq:crackmin}. Following the alternate minimization procedure common for quasi-static models, see \cite{PhamAmorMarigoMaurini:2011}, several alternate iterations could also be performed here until convergence. However in dynamics, since the time-step is in general close to the critical time-step (\emph{i.e.} very small), convergence could be reached within the first iteration. In any case when $\Delta t\to 0$, the discrete staggered model converges to the continuous one, see the remark given at the end of this section.
\end{remark}

In the explicit case when $\beta=0$, it turns out that the time evolution system in $(\uvec,\dvec)$ is automatically decoupled and the two subproblems separately in $\uvec^{n+1}$ and in $\dvec^{n+1}$ can be solved in a sequence at every time step, see \cref{fig:updatestate}.
\begin{figure}[htbp]
\centering
\[
\xymatrix{ & \vec{H}(\uvec^{n+1}) \ar[dr] & & & & & \\
\uvec^{n+1} \ar[ur] \ar[r] & \vec{b}(\uvec^{n+1}) \ar[r] & \dvec^{n+1} \ar[r] & \vec{F}_\mathrm{int}(\uvec^{n+1},\dvec^{n+1}) \ar[r] & \ddot{\uvec}^{n+1} \ar[r] & \dot{\uvec}^{n+3/2} \ar[r] & \uvec^{n+2} \\
& \dvec^n \ar[ur] & & \vec{F}_\mathrm{ext}^{n+1} \ar[ur] & \dot{\uvec}^{n+1/2} \ar[ur] & \uvec^{n+1} \ar[ur] &}
\]
\caption{Updating flow of the explicit time-stepping procedure for the discretized dynamic gradient damage model at the time step $t^{n+1}$}
\label{fig:updatestate}
\end{figure}
Indeed according to \eqref{eq:un1}, the current displacement state $\uvec^{n+1}$ can be directly determined from the previous kinematic states, which itself determines then the damage state via \eqref{eq:cracknummin}. Introducing the middle-step velocity
\[
\dot{\uvec}^{n+1/2}=\dot{\uvec}^n+\frac{\Delta t}{2}\ddot{\uvec}^n\,,
\]
the Newmark time-stepping \eqref{eq:un2} and \eqref{eq:un1} reads
\begin{align*}
\uvec^{n+1} &= \uvec^n+\Delta t\dot{\uvec}^{n+1/2}\,, \\
\dot{\uvec}^{n+1} &= \dot{\uvec}^{n+1/2}+\frac{\Delta t}{2}\ddot{\uvec}^{n+1}\,, \\
\dot{\uvec}^{n+1/2} &= \dot{\uvec}^{n-1/2}+\Delta t\ddot{\uvec}^{n}.
\end{align*}
The explicit time-stepping procedure for the discretized dynamic gradient damage model is then summarized in \cref{algo:explicit}. It is implemented by the author in the explicit dynamics software EPX, see \cite{EPX:2015}.
\begin{algorithm}[htbp]
\caption{Explicit time-stepping procedure for the dynamic gradient damage model} \label{algo:explicit}
\begin{algorithmic}[1]\linespread{1.2}\selectfont\normalsize
\For{every successive time step $n\geq 0$}
  \State Update $\dot{\uvec}^{n+1/2}=\dot{\uvec}^{n}+\frac{\Delta t}{2}\ddot{\uvec}^n$.
  \State Update $\uvec^{n+1}=\uvec^n+\Delta t\dot{\uvec}^{n+1/2}$.
  \State Solve $\dvec^{n+1}$ via \eqref{eq:cracknummin}.
  \State Solve $\ddot{\uvec}^{n+1}$ via \eqref{eq:waveeqsdistdis} with the lumped mass matrix $\overline{\vec{M}}$.
  \State Update $\dot{\uvec}^{n+1}=\dot{\uvec}^{n+1/2}+\frac{\Delta t}{2}\ddot{\uvec}^{n+1}$.
\EndFor
\end{algorithmic}
\end{algorithm}

The initialization phase for the above implicit and explicit time-stepping procedure is described in \cref{algo:init}. We observe that the initial damage is recomputed $\dvec^{-1}\mapsto\dvec^0$ in the step 2. The role of $\dvec^{-1}$ is to bring some \emph{a priori} knowledge of the damage field resulting from a previous calculation or more frequently to represent an initial crack $\dvec^{-1}=1$ on $\Gamma_0$. This step thus renders it compatible with the initial displacement condition and the energy minimization structure. Remark that in phase field approaches for example in \cite{BordenVerhooselScottHughesLandis:2012}, the initial crack is introduced via an initial history field $\mathcal{H}_0$ which is localized along the desired initial defect. It is due to the fact that the irreversible constraint is dropped during energy minimization \eqref{eq:crackmin} in their models.
\begin{algorithm}[htbp]
\caption{Initialization for the implicit and explicit time-stepping procedure described in \cref{algo:implicit,algo:explicit}} \label{algo:init}
\begin{algorithmic}[1]\linespread{1.2}\selectfont\normalsize
\State Given initial conditions $\uvec^0$, $\dot{\uvec}^0$ and $\dvec^{-1}$.
\State Reinitialize the damage $\dvec^0=\operatorname{arg min}q_{\,\uvec^0}(\cdot)$ subjected to constraints $0\leq\dvec^{-1}\leq\dvec^0\leq 1$.
\State Initialize the acceleration $\vec{M}\ddot{\uvec}^0=\vec{F}_\mathrm{ext}^0-\vec{F}_\mathrm{int}(\uvec^0,\dvec^0)$.
\end{algorithmic}
\end{algorithm}

\begin{remark}[Temporal convergence]
Remark that only the irreversiblity condition $\dot{\alpha}_t\geq 0$ and the variational inequality \eqref{eq:vi} is effectively implemented. The former condition is reinforced during energy minimization \eqref{eq:cracknummin}. It is shown in \cite{LarsenOrtnerSuli:2010} that the time-discrete numerical model summarized by \cref{algo:implicit,algo:explicit} will converge to the continuous model, \emph{i.e.} \cref{def:dynagraddama}. In particular, it will thus also balance energy as required in \eqref{eq:dyngdeb}, when the time increment becomes small. Their constructive proof makes use of the implicit Euler scheme used in \cite{Bourdin:2011}, however our experience suggests the same for the implicit and explicit Newmark schemes.
\end{remark}

\begin{remark}[Approximation of the Hencky logarithmic strain]
An approximation of the Hencky strain based on the following remarkable property established in \cite{XiaoBruhnsMeyers:1997} can be used: within a proper corotational frame the rate of the Hencky strain equals the stretching tensor. An increment of the Hencky strain can thus be given by $\Delta\vec{h}^n\approx\symgrad(\Delta\vec{u}^n)$ where $\symgrad$ denotes the symmetrized gradient operator and the displacement increment $\Delta\vec{u}^n=\vec{u}^n-\vec{u}^{n-1}$ is written in the current deformed configuration. To have better accuracy, a second-order approximation based on the incremental Almansi strain is used
\[
\Delta\vec{h}^n\approx\frac{1}{2}\bigl(\nabla\Delta\vec{u}^n+\nabla^\mT\Delta\vec{u}^n-(\nabla^\mT\Delta\vec{u}^n)(\nabla\Delta\vec{u}^n)\bigr).
\]
Finally the current Hencky strain is incremented $\vec{h}^n=\vec{h}^{n-1}+\Delta\vec{h}^n$.
\end{remark}

\section{Implementation} \label{sec:implementation}
This section provides some details on the implementation of the discretized numerical model summarized by \cref{algo:implicit,algo:explicit}. The thematic subjects covered here are thus summarized in \cref{tab:summim}.
\begin{table}[htbp]
\centering
\caption{Thematic subjects covered in this section} \label{tab:summim}
\begin{tabular}{ccccc} \toprule
& Going dynamical & $\alpha\leftrightarrow\phi$ & $\nabla\alpha\to\Gamma$ & Experimental validation \\ \midrule
Theoretics & & & & \\
Numerics & \rightthumbsup & & & \\ \bottomrule
\end{tabular}
\end{table}

\subsection{Numerical solving of the damage problem}
In this work, the damage energy minimization problem \eqref{eq:cracknummin} will be directly solved at the structural scale. It could be converted into an equivalent local problem defined for each element. This possibility requires introduction of Lagrange multipliers at every Gauss points and an identification between the nodal damage field and the damage defined at Gauss points, see \cite[p.~49]{Beaurain:2011}. The principal motivation of this kind of Lagriangian modeling is the ability in the software Code-Aster to impose more naturally the irreversibility condition, however the computational cost is higher due to additional unknown variables.

Consequently, \eqref{eq:cracknummin} can be regarded as a large scale bound-constrained minimization problem defined at the structural level. Due to the damage constitutive requirement, the problem is convex and gradient-based numerical algorithms can be applied. These methods call for the evaluation of the objective functional \eqref{eq:Hb} as well as its first and second derivatives (Hessian matrix $\vec{H}$). An efficient storage and manipulation of these objects is essential especially in an explicit dynamics context.

Here the library PETSc is adopted due to its optimized efficiency, see \cite{PETSc:2015}. As a high-level C++/Fortran numerical linear algebra backend, PETSc provides in particular the \texttt{Vec} and the \texttt{Mat} classes for defining vectors and matrices. The actual storage format is not fixed in advance and it suffices to call the function \texttt{Vec/MatSetType} to affect a particular format. The computational arrays (the CSR format for example) can be extracted for output but in general all operations of these objects \texttt{Vec}/\texttt{Mat} are independent of the storage format used and thus transparent to the user. As an example, a unique interface is available to conduct a matrix-vector product
\begin{lstlisting}[language=Fortran]
CALL MATMULT(H,X,GRAD,PETSCERR)
\end{lstlisting}
where the vector \texttt{GRAD} stocks the result of $\vec{H}\dvec$. PETSc is built on the Message Passing Interface (MPI) standard and only one generic version exists which can be run either in sequential or parallel. The calling functions are almost the same in both cases, thus the code could be parallelized with minor interface changes.

Specifically the Gradient Projection Conjugate Gradient (GPCG) algorithm proposed in \cite{MoreToraldo:1991} is used to numerically solve \eqref{eq:cracknummin} in an iterative fashion. It is already implemented in the TAO package of PETSc and is designed for quadratic bound-constrained minimization problems. Due to the bound constraint, approximate solutions $\bvec$ to \eqref{eq:cracknummin} can be defined using the projected gradient $[\vec{g}]$ of which the $i$-th component is given by
\[
0\overset{?}{\approx}[\vec{g}]_i=\begin{cases}
\partial_i q & \text{if $\bvec_i\in(\dvec_i^n,1)$}\,, \\
\min(\partial_i q, 0) & \text{if $\bvec_i=\dvec_i^n$}\,, \\
\max(\partial_i q, 0) & \text{if $\bvec_i=1$}.
\end{cases}
\]
At each solving iteration, the method consists of several gradient projections
\[
\bvec\gets\mathbb{P}\bigl(\bvec-c\nabla q(\bvec)\bigr)
\]
to \emph{approximately} identify the \emph{active} nodes, \emph{i.e.} those either $\dvec_i^{n+1}=\dvec_i^n$ or $\dvec_i^{n+1}=1$. Note that here we perform a gradient descent followed by a projection $\mathbb{P}$ onto the bound to ensure that $\dvec^n\leq\bvec\leq 1$. Then it applies the preconditioned conjugate gradient method to minimize an unconstrained reduced problem of the \emph{free} variables, \emph{i.e.} those satisfying $\dvec_i^n<\dvec_i^{n+1}<1$. The method proceeds to the next iteration until convergence. Interested readers are referred to \cite{MoreToraldo:1991} for a more detailed explanation of the algorithm. The incomplete Cholesky factorization preconditioner is applied block-wise into each decomposed subdomain. Computational load is also well balanced in parallel computations based on domain decomposition, see \cite{BensonMcInnesMore:2001}.

Using the PETSc library, both the implicit and explicit time-stepping \cref{algo:implicit,algo:explicit} are implemented by the author as a Python package named ``FEniCS Dynamic Gradient Damage'', see \cite{LiMaurini:2015}. It is based on the FEniCS Project \cite{LoggMardalWells:2012} for automated solution of partial differential equations. A specific C++ interface \texttt{PETScTAOSolver} is developed by the author which brings various mathematical optimization algorithms defined in PETSc/TAO into FEniCS. In this implementation the tension-compression asymmetry is not taken into account. A linearized strain theory is also assumed.

The explicit time-stepping procedure for the dynamic gradient damage model summarized in \cref{algo:explicit} is also implemented in EPX by the author, an explicit dynamics program dedicated to transient phenomena involving fluid-structure interaction \cite{EPX:2015}. Here all the tension-compression asymmetry models discussed in \cref{sec:TC} are implemented. The large-displacement extension, see \cref{sec:extensions}, is also by default activated in the software.

\subsection{Details of implementation in EPX} \label{sec:epx}
\paragraph{Initialization}
The initialization \texttt{INIT\_ENDOGRAD} of the gradient damage model is performed at the end of the routine \texttt{initia.ff}. It has several objectives
\begin{itemize}
\item Allocate various Fortran arrays needed in the model: the damage field \texttt{ALPHA}, the damage dissipation energy for all elements \texttt{EFISS}\ldots

\item Initialize the PETSc solver and several objects: the minimization solver \texttt{TAOSOLV}, the global Hessian matrix for damage \texttt{HALPHA}, the PETSc viewer \texttt{VIEWER} to have access to convergence information\ldots
\end{itemize}

To guarantee a good efficiency when assembling sparse matrices, it is essential that we preallocate enough memory by indicating to PETSc the number of non-zero elements at every row of the matrix. In parallel, the non-zeros elements should be treated separately depending whether they appear in the diagonal or off the diagonal of the current processor. If this preallocation phase is omitted, PETSc will allocate the needed memory by calling \texttt{PetscMalloc} every time it encounters a new unexpected non-zero term, which causes the execution time to increase significantly, see \cite{PETSc:2015}. An overestimation of the sparse structure of the mesh connectivity is performed: for every node in the mesh, we consider all the elements containing that node and count the total node numbers possessed by these elements. Only one loop on all the elements is needed during the initialization. This overestimation does virtually no harm to the overall computational efficiency, as during the final assembly of matrices in PETSc, the routines \texttt{MatAssemblyBegin/End} will be called to release all unnecessary preallocated memories.

During a MPI parallel computation, a permutation of nodes is needed according to the PETSc convention described by the documentation of the routine \texttt{MatMPIAIJSetPreallocation}. It is achieved during initialization by a loop on all the processors based on the mesh partition performed by EPX. At the end two integer Fortran arrays are provided: \texttt{CONV\_EP(I)} which gives the position of the $i$-th EPX node in the PETSc convention, while \texttt{CONV\_PE} converts the PETSc node ordering to the EPX one.

\paragraph{Time-stepping} The change to the EPX time iteration routines \texttt{calcul.ff} and \texttt{d\_calcul.ff} is minimal. We simply add the following pseudo code concerning the assembly and the solving of the damage problem before the elastodynamics part.
\begin{lstlisting}[language=Fortran]
...
IF (L_ENDOGRAD) THEN
  CALL SEQ/MPI_ASSEM_ALPHA(...)
  CALL RESO_ALPHA(...)
END IF
...
CALL CELEM(...)
...
\end{lstlisting}

We note that the routine \texttt{SEQ/MPI\_ASSEM\_ALPHA} constructs the Hessian matrix $\vec{H}$ and the second member $\vec{b}$ associated to the quadratic damage minimization problem by performing a first loop on all the elements. The copy and the correspondence between the global damage vector and the local damage vector specific to each element are conducted without modifying the arguments of all existing routines: \texttt{celem.ff}, \texttt{loopelm.ff}, \texttt{wpgen.ff} and all element routines \texttt{triang.ff} for example. The elementary variables associated with the current element $\alpha^\mathrm{e}$, $\vec{H}^\mathrm{e}$ and $\vec{b}^\mathrm{e}$ are all stored globally inside the module \texttt{m\_endograd\_data.ff}. They are preallocated during the initialization phase depending on the maximum local nodes. This current implementation is not thread-safe, thus OpenMP parallelization is not available.

\paragraph{Element routines} At every time step, the following two loops on elements are needed:
\begin{enumerate}
\item Knowing the current configuration $\vec{x}^n$ and $\vec{u}^n$, the routine \texttt{SEQ/MPI\_ASSEM\_ALPHA} opens a first loop on elements to calculate the elementary matrices $\vec{H}^\me$ and vectors $\vec{b}^\me$ for the damage problem \eqref{eq:cracknummin}. During this first loop, the current damage field is not yet known.

\item After energy minimization \eqref{eq:cracknummin}, the current damage is solved in \texttt{RESO\_ALPHA}. We then open a loop for the second time in \texttt{CELEM} on all the elements to calculate the internal force vector $\vec{F}_\mathrm{int}^\me$ which depends on the current damage state.
\end{enumerate}

Since it is needed to perform two loops on elements at every time step, the element routines should be carefully restructured to minimize the computational cost. Some logical indicators are specially used and introduced to partition the code:
\begin{itemize}
\item \texttt{MATPRI(NTH) == 115} indicates if the current element is affected by a gradient damage material. If so, the following two logical indicators are then relevant.

\item \texttt{L\_FIRSTLOOP} encloses the lines that should be run during the calling of \texttt{SEQ/MPI\_ASSEM\_ALPHA}, \emph{i.e.} in order to calculate the elementary matrices $\vec{H}^\me$ and vectors $\vec{b}^\me$ for the damage problem. In particular, the strain tensor will be calculated at this stage via \texttt{DSDX2D} or \texttt{DSDX3D}. The tension-compression asymmetry is also performed here to calculate $\sig_0^+$ and $\sig_0^-$. The positive stress is stored temporarily in the \texttt{SIG} array while the negative stress in the \texttt{ECR} vector.

\item \texttt{L\_SECONDLOOP} encloses the lines that should be run during the calling of \texttt{CELEM} where the stress tensor and the internal force vector $\vec{F}_\mathrm{int}^\me$ will be calculated. In particular the stress tensor will be updated via \eqref{eq:stressposneg}
\end{itemize}

\paragraph{Improved method of calculating eigenvalues}
For the tension-compression asymmetry model described in \cite{MieheHofackerWelschinger:2010,FreddiRoyer-Carfagni:2010}, we need to perform the spectral decomposition of the strain tensor, see for example \cref{algo:epspos3d,algo:epspos2d}. The \texttt{DSYEV} routine of LAPACK is purely numerical (iterative method) and consequently robust but slow. Here we use and implement an analytical method described in \cite{Scherzinger:2008aa} that is both robust (in the presence of two or three identical or near eigenvalues) and computationally efficient. Consider the problem
\[
\vec{A}\approx\mathbb{I}=\begin{bmatrix}
1 & &  \\
& 1+\frac{\epsilon}{4} & \frac{3\sqrt{\epsilon}}{4} \\
& \frac{3\sqrt{\epsilon}}{4} & 1+\frac{3\epsilon}{4}
\end{bmatrix}
\]
with $\epsilon=\num{1e-7}$. Theoretical solution $(d_i,\vec{v}_i)$ is given by
\begin{equation} \label{eq:eigenvalueana}
\begin{aligned}
& d_1=1+\epsilon\,,\quad d_2=d_3=1, \\
& \vec{v}_1=(0, 1/2, \sqrt{3}/2)\,,\quad \vec{v}_2=(-1, 0, 0)\quad\text{and}\quad\vec{v}_3=(0, \sqrt{3}/2, -1/2).
\end{aligned}
\end{equation}

The analytical solution \eqref{eq:eigenvalueana} is compared with the numerical ones given by \texttt{DSYEV} and the method described in \cite{Scherzinger:2008aa}:
\begin{itemize}
\item \texttt{DSYEV}: 15 significant figures for eigenvalues and eigenvectors, within \SI{63.7}{s} for \num{1e8} calculations.
\item New method: 15 significant figures for eigenvalues and 10 for eigenvectors, within \SI{23.5}{s} for \num{1e8} calculations.
\end{itemize}
As can be seen, a factor of 2.7 in time is gained.

\section*{Summary of this Chapter}
In this chapter we describe a direct numerical implementation of the dynamic gradient damage model. It applies to both 1-d, 2-d and 3-d physical problems. Its efficiency will be demonstrated via simulation results in \cref{chap:simulation}. The spatial discretization is based on classical $C^0$ finite element method and is presented in \cref{sec:spatial}. It is decoupled from the time-stepping scheme described in \cref{sec:timedis} for the $\vec{u}$-problem governed by the discrete elastic-damage dynamic wave equation \eqref{eq:waveeqsdistdis}. The explicit Newmark method will be used whenever inertial effects are dominant. Implicit methods parametrized by the $\beta$-parameter are applied for intermediate cases between explicit dynamic and quasi-static situations. Due to the same mathematical structure for the damage problem (bound-constrained convex optimization problem), \eqref{eq:cracknummin} is solved at the structural scale similarly to the quasi-static settings. The discrete numerical model is summarized in \cref{algo:implicit,algo:explicit}.

The computational architecture is summarized in \cref{sec:implementation}. The numerical solving of the damage problem is based on the numerical linear algebra library PETSc. It is introduced to the explicit dynamics software EPX for storage and manipulation of sparse matrices and vectors. The Gradient Projection Conjugate Gradient method is used to numerically solve \eqref{eq:cracknummin}.


\chapter{{\bf Summary and Outlook}}
\label{sec:summaryandoutlook}

\section{Summary}
The LHCb experiment was built to test the predictions of the SM and search for NP effects through the study of $\mathcal{CP}$ violating and rare decays of $b$-hadrons. So far measurements performed using the LHCb experiment, and other LHC experiments, have not revealed conclusive evidence for NP effects, although some interesting anomalies have been seen in measured results in heavy flavour physics~\cite{PhysRevLett.118.111801,Aaij:2014pli, Aaij:2015oid, ATLAS-CONF-2017-023,CMS-PAS-BPH-15-008,Aaij:2015esa,PhysRevLett.113.151601,Aaij:2017vbb,Aaij:2015yra,Huschle:2015rga, Lees:2012xj,Lees:2013uzd, Sato:2016svk,Altmannshofer:2017yso,Capdevila:2017bsm,Amhis:2016xyh}.
%~\cite{PhysRevLett.118.111801,Aaij:2014pli,Aaij:2015yra,Lees:2013uzd,Huschle:2015rga,Lees:2012xj,Aaij:2015oid,Aaij:2015esa,PhysRevLett.113.151601, R_K_star}. %show no significant deviations from predictions and confirm the predictive power of the SM.
The search for \bmumu decays was identified as one of the key measurements to be made with the LHCb experiment~\cite{Adeva:2009ny} as an indirect search for NP.
In 2011 LHCb joined the search for these decays, that began over 30 years ago, using the unprecedented energies available at the LHC. The first evidence for \bsmumu decays was found by the LHCb experiment with 2.1 \fb of Run~1 data from $pp$ collisions~\cite{Aaij:2012nna}. A combined analysis of the Run~1 data from the CMS and LHCb experiments produced the first observation of \bsmumu decays and the first evidence for \bdmumu decays~\cite{CMS:2014xfa}. The measured branching fractions of these decays are consistent with the SM predictions and place constraints on BSM theories. However, the precision of the measurements still leaves room for NP effects to be revealed, therefore it is important to improve the precision of the \BF measurements. With the observation of the \bs mode the search for \bsmumu decays is complete and properties of this decay, including the effective lifetime, can now be studied. The effective lifetime offers a new observable to test the SM in \bsmumu decays that is complementary to the \BF.

The measurements of the \bmumu \BFs and the \bsmumu effective lifetime with 4.4~\fb of Run~1 and Run~2 data collected by the LHCb experiment are presented in this dissertation. Figure~\ref{fig:BDT} shows a summary of the fit used to measure the \bmumu \BFs for candidates in Run~1 and Run~2 data with a global BDT value of BDT > 0.5. The measured \BFs are
\begin{equation}
%\begin{align}
\begin{split}
  \mathcal{B}(B^{0}_{s} \to \mu^{+} \mu^{-}) &= (3.0 \pm 0.6^{+0.3}_{-0.2}) \times 10^{-9\
} \\
  \mathcal{B}(B^{0} \to \mu^{+} \mu^{-}) &= (1.5^{+1.2 +0.2}_{-1.0 -0.1})    \times 10^{-\
10}.
%\end{align}
\end{split}
\label{eq:BFresults2}
\end{equation}
The \bs mode is observed with a statistical significance of 7.8$\sigma$, making this result the first single experiment observation of this decay and the most precise measurement to date. The \bd mode has a significance of 1.6$\sigma$, therefore a limit is placed on the \BF of $\mathcal{B}$(\bdmumu)$ < 3.4 \times 10^{-10}$ at the 95$\%$ confidence level. The measured values are consistent with the SM predictions. %and Figure~\ref{fig:BDT} shows \bmumu candidates with a global BDT value of BDT > 0.5. %plot of Bd bv Bs?? with the others superimposed?
\begin{figure}[tbp]
    \centering
        \includegraphics[width=0.8\textwidth]{./Figs/Summary/BDT_plot.pdf}
    \caption{\bmumu candidates with a global BDT value of BDT > 0.5 in 4.4~\fb of Run~1 and Run~2 data collected by the LHCb experiment and the mass fit used to measure the \bmumu \BFs overlaid.}
    \label{fig:BDT}
\end{figure}

The effective lifetime of \bsmumu decays is measured for the first time to be
\begin{equation}
\tau_{\mu\mu} = 2.04 \pm 0.44 \pm 0.05 \text{ ps},
\end{equation}
which is within 1.0$\sigma$ of the SM prediction. The result is consistent with \ADG = +1 hypothesis at 1.0$\sigma$ and with \ADG = $-1$ hypothesis at 1.4$\sigma$. Although the current precision of the measurement does not enable constraints to be placed on BSM theories it is important to illustrate the ability of the LHCb experiment to make this measurement.

The measured values of the \BF presented in this dissertation have already been used to help constrain parameters in BSM theories~\cite{Altmannshofer:2017wqy,Fleischer:2017ltw,Bobeth:2017xry,Chiang:2017etj}.
The possible values available for the parameters $P$ and $S$, defined in Section~\ref{sec:BFdef}, are shown in Figure~\ref{fig:NPcontss}, where the values are constrained from the \BF results in Equation~\ref{eq:BFresults2} and the results from the CMS collaboration in reference~\cite{Chatrchyan:2013bka}. The current measurements produce a circular band of possible $P$ and $S$ values, and a precise measurement of \ADG through the \bsmumu \el will help resolve the ambiguities.

\begin{figure}[tbp]
    \centering
        \includegraphics[width=0.7\textwidth]{./Figs/Summary/fig3.pdf}
    \caption{Constraints in the $P$ - $S$ plane where $P$ and $S$ are defined in Section~\ref{sec:BFdef}. The blue band corresponds to constraints from the \bmumu \BF measurements presented in this dissertation and the CMS experiment Run~1 results in reference~\cite{Chatrchyan:2013bka}. The dashed lines show different values for \ADG and the constraints assume $\varphi_{P}, \varphi_{S}\in {0, \pi}$. The figure is taken from~\cite{Fleischer:2017ltw}.}
    \label{fig:NPcontss}
\end{figure}

\section{Outlook}
The measured values of the branching fractions and the effective lifetime still leave plenty of room for NP effects to be observed with these decays. At the end of Run~2 of the LHC, the LHCb dataset will have almost doubled to 8~\fb, enabling the precision of these measurements to be improved. Looking further ahead, LHCb is expected to collect 50~\fb of data by the end of Run~4 and with the high luminosity LHC up to 300~\fb could be recorded.

With more data, the expected precision of the \BF measurements is expected to be reduced to $\sim 0.19 \times 10^{-9}$ with 50~\fb~\cite{LHCb-PUB-2014-040}. Not only are the \BF measurements in themselves interesting to test the SM but the ratio of the \BFs of the two modes is also useful to test the SM, in particular the MFV hypothesis. The current precision of the ratio of \BFs is $\sim 60\%$~\cite{CMS:2014xfa}, and future runs of the LHC will enable the precision of the ratio of \BFs to be reduced to 40$\%$ with 50~\fb of $pp$ data and 20$\%$ with 300~\fb of data~\cite{Aaij:2244311}.

The expected uncertainty achievable by the LHCb experiment for the \el at the end of Run~2 and after future runs of the LHC is estimated using pseudoexperiments based on the observed numbers of decays with 4.4~\fb and the current measurement strategy. At the end of Run~2, with 8~\fb, the median uncertainty of the \el will be $\sim$0.2 \ps which is reduced to $\sim$0.08~\ps with 50~\fb and $\sim$0.03~\ps with 300~\fb. Therefore, with 300~\fb the precision on the effective lifetime will enable \ADG = +1 to be distinguished from \ADG = $-1$ with a statistical significance of 5$\sigma$. The expected mass and decay time distributions for 8, 50 and 300~\fb are shown in Figure~\ref{fig:expected_dist}. The expected uncertainties for the \el measurement are conservative estimates because they are based on the current measurement strategy which was designed for low expected statistics. Therefore the precision of the measurement could be much better as different analysis methods can be taken advantage of with more statistics.

\begin{figure}[tbp]
    \centering
        \includegraphics[width=0.49\textwidth]{./Figs/Summary/8fb_mass.pdf}
        \includegraphics[width=0.49\textwidth]{./Figs/Summary/8fb_time.pdf}
        \includegraphics[width=0.49\textwidth]{./Figs/Summary/50fb_mass.pdf}
        \includegraphics[width=0.49\textwidth]{./Figs/Summary/50fb_time.pdf}
        \includegraphics[width=0.49\textwidth]{./Figs/Summary/300fb_mass.pdf}
        \includegraphics[width=0.49\textwidth]{./Figs/Summary/300fb_time.pdf}
    \caption{The expected mass and decay time distributions for \bsmumu candidates to measure the \el with 8, 50 and 300~\fb. The signal and background yields are determined from the observed number of \bsmumu and combinatorial background decays and the expected number of mis-identified backgrounds and \bdmumu decays in 4.4~\fb of Run~1 and Run~2 data.}
    \label{fig:expected_dist}
\end{figure}

The current systematic uncertainty on the \el is 0.05 \ps which is too large to allow possible NP effects to be observed. However, several components contributing to the total, such as the fit accuracy and the acceptance function systematic, will be reduced with the availability of more data enabling greater precision on the measurement. With more data, an alternative analysis approach could be used which would reduce the systematic uncertainties on the \el; the selection criteria can be designed so that it does not bias \bsmumu decay time distribution, therefore removing the need for an acceptance function and the associated systematic.
%Do I want to include more discussion on this?? Or prehaps add as a appendix? I don't think a long discussion will add much here but prehaps the details could be useful or someone will ask about it ....

The study of \bmumu decays has been in progress for over 30 years and with the energy and luminosity available at the LHC, the study of these decays is just as interesting as it ever was. As more data is collected by the LHCb experiment NP effects will have less and less space to hide. It will either be seen in \bmumu decays or these decays will place ever tighter constraints on BSM theories.


\section{Conclusion and Perspective}\label{conclusion}

%In this paper we extended model typing by taking into account contract-aware substitutability where contracts are defined in terms of invariants and pre-/postconditions expressed using OCL.
We propose in this paper a model typing theory where model types include contracts.
This includes a formally defined subtyping relation between model types, and a tool-supported approach supporting a safe contract-aware substitutability of models conforming to metamodels including contracts. This ensures a safe reuse of model transformations expressed on metamodels including contracts.

Contracts are defined in terms of invariants and pre-/post-conditions expressed using OCL on MOF-based metamodels.
The invariants are added on the classes of a metamodel to specify additional structural properties of the metamodel, and pre-/post-conditions are added on the operations of classes to specify model transformations.
%
Consequently, the support of invariants in the subtyping relation ensures a safe reuse of model transformations where OCL contracts are needed to precisely specify the structure on which the model transformation can be applied. The support of pre-/post-conditions paves the way for behavioral substitutability to safely reuse model transformations where OCL contracts are needed to precisely specify the applicability of the model transformation.

The subtyping relation is based on a matching relation between two MOF classes that include OCL contracts, and is checked thanks to a technique based on Alloy. The actual scope of the provided contract-aware substitutability is mainly determined by the OCL-to-Alloy translation.

%
%We also provide a research prototype to rigorously reason about the subtyping relation between two model types with contracts.
%The current version of prototype takes as input Ecore model types and OCL contracts.
We are currently extending the prototype by providing support for model types and contracts expressed using the Kermeta language workbench.  We also explore how we can extend the approach by using SMT solvers at back-end to analyze the OCL contracts that include more complex arithmetic calculation.


\chapter{Assignment: Data Inspection}
\label{hw:data-inspection}

\newthought{Understanding the data is crucial} for any data science task. And the best way to achieve this is with visualizations. With the \widget{Datasets} widget load \textit{Employee Attrition} data set, which describes employees of a company with 32 features, such as age, job role, years at company, and so on. We also know, whether an employee resigned from the company (Attrition = Yes) or not (Attrition = No).

\begin{figure}[h]
  \centering
  \includegraphics[scale=0.6]{workflow.png}%
  \caption{Workflow for the assignment.}
  \label{fig:data-inspection-workflow}
\end{figure}

Inspect the data to understand it better:
\begin{enumerate}
    \item Use \widget{Scatter Plot} and try \textit{Find Informative Projections}. How does the top combination look like? Is it useful? Why (not)?
    \item Use \widget{Box Plot} and try \textit{Order by relevance to subgroups}. Which attribute best splits the data by \textit{Attrition}? Explain it.
    \item Which widget would you use to observe relationship between two discrete variables in relation to Attrition? Use \textit{Find informative projections} in that widget and explore the top result.
\end{enumerate}