\section{Some theoretical background: }

GIFMOD is equipped with two inverse modeling modules including a maximum-likelihood (deterministic) parameter estimation and a Bayesian probabilistic parameter estimation. The deterministic parameter estimation approach uses a Hybrid Genetic Algorithm to find combination of model parameters maximizing the likelihood of observing the measured data which the Bayesian approach uses Markov Chain Monte Carlo (MCMC) algorithm to produce samples of parameters representing the joint posterior distribution of model parameters designated to be estimated by the program.

GIFMOD allows for any of the parameters in the model to be treated as unknown parameters and be estimated by providing their prior distributions while allowing the user to choose between three types of prior distributions: including normal, log-normal, and uniform.
The observed data can be specified to represent any of the state variables in the model including hydraulic (i.e.  flow rates, head, cross- sectional areas), as well as particle and constituent concentrations at each block. As is typically done in Bayesian inference, the error here is defined as a quantity encompassing measurement error, model structural error, and all other errors resulting from the uncertainties associated with the external forcing (input data) such as weather data, inflow rates, and concentrations, etc. The general form of the model error structure can be expressed using the following equation:

\begin{equation}
\label{eq:im1}
\tilde Y(t) = \xi^{-1}[\xi[Y(t)]+\epsilon]
\end{equation}
where $Y$ is the vector containing the model’s observed constituents, $\tilde Y(t)$ represents the observed data vector, $\xi$ represents the error structure function (e.g., for log-normal error structure, $\xi$ is the natural logarithm function, while for Gaussian error structure, $\xi$ is the identity function), and $\epsilon$ is a random vector containing transformed measurement, structural and external forcing errors, which is assumed to collectively follow a multivariate normal distribution.
The Bayes’ theorem \citep{kaipio2006} can be used to obtain the joint probability distribution of parameters given the observed data, namely, the posterior distribution of the parameters as:
\begin{equation}
\label{eq:im2}
p(\vec{\Psi}|\vec{\tilde Y})=\frac{p(\vec{\tilde Y}|\vec{\Psi}) p(\vec{\Psi})}{p(\vec{\tilde Y})}
\end{equation}

where $p(\vec{\tilde Y}|\Psi)$ is the likelihood function that is the probability of observing the measured data $\vec{\tilde Y}$ given a parameter set $\vec{\Psi}$, $p(\vec{\Psi})$ represents the prior knowledge about the parameters and error structure, and $p(\vec{\tilde Y})$ is a normalizing factor. $\vec{\Psi}$ in Eq. (\ref{eq:im2}) contains all the parameters that are intended to be estimated using the observed data and the elements of the variance-covariance matrix for the random error term $\epsilon$ in eq. (\ref{eq:im1}). The likelihood of observing $\vec{\tilde Y}(t)$ given a parameter set $\vec{\Psi}$ or simply the likelihood function $p(\vec{\tilde Y}|\Psi) = p(\vec{\tilde Y}|\vec{Y},\vec{\Gamma})$ can be theoretically calculated based on the error structure. If the errors due to the uncertainties associated with the external forcing are considered part of the observation error, then the external forcing can be considered deterministically, and thus the model outputs become a function of only model parameters. In this case, the error function can be re-written as $p(\vec{\tilde Y}|\vec{\Phi},\vec{\Gamma})$. In theory, each element of the variance-covariance matrix $\vec{\Gamma}$ should be estimated as part of the parameter estimation. However, this makes the total number of parameters to be estimated very large and imposes a large computational burden. Therefore, observation errors of different constituents are typically assumed to be independent, and the correlations between observed concentration errors of different constituents are ignored \citep{walsh2012}. By making this assumption, the variance-covariance matrix becomes diagonal. If it is also assumed that errors of consecutive observations of individual data-sets used in parameter estimation are independent of each other, then the likelihood function of the observed vector can be computed as a function of the model parameters:
\begin{equation}
\label{eq:im3}
p(\vec{\tilde Y}|\vec{\Psi})=\frac{\prod_{i=1}^{NT} \prod_{j=1}^{NS_i} \xi'(\tilde y_{ij})}{(2 \pi \prod_{i=1}^{NT} \sigma_i^2)^{NS_i/2}}exp\bigg[-\sum_{i=1}^{NT} \sum_{j=1}^{NS_i} \frac{(y_{ij}-\tilde y_{ij})^2}{2 \sigma_i^2}\bigg]
\end{equation}

where $\tilde y_{ij}\in \vec{\tilde Y}$ is the $j^{th}$ data point of observed concentration of observed state variable $i$, $NT$ is the total number of different measured state variables, $j$ indicates the time and location of the measurement. $NS_i$ is the number of total samples (in time or location) of observed state variable $i$. The mapping function $\xi$ in the likelihood function can be assumed to be any transformation depending on the distribution of the observed data or by performing trial and error to find the best transformation by comparing the posterior residuals with the presumed error structure, and $\xi'$ is its derivative. For example, considering $\xi$ to be the identity function implies a normally distributed and additive error structure, while assuming $\xi$ to be the logarithm function results in a log-normally distributed and multiplicative error structure. Other error structures can be obtained by selecting a corresponding mapping function. $y_{ij}\in \vec{Y}$  is the predicted concentration of constituent $i$ at time/location $j$. $\sigma_i$ is the standard deviation of observed constituent $i$. When the deterministic parameter estimation approach based on maximum likelihood approach is to be applied to estimate the parameters, the values of parameters that maximize Eq. \ref{eq:im3} \citep{montgomery2010} are found  using a hybrid genetic algorithm.
The second term in the numerator of Eq. \ref{eq:im2} represents the prior knowledge about the probability density of the parameters. This information can be obtained from literature reviews, experts’ experience, or independently performed experimental results.
In the probabilistic parameter estimation module, the MCMC algorithm \citep{gamerman2006} is used to generate random samples according to the posterior distribution in Eq. \ref{eq:im2}. The MCMC algorithm implemented in the program is based on the Metropolis-Hasting method \citep{metropolis1953}. MCMC algorithm that is implemented into GIFMOD automatically adjusts the perturbation factors to achieve an acceptance rate provided by the user. In the next section the way to set up an inverse problem in GIFMOD and the parameters adjusted by the user in order to do so are described.
\section{Defining parameters and observed data}
In order to perform an inverse modeling, parameters and observed data should first be introduced. Parameters should be assigned to the specific properties of the model and each set of observed data should be attributed to a specific output of the model.
\subsection{Defining parameters and adjusting their properties: }
In order to add a parameter to the model right click on \textbf{Inverse Modeling}$\rightarrow$\textbf{Parameters} and then click on \textbf{Add parameters}. Each parameter has the following properties:

\begin{itemize}
    \item \textbf{Name: } This indicates the name that is assigned to a parameter. The name of the parameter will be used to assign the parameter to specific properties of the model and when generating the results of inverse modeling.
    \item \textbf{Value: } This is the value that will be assigned to the parameter if the modeling is conducted in forward mode.
    \item \textbf{Maximum Value: } In deterministic inverse modeling, this indicates the upper bound for the parameter (search domain). In probabilistic inverse modeling, it indicates the 97.5 percentile of the prior distribution for this parameter.
    \item \textbf{Minimum Value: } In deterministic inverse modeling, this indicates the lower bound for the parameter (search domain). In probabilistic inverse modeling, it indicates the 2.5 percentile of the prior distribution for this parameter.
    \item \textbf{Prior distribution: } Indicates the mathematical form of the prior distribution of the parameters. The choices provided include \textbf{Normal}, and \textbf{Lognormal}. The mean and standard deviation of the prior distributions are calculated based on the maximum and minimum values.
\end{itemize}

\subsection{Defining observed data and assigning their properties: }
Observed data can be added by right-clicking on \textbf{Inverse Modeling}$\rightarrow$\textbf{Observations} and then clicking on \textbf{Add observation}. Each observation includes a time-series representing an state variable of the model. Each observation has the following properties:

\begin{itemize}
    \item \textbf{Name: } The name of an observation is used when generating results in deterministic and probabilistic inverse modeling.
    \item \textbf{Standard Deviation ID: } This is an identification that is used to group observations that will be assigned the same standard deviation. The total number of groups of observations is equivalent to the parameter $NT$ in Eq. \ref{eq:im3}. The observations having the same \textbf{Standard Deviation ID} will be grouped together and a single observation error standard deviation, $\sigma_i$ will be assigned to them.
    \item \textbf{Block/connector: } This indicates whether the state variable to the observation correspond to is a block or a connector state variable. Concentrations, storages, hydraulic heads are always block state variables, while flow rates and interface areas are connector state variables.
    \item \textbf{Error distribution: } This indicates the error structure ($\xi$) in Eq. \ref{eq:im1} that is used when computing the likelihood value.
    \item \textbf{Location: } Indicate the block or the connector that this observed data corresponds to.
    \item \textbf{Observed data: } This field allows loading the observed data time-series. The observed data time series is a .csv text file that should consist of two columns first representing times of measurement and the second the observed values.
    \item \textbf{Quantity: } This field allows selecting the state variable that the observed data corresponds to. As for water quality and particles the drop-down menu containing the selectable state variables is populated depending on the constituents, phases, and particle types available.

\end{itemize}
\subsection{Example: Estimation of bio-kinetic parameters of nitrification processes from an batch test: }
This case represent interpretation of an actual experiment to determine the rates of microbial activities during ammonia and nitrite oxidation. Four experiments were conducted using the same sludge sample. The variation of dissolved oxygen with respect to time was measured for each of the four experiments. The initial conditions for each of the experiments is shown in Table \ref{table:inverse_example} and the observed variation of DO for the four cases are shown in Figure \ref{fig:inverse_example1}.

\begin{table}[]
    \centering
    \begin{tabular}{c|c c c c}
       Experiment  & $NO_3^{-1}(mg/L) $ & $NH_3(mg/L)$ & $DO(mg/L)$ & $VSS(mg/L)$ \\
    \hline
        No Spike & 0 & 0 & UK* &  730  \\
        $NO_2^{-1}$ Spike & 50 & 0 & UK & 695 \\
        $NH_3$ Spike & 0 & 50 & UK & 645 \\
        $NO_2^{-1}$ and $NH_3$ & 50 & 50 & UK & 660\\
    \end{tabular}
    \caption{Initial condition for the declining DO test aimed at estimating AOB and NOB biokinetics. *- UK: The initial condition for DO is treated as a parameter to be estimated by the model}
    \label{table:inverse_example}
\end{table}

The processes considered in the model are shown in table \ref{table:inverse_example2}. The only constituents that will be explicitly considered in the model include $DO$, $NH_3$, $NO_2^{-1}$, and $VSS$. $VSS$ is used as a surrogate for biomass (i.e. it is assumed to be proportional to AOB, NOB and OHO) and it is also assumed to stay unchanged throughout each experiment. the Petersen matrix is shown in table \ref{table:inverse_example3}.

\begin{figure}[!ht]
\begin{center}
\includegraphics[width=6cm]{Images/Figure35.png} \\
\caption{Temporal variation of DO for the four different experiments used to estimate nitrification bio-kinetics parameters}\label{fig:inverse_example1}
\end{center}
\end{figure}

\begin{table}[]
    \centering
    \begin{tabular}{l l}
       Process & Reaction Expression \\
    \hline
        Ammonia oxidation (AO) & $21.9O_2 + \frac{21.9}{3.43}NH_3 \rightarrow \frac{21.9}{3.43}NO_2^{-1}$  \\
        Nitrite oxidation (NO) & $11.7O_2 + \frac{11.7}{1.14}NO_2^{-1} \rightarrow \frac{11.7}{1.14}NO_3^{-1}$ \\
        Ordinary heterotrophic growth (OHO) & $BOD + O_2 \rightarrow CO_2$ \\
    \end{tabular}
    \caption{Reactions considered in the nitrification bio-kinetics parameter estimation example}
    \label{table:inverse_example2}
\end{table}

\begin{table}[]
    \centering
    \begin{tabular}{l|c |c c c c}
       Process & Rate Expression & $O_2$ & $NH_3$ & $NO_2^{-1}$ & VSS  \\
    \hline
        AO & $VSS \mu_{AOB} \frac{[O_2]}{(k_{OA} + [O_2]}\frac{[NH_3]}{[NH_3]+k_{NH3}}$ & -21.9 & $-\frac{21.9}{3.43}$ & $\frac{21.9}{3.43}$ &   \\
        NO & $VSS \mu_{NOB} \frac{[O_2]}{k_{ON} + [O_2]}\frac{[NO_2^{-1}]}{[NO_2^{-1}]+k_{NO2}}$ & $-11.7$ &  & $-\frac{11.7}{1.14}$ &   \\
        OHO & $VSS \mu_{OHO} \frac{[O_2]}{k_{OH}+[O_2]}$ & $-\frac{1-0.52}{0.52}$ & & &  \\
    \end{tabular}
    \caption{Petersen matrix for the nitrification bio-kinetics parameter estimation example}
    \label{table:inverse_example3}
\end{table}

\begin{table}[]
    \centering
    \begin{tabular}{l|c c c}
       Parameter & 2.5\% & 97.5\% & Distribution  \\
    \hline
        $\mu_{AOB}$ & 0.005 & 0.5 & lognormal \\
        $k_{OA}$ & 0.05 & 2.0 & lognormal \\
        $\mu_{NOB}$ & 0.005 & 0.5 & lognormal \\
        $k_{ON}$ & 0.05 & 2.0 & lognormal \\
        $k_{NH3}$ & & FIXED = 0.5 & \\
        $k_{NO2}$ & & FIXED = 0.5 & \\
        $\mu_{OHO}$ & 0.0001 & 0.1 & lognormal \\
        $k_{OH}$ & 0.00125 & 10 & lognormal \\
    \end{tabular}
    \caption{Prior range of parameters used in nitrification bio-kinetics example}
    \label{table:inverse_example4}
\end{table}

\begin{figure}[!ht]
\begin{center}
\includegraphics[width=12cm]{Images/Figure36.png} \\
\caption{Reaction network for the nitrification parameter estimation example}\label{fig:inverse_example2}
\end{center}
\end{figure}

Table \ref{table:inverse_example4} shows the prior 95\% ranges and the prior distributions of each of the parameters. In this particular case due to lack of information about $NH_3$ and $NO_2^{-1}$ concentration variation with time if $k_{NH3}$, and $k_{NO2}$ are considered unknown, there is a risk of problem becoming over-parametrized and therefore, we here assume the values of these two parameters to be fixed.

\subsubsection{Steps to set-up the model: }
To construct the model in GIFMod follow the following steps:
\begin{itemize}
    \item \textbf{Add a pond: } We will simulate the reactor using a \textbf{Pond} block. Other media types can also be used. Add a pond by clicking on the pond icon \includegraphics[width=0.5cm]{Icons/pond_icon.png} on the top tool bar. Set the following properties for the pond: \\
    - \textbf{Area: } 1$m^2$\\
    - \textbf{Initial water depth: } 1m\\
    \item \textbf{Add the constituents: } Add constituents ($DO$, $NH_3$, $NO_2^{-1}$, $VSS$) by right-clicking on \textbf{Water quality}$\rightarrow$\textbf{Constituents}. \item \textbf{Adding experiments: } The goal of this example is to infer the values of reaction parameters using four experiments in a holistic way. This means that the best parameter set that can collectively explain the results from the four experiments in sought for. There are two ways to consider four experiments in the model. The first way is to define four independent ponds with different initial conditions and the second way is to use \textbf{Experiments}. Here we will use the second approach. \\ To add experiments click on the \textbf{Add new experiment} button \includegraphics[width=0.3cm]{Icons/newexperiment_icon.png} three times to add three new experiments.

    \item \textbf{Setting the duration of the simulation: } The duration of the experiments are all below 0.2 days. First from the experiment menu on the top tool bar select \textbf{All experiments}. This forces the program to apply any changes in the properties to all the experiments. To set the simulation duration to 0.2 from \textbf{Settings}$\rightarrow$\textbf{Project settings} and from the \textbf{Properties} window right-click on the label for \textbf{Simulation end time} and click on \textbf{Enter number} from the drop-down menu that appears. Enter 0.2 in the dialog box that appears.

    \item \textbf{Adding reaction parameters: } Add the eight reaction parameters by right-clicking on \textbf{Water quality}$\rightarrow$\textbf{Reactions}$\rightarrow$\textbf{Reaction parameters} according to table \ref{table:inverse_example4}. For the two fixed parameters $k_{NH3}$ and $k_{NO2}$, enter a value of 0.5.
    \textbf{Add reaction network: } Right click on reaction  \textbf{Water quality}$\rightarrow$\textbf{Reactions}$\rightarrow$\textbf{Reaction network}. Enter the processes according to the Petersen matrix shown in table \ref{table:inverse_example3}. The finished reaction network window should look like figure \ref{fig:inverse_example2}.\\
    \item \textbf{Setting initial conditions: } From the \textbf{Experiments} drop-down menu on the top tool bar, select \textbf{Experiment1}. Click on the pond block and then from the \textbf{Properties} window choose \textbf{Constituent initial conditions}. Enter the initial conditions for experiment 1 according to Table \ref{tab:inv_ini_cond}.
   \begin{table}[]
    \centering
    \begin{tabular}{l|c c c c}
       Experiment & $DO(mg/L)$ & $NH_3(mg/L)$ & $NO_2^{-1}(mg/L)$ & VSS (mg/L) \\
    \hline
        1 & 7.78 & 0 & 0 & 730 \\
        2 & 7.76 & 50 & 0 & 645 \\
        3 & 8.02 & 0 & 50 & 695 \\
        4 & 7.8 & 50 & 50 & 660 \\
    \end{tabular}
    \caption{Initial conditions for the four experiments used for estimation of bio-kinetics and stoichiometric parameters of nitrification}
    \label{tab:inv_ini_cond}
\end{table}

    The initial condition window for the first experiment should look like Figure \ref{fig:42}.

\begin{figure}[!ht]
\begin{center}
\includegraphics[width=6cm]{Images/Figure42.png} \\
\caption{Initial conditions for the nitrification inverse modeling example}\label{fig:42}
\end{center}
\end{figure}

Change the experiment to experiment2 from the \textbf{experiments} drop down menu and similarly set the initial condition according to the second row of table \ref{tab:inv_ini_cond}.

Similarly assign the initial conditions for experiment 3 and 4.

    \item \textbf{Setting up parameters to be estimated: } Add a parameter representing $\mu_{AOB}$ by right-clicking of \textbf{Inverse modeling}$\rightarrow$\textbf{Parameters} and clicking on \textbf{Add parameters} from the drop-down menu. Assign the following parameter to the newly added parameter:\\
    - \textbf{Name:} $\mu_{AOB}$\\
    - \textbf{Maximum value: } \textit{0.5}\\
    - \textbf{Minimum value: } \textit{0.005}\\
    - \textbf{Distribution: } \textit{Log-Normal}\\
    Repeat for the other parameters to be estimated including $k_{OA}$, $\mu_{NOB}$, $k_{ON}$, $\mu_{OHO}$,
and $k_{OH}$ and assign the ranges and the distribution according to table \ref{table:inverse_example4}. As for the \textbf{Value} of the parameters respectively use $\mu_{AOB}=0.05$, $k_{OA}=0.31$, $\mu_{NOB}=0.05$, $k_{ON}=0.31$, $\mu_{OHO}=0.003162$, $k_{OH}=0.111$. These values are used when the model is run in forward mode. \\Please note that the name of the parameters should not be identical to their corresponding reaction parameters. \\

\item \textbf{Assigning the parameters to the corresponding model properties: } The parameters defined in the previous step should not be assigned to their corresponding properties in the model. Choose the reaction parameter $\mu_{AOB}$ from \textbf{Water quality}$\rightarrow$\textbf{Reactions}$\rightarrow$\textbf{Reaction parameters}, and the right click on the label of \textbf{Value} property and from the drop-down menu select \textbf{Parameters}$\rightarrow$\textbf{$\mu_{AOB}$} figure \ref{fig:inverse_example3}.

\item \textbf{Setting observations: } Here we specify the properties of the observed data used to perform the parameter estimation. \\ Right-click on \textbf{Project explorer}$\rightarrow$\textbf{Inverse modeling}$\rightarrow$\textbf{Observations} and click on \textbf{Add Observation}.\\
Set the following properties for the first observation: \\
\underline{observation 1:} \\
- \textbf{Name: } \textit{DO\_no\_spike}\\
- \textbf{Standard deviation ID: } \textit{std} \\
- \textbf{Block/Connector: } \textit{Block} \\
- \textbf{Error Distribution: } \textit{Normal} \\
- \textbf{Location: } \textit{Pond (1)} \\
- \textbf{Experiment: } \textit{experiment1} \\
- \textbf{Observed data: }  \textit{Obs\_nospike.txt} \\
\textbf{Quantity} \textit{DO:Aqueous} \\

Add three more observation and set the properties as follows:

\underline{observation 2:} \\
- \textbf{Name: } \textit{DO\_NH3\_spike}\\
- \textbf{Standard deviation ID: } \textit{std} \\
- \textbf{Block/Connector: } \textit{Block} \\
- \textbf{Error Distribution: } \textit{Normal} \\
- \textbf{Location: } \textit{Pond (1)} \\
- \textbf{Experiment: } \textit{experiment2} \\
- \textbf{Observed data: }  \textit{Obs\_NH3.txt} \\
\textbf{Quantity} \textit{DO:Aqueous} \\

\underline{observation 3:} \\
- \textbf{Name: } \textit{DO\_NO2\_spike}\\
- \textbf{Standard deviation ID: } \textit{std} \\
- \textbf{Block/Connector: } \textit{Block} \\
- \textbf{Error Distribution: } \textit{Normal} \\
- \textbf{Location: } \textit{Pond (1)} \\
- \textbf{Experiment: } \textit{experiment3} \\
- \textbf{Observed data: }  \textit{Obs\_NO2.txt} \\
\textbf{Quantity} \textit{DO:Aqueous} \\

\underline{observation 4:} \\
- \textbf{Name: } \textit{DO\_both}\\
- \textbf{Standard deviation ID: } \textit{std} \\
- \textbf{Block/Connector: } \textit{Block} \\
- \textbf{Error Distribution: } \textit{Normal} \\
- \textbf{Location: } \textit{Pond (1)} \\
- \textbf{Experiment: } \textit{experiment3} \\
- \textbf{Observed data: }  \textit{Obs\_both.txt} \\
\textbf{Quantity} \textit{DO:Aqueous} \\

\textbf{Note: } Entering the same \textbf{Standard deviation ID} for all the observation forces the program to find a single observation error standard deviation for all the observation. In this case because the measured quantity for all observations is dissolved oxygen it is expected that the measurement error for all observation to have the same statistical distribution.

\begin{figure}[!ht]
\begin{center}
\includegraphics[width=6cm]{Images/Figure37.png} \\
\caption{Assigning parameters to model properties}\label{fig:inverse_example3}
\end{center}
\end{figure}

Repeat for the other parameters to be estimated including $k_{OA}$, $\mu_{NOB}$, $k_{ON}$, $\mu_{OHO}$,
and $k_{OH}$.

\item \textbf{Running in forward mode: } When the model is run in forward mode, the values of the parameters as specified in the \textbf{Value} field are used and a single simulation is performed. Click on the forward run icon \includegraphics[width=0.3cm]{Icons/run_icon.png} and wait for the simulation to end. Choose an experiment from the \textbf{Experiments} drop-down menu and then right click on the block and choose \textbf{Plot water quality results}$\rightarrow$\textbf{DO}. \\

- Right-click on \textbf{Project Explorer}$\rightarrow$\textbf{Inverse modeling}$\rightarrow$ \textbf{Observations}$\rightarrow$ \textit{DO\_NH3\_spike} and then click on \textbf{Plot modeled data}. This shows the corresponding model prediction to observations for experiment 2. You can also check the agreement plot.

\item \textbf{Change the initial time-step: } The model results that are used to calculate the likelihood are interpolated at time-intervals specified in the \textbf{initial time-step field}. Because the experiments in this example are short (0.1 day), the default initial time-step of 0.01 day is not adequate. From the project exploder choose \textbf{Settings}$\rightarrow$\textbf{Solver Settings} and then from the \textbf{Properties} window find \textbf{Initial time step size} and change the value to 0.001 day.

\item{Inverse modeling: }\\ - Choose \textbf{number of generations} from \textbf{Inverse modeling}$\rightarrow$\textbf{Genetic Algorithm} and change the value to 100. This makes the number of generations in the Genetic Algorithm to 100. Keep the rest of the parameters unchanged.

 - Choose \textbf{number of realizations} from \textbf{Inverse modeling}$\rightarrow$\textbf{Markov chain Monte Carlo} and change the value to 1000. This makes the number of posterior prediction realizations to 1000. Keep the rest of the parameters unchanged.

- Click on the inverse modeling icon \includegraphics[width=0.5cm]{Icons/inverse_icon.png} on the left side tool bar. Inverse simulation can take up to one hour depending on the number and speed of the CPUs of the computer being used for the simulation. Wait until the deterministic inverse modeling and the MCMC is finished. The progress bars on the \textbf{Simulation} window shows the percentage of each stage on inverse modeling being completed (Figures \ref{fig:43} and \ref{fig:44}).

\begin{figure}[!ht]
\begin{center}
\includegraphics[width=8cm]{Images/Figure43.png} \\
\caption{Inverse modeling progress window during deterministic parameter estimation stage}\label{fig:43}
\end{center}
\end{figure}

\begin{figure}[!ht]
\begin{center}
\includegraphics[width=8cm]{Images/Figure44.png} \\
\caption{Inverse modeling progress window during probabilistic parameter estimation}\label{fig:44}
\end{center}
\end{figure}

\item \textbf{Estimated values of the parameters: } To see the estimated values of the parameters select a parameter from \textbf{Project Explorer}$\rightarrow$\textbf{Inverse Modeling}$\rightarrow$\textbf{Parameters} and look at the \textbf{Value} field. The value should be replaced by the estimated parameter value.

\item \textbf{Checking the model vs. observed agreement: } Right-click on \textbf{Project Explorer}$\rightarrow$\textbf{Inverse Modeling}$\rightarrow$\textbf{Observations}$\rightarrow$\textit{DO\_nospike} and the choose \textbf{Plot modeled data}. A graph will appear that will show observed data and the model prediction based on the estimated parameters. Do the same for other observation data (Figure \ref{fig:48}).

\begin{figure}[!ht]
\begin{center}
\includegraphics[width=10cm]{Images/Figure48.png} \\
\caption{Observed and predicted DO variation in all four experiments}\label{fig:48}
\end{center}
\end{figure}

\item \textbf{Checking the posterior distributions of the parameters: }\\ - From \textbf{Project Explorer}$\rightarrow$\textbf{Inverse Modeling}$\rightarrow$\textbf{Parameters} right-click on $\mu_{NOB}$ and select \textbf{Plot posterior distribution histogram}. A graph will appear that shows the posterior distribution of parameter $\mu_{NOB}$ (Figure \ref{fig:45}). Similarly inspect the posterior distribution for other parameters. (Figure \ref{fig:46}). In this figure the box plot shows the 95\% credible interval for the parameter while the solid line shows the median and the dot shows the expected value of the parameter.

\begin{figure}[!ht]
\begin{center}
\includegraphics[width=11cm]{Images/Figure45.png} \\
\caption{Posterior distribution of $k_{OH}$,  $\mu_{OHO}$, $k_{ON}$,  $\mu_{NOB}$}\label{fig:45}
\end{center}
\end{figure}

\begin{figure}[!ht]
\begin{center}
\includegraphics[width=8cm]{Images/Figure46.png} \\
\caption{Posterior credible interval for }\label{fig:46}
\end{center}
\end{figure}

- To see the 95\% credible intervals for all the parameters right-click on \textbf{Project Explorer}$\rightarrow$\textbf{Inverse Modeling}$\rightarrow$\textbf{Parameters} and select \textbf{Plot Percentile data} (Figure \ref{fig:47}).

-  From \textbf{Project Explorer}$\rightarrow$\textbf{Inverse Modeling}$\rightarrow$\textbf{Parameters} right-click on $\mu_{NOB}$ and select \textbf{Plot percentiles}.

\begin{figure}[!ht]
\begin{center}
\includegraphics[width=8cm]{Images/Figure47.png} \\
\caption{Posterior credible interval for all parameters}\label{fig:47}
\end{center}
\end{figure}

\end{itemize}
\section{Maximum-likelihood Inverse Modeling Control Parameters}

This section describes the control parameters for the maximum-likelihood (deterministic) inverse modeling feature of GIFMod. The maximum-likelihood control parameters can be accessed through \textbf{Project Explorer}$\rightarrow$\textbf{Inverse Modeling}$\rightarrow$\textbf{Genetic Algorithm}.

\begin{itemize}
    \item \textbf{Name: } The name of Genetic Algorithm object. It can be ignored.
    \item \textbf{Cross-over probability: } This indicates the probability of \textit{individuals} undergoing a cross-over to generate the \textit{off-springs}. The reminder of the \textit{individuals} selected for mating will be copied to the next generations with only \textit{mutation}. For example if the value of \item \textbf{Cross-over probability} is set to 0.8, 80\% of the selected individuals will undergo cross-over while 20\% will be directly copied to the next generation.
    \item \textbf{GA output file: } The results of genetic algorithm optimization will be written in this file. The file that will be generated can be used for diagnosis of the GA performance.
    \item \textbf{Initial GA population: } If a file name is provided here it will be considered as the initial population for the GA analysis. If this field is empty, the initial GA population will be created randomly.
    \item{Mutation probability: } This indicates the probability of mutation of each bit when copying from one generation to the other.
    \item{Number of generations: } Indicates the number of generation in the GA algorithm to obtain the optimal parameter set.
    \item{Perform local sensitivity analysis: } Indicates whether a local sensitivity analysis should be performed based on the deterministic parameter obtained by the GA algorithm. The results of the local sensitivity analysis will be saved in a text file called "sensitivity\_mat\_lumped.txt" in the working path.
    \item{Population: } Indicates the population number that will be used for the GA optimization.
    \item{Read GA analysis from file: } This field is used when the GA analysis has been previously done and only the post-processing is intended to be done using the results of the previous inverse modeling. The file name that will be entered is where GIFMod will load the results of the previous GA analysis from.
    \item{Shake scale: } When the maximum fitness in the population is not improved in three sequential generations, an offspring of the fittest individual is produced by adding a random noise to the parameter values. This scale indicate the magnitude of the random noise. The default value is 0.05 which indicate adding a normally distributed noise with a standard deviation equal to 5\% of the value of each parameter. The purpose of this \textit{shaking} is to find possible optimal solution in the neighborhood of the parameter set represented by the fittest individual.
    \item{Shake scale reduction factor: } If adding the noise does not result in an improved fitness, the shake-scale is gradually reduced (so a closer neighborhood is searched). The shake scale reduction factor is the factor by which this reduction in the shake scale is done.
    \item{Number of threads: } The number of CPU threads used for the GA and MCMC parameter estimation.
\end{itemize}

\section{Probabilistic Inverse Modeling Control Parameters}
This section describes the control parameters for the MCMC (stochastic) inverse modeling feature of GIFMod. The MCMC control parameters can be accessed through \textbf{Project Explorer}$\rightarrow$\textbf{Inverse Modeling}$\rightarrow$\textbf{Markov chain Monte Carlo}.

\begin{itemize}
\item \textbf{Create output realization including observation errors} Switching this property to \textit{Yes} results in GIFMod to generate prediction confidence intervals while considering the observation error.

\item \textbf{Generate output realizations: } Determines whether output realizations based on parameter samples from the posterior distribution should be generated. These realization will be used to determine confidence intervals of model predictions while accounting for parameter uncertainty.
\item \textbf{Initial perturbation: } Determines whether the initial MCMC parameter sets should be obtained by perturbing the deterministically estimated parameters using GA or by using the non-perturbed values. MCMC used the parameters estimated by the GA as the initial point wither after perturbing the parameters or not.
 \item{Save realization outputs in a file: } Switching this property to \textit{Yes}, forces the program to create a file where the sampled parameter values from the posterior distribution to generate the posterior realizations are saved.


\graphicspath{ {./img/TheFEM/} }
\chapter{The Finite Element Method}
\section{Weighted residual methods}
This section introduces the concept of residual or difference from zero in a differential equation once its solution is approximated. For that purpose we will take as prototype equation the one obtained as our general model of BVP (see \cref{eq:GenPDE}) and recalled here for completeness
\begin{equation}
\rho(\vb x) \pdv{u(\vb x, t)}{t} + \mathcal{L}u(\vb x, t) = \rho (\vb x)F(\vb x, t) \enspace .
\label{eq:GenPDE2}
\end{equation}

We will assume that the actual solution to the generalized BVP given by \cref{eq:GenPDE2} is approximated by $\tilde u(\vb{x})$ through a superposition like
\begin{equation}
\tilde u (\vb{x}) = {N^I}(\vb{x}){u^I}
\label{basicsuper}
\end{equation}
where ${N^I}(\vb{x})$ are interpolating functions and $I$ denotes a superposition index  varying like $I=1,2,...,K$ with $K$ being the number of points where the solution is known. In what follows we will use $u(\vb{x})$ instead of $\tilde u (\vb{x})$ but will keep in mind that we are actually using the approximation given by \cref{basicsuper}. Similarly, in order to keep the discussion simple for the time being we will drop the time effects reducing the generalized PDE to the simple form:
\begin{equation}
\mathcal{L}u(\vb x) = \rho (\vb x)F(\vb x) \enspace .
\label{eq:GenPDE3}
\end{equation}

Now, since we are using the approximation given by \cref{basicsuper} this equation is not strictly satisifed but instead we will have the following ``unbalanced" condition
\[\mathcal{L}u(\vb{x}) - \rho (\vb{x})F(\vb{x}) \equiv R \ne 0\]
where the term $R$ corresponds to a residual error which is to be distributed throughout the solution domain. The so-called weighted residual methods differ in the form in which they distribute the residual between the different $K$ points conforming the computational domain.

Using \cref{basicsuper} in \cref{eq:GenPDE2} and the linearity in the differential operator yields
\[R = \mathcal{L}({N^P}){u^P} - \rho F\, .\]

We can see that the residual $R$ is a function defined over the domain of interest. The residual would be exactly zero for the solution of the differential equation, but it will not be zero in general. Thus, we want to make the function $R$ as close to zero as possible. To make $R$ as small as possible we need a function (a functional) where we can compare different approximation functions. After getting this functional we can minimize its value. For this minimization we could use the norm of the function, another option is to compute a weighted \emph{average} of the function over the domain. This is what we call a weighted residual
\[\Pi[u, w] = \int\limits_V w R(u) \dd{V}\, ,\]
and we want to minimize it by making
\[\var{\Pi}[u, w] = 0\, ,\]

In what follows we will consider different strategies to distribute or weight the residual $R$ over the computational domain.

\subsubsection{Galerkin method}
In the Galerkin scheme the interpolation functions are used also as weighting functions leading to:
\[\int\limits_V N^Q R\dd{V} = 0 \]
or explicitly
\begin{equation}
  \int\limits_V N^Q \mathcal{L} (N^P)\dd{V} u^P = \int\limits_V N^Q\rho F\dd{V}\, .
  \label{eq:Galer}
\end{equation}

Imposing \cref{eq:Galer} in the $K$ points conforming the computational domain or equivalently ranging $Q$ from $1$ to $K$ leads to the following system of algebraic equation
\begin{equation}
{K^{QP}}{U^P} = {f^Q}
\label{eq:DGaler}
\end{equation}

where $U^P$ is a vector that stores the point values of the function $u$ along te $K$ points of the computational domain, while $f^Q$ stores the corresponding point excitations.

\subsubsection{Least squares method}
In this method the integral of the square of the residual is minimized with respect to the $K$ point parameters or nodal values of the function. Accordingly,
\begin{align*}
  &\pdv{u^I}\int\limits_V R^2 \dd{V} = 0\\
  &\int\limits_V R \pdv{R}{u^I} \dd{V} = 0\, ,
\end{align*}

The least squares method is a special case of the weighted residual method for the weight functions
\[w^I = \pdv{R}{u^I}\, .\]

Expanding the residual, and considering the operator $\mathcal{L}$ as linear, we obtain
\begin{align*}
  &\pdv{u^I}\int\limits_V [\mathcal{L}(N^P u^P) - \rho F]^2 \dd{V} = 0\\
  &\int\limits_V [\mathcal{L} N^P u^P - \rho F] \mathcal{L}(N^I) \dd{V} = 0\\
 &\int\limits_V \mathcal{L}(N^I) \mathcal{L}(N^P) \dd{V} u^P - \rho \int\limits_V \mathcal{L}(N^I) F \dd{V} = 0
\end{align*}
which can be written like
\begin{equation}
  K^{IP} U^P = f^I
  \label{eq:Dsquares}
\end{equation}

\subsubsection{Collocation method}
In the collocation method the coefficients of the approximation are determined by forcing the residual to be exactly zero at $K$ points over the computational domain, i.e.,
\[\mathcal{L}(N^I) u^I - \rho F = 0\, ,\]
or
\[\mathcal{L}[N^I(x^J)] u^I - \rho F(x^J) = 0\, ,\]
where $J$ ranges between $1$ and $K$. This equation can be rewritten as a weighted-residual if we consider the residual to be $\delta(x - x^I)$, the Dirac delta function over the selected points

The resulting system of algebraic equation can be written as
\begin{equation}
K^{IP} U^P = f^I\, .
\label{eq:Colo}
\end{equation}

\subsubsection{Subdomain method}
The zero value of the residual is imposed upon $K$ subdomains
\[\int\limits_{V^I} \mathcal{L}(N^P)\dd{V^I} u^P  - \rho \int\limits_{V^I}  F\dd{V^I}  = 0 \qquad I=1,\cdots,K\, .\]

For instance, for the $N$-th element it follows that
\[\int\limits_{V^N} \mathcal{L}(N^P)\dd{V^N} u^P  - \rho \int\limits_{V^N} F\dd{V^N}  = 0 \qquad P=1,\cdots,K\, .\]

Applying the equation over the $K$ subdomains leads to the discrete system;
\begin{equation}
K^{IP} U^P = f^I \quad \quad I=1,\cdots,K\, .
\label{eq:Subdomain}
\end{equation}

\subsubsection{Ritz method}
It operates directly upon the variational statement of the problem. For a given functional
\[\Pi  = \Pi (N^Q u^Q)\]
the variational equation reads
\[\var{\Pi}  \equiv \pdv{\Pi}{u^Q} \var{u^Q} = 0\]
from which
\[\pdv{\Pi}{u^Q} = 0\, .\]

\subsubsection*{Problem: Discretization of the generalized parabolic equation.}
Let us consider the case of the generalized parabolic equation and its discretization following the Galerkin method
\[\rho(\vb x) \pdv{u(\vb x,t)}{t} + \mathcal{L}u(\vb x, t) = 0\]
which can also be written using indicial notation
\[\pdv{x_i}\left[{p(x)\pdv{u}{x_i} \right] + q(x)u + \rho \pdv{u}{t} = \rho F.\]

Assuming that $p(x)=1$ yields
\[ - \int\limits_V N^P N_{,ii}^Q \dd{V} u^Q + \int\limits_V {q{N^P}{N^Q}dV{u^Q}}  + \rho \int\limits_V {{N^P}{N^Q}dV{v^Q}}  - \rho \int\limits_V {{N^P}FdV = 0} \]

\[\int\limits_V {N_{,i}^PN_{,i}^Q} dV{u^Q} - \int\limits_S {{N^P}N_{,i}^Q{{\hat n}_i}} dS{u^Q} + \int\limits_V {q{N^P}{N^Q}dV{u^Q}}  + \rho \int\limits_V {{N^P}{N^Q}dV{v^Q}}  - \rho \int\limits_V {{N^P}FdV = 0} \]

\[\int\limits_V {\left( {N_{,i}^PN_{,i}^Q + q{N^P}{N^Q}} \right)dV{u^Q}}  + \rho \int\limits_V {{N^P}{N^Q}dV{V^Q}}  = \int\limits_S {{N^P}N_{,i}^Q{{\hat n}_i}} dS{u^Q} + \rho \int\limits_V {{N^P}FdV} \]

which can be written in discrete form;

\[{K^{PQ}}{U^Q} + {C^{PQ}}{V^Q} = {f^p}\]

\subsubsection*{Problem: Discretization of Navier equations.}
In this case the differential equations are written as
\[(\lambda  + \mu ){u_{j,ij}} + \mu {u_{i,jj}} + {f_i} = 0 \enspace .\]

We can write the differential operator as
\[L_{ij} \equiv (\lambda  + \mu )\pdv[2]{}{x_i}{x_j} + \mu \pdv[2]{}{x_k}{x_k}\delta_{ij}\]

\begin{align*}
&r_i =  - f_i\\
&u_i = N_i^Q u^Q\\
&\mathcal{L}_{ij}(u_j) \equiv (\lambda  + \mu )(N_j^Q{u^Q})_{,ij} + \mu (N_j^Q{u^Q})_{,kk}\delta_{ij}\\
&\mathcal{L}_{ij}(u_j) \equiv (\lambda  + \mu )N_{j,ij}^Q u^Q + \mu N_{i,kk}^Q u^Q\\
&\mathcal{L}_{ij}(u_j) \equiv \mathcal{L}_{ij}(N_j^Q) u^Q \enspace .
\end{align*}

In the Galerkin scheme we use the trial function as weighting function.
\[R_i \equiv L_{ij}(N_j^Q) u^Q + f_i\]
and we state \dots

\[\int\limits_V {N_i^P{R_i}dV = 0} \quad \quad P=1,2,...,N. \]

\[\int\limits_V {N_i^P{\mathcal{L}_{ij}}(N_j^K)dV{u^K}}  + \int\limits_V {N_i^P{f_i}dV = 0} \]

\[(\lambda  + \mu )\int\limits_V {N_i^PN_{j,ij}^KdV{u^K}}  + \mu \int\limits_V {N_i^PN_{i,kk}^KdV{u^K}}  + \int\limits_V {N_i^P{f_i}dV = 0} \]

integrating by parts;

\begin{align*}
- (\lambda  + \mu )\int\limits_V {N_{i,j}^PN_{j,i}^KdV{u^K} + (\lambda  + \mu )\int\limits_S {N_i^PN_{j,i}^K{{\hat n}_j}dS{u^K} - \mu \int\limits_V {N_{i,k}^PN_{i,k}^KdV{u^K}} } } \\
+ \mu \int\limits_S {N_i^PN_{i,k}^K{{\hat n}_k}dS{u^K}}  + \int\limits_V {N_i^P{f_i}dV = 0}
\end{align*}

which can be written like;

\[{K^{PQ}}{U^Q} = {F^P}\]

where

\[{K^{PQ}} = (\lambda  + \mu )\int\limits_V {N_{i,j}^PN_{j,i}^QdV}  + \mu \int\limits_V {N_{i,k}^PN_{i,k}^QdV} \]

\[{F^P} = \int\limits_S {N_i^Pt_i^{(\hat n)}dS + \int\limits_V {N_i^P{f_i}dV = 0} } \]

\subsubsection*{Problem: Discretization of the acoustic wave equation.}

\[\vec \nabla  \cdot \left[ {\frac{1}{\rho }\vec \nabla p(\vb{x})} \right] - \frac{\partial }{{\partial t}}\left( {\frac{1}{\lambda }\frac{{\partial \rho }}{{\partial t}}} \right) - q(\vb{x}) = 0\]

where we recognize;

\[\mathcal{L}() \equiv \vec \nabla  \cdot \left( {\frac{1}{\rho }\vec \nabla } \right) - \frac{\partial }{{\partial t}}\left( {\frac{1}{\lambda }\frac{\partial }{{\partial t}}} \right)\]

Let

\[p(x) = {N^K}{p^K}\]

then

\[\mathcal{L}(p) \equiv \vec \nabla  \cdot \left( {\frac{1}{\rho }\vec \nabla {N^K}{p^K}} \right) - \frac{\partial }{{\partial t}}\left( {\frac{1}{\lambda }\frac{{\partial {N^K}{p^K}}}{{\partial t}}} \right)\]

or in indicial notation

\[\mathcal{L}(p) \equiv {\left( {\frac{1}{\rho }N_{,i}^K} \right)_{,i}}{p^K} - \frac{\partial }{{\partial t}}\left( {\frac{1}{\lambda }\frac{{\partial {N^K}}}{{\partial t}}} \right){p^K}\]

which is equivalent to having;

\[\mathcal{L}(p) \equiv \mathcal{L}({N^K}){p^K}\]

using the trial functions as weighting functios and recalling the definition of the residual which in this case reads;

\[R = \mathcal{L}({N^K}){p^K} - q\]

yields;

\[\int\limits_V {{N^J}RdV = 0} \quad \quad J=1,2,...,K \]

\[\int\limits_V {{N^J}L({N^K})dV{p^K}}  - \int\limits_V {{N^J}qdV}  = 0\]

\[\int\limits_V {{N^J}{{\left( {\frac{1}{\rho }N_{,i}^K} \right)}_{,i}}dV{p^K} - \int\limits_V {{N^J}\frac{\partial }{{\partial t}}\left( {\frac{1}{\lambda }\frac{{\partial {N^K}}}{{\partial t}}} \right)dV{p^K}} }  - \int\limits_V {{N^J}qdV}  = 0\]

Integrating by parts the first term on the R.H.S gives us;

\[ - \int\limits_V {N_{,i}^J\frac{1}{\rho }N_{,i}^KdV} {p^K} + \int\limits_S {{N^J}\frac{1}{\rho }N_{,i}^K{{\hat n}_i}dS{p^K}}  = \int\limits_V {{N^J}\frac{\partial }{{\partial t}}\left( {\frac{1}{\lambda }\frac{{\partial {N^K}}}{{\partial t}}} \right)dV{p^K}}  + \int\limits_V {{N^J}qdV} \]

\[\int\limits_V {N_{,i}^J\frac{1}{\rho }N_{,i}^KdV} {p^K} + \int\limits_V {{N^J}\frac{1}{\lambda }{N^K}dV{{\ddot p}^K}}  = \int\limits_S {{N^J}\frac{1}{\rho }N_{,i}^K{{\hat n}_i}dS{p^K}}  + \int\limits_V {{N^J}qdV} \]

\[{K^{JK}}{P^K} + {M^{JK}}{{\ddot P}^K} + {f^J} = 0\]

\section{A simple discrete system}
The simple problem of a spring-mass system considered next resembles most of the algorithmic aspects of a finite element code with the advantage that the problem is already a discrete mechanical system. The problem consists of an assemblage of masses joined by different springs submitted to time varying loads. Each spring plays the role of a finite element and each mass is analogous to a nodal point in a finite element algorithm. For instance the full system may be like the one shown in \cref{fig:bathe}
\begin{figure}[h]
\centering
\includegraphics[width=10cm]{spring_system.pdf}
\caption{Typical assemblage of springs and masses.}
\label{fig:bathe}
\end{figure}

Consider a typical spring (finite element) like the one shown in \cref{fig:springel}
\begin{figure}[h]
\centering
\includegraphics[width=8cm]{springel.pdf}
\caption{Typical spring element.}
\label{fig:springel}
\end{figure}

The relation between the force and the relative displacement can be written like
\[f_1 = K(u_1 - u_2)\]
and from equilibrium we have
\[f_1 + f_2 = 0\]
which yields the following force-displacement relationship for a typical spring element:
\begin{equation}
    \begin{Bmatrix}
        f_1\\
        f_2
    \end{Bmatrix} =
    K\begin{bmatrix}
          1.0 & -1.0\\
        - 1.0 & 1.0
    \end{bmatrix}
    \begin{Bmatrix}
        u_1\\
        u_2
    \end{Bmatrix}
    \label{eq:Kspring}
\end{equation}

On the other hand, the equilibrium equation for a typical mass with displacement $u_j$  (see \cref{fig:dclmass}) and attached to springs $i$ and $i+1$ reads
\begin{equation}
f_2^i + f_1^{i + 1} + m_j \dv{V_j}{t} = P_j.
\label{eq:equilmass}
\end{equation}
which can be written in terms of displacements using \cref{eq:Kspring} like
\[(K^i + K^{i + 1}) u_j - K^i u_{j - 1} - K^{i + 1} u_{j + 1} + m_j\dv{V_j}{t} = P_j .\]

\begin{figure}[H]
\centering
\includegraphics[width=8cm]{dcl_mass.pdf}
\caption{Free body diagram for a typical mass connected to springs $i$ and $i+1$.}
\label{fig:dclmass}
\end{figure}

Writing the elemental equilibrium equations in terms of $u_{j - 1}$, $u_j$ and $u_{j + 1}$ we have:

\[\left\{ {\begin{array}{*{20}{c}}
{f_1^i}\\
{f_2^i}
\end{array}} \right\} = \left[ {\begin{array}{*{20}{c}}
{k_{11}^i}&{k_{12}^i}\\
{k_{21}^i}&{k_{22}^i}
\end{array}} \right]\left\{ {\begin{array}{*{20}{c}}
{{u_{j - 1}}}\\
{{u_j}}
\end{array}} \right\}\]

and

\[\left\{ {\begin{array}{*{20}{c}}
{f_1^{i + 1}}\\
{f_2^{i + 1}}
\end{array}} \right\} = \left[ {\begin{array}{*{20}{c}}
{k_{11}^{i + 1}}&{k_{12}^{i + 1}}\\
{k_{21}^{i + 1}}&{k_{22}^{i + 1}}
\end{array}} \right]\left\{ {\begin{array}{*{20}{c}}
{{u_j}}\\
{{u_{j + 1}}}
\end{array}} \right\}\]

which gives for the equilibrium equation of the $m_j$ mass:

\[k_{21}^i{u_{j - 1}} + (k_{22}^i + k_{11}^{i + 1}){u_j} + k_{12}^{i + 1}{u_{j + 1}} + {m_j}\frac{{d{V_j}}}{{dt}} = {P_j}.\]

Considering also the contributions from the springs $K^i$ and $K^{i+1}$ to the equilibrium of masses $m_{j-1}$ and $m_{j+1}$ respectively we have the following matrix block:

\[\left[ {\begin{array}{*{20}{c}}
{}&{}&{}&{}\\
{}&{k_{11}^i}&{k_{12}^i}&{}\\
{}&{k_{21}^i}&{k_{22}^i + k_{11}^{i + 1}}&{k_{12}^{i + 1}}\\
{}&{}&{k_{21}^{i + 1}}&{k_{22}^{i + 1}}
\end{array}} \right]\]

Considering now the complete system of masses and springs leads to a system of linear equations of the form
\begin{equation}
\left[ {{K_G}} \right]\left\{ {{U_G}} \right\} + \left[ M \right]\left\{ {{A_G}} \right\} = \left\{ {{F_G}} \right\}.
\label{eq:global}
\end{equation}
where each equation represents the equilibrium of a given mass. The coefficient matrix in \cref{eq:global} can be assembled in a systematic way by establishing the connection between the global and local degrees of freedom. This can be accomplished through an operator storing in each row the global degrees of freedom corresponding to each element. For instance, \cref{fig:IBC} shows elements $K^i$ and $K^{i+1}$ and the global degrees of freedom corresponding to masses $m_{j-1}$, $m_j$ and $m_{j+1}$. The corresponding entries of the $DME$ operator for these elements are given by:

\[DME = \left[ {\begin{array}{*{20}{c}}
{}&{}\\
{j - 1}&j\\
j&{j + 1}\\
{}&{}
\end{array}} \right]\]

\begin{figure}[H]
\centering
\includegraphics[width=12cm]{ibc}
\caption{Global degrees of freedom connected to the spring elements $K^i$ and $K^{i+1}$ respectively.}
\label{fig:IBC}
\end{figure}

and the contribution from these elements to the global coefficient matrix for elements $i$ and $i+1$ reads respectively:

\[\begin{array}{l}
{K_{j - 1,j - 1}} \leftarrow {K_{j - 1,j - 1}} + k_{11}^i\\
{K_{j - 1,j}} \leftarrow {K_{j - 1,j}} + k_{12}^i\\
{K_{j,j - 1}} \leftarrow {K_{j,j - 1}} + k_{21}^i\\
{K_{j,j}} \leftarrow {K_{j,j}} + k_{22}^i
\end{array}\]

and

\[\begin{array}{l}
{K_{j,j}} \leftarrow {K_{j,j}} + k_{11}^{i + 1}\\
{K_{j,j + 1}} \leftarrow {K_{j,j + 1}} + k_{12}^{i + 1}\\
{K_{j + 1,j}} \leftarrow {K_{j + 1,j}} + k_{21}^{i + 1}\\
{K_{j + 1,j + 1}} \leftarrow {K_{j + 1,j + 1}} + k_{22}^{i + 1}
\end{array}\]

The system given by \cref{eq:global}, assembled with the aid of the $DME$ operator can be solved for the global displacements $U_G$. The pseudo-code shown in \cref{algo:springs} presents all the steps required to solve the problem in the context of the finite element method. In that code the so-called DME operator is an equation assembly array indicating how each element contributes to the global stiffness and mass matrix.

\begin{algorithm}[H]\label{algo:springs}
    \SetAlgoLined
    \KwData{Problem parameters; NUMNP, NUMEL, NMATP}
    \KwResult{Displacements and spring forces}
    Create $DM$E operator\;
    Assemble $K^G$, $F^G$\;
    \While{$j \leq 1, NUMEL$}{
        $K^G \leftarrow K^G+K^i$\\
        $F^G \leftarrow F^G+F^i$\\
    }
    Solve $[K^G]U=F^G$\\
    Find internal forces
    \caption{Springs Algorithm.}
\end{algorithm}

\newpage

\inputminted[]{python}{src/engine.py}

\newpage

\section{Basic elements of interpolation theory}
Let $f(x)$ be a function whose values are known at n discrete points ${x_1, x_2,...,x_n}$. We want to know (interpolate) the value of $f(x)$ at an arbitrary point $x \in \left[ {{x_1},{x_n}} \right]$.

\begin{figure}[h]
\centering
\includegraphics[width=12cm]{img/interpol1.pdf}
\caption{Global interpolation of a function}
\label{fig:interpol1}
\end{figure}

The process of interpolation or computation of the unknown value of $f(x)$ using the known values $\left\{ {{f^1},{f^2},...,{f^n}} \right\}$ involves two steps:

\begin{itemize}
\item[i]  Fitting an interpolating function to the known data points.
\item[ii] Evaluating the function at the arbitrary point.
\end{itemize}

We can (i) use all the n-data points and fit an $(n-1)$-th order polynomial (which is cumbersome and difficult to code) or (ii) split the domain in sub-intervals and use local polynomials within each sub-interval. This last approach involves only a couple of polynomials and it is easy to code, however it may have some continuity issues.

In finite element analysis local interpolation is used in order to proceed systematically. Local interpolation uses a finite number of nearest-neighbors and generates interpolated values $f(x)$ that do not in general have continuous first or higher derivatives.

\begin{figure}[h]
\centering
\includegraphics[width=12cm]{img/interpol2.pdf}
\caption{Local or piecewise interpolation of a function}
\label{fig:interpol2}
\end{figure}

\subsubsection{Lagrange interpolation theorem}
Given a set of n-points $\{ (x^1, y^1),\cdots,(x^n, y^n)\}$ where $y^n \equiv f({x^n})$ then: ``there exists a unique polynomial $p(x)$ of order at most $(n-1)$ such $p(x^I) = f(x^I)$ for $I=1,2,\cdots,n$". The polynomial is given by;
\begin{equation}\label{eq:pol}
  p(x^I) = L^I(x) f(x^I)
\end{equation}
for $I=1,2,...,n$ where
\begin{equation}\label{eq:coef}
  L^I(x) = \prod_{\substack{J = 1\\ I \ne J}}^n \frac{(x - x^J)}{(x^I - x^J)}
\end{equation}
and where it should be noticed that
\[L^I(x^J) = \delta^{IJ}.\]

\subsubsection*{Example for n=3}
Consider the domain $[ - 1,1]$ and the data points at ${x^1} =  - 1.0$, ${x^2} =  + 1.0$ and ${x^3} = 0.0$. We have
\begin{align*}
& L^1(x) = \frac{(x - x^2)(x - x^3)}{(x^1 - x^2)(x^1-x^3)} \equiv  - \frac{1}{2}(1 - )x\\
& L^2(x) = \frac{(x - x^1)(x - x^3)}{(x^2 - x^1)(x^2 - x^3)} \equiv  + \frac{1}{2}(1 + x)x
\end{align*}
and
\[L^3(x) = \frac{(x - x^1)(x - x^2)}{(x^3 - x^1)(x^3 - x^2)} \equiv 1 - x^2.\]

The resulting interpolating polynomials $L^I(x)$ and the interpolating function  are shown in \cref{fig:pols} below
\begin{figure}[H]
  \centering
  \includegraphics[width=16cm]{func.pdf}
  \caption{Interpolating polynomials and the resulting interpolating function}
  \label{fig:pols}
\end{figure}

\subsubsection*{Example: Interpolation of a function using a global and a local scheme}
Assume we have known values of the function:

\[f(x) = {x^3} + 4{x^2} - 10\]

in the interval $[-1.0, 1.0]$ and we wish to obtain an interpolated version of the function using different schemes.

\begin{figure}[H]
\centering
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{lineal.pdf}
		\caption{Interpolation polynomials. }
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{interlin.pdf}
		\caption{Actual and interpolated function.}
	\end{subfigure}\\
%
\centering
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{quadra.pdf}
		\caption{Interpolation polynomials.}
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{interqua.pdf}
		\caption{Actual and interpolated function.}
	\end{subfigure}\\
%
\centering
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{third.pdf}
		\caption{Interpolation polynomials.}
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{intertri.pdf}
		\caption{Actual and interpolated function.}
	\end{subfigure}
\caption{Lineal interpolation of the function $f(x) = {x^3} + 4{x^2} - 10$.}
\label{fig:several interpol}
\end{figure}

\Cref{fig:several interpol} shows the interpolating polynomials and the resulting function for the case of first, second and fourth order polynomials respectively. Similarly, the first order derivative obtained out of the interpolated function is displayed in \cref{fig:first der}.

\begin{figure}[H]
\centering
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{deriv2.pdf}
		\caption{First order derivatives of the interpolation polynomials. }
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{firstder.pdf}
		\caption{Interpolated first order derivative of the function.}
	\end{subfigure}\\

\caption{Lineal interpolation of the function $f(x) = {x^3} + 4{x^2} - 10$.}
\label{fig:first der}
\end{figure}

We now proceed like in the finite element method and use the local first order polynomials shown in \cref{fig:loc-pols} to interpolate the function under study.

\begin{figure}[H]
  \centering
  \includegraphics[width=10cm]{localone.pdf}
  \caption{Local interpolating polynomials}
  \label{fig:loc-pols}
\end{figure}

The resulting function and its numerically obtained first order derivateive are shown in \cref{fig:fully local}. It is clear how the local interpolation destroys the global continuity in the first order derivative while the function itself remains continous.

\begin{figure}[H]
\centering
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{localfun.pdf}
		\caption{Function interpolated with local first order polynomials. }
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{localfirst.pdf}
		\caption{Interpolated first order derivative of the function.}
	\end{subfigure}\\

\caption{Lineal interpolation of the function $f(x) = {x^3} + 4{x^2} - 10$.}
\label{fig:fully local}
\end{figure}

\subsection*{Extension to 2D domains}
Assume we are now interested in conducting interpolation of a function over a spatial 2-dimensional domain where every point is specified by a position vector of the form $\vb{x} = x \hat{\imath} + y\hat{\jmath}$. We want to know, via interpolation, the value of a function $f(\vb{x})$ at an arbitrary point $\vb{x}$ provided we know the set of n-points $\{(\vb{x}^1, f^1),\cdots,(\vb{x}^n, f^n)\}$.
\begin{figure}[H]\label{fig:element}
  \centering
  \includegraphics[width=10cm]{element.pdf}
  \caption{Basic square domain}
\end{figure}

We first fix $x = x^A$ and conduct 1-dimensional interpolation along the $y$ direction as discussed in the previous section as follows (see \cref{fig:onedimn})
\[f(x^A,y) = L^1(y)f^1 + L^4(y)f^4\]

\begin{figure}[H]
\centering
\includegraphics[width=10cm]{inter1D.pdf}
\caption{Interpolation along the $y$-direction}
\label{fig:onedimn}
\end{figure}

Similarly, we can fix $x = x^B$ and interpolate once again along the $y$ direction
\[f(x^B,y) = L^2(y)f^2 + L^3(y)f^3.\]

We now conduct the interpolation along the $x$-direction using the functions $f(x^A,y)$ and $f(x^B,y)$ respectively as follows
\begin{align*}
  &f(x,y) = L^A(x) f(x^A,y) + L^B(x)f(x^B,y)\\
  &f(x,y) = L^A(x)\{L^1(y)f^1 + L^4(y)f^4\} + L^B(x)\{L^2(y)f^2 + L^3(y)f^3\}\\
  &f(x,y) = L^A(x)L^1(y)f^1 + L^A(x)L^4(y)f^4 + L^B(x)L^2(y)f^2 + L^B(x)L^3(y)f^3 \enspace ,
\end{align*}
where
\begin{align*}
L^A(x) & \equiv L^1(x)\\
L^B(x) & \equiv L^2(x)\\
L^1(y) & \equiv L^1(y)\\
L^2(y) & \equiv L^1(y)\\
L^3(y) & \equiv L^2(y)\\
L^4(y) & \equiv L^2(x) \enspace .
\end{align*}

The function of two variables is then written as the product of one-dimensional interpolations
\[f(x,y) = N^1(x,y)f^1 + N^2(x,y)f^2 + N^3(x,y)f^3 + N^4(x,y)f^4\]
with
\begin{align*}
N^1(x,y) & = L^1(x)L^1(y)\\
N^2(x,y) & = L^2(x)L^1(y)\\
N^3(x,y) & = L^2(x)L^2(y)\\
N^4(x,y) & = L^1(x)L^2(y) \enspace .
\end{align*}

See Figure \ref{fig:four-nodes-interp} for the shape functions of a 4-nodes element.
\begin{figure}[H]
\centering
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-4-nodes-1.pdf}
		\caption{Shape function ${N^1(x,y)=\frac{1}{4}(1-x)(1-y)}$. }
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-4-nodes-2.pdf}
		\caption{Shape function ${N^2(x,y)=\frac{1}{4}(1+x)(1-y)}$.}
	\end{subfigure}\\
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-4-nodes-3.pdf}
		\caption{Shape function ${N^3(x,y)=\frac{1}{4}(1+x)(1+y)}$.}
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-4-nodes-4.pdf}
		\caption{Shape function ${N^4(x,y)=\frac{1}{4}(1-x)(1+y)}$.}
	\end{subfigure}
\caption{Shape functions for a 4-nodes element.}
\label{fig:four-nodes-interp}
\end{figure}

See Figure \ref{fig:nine-nodes-interp} for the shape functions of a 9-nodes element.
\begin{figure}[H]
\centering
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-9-nodes-1.pdf}
		\caption{Shape function $N^1(x,y)$. }
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-9-nodes-2.pdf}
		\caption{Shape function $N^2(x,y)$.}
	\end{subfigure}\\
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-9-nodes-3.pdf}
		\caption{Shape function $N^3(x,y)$.}
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-9-nodes-4.pdf}
		\caption{Shape function $N^4(x,y)$.}
	\end{subfigure}\\
	%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-9-nodes-5.pdf}
		\caption{Shape function $N^5(x,y)$.}
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-9-nodes-6.pdf}
		\caption{Shape function $N^6(x,y)$.}
	\end{subfigure}
	\caption{Shape functions for a 9-nodes element.}
	\label{fig:nine-nodes-interp}
\end{figure}
%
\begin{figure} [H]
	\ContinuedFloat
	\centering
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-9-nodes-7.pdf}
		\caption{Shape function $N^7(x,y)$.}
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-9-nodes-8.pdf}
		\caption{Shape function $N^8(x,y)$.}
	\end{subfigure}\\
	%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-9-nodes-9.pdf}
		\caption{Shape function $N^9(x,y)$.}
	\end{subfigure}
\caption{Shape functions for a 9-nodes element. (Continued)}
\end{figure}

\section[Discretization of the PVW using FEM]{Discretization of the PVW via the FEM}
\subsection{Formulation of the finite element matrices}
We now discretize the principle of virtual work repeated below for completeness:
\begin{equation} \label{pvw_2}
\intL_V \sigma_{ij} \delta u_{i,j} \dd{V} - \intL_V f_i \delta u_i \dd{V} - \intL_{S_t} t_i^n \delta u_i \dd{S} = 0.
\end{equation}

For that purpose we will divide the complete domain $V$ into $N$-finite non-overlapping sub-domains over each one of which we will approximate the solution in terms of local interpolating functions (see \cref{fig:fully local}). Since the PVW (or weak form of the BVP) has been cast into an integral representation, it is possible to build the total integral considering the contribution of the $N$-sub-domains like:
\begin{equation}\label{pvw_dis}
\sum_{e=1}^{NEL} \intL_{V^e} \sigma_{ij} \var{u}_{i,j} \dd{V^e} - \intL_V f_i \var{u}_i \dd{V^e} - \intL_{S_t} t_i^n \var{u}_i\dd{S^e} = 0
\end{equation}

For simplicity we consider only a single element or sub-domain, thus;

\begin{equation}\label{pvw_sing}
\intL_V \sigma_{ij} \delta\epsilon_{i,j} dV - \intL_V f_i\delta u_idV - \intL_{S_t} t_i^n \delta u_i dS = 0
\end{equation}

where it is assumed that the discretized version of each element is later added up (or assembled) into the global equations for the complete model and where we have already used the fact that $\sigma_{ij}$ is symmetric impliying that $\sigma_{ij} \delta u_{i,j} = \sigma_{ij} \delta\epsilon_{i,j}$.

The involved functions (e.g., displacements, strain, stresses) will be approximated via interpolation of the solution over a determined number of points termed in what follows nodes. Assume for instance that over element $e$ containing $n$ such nodes we know the displacements vector $u_i$. For instance, \cref{fig:simple element} below shows a typical square 4-noded element of side $2h$. The rectangular components of the displacement vector at each node are labeled like $[u_1, u_2,...,u_7, u_8]$.

\begin{figure}[H]
\centering
\includegraphics[width=7cm]{lado2h.pdf}
\caption{Square element of side $2h$.}
\label{fig:simple element}
\end{figure}

Furthermore, in a more general treatment, we let the displacements for an arbitrary $p$-node of a $3D$ problem $u^P=[u^P, v^P, w^P]$. Using ideas from interpolation theory it is now possible to approximate the displacements vector over an arbitrary point $\vb{x}$ inside the element by;

\[u_i(\vb x) = N_i^1(\vb x)u^1 + N_i^2(\vb x)u^2 + \cdots + N_i^P(\vb x)u^P + \cdots + N_i^n(\vb x)u^n\]

or in more general form

\begin{equation} \label{bas_interpol}
{u_i}(\vb x) = N_i^Q(\vb x){u^Q}
\end{equation}

and where the caption superscripts indicate summation over the number of nodes of the element while the subscript refers to the physical character of the variable being interpolated.

Now let us use the kinematic relationshio between strains and displpacements:

\[ \varepsilon_{ij}(\vb x) = \frac{1}{2}( u_{i,j} + u_{j,i} ) \]

together with \cref{bas_interpol}. Carrying the derivatives into the shape functions yields;

\[ \varepsilon_{ij}(\vb x) = \frac{1}{2}\left(\pdv{N_i^Q}{x_j} + \pdv{N_j^Q}{x_i} \right){u^Q} \]

which can be written like

\begin{equation}
\varepsilon_{ij}(\vb x) = B_{ij}^Q(\vb x){u^Q}
\label{str-dis}
\end{equation}

after letting

\[ B_{ij}^Q = \frac{1}{2}\left(\pdv{N_i^Q}{x_j} + \pdv{N_j^Q}{x_i} \right). \]

Proceeding analogously for the virtual fields after using
\[ \delta {u_i} = N_i^Q(\vb x)\delta {u^Q} \]
gives us:
\[ \intL_V C_{ijkl} B_{kl}^P u^P B_{ij}^Q\delta u^Q \dd{V} - \intL_V f_i N_i^Q\delta {u^Q}\dd{V}  - \intL_{S_t} t_i^n N_i^Q\delta {u^Q} \dd{S} = 0 \]
which can be re-organized into
\[ \delta {u^Q}\intL_V B_{ij}^Q C_{ijkl} B_{kl}^P\dd{V}{u^P} - \delta {u^Q}\intL_V N_i^Q{f_i}\dd{V}  - \delta {u^Q}\intL_{S_t} N_i^Qt_i^n\dd{S} = 0\]
which is the generalized discrete version of the PVW for a single element consistent with \cref{pvw_sing}. Defining the terms corresponding to the integrals like:
\begin{equation}
\begin{aligned}
{K^{QP}} & = \int\limits_V {{C_{ijkl}}B_{kl}^PB_{ij}^QdV} \\
f_c^Q    & = \int\limits_S {N_i^Qt_i^ndS} \\
f_V^Q    & = \int\limits_V {N_i^Q{f_i}dV}
\label{Rigi_3}
\end{aligned}
\end{equation}

allows us to write:

\[ \delta u^Q f_\sigma ^Q - \delta {u^Q}f_V^Q - \delta u^Q f_c^Q = 0. \]

This equation is once again the generalized discrete version of the PVW corresponding to the $Q$-displacement degree of freedom. The equation quantifies the work of the forces along the $Q$-th degree of freedom over the virtual displacements $\delta u^Q$. If we now use the arbitrary character of the virtual field $\delta u^Q $ we can write;

\begin{equation}
f_\sigma ^Q - f_V^Q - f_c^Q = 0.
\label{forces}
\end{equation}

This is now a mechanical equilibrium equation relating the forces along the $Q$-th degree of freedom associated to the internal element stresses $f_\sigma ^Q$; the external body forces $f_V^Q$ and the applied external tractions $f_c^Q$. Introducing the stiffness of the element gives:

\begin{equation}
K^{QP} u^P = f_V^Q + f_c^Q.
\label{Discreta}
\end{equation}

\subsubsection{Formulation of the stiffness matrix in perfectly square elements}
In the previous section we recognized:

\begin{equation}
\begin{aligned}
{K^{QP}} & = \int\limits_V {{C_{ijkl}}B_{kl}^PB_{ij}^QdV} \\
f_c^Q    & = \int\limits_S {N_i^Qt_i^ndS} \\
f_V^Q    & = \int\limits_V {N_i^Q{f_i}dV}
\label{Rigi}
\end{aligned}
\end{equation}

as the elemental stiffness matrix and the vectors of consistent contact and body forces for a single element. Let us assume that the domain of the single element is a perfect square of side $2h$ (see \cref{fig:lado2h}), thus;

\begin{equation}
\begin{aligned}
K & = \int\limits_{ - h}^{ + h} {\int\limits_{ - h}^{ + h} {{B^T}CBdxdy} } \\
{f_c} & = \int\limits_{ - h}^{ + h} {{N^T}{t^n}ds} \\
{f_v} & = \int\limits_{ - h}^{ + h} {\int\limits_{ - h}^{ + h} {{N^T}fdxdy} }
\label{ele2}
\end{aligned}
\end{equation}

\begin{figure}[H]
\centering
\includegraphics[width=7cm]{lado2h.pdf}
\caption{Square element of side $2h$.}
\label{fig:lado2h}
\end{figure}

and where $N$ and $B$ are the displacements interpolation matrix and the strain-displacements interpolation matrix respectively. For the linear 4-noded element shown in \cref{fig:lado2h} these matrices have the form defined in the following interpolation equations:

\[\left\{ {\begin{array}{*{20}{c}}
{u(\vec x)}\\
{v(\vec x)}
\end{array}} \right\} = \left[ {\begin{array}{*{20}{c}}
{{N^1}(\vec x)}&0&{{N^2}(\vec x)}&0&{{N^3}(\vec x)}&0&{{N^4}(\vec x)}&0\\
0&{{N^1}(\vec x)}&0&{{N^2}(\vec x)}&0&{{N^3}(\vec x)}&0&{{N^4}(\vec x)}
\end{array}} \right]\left\{ {\begin{array}{*{20}{c}}
{{u^1}}\\
{{u^2}}\\
{{u^3}}\\
{{u^4}}\\
{{u^5}}\\
{{u^6}}\\
{{u^7}}\\
{{u^8}}
\end{array}} \right\}\]

\[\left\{ {\begin{array}{*{20}{c}}
{\frac{{\partial u}}{{\partial x}}}\\
{\frac{{\partial v}}{{\partial y}}}\\
{\frac{{\partial v}}{{\partial x}} + \frac{{\partial u}}{{\partial y}}}
\end{array}} \right\} = \left[ {\begin{array}{*{20}{c}}
{\frac{{\partial {N^1}(\vec x)}}{{\partial x}}}&0&{\frac{{\partial {N^2}(\vec x)}}{{\partial x}}}&0&{\frac{{\partial {N^3}(\vec x)}}{{\partial x}}}&0&{\frac{{\partial {N^4}(\vec x)}}{{\partial x}}}&0\\
0&{\frac{{\partial {N^1}(\vec x)}}{{\partial y}}}&0&{\frac{{\partial {N^2}(\vec x)}}{{\partial y}}}&0&{\frac{{\partial {N^3}(\vec x)}}{{\partial x}}}&0&{\frac{{\partial {N^4}(\vec x)}}{{\partial x}}}\\
{\frac{{\partial {N^1}(\vec x)}}{{\partial x}}}&{\frac{{\partial {N^1}(\vec x)}}{{\partial y}}}&{\frac{{\partial {N^2}(\vec x)}}{{\partial x}}}&{\frac{{\partial {N^2}(\vec x)}}{{\partial y}}}&{\frac{{\partial {N^3}(\vec x)}}{{\partial x}}}&{\frac{{\partial {N^3}(\vec x)}}{{\partial y}}}&{\frac{{\partial {N^4}(\vec x)}}{{\partial x}}}&{\frac{{\partial {N^4}(\vec x)}}{{\partial y}}}
\end{array}} \right]\left\{ {\begin{array}{*{20}{c}}
{{u^1}}\\
{{u^2}}\\
{{u^3}}\\
{{u^4}}\\
{{u^5}}\\
{{u^6}}\\
{{u^7}}\\
{{u^8}}
\end{array}} \right\}\]

with the independent shape functions being:

\begin{align*}
{N^1}(x) & = \frac{1}{4}(1 - x)(1 - y) \\
{N^2}(x) & = \frac{1}{4}(1 + x)(1 - y) \\
{N^3}(x) & = \frac{1}{4}(1 + x)(1 + y) \\
{N^4}(x) & = \frac{1}{4}(1 - x)(1 + y)
\end{align*}

Notice that in the computation of the stiffness matrix we actually require the spatial derivatives of the shape functions given by;

\begin{align*}
\frac{{\partial {N^1}(x)}}{{\partial x}} & =  - \frac{1}{4}(1 - y)           &  \frac{{\partial {N^1}(x)}}{{\partial y}} & =  - \frac{1}{4}(1 - x)\\
\frac{{\partial {N^2}(x)}}{{\partial x}} & =  + \frac{1}{4}(1 - y)           &  \frac{{\partial {N^2}(x)}}{{\partial y}} & =  - \frac{1}{4}(1 + x)\\
\frac{{\partial {N^3}(x)}}{{\partial x}} & =  + \frac{1}{4}(1 + y)           &  \frac{{\partial {N^3}(x)}}{{\partial y}} & =  + \frac{1}{4}(1 + x)\\
\frac{{\partial {N^4}(x)}}{{\partial x}} & =  - \frac{1}{4}(1 + y)           &  \frac{{\partial {N^4}(x)}}{{\partial y}} & =  + \frac{1}{4}(1 - y)
\end{align*}

The contribution to the element matrix from a typical nodal point is thus given by;

\[K = \int\limits_{ - h}^{ + h} {\int\limits_{ - h}^{ + a} {\begin{bmatrix}

 \vdots &  \vdots & \vdots &  \vdots & \vdots & \vdots \\
 \cdots &  \frac{{\partial {N^K}(x)}}{{\partial x}} & 0   & \frac{{\partial {N^K}(x)}}{{\partial y}} &  \cdots & \cdots \\
  \cdots &  0 & \frac{{\partial {N^K}(x)}}{{\partial y}}   & \frac{{\partial {N^K}(x)}}{{\partial x}} &  \cdots & \cdots \\
 \vdots &  \vdots & \vdots &  \vdots & \vdots & \vdots

\end{bmatrix}
%
\begin{bmatrix}
 A & B &  0\\
 B & A & 0 \\
 0 &  0 & C
\end{bmatrix}
%
\begin{bmatrix}
 \cdots & \frac{{\partial {N^Q}(x)}}{{\partial x}} & 0  & \cdots \\
 \cdots & 0 & \frac{{\partial {N^Q}(x)}}{{\partial y}}  & \cdots \\
 \cdots & \frac{{\partial {N^Q}(x)}}{{\partial y}} & \frac{{\partial {N^Q}(x)}}{{\partial x}}  & \cdots
\end{bmatrix} dxdy} } \]

while a typical element of the resultant stiffness matrix takes the general form:
\[K \equiv A\int\limits_{- h}^{ + h} \int\limits_{-h}^{+h} \pdv{N^K}{x}\pdv{N^Q}{x}\dd{x}\dd{y}  + C\int\limits_{-h}^{+h} \int\limits_{-h}^{+h}\pdv{N^K}{y}\pdv{N^Q}{x}\dd{x}\dd{y}.\]
It should be noticed that for the considered perfectly square element the resulting stiffness matrix can be partitioned into factors which depend upon the material properties only and into a single factor which depends on the size $h$. This last contribution can be obtained analytically.

Once the elemental matrices are obtained these are assembled into the final global system of equations. Details of the assembly process are discussed in \autoref{chap: Computational Aspects}. The following Python script computes the stiffness matrix for the 4-noded element.

\inputminted[]{python}{src/stiff_4nodes.py}

\subsubsection{Formulation of the stiffness matrix for distorted elements: the continuum mechanics analogy}
In typical finite element equilibrium equations we need to perform integration over the reference element domain $V_0(\vb{x})$ corresponding to originally arbitrarily shaped sub-domains as created during the meshing process.  In order to proceed with this integration it is useful to consider the following continuum mechanics analogy.

First assume that the actual physical domain $V_0(\vb{x})$ is the result of a deformation process imparted upon the natural domain as shown in \cref{fig:natural domain}. In this analogy, the physical domain $V_0(\vb{x})$ is regarded like a ``deformed'' configuration at an imaginary time $t=t$, while the natural ``undeformed'' domain $V(\vb{r})$   is treated like a reference undeformed configurations at time $t=0$. Both configurations are assumed to be connected through a deformation process
\begin{equation}
\begin{aligned}
\vb{X}&=\vb{X}(\vb{r})\\
\vb{r}&=\vb{r}(\vb{X})
\end{aligned}
\label{eq:motion}
\end{equation}
\begin{figure}[h]
\centering
\includegraphics[width=8cm]{figure1.pdf}
\caption{Definition of the natural domain}
\label{fig:natural domain}
\end{figure}

In \cref{eq:motion} we can understand $\vb{r}$ like a material (Lagrangian) variable and $\vb{X}$ like a spatial (or Eulerian) variable. Using the continuum mechanics analogy it is clear that the ``deformation'' process at the continuum level is fully characterized by the ``deformation'' gradient or Jacobian of the transformation \cref{eq:motion} and defined according to
\begin{equation}
\dd{X}_i=\pdv{X_i}{r_J}\dd{r}_J\equiv J_{iJ}\dd{r}_{J}
\label{eq:gradient}
\end{equation}
where $\dd{r}_{J}$ and $\dd{X}_i$ represent material vectors in the original and deformed configuration. From \cref{eq:gradient} it is evident that the Jacobian contains all the information describing the change of the physical sub-domain with respect to the natural element. For the element integration process we will assume that every element $V(\vb{r})$ in the natural domain deforms into the physical element $V_0(\vb{X})$, thus allowing us to write typical terms like the ones in the material stiffness matrix
\begin{equation}
\intL_{V(\vb{X})} \hat{B}_{ij}^K(\vb{X}) C_{ijkl} \hat{B}_{kl}^P(\vb{X}) dV(\vb{X})\equiv \intL_{V_0(\vb{r})} \hat{B}_{ij}^K(\vb{r}) C_{ijkl} \hat{B}_{kl}^P(\vb{r})J dV_0(\vb{r})
\label{eq:matmatrix}
\end{equation}
where we have used $dV(\vb{X})=JdV(\vb{r})$, with $J$ being the determinant of the deformation gradient and in general we transform functions between the natural and physical space making use of \cref{eq:motion} according to
\begin{equation}
f(\vb{r})=F[\vb{X}(\vb{r})]
\label{eq:funtrans}
\end{equation}

\subsubsection*{Interpolation scheme}
Having identified the fact that the integration process will take place in the natural domain, we will approach the interpolation process directly in this natural space. In the case of the displacement based finite element method all the involved variables will then be obtained via interpolation of nodal displacements. For instance, assume that a given problem variable is defined in the physical space by the tensor $\Phi_{ik...p}(\vb{X})$. The interpolated variable is then written like;

\begin{equation}
\Phi_{ij...p}(\vb{X})=H_{ij...p}^K(\vb{r})\hat{u}^K
\label{eq:interpol}
\end{equation}

where $\hat{u}^K$ represents a vector of nodal points displacements, see \cref{fig:interpol nat dom}, and $H_{ij...p}^K(\vb{r})$ is an interpolator which keeps the tensorial character of the original physical variable $\Phi_{ik...p}(\vb{X})$ and where the super-index makes reference to a nodal identifier (with the summation convention in place).

\begin{figure}[h]
\centering
\includegraphics[width=4cm]{figure2.pdf}
\caption{General interpolation strategy in the natural domain}
\label{fig:interpol nat dom}
\end{figure}

Since the primary variable corresponds to displacements it must be kept in mind that $H_{ij...p}^K(\vb{r})$ corresponds to combinations of derivatives (or other arbitrary combinations) of the basic element shape functions defined in;

\begin{equation}
u_i(\vb{X})=N_i^K(\vb{r})\hat{u}^K
\label{eq:el interpol}
\end{equation}

For the general interpolation process we need two kinds of transformations.  First we need to transform integrals over the physical space into integrals into the natural space which corresponds to
\begin{equation}
\intL_{V(\vb{X})} F(\vb{X})dV(\vb{X})\equiv \intL_{V_0(\vb{r})} f(\vb{r})J dV_0(\vb{r})
\label{gen trans}
\end{equation}

Second we need to relate spatial differentiation in both, the physical and spatial domains.  Let us define these operators like $\nabla_i^X$ and $\nabla_I^r$ respectively. It then follows from \cref{eq:funtrans} that
\begin{equation}
\dfrac{\partial F}{\partial X_i}=\dfrac{\partial f}{\partial r_J}\dfrac{\partial r_J}{\partial X_i}
\label{eq:chain}
\end{equation}
from where we can establish the connection between the two operators like

\begin{equation}
\nabla_i^X=J_{iJ}^{-1}\nabla_J^r
\label{eq:fundamental}
\end{equation}

\subsubsection*{The fundamental interpolator}
We further define the fundamental interpolator giving rise to gradients of the primary displacement variable in the physical space according to
\begin{equation}
u_{i,j}(\vb{X})=L_{ij}^K(\vb{r})\hat{u}^K
\label{eq:fund operator}
\end{equation}

This fundamental interpolator  $L_{ik}^K(\vb{r})$ is derived after using \cref{eq:el interpol} and \cref{eq:fundamental} in the physical displacement gradient definition as shown next
\begin{align*}
u_{i,j}(\vb{X})&=\nabla_j^X u_i(\vb{X})\\
u_{i,j}(\vb{X})&=\nabla_j^X N_i^K(\vb{r})\hat{u}^K\\
u_{i,j}(\vb{X})&=J_{jQ}^{-1}\nabla_Q^r N_i^K(\vb{r})\hat{u}^K\\
u_{i,j}(\vb{X})&=J_{jQ}^{-1}N_{i,Q}^K(\vb{r})\hat{u}^K
\end{align*}
then
\begin{equation}
L_{ij}^K(\vb{r})=J_{jQ}^{-1}N_{i,Q}^K(\vb{r})
\label{eq:fundamental interpolator}
\end{equation}

\subsubsection*{Elemental stiffness matrix}
The elemental material stiffness matrix computed in the natural domain of \cref{fig:Nat domain} reads

\begin{equation}
K^{KP}=\intL_{V_0(\vb{r})} \hat{B}_{ij}^K(\vb{r}) C_{ijkl} \hat{B}_{kl}^P(\vb{r})J dV_0(\vb{r})\equiv \intL_{r=-1}^{r=+1}\intL_{s=-1}^{s=+1} \hat{B}_{ij}^K(r,s) C_{ijkl} \hat{B}_{kl}^P(r,s)J(r,s) \mathrm{d}r\mathrm{d}s
\label{eq:elematrix}
\end{equation}

\begin{figure}[h]
\centering
\includegraphics[width=6cm]{figure3.pdf}
\caption{Natural domain of integration}
\label{fig:Nat domain}
\end{figure}

Once the interpolator $\hat{B}_{ij}^K(\vb{r})$ has been identified the elemental stiffness matrix is obtained via numerical integration (quadrature) as described in \eqref{eq:eleintegration}

\begin{equation}
\intL_{r=-1}^{r=+1}\intL_{s=-1}^{s=+1} \hat{B}_{ij}^K(r,s) C_{ijkl} \hat{B}_{kl}^P(r,s)J(r,s) \mathrm{d}r\mathrm{d}s\approx \sum_{i,j=1}^\text{NGPTS} \alpha_i \alpha_j \hat{B}_{kl}^K(r_i,s_j)C_{ijkl} \hat{B}_{kl}^P(r_i,s_j) J(r_i,s_j)
\label{eq:eleintegration}
\end{equation}

and where NGPTS corresponds to the number of integration points, $\alpha_j$ is a weighting factor and $r_i,s_j$   are the coordinates of a typical point $\vb{r}$ in the natural space of \cref{fig:Nat domain}.

\begin{figure}[h]
\centering
\includegraphics[width=6cm]{figure4.pdf}
\caption{Natural integration domain showing quadrature evaluation nodes}
\label{fig:integration domain}
\end{figure}

One important aspect of the numerical integration that has to be kept in mind is accuracy.  Depending on the particularly selected integration scheme, the number of introduced integration points fixes the maximum polynomial order of the considered functions that can be integrated accurately.  In the case of the integrand in \cref{eq:eleintegration}, it is clear that this order increases as the distortion of the physical element  with respect to the natural element increases.  One way of dealing with this dependency of accuracy with element distortion is to make use of adaptative integration techniques which are numerically expensive.  What is actually done in standard FEM analysis is to choose the number of quadrature points beforehand and introduce distortion related error criteria inside the code in such a way that some sort of validation is performed before the numerical integration process is started.

\subsubsection*{Strain displacement interpolator for the infinitesimal strain tensor}
The $Q$-th nodal contribution to the infinitesimal strain-displacement interpolator can be obtained in explicit form as follows. Let $L_x^Q$ and $L_y^Q$ be the spatial differential operators in $x$ and $y$ respectively. We have after expanding \cref{eq:fundamental interpolator}
%
\begin{align*}
L_x^Q & = J_{xP}^{-1}\frac{\partial N^Q}{\partial r_P} \equiv J_{xr}^{-1}\frac{\partial N^Q}{\partial r} + J_{xs}^{-1}\frac{\partial N^Q}{\partial s}\\
L_y^Q & = J_{yP}^{-1}\frac{\partial N^Q}{\partial r_P} \equiv J_{yr}^{-1}\frac{\partial N^Q}{\partial r} + J_{ys}^{-1}\frac{\partial N^Q}{\partial s}
\end{align*}
%
or in matrix form
%
\begin{equation}
\begin{Bmatrix}
L_x^Q\\
L_y^Q
\end{Bmatrix} =
\begin{bmatrix}
J_{xP}^{-1} &J_{xs}^{- 1}\\
J_{yr}^{-1} &J_{ys}^{- 1}
\end{bmatrix}
\begin{Bmatrix}
\frac{\partial N^Q}{\partial r}\\
\frac{\partial N^Q}{\partial s}
\end{Bmatrix}
\end{equation}

The $Q$-th nodal contribution is then assembled as follows;

\begin{equation}
\begin{Bmatrix}
\pdv{u}{x}\\
\pdv{v}{y}\\
\pdv{u}{y} + \pdv{v}{x}
\end{Bmatrix} =
\begin{bmatrix}
 &L_x^Q &0 \\
\cdots &0 &L_y^Q &\cdots\\
 &L_y^Q &L_{xy}^Q
\end{bmatrix}
\begin{Bmatrix}
\vdots\\
u^Q\\
v^Q\\
\vdots
\end{Bmatrix}
\label{eq:strain inter}
\end{equation}

\begin{algorithm}[H]
    \SetAlgoLined
    \KwData{Nodal coordinates $x^Q$}
    \KwResult{Strain-displacement interpolator $B_{ij}^Q$ }
    Compute Jacobian ${J_{iJ}} = \pdv{N_i^Q}{r_J}{\hat x}^Q$\\
    Invert Jacobian  ${J_{iJ}} \to J_{iJ}^{ - 1}$\\
    Compute fundamental interpolator $L_{ij}^Q = J_{jP}^{ - 1}\pdv{N_i^Q}{r_P}$\\
    Assemble $B_{ij}^Q = \frac{1}{2}\left( {L_{ij}^Q + L_{ji}^Q} \right)$
    \caption{Strain-displacement interpolator}
\end{algorithm}
\newpage
\subsection*{Problems}