\chapter{fMRI model estimation}

Model parameters can be estimated using classical (ReML - Restricted Maximum Likelihood) or Bayesian algorithms. After parameter estimation, the RESULTS button can be used to specify contrasts that will produce Statistical Parametric Maps (SPMs), Effect Size Maps (ESMs) or Posterior Probability Maps (PPMs) and tables of statistics.

\section{Select SPM.mat}

Select the SPM.mat file that contains the design specification. SPM will output the results of its analysis into this directory. This includes overwriting the SPM.mat file. When the estimation job is run, no warning will be given that the SPM.mat file will be overwritten. A warning is given at the specification stage. When it comes to estimation, SPM assumes that you've now sorted out your directory structures.

\begin{figure}
\begin{center}
\includegraphics[width=100mm]{fmri_est/est_method}
\end{center}
\caption{\em After starting SPM in fMRI mode, pressing the `Estimate' button, and then double-clicking on the `+fMRI model estimation' text, the SPM graphics window should appear as above. The options under `-fMRI model estimation' can be examined by clicking on them. A single click will bring up some help text in the lower subwindow (not shown in the above graphic). A double-click on options prefixed by a '+' will allow you to specify options at a greater level of detail. Options highlighted with a `$<$-X' are mandatory and must be filled in by the user. Each of the options shown above is described in this chapter. \label{est}}
\end{figure}

\section{Method}

There are three possible estimation procedures for fMRI models (1) classical (ReML) estimation of first or second level models, (2) Bayesian estimation of first level models and (3) Bayesian estimation of second level models. Option (2) uses a Variational Bayes (VB) algorithm that is new to SPM5. Option (3) uses the Empirical Bayes algorithm with global shrinkage priors that was also in SPM2.

To use option (3) you must have already estimated the model using option (1). That is, for second-level models you must run a ReML estimation before running a Bayesian estimation. This is not necessary for option (2). Bayesian estimation of 1st-level models using VB does not require a prior ReML estimation.

\subsection{Classical}

Model parameters are estimated using Restricted Maximum Likelihood (ReML). This assumes the error correlation structure is the same at each voxel. This correlation can be specified using either an AR(1) or an Independent and Identically Distributed (IID) error model. These options are chosen at the model specification stage. ReML estimation should be applied to spatially smoothed functional images. See \cite{peb1,peb2} for further details of the ReML estimation scheme. After estimation, specific profiles of parameters are tested using a linear compound or contrast with the T or F statistic. The resulting statistical map constitutes an SPM. The SPM{T}/{F} is then characterised in terms of focal or regional differences by assuming that (under the null hypothesis) the components of the SPM (ie. residual fields) behave as smooth stationary Gaussian fields.

The rest of this chapter describes the Bayesian estimation options. So, please skip to the next chapter if you are interested only in classical estimation and inference.

\subsection{Bayesian 1st-level}

Model parameters are estimated using Variational Bayes (VB). This allows you to specify spatial priors for regression coefficients and regularised voxel-wise AR(P) models for fMRI noise processes. The algorithm does not require functional images to be spatially smoothed. Estimation will take about 5 times longer than with the classical approach. This is why VB is not the default estimation option. The VB approach has been described in a number of papers \cite{vb_fmri_ar,vb2,vb3,will_bayes_srglm}.

After estimation, contrasts are used to find regions with effects larger than a user-specified size eg. 1 per cent of the global mean signal. These effects are assessed statistically using a Posterior Probability Map (PPM) \cite{karl_posterior}.

\begin{figure}
\begin{center}
\includegraphics[width=100mm]{fmri_est/bayes_options}
\end{center}
\caption{\em After choosing Bayesian 1st-level under `Method' and then double-clicking on the `+Bayesian 1st-level' text, the SPM graphics window should appear as above. Each of the options shown above is described in this chapter. \label{bayes_options}}
\end{figure}

\subsubsection{Analysis Space}

Because estimation can be time consuming, an option is provided to analyse selected slices rather than the whole volume.

\paragraph{Volume}

You have selected the Volume option. SPM will analyse fMRI time series in all slices of each volume.

\paragraph{Slices}

Enter Slice Numbers. This can be a single slice or multiple slices. If you select a single slice or only a few slices you must be aware of the interpolation options when, after estimation, displaying the estimated images eg. images of contrasts or AR maps. The default interpolation option may need to be changed to nearest neighbour (NN) (see bottom right hand of graphics window) for your slice maps to be visible.

\subsubsection{Signal priors}

\begin{itemize}

\item{[GMRF] Gaussian Markov Random Field. This spatial prior is the recommended option. Regression coefficients at a given voxel are (softly) constrained to be similar to those at nearby voxels. The strength of this constraint is determined by a spatial precision parameter that is estimated from the data. Different regression coefficients have different spatial precisions allowing each putative experimental effect to have its own spatial regularity. }

\item{[LORETA] Low Resolution Tomography Prior. This spatial prior is very similar to the GMRF prior and is a standard choice for MEG/EEG source localisation algorithms. It does, however, have undesirable edge effects.}

\item{[Global] Global Shrinkage prior. This is not a spatial prior in the sense that regression coefficients are constrained to be similar to neighboring voxels. Instead, the average effect over all voxels (global effect) is assumed to be zero and all regression coefficients are shrunk towards this value in proportion to the prior precision. This is the same prior that is used for Bayesian estimation at the second level (see also \cite{karl_posterior}), except that here the prior precision is estimated separately for each slice. }

\item{[Uninformative] A flat prior. Essentially, no prior information is used. If you select this option then VB reduces to Maximum Likelihood (ML) estimation. This option is useful if, for example, you do not wish to use a spatial prior but wish to take advantage of the voxel-wise AR(P) modelling of noise processes. In this case, you would apply the algorithm to images that have been spatially smoothed. For P=0, ML estimation in turn reduces to Ordinary Least Squares (OLS) estimates, and for P$>$0, ML estimation is equivalent to a weighted least squares (WLS) algorithm but where the weights are different at each voxel. This reflects the different noise correlations at each voxel. }

\end{itemize}

\subsubsection{AR model order}

An AR model order of 3 is the default. Cardiac and respiratory artifacts are periodic in nature and therefore require an AR order of at least 2. In previous work, voxel-wise selection of the optimal model order showed that a value of 3 was the highest order required \cite{vb_fmri_ar}.

Higher model orders have little effect on the estimation time. If you select a model order of zero this corresponds to the assumption that the errors are Independent and Identically Distributed (IID). This AR specification overrides any choices that were made in the model specification stage.

Voxel-wise AR models are fitted separately for each session of data. For each session this therefore produces maps of AR(1), AR(2) etc coefficients in the output directory.

\subsubsection{Noise priors}

There are three noise prior options.

\begin{itemize}

\item{[GMRF] Gaussian Markov Random Field. This is the default option. This spatial prior is the same as that used for the regression coefficients. Spatial precisions are estimated separately for each AR coefficient eg. the AR(1) coefficient over space, AR(2) over space etc.}

\item{[LORETA] Low Resolution Tomography Prior. See comments on LORETA priors for regression coefficients.}

\item{[Tissue-type] This provides an estimation of AR coefficients at each voxel that are biased towards typical values for that tissue type (eg. gray, white, CSF). If you select this option you will need to then select files that contain tissue type maps (see below). These are typically chosen to be Grey Matter, White Matter and CSF images derived from segmentation of registered structural scans.}

\end{itemize}

Previous work has shown that there is significant variation in AR values with tissue type. However, GMRF priors have previously been favoured by Bayesian model comparison \cite{will_bayes_srglm}.

\subsubsection{ANOVA}

Perform 1st or 2nd level Analysis of Variance.

\paragraph{First level}

This is implemented using Bayesian model comparison as described in \cite{will_bayes_srglm}. For example, to test for the main effect of a factor two models are compared, one where the levels are represented using different regressors and one using the same regressor. This therefore requires explicit fitting of several models at each voxel and is computationally demanding (requiring several hours of computation). The recommended option is therefore NO.

To use this option you must have already specified your factorial design during the model specification stage.

\paragraph{Second level}

This option tells SPM to automatically generate the simple contrasts that are necessary to produce the contrast images for a second-level (between-subject) ANOVA. Naturally, these contrasts can also be used to characterise simple effects for each subject.

With the Bayesian estimation option it is recommended that contrasts are computed during the parameter estimation stage (see 'simple contrasts' below). The recommended option here is therefore YES.

To use this option you must have already specified your factorial design during the model specification stage.

If you wish to use these contrast images for a second-level analysis then you will need to spatially smooth them to take into account between-subject differences in functional anatomy ie. the fact that one persons V5 may be in a different position than anothers.

\subsubsection{Simple contrasts}

`Simple' contrasts refers to a contrast that spans one-dimension ie. to assess an effect that is increasing or decreasing.

If you have a factorial design then the contrasts needed to generate the contrast images for a 2nd-level ANOVA (or to assess these simple effects within-subject) can be specified automatically using the ANOVA-$>$Second level option.

When using the Bayesian estimation option it is computationally more efficient to compute the contrasts when the parameters are estimated. This is because estimated parameter vectors have potentially different posterior covariance matrices at different voxels and these matrices are not stored. If you compute contrasts post-hoc these matrices must be recomputed. This uses an approximate reconstruction based on a Taylor series expansion described in \cite{vb3}. It is therefore recommended to specify as many contrasts as possible prior to parameter estimation.

If you wish to use these contrast images for a second-level analysis then you will need to spatially smooth them to take into account between-subject differences in functional anatomy ie. the fact that one persons V5 may be in a different position than anothers.

\paragraph{Simple contrast}

\subparagraph{Name}

Name of contrast eg. `Positive Effect'

\subparagraph{Contrast vector}

These contrasts are used to generate PPMs which characterise effect sizes at each voxel. This is different to SPMs in which eg. maps of t-statistics show the ratio of the effect size to effect variability (standard deviation). SPMs are therefore a-dimensional. This is not the case for PPMs as the size of the effect is of primary interest. Some care is therefore needed about the scaling of contrast vectors. For example, if you are interested in the differential effect size averaged over conditions then the contrast $[0.5, 0.5, -0.5, -0.5]$ would be more suitable than the $[1, 1, -1, -1]$ contrast which looks at the differential effect size summed over conditions.

\subsection{Bayesian 2nd-level}

Bayesian estimation of 2nd level models. This option uses the Empirical Bayes algorithm with global shrinkage priors that was previously implemented in SPM2. It is described in detail in \cite{karl_posterior}.

Use of the global shrinkage prior embodies a prior belief that, on average over all voxels, there is no net experimental effect. Some voxels will respond negatively and some positively with a variability determined by the prior precision. This prior precision can be estimated from the data using Empirical Bayes.

\section{Output files}

After estimation a number of files are written to the output directory. These are

\begin{itemize}
\item{An \verb!SPM.mat! file containing specification of the design and estimated model parameters}
\end{itemize}

\subsection{Classical 1st-level}

For classical 1st-level models the following files are also produced

\begin{itemize}

\item{Images of estimated regression coefficients  \verb!beta_000k.img! where $k$ indexes the $k$th regression coefficient.}

\item{An image of the variance of the error \verb!ResMS.img!.}

\item{An image \verb!mask.img! indicating which voxels were included in the analysis.}

\item{The image \verb!RPV.img!, the estimated resels per voxel.}

\item{If contrasts have been specified SPM also writes \verb!con_000i.img!  if the $i$th contrast is a t-contrast and the extra sum of squares image \verb!ess_000i.img! if it is an F-contrast.}

\end{itemize}

Type \verb!help spm_spm! at the matlab command prompt for further information.

\subsection{Bayesian 1st-level}

For Bayesian 1st-level models the following files are also produced

\begin{itemize}

\item{Images of estimated regression coefficients  \verb!Cbeta_000k.img! where $k$ indexes the $k$th regression coefficient. These filenames are prefixed with a `C' indicating that these are the mean values of the `Conditional' or `Posterior' density.}

\item{Images of error bars/standard deviations on the regression coefficients \verb!SDbeta_000k.img!.}

\item{An image of the standard deviation of the error \verb!Sess1_SDerror.img!.}

\item{An image \verb!mask.img! indicating which voxels were included in the analysis.}

\item{If a non-zero AR model order is specified then SPM also writes images \verb!Sess1_AR_000p.img! where $p$ indexes the $p$th AR coefficient.}

\item{If contrasts have been specified SPM also writes \verb!con_000i.img! and \verb!con_sd_000i.img! which are the mean and standard deviation of the $i$th pre-defined contrast.}

\end{itemize}

Each of these images can be inspected using the `Display' button. Type \verb!help spm_spm_vb! at the matlab command prompt for further information.

\section{Model comparison}

Once you have estimated a model you can use SPM's results button to look at the results. You can also extract fMRI data from regions of interest using the ROI button. You can then compare GLMs based on different hemodynamic basis sets using the Bayesian model evidence.

This is described in \cite{will_bayes_srglm} and implemented using the command line option `spm\_vb\_roi\_basis'. This requires a VOI filename (created using the ROI button) and an SPM data structure. Type `help spm\_vb\_roi\_basis' at the matlab command prompt for further information. Figure~\ref{basis} shows an example output from the function indicating that, for the data in this brain region, an informed basis set has the highest model evidence.

\begin{figure}
\begin{center}
\includegraphics[width=150mm]{fmri_est/basis}
\end{center}
\caption{\em This plot shows the model evidence for a number of different hemodynamic basis sets: Inf1 - Canonical HRF, Inf2 - Canonical plus temporal derivative, Inf3 - Canonical plus temporal and dispersion derivatives, F - Fourier, FH - Fourier with a Hanning Window, Gamm3 - 3 Gamma basis functions and FIR - a Finite Impulse Response function. An informed basis set provides the best model of the data for the selected region.  \label{basis}}
\end{figure}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{pgfplots}
\usepackage{fancyhdr}
\usepackage{enumitem}
\usepackage{tikz}
\usepackage{xparse}
\usepackage{siunitx}

\pgfplotsset{width=10cm,compat=1.9}

\pagestyle{fancy}
\fancyhf{}
\lhead{Steven Glasford}
\chead{Homework 1.4}
\rhead{Page \thepage}

\title{Homework 1.4}
\author{Steven Glasford}
\date{\parbox{\linewidth}{\centering%
    \today\endgraf\medskip
    Math-451-M001}}
\newcommand{\rpm}{\sbox0{$1$}\sbox2{$\scriptstyle\pm$}
  \raise\dimexpr(\ht0-\ht2)/2\relax\box2 }

\newlist{steps}{enumerate}{1}
\setlist[steps, 1]{label = Step \arabic*:}
\ExplSyntaxOn
\newcommand*{\prlen}[1]{%
   % round to 1 digit:
    \pgfmathparse{round(10)/10.0}%
    %\pgfkeys{/pgf/number format/precision=1}
    %\pgfmathresult
    \pgfmathprintnumber[fixed, precision=2]{\pgfmathresult}
}
\ExplSyntaxOff

\begin{document}

\maketitle

I choose to do problems 3 and 9 from the options of 3, 4, 5 and 9.

\section{Problem 3}

\begin{enumerate}[label=\alph*]
    \item
    \begin{tikzpicture}
        \begin{axis}[
            axis lines = left,
            xlabel = Time in $t$ day,
            ylabel = Price per pig $P(t)$,
        ]
        \addplot [
            domain=0:15,
            samples=100,
            color=red,
        ]{.65 - .01*x + .00004*x^2};
        \addlegendentry{New: $.65-.01t+.00004t^2$}

        \addplot[
            domain=0:15,
            samples=100,
            color=blue,
        ]{.65-.01*x};
        \addlegendentry{Old: $.65-.01*x$}

        \end{axis}
    \end{tikzpicture}
    The graph above shows the price per pig using both the old linear model (blue) and the newer model (the red line). The red line is more accurate to the actual price per pig in real life. But as is obvious in the graph the values for the old equation are very similar to the more accurate model within the appropriate time domain.
    \item
        \begin{center}
            \begin{tabular}{ |c|c|c| }
            \hline
            Variables & Constants & Assumptions \\
            \hline
            $t$ - Time (Days) & $w_0 = 200$ initial weight (pounds) & $w=w_0+5t$ \\
            $f$ - profit (in dollars) &  & $p=.65-.01t+.00004t^2$ \\
            $w$ - Weight (pounds) && $c=.45t$ \\
            $p$ - Price per pound && $r=p*w$\\
            $c$ - cost (in dollars) && $f=r-c$\\
            $r$ - Revenue (in dollars) &&\\
            \hline
            \end{tabular}
        \end{center}
        If we want to maximize $f(t)$ then we should try to add all of the assumptions into a single equation, this way all of the assumptions are considered. We will try to use $t$ as the isolating factor. If $$f=r-c$$ \parbox{\linewidth}{\centering%
            $r=pw$ \hspace*{3cm} $c=.45t$ \endgraf \bigskip
            $f=pw-.45t$ \endgraf\bigskip
            $p=.65-.01t$ \hspace*{3cm}
            $w=w_0+5*t$ \endgraf \bigskip
            $f=(.65-.01t+.00004t^2)(w_0+5t)-.45t$ \endgraf\bigskip
            $w_0=200$ \endgraf \bigskip
            $f=(.65-.01t+.00004t^2)(200+5t)-.45t$ \endgraf\bigskip
        }
        Which simplifies to:$$f(t)=.0002t^3-.042t^2+.8t+130$$
        Now we need to take the derivative of $f(t)$ and find where it is equal to zero. $$\frac{df}{dt}=\frac{3t^2}{5000}-\frac{21t}{250}+\frac{4}{5}$$

        \begin{tikzpicture}
        \begin{axis}[
            axis lines = left,
            xlabel = Time in $t$ day,
            ylabel = $y$,
        ]

        \addplot[
            domain=0:20,
            samples=100,
            color=blue,
        ]{(3*x^2-420*x+4000)/5000};
        \addlegendentry{$\frac{df}{dt}$}
        \draw[ultra thin] (axis cs:\pgfkeysvalueof{/pgfplots/xmin},0) -- (axis cs:\pgfkeysvalueof{/pgfplots/xmax},0);
        \end{axis}

    \end{tikzpicture}
    \endgraf
    Using the quadratic equation we try to determine the extreme somewhere between 8 and 14, as that point will be a maximum, this is known since the derivative is a quadratic and that point the derivative goes from positive to negative, indicating a maxima. The other root would in turn be a minima.
    $$\frac{df}{dt}=0=\frac{3t^2-420t+4000}{5000}$$ $$t=\frac{10(21+\sqrt{321})}{3},\frac{10(21-\sqrt{321})}{3}$$
    $$t \approx \pgfmathparse{(10*(21+321^(1/2)))/3}\pgfmathresult , \pgfmathparse{(10*(21-321^(1/2)))/3}\pgfmathresult $$
    \textbf{Therefore, the best day to sell your pig is at about 10 days.}

    \item \underline{Sensitivity analysis}
    \endgraf
    $$p(t)=.65-.01t+.00004t^2$$
    $$f(t)=.0002t^3-.042t^2+.8t+130$$
    $$y=(.65-.01x+xt^2)(200+5x)-.45x$$
    $$y=5rx^3-.05x^2+200rx^2+.8x+130=0$$
    $$\frac{dy}{dx}=15rx^2+400rx-\frac{x}{10}+\frac{4}{5}=0$$
    $$x=\frac{-4000r+1+ 10\sqrt{160000r^2-128r+1/100}}{300r}$$
    $$\frac{dx}{dr}=\frac{-6400r+1+ \sqrt{16000000r^2-12800r+1}}{300r^2\sqrt{16000000r^2-12800r+1}}$$
    \begin{center}
            \begin{tabular}{ |c|c|c| }
            \hline
            r & x & $\frac{\Delta x}{\Delta r} * 100$ \\
            \hline
            .00002 & 297.709385518&
            %\pgfkeys{/pgf/fpu} \pgfmathparse{}\pgfmathresult
            %\edef\tmp{\pgfmathresult}
            %\pgfmathresult
            %\pgfkeys{/pgf/fpu=false} &
            %1/(-6400*.00004+1+sqrt(16000000*.00004^2-128*.00004+1))/(300*.00004^2*sqrt(16000000*.00004^2-12800*.00004+1))
            %(-6400*r+1+\sqrt{16000000*r^2-128*r+1})/(300*r^2*\sqrt{16000000*r^2-12800r+1})

            \\
            .00003 & 185.997481072  & -1117119044.46
            %\pgfmathparse{((-4000*.00003+1+10*(160000*.00003^2-128*.00003+1/100)^(1/2))/(300*.00003)-(-4000*.00002+1+10*(160000*.00002^2-128*.00002+1/100)^(1/2))/(300*.00002))/(.00003-.00002)*100}\pgfmathresult &
            \\
            .00004 &129.721576224&-562759048.485
           % \pgfmathparse{(-4000*.00004+1+10*(160000*.00004^2-128*.00004+1/100)^(1/2))/(300*.00004)}\pgfmathresult &
           % \pgfmathparse{(-4000*.00004+1-10*(160000*.00004^2-128*.00004+1/100)^(1/2))/(300*.00004)}\pgfmathresult &
           % \pgfmathparse{((-4000*.00004+1+10*(160000*.00004^2-128*.00004+1/100)^(1/2))/(300*.00004)-(-4000*.00003+1+10*(160000*.00003^2-128*.00003+1/100)^(1/2))/(300*.00003))/(.00004-.00003)*100}\pgfmathresult &
           % \pgfmathparse{((-4000*.00004+1-10*(160000*.00004^2-128*.00004+1/100)^(1/2))/(300*.00004)-(-4000*.00003+1-10*(160000*.00003^2-128*.00003+1/100)^(1/2))/(300*.00003))/(.00004-.00003)*100}\pgfmathresult
            \\
            .00005 & 95.4970354689 &-342245407.55
            %\pgfmathparse{(-4000*.00005+1+10*(160000*.00005^2-128*.00005+1/100)^(1/2))/(300*.00005)}\pgfmathresult &
           % \pgfmathparse{(-4000*.00005+1-10*(160000*.00005^2-128*.00005+1/100)^(1/2))/(300*.00005)}\pgfmathresult &
           % \pgfmathparse{((-4000*.00005+1+10*(160000*.00005^2-128*.00005+1/100)^(1/2))/(300*.00005)-(-4000*.00004+1+10*(160000*.00004^2-128*.00004+1/100)^(1/2))/(300*.00004))/(.00005-.00004)*100}\pgfmathresult&
           % \pgfmathparse{((-4000*.00005+1-10*(160000*.00005^2-128*.00005+1/100)^(1/2))/(300*.00005)-(-4000*.00004+1-10*(160000*.00004^2-128*.00004+1/100)^(1/2))/(300*.00004))/(.00005-.00004)*100}\pgfmathresult
            \\
            .00006 &72.1191645491&-233778709.199
            %\pgfmathparse{(-4000*.00006+1+10*(160000*.00006^2-128*.00006+1/100)^(1/2))/(300*.00006)}\pgfmathresult &
           % \pgfmathparse{(-4000*.00006+1-10*(160000*.00006^2-128*.00006+1/100)^(1/2))/(300*.00006)}\pgfmathresult &
           % \pgfmathparse{((-4000*.00006+1+10*(160000*.00006^2-128*.00006+1/100)^(1/2))/(300*.00006)-(-4000*.00005+1+10*(160000*.00005^2-128*.00005+1/100)^(1/2))/(300*.00005))/(.00006-.00005)*100}\pgfmathresult&
           % \pgfmathparse{((-4000*.00006+1-10*(160000*.00006^2-128*.00006+1/100)^(1/2))/(300*.00006)-(-4000*.00005+1-10*(160000*.00005^2-128*.00005+1/100)^(1/2))/(300*.00005))/(.00006-.00005)*100}\pgfmathresult
            \\
            \hline
            \end{tabular}
    \end{center}
    \textbf{Therefore the best sort of sensitivity is somewhere between .00004 and .00005}

    %% \pgfmathparse{(10*(21-321^(1/2)))/3}\pgfmathresult
    \item The robustness is fairly good since the values obtained from both the linear and the quadratic models are roughly similar.

\end{enumerate}

\section{Problem 9}

\begin{enumerate}[label=\alph*]
    \endgraf
    \item\endgraf\bigskip

        \begin{steps}
            \endgraf
            \item \emph{Ask the Question, determine variables, constants, assumptions}\endgraf
            \begin{center}
                \begin{tabular}{ |c|c|c|c| }
                \hline
                Variables & Constants & Assumptions \\
                \hline
                $s$ - Subscribers & $i_0 = 1.50$ & $f=sp$\\
                $p$ - Subscription price & $s_0=80000$&$s=s_0-50000(p-p_0)$\\
                $f$ - Profits &&$p\geq0$\\
                && $s\geq0$\\

            \hline
            \end{tabular}
        \end{center}
            \item \emph{Select the model}\endgraf
            One-variable optimization
            \item \emph{Formulate the model} \endgraf
            $$f=sp$$
            $$s=s_0-50000(p-p_0)$$
            $$f=p(s_0-50000(p-p_0))$$
            \parbox{\linewidth}{\centering%
            $s_0=80000$ \hspace*{3cm} $p_0=1.50$ \endgraf \bigskip
            }
            $$f=p(80000-50000(p-1.5))$$
            or simplified
            $$f=-50000p^2+155000p$$
            \item \emph{Solve the model}\endgraf
            Find the extrema for f:
            $$\frac{df}{dp}=0=-100000p+155000$$
            $$p=1.55$$
            Determine if 1.55 is a max or min:
            \begin{tikzpicture}
                \begin{axis}[
                axis lines = left,
                xlabel = Price per paper,
            ]
            \addplot [
                domain=0:2,
                samples=100,
                color=red,
            ]{-100000*x+155000};
            \addlegendentry{$\frac{df}{dp}$}
            \draw[ultra thin] (axis cs:\pgfkeysvalueof{/pgfplots/xmin},0) -- (axis cs:\pgfkeysvalueof{/pgfplots/xmax},0);
        \end{axis}
    \end{tikzpicture}
        Since the derivative of $f$ is going from positive to negative at the point 1.55, \textbf{$p=1.55$ is a Maximum.}\bigskip
            \item \emph{Answer the question}\endgraf
            The best price to sell the newspaper is \$1.55 for a total of \$$120125$ per week, and a total of $77500$ subscribers.
        \end{steps}

    %% \pgfmathparse{(10*(21-321^(1/2)))/3}\pgfmathresult

    \item \endgraf \bigskip
        \emph{What if the newspaper lost a different number of subscribers, other than 5000?}
        $$f=p(80000-r*10(p-1.5))$$
        $$\frac{df}{dp}=0=-20rp+15r+80000$$
        $$p=\frac{80000+15r}{20r}$$
        replace $p$ in the $f$ equation:
        $$f=\frac{80000+15r}{20r}\left(80000-10r\left(\frac{80000+15r}{20r}-1.5\right)\right)$$
        Which doesn't really simplify down to anything worth mentioning.
        \begin{center}
                \begin{tabular}{ |c|c| }
                \hline
                Lost Subscribers ($r$) & Max Profit \\
                \hline
                3000 &\$130208.33\\
                4000 &\$122500.00\\
                5000 &\$120125.00\\
                6000 &\$120416.67\\
                7000 &\$122232.14\\

            \hline
        \end{tabular}
    \end{center}
    \item
    $$f=\frac{80000+15r}{20r}\left(80000-10r\left(\frac{80000+15r}{20r}-1.5\right)\right)$$
    $$\frac{df}{dr}=\frac{45r^2-1280000000}{8r^2}$$
    Sensitivity=$\frac{df}{dr}*\frac{r}{x}=S(p,n)$
    $$S(p,n)=\frac{45n^2-1280000000}{8n^2}\left(n/\left(\left(\frac{80000+15n}{20n}\left(80000-10n\left(\frac{80000+15n}{20n}-1.5\right)\right)\right)\right)\right)$$
    \item \emph{The Newspaper shouldn't need to change its prices.} The price is already near the optimal rate, but the optimal rate is just \$125 more than the basic, which is basically nothing when considering the amount of profit they received.

\end{enumerate}

\end{document}


\documentclass[a4paper]{article}

\usepackage{amsmath, amssymb}
\usepackage{circuitikz}
\usepackage{csvsimple}
\usepackage{pgfplots}
\usepackage{siunitx}

\title{Analyzing Circuit Resistance with Various Lengths of Nichrome Wire}
\date{1 December 2014}
\author{Tarik Onalan}

\def\mean#1{\left< #1 \right>}
\def\abs#1{\left| #1 \right|}
\numberwithin{equation}{subsection}

\begin{filecontents}{data.dat}
    Len  Len_Inv T_1   T_2   T_3   T_4   T_5   Avg   Err   Len_Err
    12.2 0.0820  0.620 0.621 0.619 0.622 0.622 0.621 0.002 0.05
    18.6 0.0538  0.406 0.430 0.423 0.422 0.423 0.421 0.017 0.05
    31.6 0.0316  0.232 0.231 0.229 0.234 0.233 0.232 0.003 0.05
    42.8 0.0234  0.210 0.208 0.206 0.206 0.208 0.208 0.002 0.05
    48.6 0.0206  0.182 0.181 0.180 0.179 0.181 0.181 0.002 0.05
\end{filecontents}

\begin{document}
    \maketitle
    \section{Introduction}
        \textbf{How does the length of nichrome wire in a circuit affect the current
        of the circuit?}\\
        The goal of this lab was to analyze how varying lengths of a resistor, nichrome
        wire, affected current flow in a circuit. My hypothesis is that as resistor
        length increases, current flow within the circuit will decrease, as the more
        resistance there is in a circuit, the lower the current will be. For the experiment,
        increasing lengths of nichrome wire will be inserted into a circuit; if the
        current in the circuit decreases as the length of the nichrome wire increases,
        my hypothesis will be validated. The manipulated and responding variables are
        the length of the nichrome wire and the current through the circuit, respectively.
        The controlled variables are the battery used (for power level), the voltage of
        the battery, the length of the low-resistance wire, the ammeter used, and the
        sensitivity setting used on the ammeter.

    \section{Materials}
        \begin{itemize}
            \item 1 Ammeter
            \item 1 Battery
            \item 1 Section of Low-Resistance Wire
            \item 5 Sections of Nichrome Wire (Varying lengths)
        \end{itemize}

    \section{Procedure}
        \begin{enumerate}
            \item Set up ammeter with battery and low-resistance wire:
            \begin{itemize}
                \item Positive terminal of battery connected to positive terminal
                    of ammeter
                \item Negative terminal of battery connected to negative terminal
                    of ammeter
                \item Low-resistance wire attached to one of two terminals, other
                    terminal reserved for nichrome wire
                \begin{itemize}
                    \item Note: Use \SI{1}{\A} sensitivity instead of \SI{5}{\A}
                        sensitivity
                \end{itemize}
            \end{itemize}
            \item Attach ends of nichrome wire to open terminals of battery and ammeter
            \begin{itemize}
                \item For simplicity, start with the shortest length of nichrome wire
            \end{itemize}
            \item Record amperage shown on ammeter
            \item Repeat steps 2-3 as necessary for data averaging
            \item Repeat steps 2-4 with increasing nichrome wire lengths
        \end{enumerate}

    \section{Diagram}
        \begin{circuitikz}
            \draw
                (0,0) to [battery]  (0,4)
                      to [ammeter]  (4,4) -- (4,0)
                      to [vR] (0,0)
                ;
        \end{circuitikz}

    \section{Data}
        \begin{tabular}{|c|||c|c|c|c|c||c|}
            \hline
            \bfseries Length & \bfseries Trial 1 & \bfseries Trial 2 & \bfseries Trial 3 & \bfseries Trial 4 & \bfseries Trial 5 & \bfseries Average
            \\\hline
            \SI{12.2}{\cm} & \SI{0.620}{\A} & \SI{0.621}{\A} & \SI{0.619}{\A} & \SI{0.622}{\A} & \SI{0.622}{\A} & \SI{0.621}{\A}
            \\\hline
            \SI{18.6}{\cm} & \SI{0.406}{\A} & \SI{0.430}{\A} & \SI{0.423}{\A} & \SI{0.422}{\A} & \SI{0.423}{\A} & \SI{0.421}{\A}
            \\\hline
            \SI{31.6}{\cm} & \SI{0.232}{\A} & \SI{0.231}{\A} & \SI{0.229}{\A} & \SI{0.234}{\A} & \SI{0.233}{\A} & \SI{0.232}{\A}
            \\\hline
            \SI{42.8}{\cm} & \SI{0.210}{\A} & \SI{0.208}{\A} & \SI{0.206}{\A} & \SI{0.206}{\A} & \SI{0.208}{\A} & \SI{0.208}{\A}
            \\\hline
            \SI{48.6}{\cm} & \SI{0.182}{\A} & \SI{0.181}{\A} & \SI{0.180}{\A} & \SI{0.179}{\A} & \SI{0.181}{\A} & \SI{0.181}{\A}
            \\\hline
        \end{tabular}
        \\
        \begin{tabular}{|c|c|}
            \hline
            \bfseries Length Error & \bfseries Amperage Error
            \\\hline
            0.05 & 0.002
            \\\hline
            --- & 0.017
            \\\hline
            --- & 0.003
            \\\hline
            --- & 0.002
            \\\hline
            --- & 0.002
            \\\hline
        \end{tabular}
        \\
        \begin{tikzpicture}[trim axis left, trim axis right]
            \begin{axis}[
                scale=1.75,
                title={Current Relative to Resistor Length},
                xlabel={Length [\si{\cm}]},
                ylabel={Current [\si{\A}]},
                xmin=0.0, xmax=50.0,
                ymin=0.0, ymax=0.7,
                legend pos=north east,
                ymajorgrids=true,
                grid style=dashed
            ]
                \addplot[
                    color=blue,
                    mark=*,
                    only marks
                ] plot [
                    error bars/.cd,
                        x dir=both,
                        y dir=both,
                        x explicit,
                        y explicit
                ] table [
                    x=Len,
                    y=Avg,
                    x error=Len_Err,
                    y error=Err
                ]{data.dat};

                \addplot[
                    color=red,
                    mark=none,
                    domain=0:50
                ]{6.28197/x^0.926264};

                \addlegendentry{Average}
                \addlegendentry{\(6.28197x^{-0.926264}\)}
            \end{axis}
        \end{tikzpicture}
        \\
        \begin{tikzpicture}
            \begin{axis}[
                scale=1.75,
                title={Current Relative to Inverse of Resistor Length},
                xlabel={\(\frac{1}{Length [\si{\cm}]}\)},
                ylabel={Current [\si{\A}]},
                xmin=0.0, xmax=0.1,
                ymin=0.0, ymax=0.7,
                x dir=reverse,
                legend pos=north east,
                ymajorgrids=true,
                grid style=dashed
            ]
                \addplot[
                    color=blue,
                    mark=*,
                    only marks
                ] plot [
                    error bars/.cd,
                    x dir=both,
                    y dir=both,
                    x explicit,
                    y explicit
                ] table [
                    x=Len_Inv,
                    y=Avg,
                    y error=Err
                ]{data.dat};

                \addplot[
                    color=red,
                    mark=none,
                    domain=0:0.1
                ]{7.2379*x+0.0265815};

                \addlegendentry{Average}
                \addlegendentry{\(7.2379x+0.0265815\)}
            \end{axis}
        \end{tikzpicture}

        \section{Calculations}
            \centerline{For below calculations, \(t_{i}=t_{1}\)}
            \subsection{Average}
                \begin{equation} \label{eq:avg}
                    t_{avg}=\mean{t_{i}}
                \end{equation}
                \centerline{where \(t_{i}\) is the set of results for wire \(i\).
                Therefore,}
                \begin{equation}
                    t_{avg}=\mean{[0.620,0.621,0.619,0.622,0.622]}=0.621
                \end{equation}

            \subsection{Error}
                \begin{equation} \label{eq:err}
                    t_{err}=\pm\abs{t_{avg}-t_{E}}
                \end{equation}
                \centerline{where \(t_{E}\) is the element of set \(t_{i}\) with
                the highest deviation from \(t_{avg}\). Therefore,}
                \begin{equation}
                    t_{err}=0.621-0.619=\pm0.002
                \end{equation}

        \section{Conclusion}
            It can be observed through the data that the current through the example
            circuit decreased as the length of the nichrome wire increased. When
            the nichrome wire was \SI{12.2}{\cm} long, the average current was
            \SI{0.621}{\A}. However, when the length increased to \SI{18.6}{\cm},
            the current decreased to \SI{0.421}{\A} This was to be expected, as we
            observed through tests with bulbs that as bulbs were added to the circuit,
            resistance increased, lowering the current of the circuit. In this case,
            however, bulbs are replaced by nichrome wire, and the number of bulbs
            is analogous to the length of the wire. The graphs show the inverse
            relationship between resistor length and current; the best-fit graph
            is nearly equivalent to an inverse function: \(y=ax^{-1}\) and
            \(y=6.28197x^{-0.926264}\). This validates my prediction that current
            would decrease as resistance increased.\\
            One of the major difficulties with the lab was getting an accurate length
            for the nichrome wire. As our group used longer lengths of wire, the wire
            curved more, making accurate measurement of its length more difficult. This
            is slightly reflected in the graph, with more deviation from the best-fit
            line as the length of the wire increased.\\
            The goal of this lab is to understand how resistance affects current flow
            in a circuit. Noting that, the length of nichrome wire in a circuit seems
            less important. Instead, the effective \textit{resistance} of the wire
            seems more important. So instead of using resistors of inconsistent resistance,
            it would be simpler to use resistors of constant resistance, and simply
            add more resistors in series to increase the resistance. This would remove
            the variable of the length of the wire, removing a source of possible error.
\end{document}


\chapter{Conclusions}\label{conclusionsChapter}

This thesis describes LeapGesture, which is the hand gesture recognition library dedicated for Leap Motion Controller.
This library includes modules for static and dynamic gesture recognition and also fingers recognition.
Developed library provides recognition of all types of action gestures.
Static action gestures are supported by static gesture processing module, while the dynamic gesture processing module provides recognition of dynamic action gestures.
Additionally, comparison of existing gesture recognition methods, implementation of additional modules enabling the recording and reviewing of gestures, creation of sample gestures database and performation of tests has been conducted.
%This thesis contains a thorough evaluation of proposed approaches to the gesture recognition.

The results obtained for the static gesture recognition suggest than those gestures are easy to recognize using the SVM with appropriate pre and postprocessing.
The differences between the recognition rates for different feature sets even values reaching up to $13\%$ show that choosing different feature sets can have a major impact on the performance of recognition.
Additionally, it is important not to undermine the influence of the data preprocessing.
Even the usage of the relatively simple median filter allowed to boost the recognition rate by $6\%$.
Although, the authors believe that the proposed parameters of processing module work well in many scenarios, there might be some specific cases that need different parameters or different feature sets to achieve good results.
It is also worth noting, that this and any another approach will not work properly if during the performance of static gestures, hands and fingers are not correctly detected by the Leap Motion.
Therefore it is recommended to always choose gestures that have visible and separated fingers from the sensor's perspective.
For the proposed approach, recognition rate of $99\%$ for five classes of gestures and $85\%$ for ten classes of gestures in the static gesture recognition task were achieved. Satisfactory results of $93\%$ were also obtained for the task of fingers recognition for $15$ of $32$ classes of fingers arrangements.

When it comes to the task of dynamic gesture recognition, Leap Motion might not be the best sensor choice.
Leap Motion provides great accuracy for static hand and fingers, but the data for dynamically moved hands are usually noisy.
The problems arise with short, lost finger tracking or temporal finger occlusions that are hard to detect and cope with on the library-level.
The proposed preprocessing module tries to alleviate those negative impacts, but the preprocessed data is still far from ideal.
Nevertheless the proposed approach with Hidden Markov Models allowed to recognize five classes of dynamic gestures with $80\%$ accuracy.

As part of the further development of the library, the authors intend to improve the efficiency of dynamic gesture recognition by testing other possible feature sets and to develop methods that will allow to preprocessed data derived from Leap Motion more efficiently. Supporting of parameterized gestures is also envisaged.


\section{General concepts using 1D DC resistivity inversion}\label{sec:dc1d}
{\em See cpp/py files in the directory \file{doc/tutorial/code/dc1d}.}
\subsection{Smooth inversion}\label{sec:dc1dsmooth}
{\em Example file \file{dc1dsmooth.cpp}.}\\
Included into the GIMLi library are several electromagnetic 1d forward operators.
For direct current resistivity there is a semi-analytical solution using infinite sums that are approximated by Ghosh filters.
The resulting function calculates the apparent resistivity of any array for a given resistivity and thickness vector.
There are two main parameterisation types:
\begin{itemize}
	\item a fixed parameterisation where only the parameters are varied
	\item a variable parameterisation where parameters and geometry is varied
\end{itemize}

Although for 1d problems the latter is the typical one (resistivity and thickness), we start with the first since it is more general for 2d/3d problems and actually easier.
Accordingly, in the file dc1dmodelling.h/cpp two classes called \cw{DC1dRhoModelling} and \cw{DC1dBlockModelling} are defined.
For the first we first define a thickness vector and create a mesh using the function \cw{createMesh1d}.
The the forward operator is initialized with the data.

\begin{lstlisting}[language=C++]
    RMatrix abmnr; loadMatrixCol( abmnr, dataFile ); //! read data
    RVector ab2 = abmnr[0], mn2 = abmnr[1], rhoa = abmnr[2]; //! 3 columns
    RVector thk( nlay-1, max(ab2) / 2 / ( nlay - 1 ) ); //! const. thickn.
    DC1dRhoModelling f( thk, ab2, mn2 ); //! initialise forward operator
\end{lstlisting}

Note that the mesh generation can also be done automatically using another constructor.
However most applications will read or create a mesh from in application and pass it.

By default, the transformations for data and model are identity transformations.
We initialise two logarithmic transformations for the resistivity and the apparent resistivity by
\begin{lstlisting}[language=C++]
    RTransLog transRho;
    RTransLog transRhoa;
\end{lstlisting}

Alternatively, we could set lower/upper bounds for the resistivity using
\begin{lstlisting}[language=C++]
    RTransLogLU transRho( lowerbound, upperbound);
\end{lstlisting}
Appendix \ref{app:trans} gives an overview on available transformation functions.

Next, the inversion is initialized and a few options are set
\begin{lstlisting}[language=C++]
    RInversion inv( data.rhoa(), f, verbose );
    inv.setTransData( transRhoa );           //! data transform
    inv.setTransModel( transRho );           //! model transform
    inv.setRelativeError( errPerc / 100.0 ); //! constant relative error
\end{lstlisting}

A starting model of constant values (median apparent resistivity) is defined
\begin{lstlisting}[language=C++]
    RVector model( nlay, median( data.rhoa() ) ); //! constant vector
    inv.setModel( model );                        //! starting model
\end{lstlisting}

Finally, the inversion is called and the model is retrieved using \lstinline|model = inv.run();|

A very important parameter is the regularisation parameter $\lambda$ that controls the strength of the smoothness constraints (which are the default constraint for any 1d/2d/3d mesh).
Whereas $w^c$ and $w^m$ are dimensionless and 1 by default, $\lambda$ has, after eq. (\ref{eq:min}), the reciprocal and squared unit of $m$ and can thus have completely different values for different problems\footnote{The regularisation parameter has therefore to be treated logarithmically.}.
However, since often the logarithmic transform is used, the default value of $\lambda=20$ is often a first guess.
Other values are set by
\begin{lstlisting}
    inv.setLambda( lambda ); //! set regularisation parameter
\end{lstlisting}

In order to optimise $\lambda$, the L-curve \citep{guentherruecker06,guentherdiss} can be applied to find a trade-off between data fit and model roughness by setting \lstinline|inv.setOptimizeLambda(true);|.
For synthetic data or field data with well-known errors we can also call \lstinline|model = inv.runChi1();|, which varies $\lambda$ from the starting value such that the data are fitted within noise ($\chi^2=1$).
We created a synthetic model with resistivities of 100(soil)-500(unsaturated)-20(saturated)-1000(bedrock) $\Omega$m and thicknesses of 0.5, 3.5 and 6 meters.
A Schlumberger sounding with AB/2 spacings from 1.0 to 100\,m was simulated and 3\% noise were added.
Data format of the file \file{sond1-100.dat} is the unified data format\footnote{See \url{www.resistivity.net?unidata} for a description.}.

\begin{figure}[htbp]
\includegraphics[width=\textwidth]{sond1-100-3lambdas}\\[-3ex]
~~~a\hfill ~~~b \hfill ~~~c \hfill ~
\caption{Smooth 1d resistivity inversion results for a) $\lambda=200\Rightarrow \chi^2=11.1$/rrms=10.1\%, b) $\lambda=20\Rightarrow \chi^2=1.2$/rrms=3.3\%, and c) $\lambda=2\Rightarrow \chi^2=0.6$/rrms=2.4\%, red-synthetic model, blue-estimated model}\label{fig:dc1d-3lambda}
\end{figure}

Figure~\ref{fig:dc1d-3lambda} shows the inversion result for the three different regularisation parameters 300, 30 and 3.
Whereas the first is over-smoothed, the other are much closer at the reality.
The rightmost figure over-fits the data ($\chi^2=0.3<1$) but is still acceptable.
The L-curve method yields a value of $\lambda=2.7$, which is too low.
However, if we apply the $\chi^2$-optimization we obtain a value of $\lambda=15.2$ and with it the data are neither over- nor under-fitted.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Block inversion}\label{sec:dc1dblock}
{\em Example file \file{dc1dblock.cpp} in the directory \file{doc/tutorial/code/dc1d}.}\\
Alternatively, we might invert for a block model with unknown layer thickness and resistivity.
We change the mesh generation accordingly and use the forward operator \lstinline|DC1dModelling|:
\begin{lstlisting}[language=C++]
    DC1dModelling f( nlay, ab2, mn2 );
\end{lstlisting}

\lstinline|createMesh1DBlock| creates a block model with two regions\footnote{With a second argument \lstinline|createMesh1DBlock| can create a block model with thickness and several parameters for a multi-parameter block inversion.}.
Region 0 contains the thickness vector and region 1 contains the resistivity vector.
There is a region manager as a part of the forward modelling class that administrates the regions.
We first define different transformation functions for thickness and resistivity and associate it to the individual regions.
\begin{lstlisting}[language=C++]
    RTransLog transThk;
    RTransLogLU transRho( lbound, ubound );
    RTransLog transRhoa;
    f.region( 0 )->setTransModel( transThk );
    f.region( 1 )->setTransModel( transRho );
\end{lstlisting}

For block discretisations, the starting model can have a great influence on the results.
We choose the median of the apparent resistivities and a constant thickness derived from the current spread as starting values.
\begin{lstlisting}[language=C++]
    double paraDepth = max( ab2 ) / 3;
    f.region( 0 )->setStartValue( paraDepth / nlay / 2.0 );
    f.region( 1 )->setStartValue( median( rhoa ) );
\end{lstlisting}

For block inversion a scheme after \cite{marquardt}, i.e. a local damping of the changing without interaction of the model parameters and a decreasing regularisation strength is favourable.
\begin{lstlisting}[language=C++]
    inv.setMarquardtScheme( 0.9 ); //! local damping with decreasing lambda
\end{lstlisting}

The latter could also be achieved by \begin{enumerate}
	\item setting the constraint type to zero (damping) by \lstinline|inv.setConstraintType(0)|
	\item switching to local regularization by \lstinline|inv.setLocalRecularization(true)|
	\item defining the lambda decreasing factor by \lstinline|inv.setLambdaDecrease(0.9)|
\end{enumerate}

%The choice of appropriate regularisation parameters is somewhat more complicated since the result strongly depends on the starting model and the preceding models \citep{guentherdiss}.
%The latter can be done by \lstinline|inv.setLambdaDecrease( factor );|

With the default regularization strength $\lambda=20$ we obtain a data fit slightly below the error estimate.
The model (Fig.~\ref{fig:dc1dblock-resres}a) clearly shows the four layers (blue) close to the synthetic model (red).
%Some parameters are overestimated and some are underestimated.

\begin{figure}[htbp]
\centering\includegraphics[width=0.7\textwidth]{dc1dblock-resres}\\[-3ex]
~\hfill a\hfill ~ \hfill ~~~~~b \hfill ~ \hfill ~ \hfill ~
\caption{a) Block 1d resistivity inversion result (red-synthetic model, blue-estimated model)) and b) resolution matrix}\label{fig:dc1dblock-resres}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Resolution analysis}\label{sec:dc1dresolution}
One may now be interested in the resolution properties of the individual model parameters.
The resolution matrix $\R^M$ defines the projection of the real model onto the estimated model:
\begin{equation}
	\m^{est} = \R^M \m^{true} + (\I - \R^M) \m^R + \S^\dagger\D\n\quad,
\end{equation}
\citep{guentherdiss} where $\S^\dagger\D\n$ represents the generalised inverse applied to the noise.
Note that $\m^R$ changes to $\m^k$ for local regularisation schemes \citep{friedel03}.

\citet{guentherdiss} also showed that the model cell resolution (discrete point spread function) can be computed by solving an inverse sub-problem with the corresponding sensitivity distribution instead of the data misfit.
This is implemented in the inversion class by the function \lstinline|modelCellResolution( iModel )| where \lstinline|iModel| is the number of the model cell.
This approach is feasible for bigger higher-dimensional problems and avoids the computation of the whole resolution matrix.
A computation for representative model cells can thus give insight of the resolution properties of different parts of the model.

For the block model we successively compute the whole resolution matrix.
\begin{lstlisting}
    RVector resolution( nModel ); //! create single resolution vector
    RMatrix resM;                 //! create empty matrix
    for ( size_t iModel = 0; iModel < nModel; iModel++ ) {
        resolution = inv.modelCellResolution( iModel );
        resM.push_back( resolution );  //! push back the single vector
    }
    save( resM, "resM" ); //! save resolution matrix
\end{lstlisting}

In Figure~\ref{fig:dc1dblock-resres} the model resolution matrix is shown and the diagonal elements are denoted.
The diagonal elements show that the resolution decreases with depth.
The first resistivity is resolved nearly perfect, whereas the other parameters show deviations from 1.
$\rho_2$ is positively connected with $d_2$, i.e. an increase of resistivity can be compensated by an increased resistivity.
For $\rho_3$ and $d_3$ the correlation is negative.
These are the well known H- and T-equivalences of thin resistors or conductors, respectively, and show the equivalence of possible models that are able to fit the data within noise.
%Similarly, we can obtain the resolution kernels also for smooth inversion.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Structural information}\label{sec:dc1dstruct}
{\em Example file \file{dc1dsmooth-struct.cpp} in the directory \file{doc/tutorial/code/dc1d}.}\\
Assume we know the ground water table at 4\,m from a well.
Although we know nothing about the parameters, this structural information should be incorporated into the model.
We create a thickness vector of constant 0.5\,m.
The first 8 model cells are located above the water table, so the 8th boundary contains the known information.
Therefore we set a marker different from zero (default) after creating the mesh
\begin{lstlisting}
    //! variant 1: set mesh (region) marker
    f.mesh()->boundary( 8 ).setMarker( 1 );
\end{lstlisting}
This causes the boundary between layer 8 and 9 being disregarded, the corresponding $w^c_8$ is zero and allows for arbitrary jumps in the otherwise smoothed model.
Figure~\ref{fig:dc1dsmooth-struct} shows the result, at 4\,m the resistivity jumps from a few hundreds down to almost 10.

\begin{figure}[htbp]
\centering\includegraphics[width=0.45\textwidth]{sond1-100-struct.pdf}
%\\[-3ex]
%~\hfill a\hfill ~ \hfill ~~~~~b \hfill ~ \hfill ~ \hfill ~
\caption{Inversion result with the ground water table at 4\,m as structural constraint.}\label{fig:dc1dsmooth-struct}
\end{figure}

Note that we can set the weight to zero also directly, either as a property of the inversion
\begin{lstlisting}
    //! variant 2: application of a constraint weight vector to inversion
    RVector bc( inv.constraintsCount(), 1.0 );
    bc[ 6 ] = 0.0;
    inv.setCWeight( bc );
\end{lstlisting}
or the (only existing) region.
\begin{lstlisting}
    //! variant 3: application of a boundary control vector to region
    RVector bc( f.regionManager().constraintCount(), 1.0 );
    bc[ 7 ] = 0.0;
    f.region( 0 )->setConstraintsWeight( bc );
\end{lstlisting}
Of course, in 2d/3d inverse problems we do not set the weight by hand.
Instead, we put an additional polygon (2d) or surface (3d) with a marker $\neq 0$ into the PLC before the mesh generation.
By doing so, arbitrary boundaries can be incorporated as known boundaries.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Regions}\label{sec:dc1dregion}
{\em Example file \file{dc1dsmooth-region.cpp} in the directory \file{doc/tutorial/code/dc1d}.}\\
In the latter sections we already used regions.
A default mesh contains a region with number 0.
A block mesh contains a region 0 for the thickness values and regions counting up from 1 for the individual parameters.
Higher dimensional meshes can be created automatically by using region markers, e.g. for specifying different geological units.

In our case we can divide part above and a part below water level.
In 2d or 3d we would, similar to the constraints above, just put a region marker $\neq 0$ into the PLC and the mesh generator will automatically associate this attribute to all cells in the region.

Here we set the markers of the cells $\geq8$ to the region marker 1.
\begin{lstlisting}
    Mesh * mesh = f.mesh();
    mesh->boundary( 8 ).setMarker( 1 );
    for ( size_t i = 8; i < mesh->cellCount(); i++ )
        mesh->cell( i ).setMarker( 1 );
\end{lstlisting}

Now we have two regions that are decoupled automatically.
The inversion result is identical to the one in Figure~\ref{fig:dc1dsmooth-struct}.
However we can now define the properties of each region individually.
For instance, we might know the resistivities to lie between 80 and 800\,$\Omega$m above and between 10 and 100\,$\Omega$m below.
Consequently we define two transformations and apply it to the regions.
\begin{lstlisting}
    RTransLogLU transRho0( 80, 800 );
    RTransLogLU transRho1( 10, 1000 );
    f.region( 0 )->setTransModel( transRho0 );
    f.region( 1 )->setTransModel( transRho1 );
\end{lstlisting}

Additionally we might try to improve the very smooth transition between groundwater and bedrock.
We decrease the model control (strength of smoothness) in the lower region by a factor of 10.
\begin{lstlisting}
    f.region( 1 )->setModelControl( 0.1 );
\end{lstlisting}

The result is shown in Figure~\ref{fig:dc1dsmooth-region}.
The resistivity values are much better due to adding information about the valid ranges.
Furthermore the transition zone in the lower region is clear.
\begin{figure}[htbp]
\centering\includegraphics[width=0.45\textwidth]{sond1-100-region.pdf}
\caption{Inversion result using two regions of individual range constraint transformations and regularization strength (model control).}%
\label{fig:dc1dsmooth-region}
\end{figure}


\subsection{The Keynesian cross and the Investment Saving (IS) curve}

\subsubsection{The Keynesian cross}

We have:

\(Y=C(Y-T(Y))+I(r)+G+NX(Y)\)

Where:

\begin{itemize}
\item \(Y\) is output
\item \(C\) is consumption
\item \(T\) is taxes
\item \(I\) is investment
\item \(r\) is the real interest rate
\item \(G\) is government spending
\item \(NX\) is net exports
\end{itemize}

The Keynesian cross plots:

\(Y\)

Against:

\(C(Y-T(Y))+I(r)+G+NX(Y)\)

This identifies an equilibrium level of output.

\subsubsection{The IS curve}

The IS curve plots the equilibrium level of output from the Keynesian cross against the real interest rate.

As the real interest rate rises, investment and therefore output falls.

\subsubsection{The slope of the IS curve}

The slope of the IS curve depends on taxes and net exports.


\documentclass[twoside, 11pt]{article}

\usepackage{jmlr2e}

\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{mathtools}

% ensure sufficient marginspace for todos
\setlength {\marginparwidth }{2cm}
\usepackage[obeyFinal]{todonotes}
% \setuptodonotes{inline}

% define notation for norm and abs that scale nicely.
% ref: https://tex.stackexchange.com/a/297263
\let\oldnorm\norm
\let\norm\undefined
\DeclarePairedDelimiter\norm{\lVert}{\rVert}

\let\oldabs\abs
\let\abs\undefined
\DeclarePairedDelimiter\abs{\lvert}{\rvert}

\DeclarePairedDelimiter\card{\lvert}{\rvert}

\newcommand{\xx}[0] {\mathbb{X}} % decision variable space
\newcommand{\hh}[0] {\mathbb{H}} % stochastic process for state
\newcommand{\zz}[0] {\mathbb{Z}} % hidden event space?
\newcommand{\mm}[0] {\mathbb{M}} % HMM model type space
\newcommand{\TT}[0] {\mathbb{T}} % time indices
\newcommand{\II}[0] {\mathbb{I}} % factorial model indices
\newcommand{\traj}[1] {H^{(#1)}}
\newcommand{\state}[2] {H_{#2}^{(#1)}}
\newcommand{\event}[2] {Z_{#2}^{(#1)}}
\newcommand{\eventseq}[1] {Z^{(#1)}}
\newcommand{\reals}[0] {\mathbb{R}}
\newcommand{\naturals}[0] {\mathbb{N}}
\newcommand{\events}[0] {\mathbb{Y}}

\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}

\begin{document}

\author{\name Reuben Fletcher-Costin}

\editor{}

\title{Approximate MAP inference of sparse factorial hidden Markov models through set cover decomposition}

\maketitle

\begin{abstract}%
Our goal is to infer a sparse subset of hidden Markov models (HMMs) that collectively explain a sequence of observations. We define a probabilistic model for a simple form of factorial HMMs and pose a maximum a posteriori (MAP) estimation problem to infer a sparse subset of component HMM, their hidden states, and separate the observations into component signals associated with each component HMM. We show that the MAP estimation problem is equivalent to an exact cover problem, where each component HMM is regarded as a set that covers a portion of the observed signal. This permits a relaxed approximation of the problem to be decomposed as a master set cover problem and auxiliary problem using column generation. Each auxiliary problem can be solved efficiently using dynamic programming by a "prize-collecting" modified Viterbi algorithm that recovers a HMM trajectory that incorporates prizes from the master problem for explaining portions of the observed signal.
\end{abstract}

% keywords could go here

\section{Introduction}

\todo[inline]{why care about HMMs}
\todo[inline]{why care about separation problems}
\todo[inline]{discuss factorial HMMs}
\todo[inline]{discuss alternatives to factorial HMMs}
\todo[inline]{intro linear programming decompositions, convex optimisation decompositions}

\section{Probabilistic model}

\subsection{Hidden Markov models}
We start by assuming a family of hidden Markov models, indexed by $m \in \mm$. Each model $m \in \mm$ in the family has a finite state space $S_m = \{s_1, \ldots, s_{K_m}\}$ consisting of $K_m \in \naturals$ states. The state of the model $m$ at time $t \in \TT = \{ 1, \ldots, T \}$ is denoted by the random variable $H_{m, t}$ which takes values in the state space $S_m$. We regard the states as not directly observable and refer to them as hidden (aka latent) states. The hidden states of model $m$ evolve in time independently from other models according to a discrete time first-order Markov process
\begin{equation}
P(H_{m,t+1} \mid \{ H_{m^{\prime},t^{\prime}} \}_{m^{\prime}, t^{\prime} \in \mm \times \TT \setminus \{m, t\}} )
=
P(H_{m,t+1} \mid H_{m,t} )
\end{equation}
We assume that the transition model $P(H_{m,t+1} \mid H_{m,t} )$ for each $m \in \mm$ is stationary, that is, there exists some $K_m$ by $K_m$ stochastic matrix $A^{m}$ with elements $A^{m}_{s^{\prime}, s}$ such that for all $t \in \TT$, $A^{m}_{s^{\prime}, s} = P(H_{m, t+1}=s^{\prime} \mid H_{m,t}=s)$. We write $\pi_m$ to denote a prior distribution over $H_{m,1}$ at $t=1$. In the simple case where we believe a-priori that the $m$th HMM is present \footnote{Once we consider a sparse factorial model composed of multiple component HMMs from the family $\mm$, we may no longer believe a-priori that all or any HMM $m \in \mm$ necessarily participates in the factorial model.}, we could write $P(H_{m,1}) = \pi_m$.

Each hidden Markov model $m$ emits a sequence of signals $Z_{m,1}, \ldots, Z_{m,T}$, where each $Z_{m,t} \in \events$. The space of signals $\events$ is assumed to be a vector space over $\reals$. Each signal $Z_{m,t}$ is assumed to be caused solely by the corresponding hidden state $H_{m,t}$:
\begin{equation}
P\left(Z_{m,t}
\mid
\{ H_{m^{\prime},t^{\prime}} \}_{m^{\prime}, t^{\prime} \in \mm \times \TT} \;
\{ Z_{m^{\prime},t^{\prime}} \}_{m^{\prime}, t^{\prime} \in \mm \times \TT \setminus \{m, t\}}
\right)
=
P(Z_{m,t} \mid H_{m,t} ) .
\end{equation}
We refer to $P(Z_{m, t} \mid H_{m,t} )$ as the observation model for the $m$th model. We assume the observation model for each $m \in \mm$ is stationary with respect to $t$.

\todo{add a clear graphical model of a single HMM for some $m \in \mm$}

\subsection{Factorial hidden Markov models}

We consider factorial hidden Markov models consisting of a multiset of component hidden Markov models, where the possible component HMMs are indexed by $i$ over some abstract index set $\II$. Note that the cardinality of $\II$ may not be finite. We defer explictly constructing $\II$ until section \ref{section:inference}. Let $m(i) \in \mm$ denote the component HMM associated with the $i$th index. We represent a given factorial model as the pair $X, M$, where
\begin{align}
X & := (X_i)_{i \in \II} \quad X_i \in \{0, 1\} , \\
M & := (m(i))_{i \in II} \quad m(i) \in \mm .
\end{align}
We regard $X$ as a collection of random variables over binary values that address a subset of indices from $\II$, and $M$ as a collection of random variables over $\mm$ that associate a particular type of HMM $m(i) \in \mm$ to each selected index. Note this representation of factorial models is not unique to permutations of the indices $\II$.

We restrict our focus to \emph{sparse} factorial models where $\norm{X}_1$ is bounded, that is, where there exists $n \in \naturals$ such that $\sum_{i \in \II} \abs{X_i} \leq n$. We can further express a preference for sparsity through a choice of prior over $X$.

We are interested in estimating a factorial hidden Markov model with components $(X_i)_{i \in \II}$ that best "explains" some given observed data $Y_t \in \events$ for $t \in \TT$. The standard probabilistic model for a single HMM $m \in \mm$ assumes that at each time $t in \TT$ the output signal $Z_{m,t}$ is directly observable. In contrast, with a factorial HMM the output $Z_{m,t}$ of each component HMM $m$ is not directly observable. Instead, we observe a signal $Y_t$ each $t \in \TT$ that aggregates the hidden output signals of all component HMMs:
\begin{equation}
Y_t = \sum_{i \in \II} X_i \event i t ,
\end{equation}
where $\event i t := Z_{m(i), t}$. Note that unlike the definition of the more general factorial hidden Markov model considered by {Ghahramani and Jordan 1997}\todo{CITE} we assume there is no top-level error term and assume a different dependency structure between $Y$ and $Z$. Error terms can be expressed by $\event i t$ through the observation models of the component HMMs.

\todo{add graphical model of this factorial HMM construction using Y X Z M H}

\section{Inference} \label{section:inference}

\todo{rework in terms of index set $\II$. that will allow multiple copies of a single class of markov model to appear in solution. that's fine, as long as they each pay their way by justifying another copy of the log prior in the objective function}

Our goal is to infer a sparse factorial HMM, represented by $X$ and $M$, that explains a sequence of outputs $Y = (Y_t)_{t \in \TT}$. Given the framework of Bayesian inference, this amounts to computing the posterior distribution $P(X, M | Y=y)$ given observed data $y \in \events^{\TT}$. Computing this posterior distribution exactly is computationally challenging as it requires integrating over all the hidden states $\state i t$ and all outputs $\event i t$ for all $t \in \TT$ and all potential component models $i \in \II$. A less useful but more computationally feasible task is to instead compute a maximum a posteriori parameter estimate of the quad $(X, M, H, Z) = (X_i, m(i), \traj i, Z^{(i)})_{i \in \II}$ given the observed data $Y=y$. The elements of the quad define which component models participate in the factorial model ($X$), the type of each component from the family $\mm$ ($M$), the hidden states of each component ($H$), and the hidden outputs emitted by each component ($Z$).

This form of MAP estimate is computationally tractable because the probabilities of each component HMM state $(X_i, Z_i)$ are conditionally independent from $(X_j, Z_j)$ $j \neq i$ given $Z$, allowing conditional probabilities to be calculated independently component by component once some value of $Z$ is fixed. This leads to a tractable decomposition. \todo{explain in terms of causal graph diagram, markov blankets, d separation}

We demonstrate the decomposition more formally. Consider the following factorisation of the conditional probability of the variables $X, H, Z$ given the data $Y$:
\begin{align}
P(X, M, H, Z \mid Y)
& = P(Y \mid X, M, H, Z) P(X, M, H, Z) / P(Y) \label{map1} \\
& \propto P(Y \mid X, M, H, Z) P(X, M, H, Z) \label{map2} \\
& = P(Y \mid X, Z) P(X, M, H, Z) \label{map3} \\
& = P(Y \mid X, Z) P(Z | X, M, H) P(X, M, H) \label{map4} \\
& = P\left(Y \mid X, Z\right) \prod_{i \in \II} P\left(\eventseq i | M_i, \traj i\right)^{X_i} P(X, M, H) \label{map5} \\
& = P\left(Y = \sum_{i \in \II} X_i \eventseq i \mid X, Z \right)
\prod_{i \in \II} P\left(\eventseq i | M_i, \traj i\right)^{X_i} P(X, M, H) \label{map6}
\end{align}
\todo{fixup p a given b bar scaling}
where \ref{map1} uses Bayes' theorem, \ref{map2} drops the factor $P(Y)$ as it is invariant during maximisation over $(X, H, Z)$, \ref{map3} applies the conditional independence of $Y$ from $H$ and $M$ given $X$ and $Z$, \ref{map5} is due to the conditional independence of the component observation models given $Z$, and \ref{map6} applies the definition of the aggregated observation $Y$ from the component hidden outputs.

To complete the decomposition of the conditional probability $P(X, M, H, Z \mid Y)$ in terms the distributions defined by the family of component Markov processes, we need to decide on $P(X, M, H)$, a prior joint distribution over $X$, $M$ and $H$. We assume the prior probabilities of $\traj m$ and $\traj {m^{\prime}}$ are conditionally independent given $X$ for each distinct $m, m^{\prime} \in \mm$. Therefore we have
\begin{align}
P(X, M, H)
& = \prod_{i \in \II} P(\traj i \mid m(i)) P(m(i) \mid X_i ) P(X) \\
& = \prod_{i \in \II} \left( \frac{1}{\card{\mm}} P(\traj i \mid m(i)) \right)^{X_i} P(X) \\
\end{align}
where $P(\traj i \mid m(i))$ is the prior probability of the trajectory $\traj {m(i)} = \pi_{m(i)}$ as supplied by the component HMM. The $\frac{1}{\card{\mm}}$ factor is due to assuming a uniform prior over $m(i)$ that does not prefer any particular model $m$ of the family $\mm$. We tentatively define a prior $P(X)$ over the binary sequence $X = \{X_i\}_{i \in \II} \in \{0, 1\}^{\card{I}}$ in terms of a prior $P(n)$ over the number of active processes $n := \norm{X}_1$ that assigns higher prior probability to explanations involving fewer active Markov processes:
\begin{align}
P(X)
& = P(X \mid n) P(n) \\
& \propto \frac{1}{|\mm|^n \; 2^{n+1}}
\end{align}
where we have omitted a normalisation constant required so that $\sum_{X} P(X) = 1$
\todo{rework, support nonuniform prior on $\mm$ as it is easy and probably quite useful in applications}

\todo{add $M$ into the argmax}

Consider the MAP parameter estimation problem:
\begin{equation}
(X^{\star}, M^{\star}, H^{\star}, Z^{\star}) = \argmax_{X, M, H, Z} P\left(X, M, H, Z \mid Y\right)
\end{equation}
Substituting our decomposition of $P(X, M, H, Z | Y)$ and the definition of our prior $P(X, M, H)$ gives
\begin{align*}
& \argmax_{X, M, H, Z} P\left(X, H, Z \mid Y\right) \\
= & \argmax_{X, M, H, Z}
P\left(Y = \sum_{i \in \II} X_i \eventseq i \mid X, Z \right)
\prod_{i \in \mm} P\left(\eventseq i | m(i), \traj i\right)^{X_i} P(X, M, H) \\
= & \argmax_{X, M, H, Z} \prod_{i \in \II} P\left(\eventseq i | m(i), \traj i\right)^{X_i}
\prod_{i \in \II} P(\traj i \mid m(i))^{X_i}
\frac{1}{|\mm|^n \; 2^{n+1}} \\
 & \mathrm{s.t.} \sum_{i \in \II} X_i \eventseq i = Y
\end{align*}
where $n = \norm{X}_1$ and the factor for the conditional probability of the observation vector $Y$ is equivalently expressed by the constraining the max to consider only those quads $(X, M, H, Z)$ such that $\sum_{i \in \II} X_i \eventseq i = Y$.

Since $\argmax$ is invariant under transformation of the objective function by a monotonic function, by the monotonicity of the logarithm we have
\begin{align}
(X^{\star}, M^{\star}, H^{\star}, Z^{\star})
= & \argmax_{X, M, H, Z} \sum_{i \in \II} X_i C(m(i), \eventseq i, \traj i) \\
 & \mathrm{s.t.} \sum_{i \in \II} X_i \eventseq i = Y ,
\end{align}
where $C(m(i), \eventseq i, \traj i)$ is defined by
\begin{equation*}
C(m(i), \eventseq i, \traj i) =
\log P\left(\eventseq i | m(i), \traj i\right) +
\log P(\traj i \mid m(i)) -
\left ( \log |\mm| + \log 2 \right) ,
\end{equation*}
and we have dropped terms from the $\argmax$ that are constant
with respect to $(X, M, H, Z)$.

\todo{explicitly construct the index and show this is a linear program}

\section{Practical considerations}

{bootstrapping initial feasible solution so there is a dual solution that can be used to set initial prizes}\todo{elabourate}

{decomposition of MAP estimation problem into master relaxed exact cover problem and auxiliary prize-collecting Viterbi dynamic programming problem allows many complex problem specific details to be handled as part of the dynamic programming}\todo{elabourate}

{degeneracy / slow convergence of restricted relaxed master linear program}\todo{investigate and mitigate}

{exactly when is the dual solution used to set prizes well defined -- over which range?}\todo{investigate}

{efficient search over particular families of HMMs}\todo{investigate}

{embed into full branch and price regime}\todo{investigate}

{reducing wasted effort by reusing LP bookkeeping and prior soln to warm-start over successive LP solves}\todo{investigate}

\section{Variations}

{maybe hard equality constraint linking Y to Z is not pragmatic. if soften constraint and allow nonzero error $r$ one naive implementation might add factor $k \exp(-\frac{\norm{r}^2}{{\sigma^2}})$ to unlogged objective function. equivalently when objective is logged, this adds some $- \norm{r}^2$ term. seems to give some kind of quadratic optimisation problem. can decomposition still work?}\todo{investigate}

\section{bibliography}
\todo{setup bibtex etc}

\end{document}


\documentclass{article}
\usepackage{tocloft}
\include{common_symbols_and_format}
\renewcommand{\cfttoctitlefont}{\Large\bfseries}

\begin{document}
\logo
\rulename{Exponential Moving Average Trading Rule} %Argument is name of rule
\tblofcontents

\ruledescription{Exponential Moving Average (EMA) is a type of moving average similar to simple moving average, but it reduce the lag by applying more weight to recent prices. The weighting applied varies depending on the number of periods use in the moving average.
                Traders used this more to get a clearer view of the most recent price change of an asset.}

\section{Equation}
\begin{equation}
    EMA_{\currenttime} = \Big(V_{\currenttime} * \Big(\frac{S}{1 + \lookbacklength}\Big)\Big) + EMA_{\currenttime-1} * \Big(1-\frac{S}{1 + \lookbacklength}\Big)
\end{equation}
\\

where: \\

$EMA$ is exponentially weighted moving average. \\

$V_{\currenttime}$ is current stock value. \\

$\lookbacklength$ \ is look back length. \\

$S$ is the smoothing factor.

\ruleparameters
{Window size}{50}{This is the number of time steps over which exponential contributions are sourced.}{$\lookbacklength$}
{Smoothing Factor}{2}{Smoothing factor represents the weighting applied to the most recent period’s value.}{$S$}
\stoptable

\keyterms
\furtherlinks

\end{document}


\documentclass{article}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{geometry}
\usepackage[utf8]{inputenc}
\usepackage{enumitem}
\usepackage{physics}
\setlength{\parindent}{0pt}
\setlength{\parskip}{1em}
\relpenalty=10000
\binoppenalty=10000

\begin{document}
\section{Geometry revision}
\begin{enumerate}
  \item % http://www.pdmi.ras.ru/EIMI/2018/Baltic_way/bw18coord.pdf
  The points $A$, $B$, $C$, $D$ lie, in this order, on a circle $\omega$, where $AD$ is a diameter of $\omega$. Furthermore, $AB = BC = a$ and $CD = c$ for some relatively prime integers $a$ and $c$. Show that if the diameter $d$ of $\omega$ is also an integer, then either $d$ or $2d$ is a perfect square.

  \item %IMO SL2018
  Let $ABCDE$ be a convex pentagon such that $AB = BC = CD$, $\angle EAB =  \angle BCD$, and $\angle EDC = \angle CBA$. Prove that the perpendiular line from $E$ to $BC$ and the line segments $AC$ and $BD$ are concurrent.

  \item % http://www.math.olympiaadid.ut.ee/arhiiv/varia/bwtr/bw18tr/bw18tren.pdf
  The heights of triangle $ABC$ for triangle $A_1B_1C_1$. The heights of triangle $A_1B_1C_1$ form triangle $A_2B_2C_2$. Prove that $ABC \sim A_2B_2C_2$

  \item % http://www.math.olympiaadid.ut.ee/arhiiv/varia/bwtr/bw18tr/bw18tren.pdf
  Vertex $A$ of square $ABCD$ is symmetric to the midpoint of side $CD$ with respect to line $l$. Find the ratio of the areas of the two quadrilaterals on either side of line $l$ which make up the square.

  \item % http://www.math.olympiaadid.ut.ee/arhiiv/varia/bwtr/bw18tr/bw18tren.pdf
  Acute triangle $ABC$ has circumcircle $\omega$. The tangents of $\omega$ at points $B$ and $C$ intesect at $P$. $D$ and $E$ are the projections of $P$ to lines $AB$ and $AC$ respectively. Prove that the orthocentre of triangle $ADE$ coincides with the midpoint of line $BC$.

  \item % http://www.pdmi.ras.ru/EIMI/2018/Baltic_way/bw18coord.pdf
  The bisector of the $\angle A$ of a triangle $ABC$ intersects $BC$ in a point $D$ and intersects the circumcircle of the triangle $ABC$ in a point $E$. Let $K, L, M$ and $N$ be the midpoints of the segments $AB, BD, CD$ and $AC$, respectively. Let $P$ be the circumcenter of the triangle $EKL$, and $Q$ be the circumcenter of the triangle $EMN$. Prove that $\angle PEQ = \angle BAC$.

\end{enumerate}

\section{Auxilliary constructions} % http://www.math.olympiaadid.ut.ee/arhiiv/oppemat/eesti/lisakon.pdf
\begin{enumerate}[resume]
  \item
  Vertices $A,B,C,D$ of parallelogram are connected to the midpoints of sides $BC,CD,DA,AB$ respectively. The segments intersect at points $K,L,M,N$. Find the ratio of the areas of $KLMN$ and $ABCD$.

  \item
  In triangle $ABC$ $\angle A = 90 ^\circ$. Let $M$ be the midpoint of $AB$. Perpendicular line to $CM$ through $A$ intersects $AB$ at $P$. Prove that $\angle AMC = \angle BMP$.

  \item
    Let $ABC$ be an acute triangle and $D$ the projection of $A$ to side $BC$. Point $E$ lies on segment $AD$ and satisfies
    $$\frac{AE}{ED}=\frac{CD}{DB}$$
    Let $F$ be the projection of point $D$ to $BE$. Prove that $\angle AFC = 90 ^\circ$.
  \item
  Points $E$ and $F$ are chosen on the sides $CD$ and $BC$ of square $ABCD$, such that $\angle AEB = \angle AEF$. Find $\angle EAF$.
  \item
  The angle bisectors from points $A$ and $B$ of triangle $ABC$ intersect sides $BC$ and $AC$ at points $D$ and $E$ respectively. Find $\angle C$, if $AE + BD = AB$

  \item
  Let us have a parallelogram $ABCD$. A circle which goes through $A$ intersects segments $AB$, $AC$ and $AD$ at points $M$, $K$ and $N$. Prove that $|AB|\cdot|AM| + |AD|\cdot |AN| = |AK|\cdot|AC|$

\end{enumerate}

\end{document}


%!TEX root = ../notes.tex
\section{February 15, 2022}

\subsection{Cyclicity of Groups}
\subsubsection{mod odd \texorpdfstring{$p$}{p}}
\recall from last class, we had \cref{prop:d-roots}:
\begin{proposition*}
    If $p$ is a prime and if $d\mid p-1$, then the polynomial
    \[x^{d-1}\in (\ZZ/p\ZZ)[x]\] has exactly $d$ roots in the base field $\FF_p = \ZZ/p\ZZ$.
\end{proposition*}
\begin{corollary}\label{cor:generators-in-p}
    $G:=(\ZZ/p\ZZ)^\times$ is cyclic.
\end{corollary}
\begin{proof}
    For $d\mid (p-1)$, we write $\psi(d)$ for the number of elements of $G$ having order $d$.

    Proposition 2 implies that\footnote{We throw in Lagrange's theorem, and essentially count the number of solutions to $x^d\equiv 1$.}
    \[\sum_{c\mid d}\psi(c) = d\qquad(\psi* i = \id, \psi = \id * \mu)\]
    M\"obius inversion gives
    \[\psi(d) = \sum_{c\mid d}\mu(c)\frac{d}{c}.\]
    On the other hand, we have $\id = \phi * i\Rightarrow \phi = \mu * \id$. Thus $\psi(d) = \phi(d)$ for all $d\mid (p-1)$.
    So in particular, $\psi(p-1) = \phi(p-1)\geq 1$ for any prime $p$.
\end{proof}

\subsubsection{mod odd power \texorpdfstring{$p^e$}{p\^e}}
\begin{theorem}\label{thm:generators-in-powers-of-p}
    Let $p\in \ZZ_+$ be an odd prime, and let $e\geq 1$. Then $U(p^e)$ is cyclic.
\end{theorem}
Proof overview:
\begin{enumerate}[1.]
    \item Pick a primitive root mod $p$. We call it $g$ (for generator).
    \item Show that either $g$ or $g + p$ is a primitive root mod $p^2$.
    \item Show that if $h$ is any primitive root mod $p^2$, then $h$ is a primitive root mod $p^e$ $\forall e\geq 2$.
\end{enumerate}
\begin{proof}[Proof of \cref{thm:generators-in-powers-of-p}]
    ~\begin{description}
        \item[Step 1.] Let $g$ be a primitive root modulo $p$ given by \cref{cor:generators-in-p}.
        \item[Step 2.]  Let $d$ be the order of $g$ mod $p^2$. Since $\phi(p^2) = p(p-1)$, we have that
            \[d\mid p(p-1)\qquad\text{by Lagange}.\]
            By definition of $d$,
            \begin{align*}
                g^d\equiv 1\mod{p^2}
                \intertext{so we also have}
                g^d\equiv 1\mod{p}
            \end{align*}
            Thus $(p-1)\mid d$ since $g$ has order $p-1$ mod $p$. Altogether, $d$ is either $p-1$ or $p(p-1)$. If $d = p(p-1)$, then we are done with step 2. So we assume the former that $d = p-1$.

            Let $h = g + p$. We know that $h$ is a primitive root mod $p$, so we do the same [yoga] as above and conclude that the order of $h$ mod $p^2$ is either $p-1$ or $p(p-1)$.

            By our new hypothesis,
            \[g^{p-1}\equiv 1\pmod{p^2}\]
            so modulo $p^2$, we have
            \begin{align*}
                h^{p-1} = (g+p)^{p-1} & = g^{p-1} + (p-1)g^{p-2}p + \cdots + p^{p-1}
                \intertext{Modulo $p^2$, the only terms that survive are (expand and all $p^2$ terms die):}
                                      & \equiv 1 - pg^{p-2}\pmod{p^2}
            \end{align*}
            But $p\nmid g$, so $pg^{p-2}\not\equiv 0\mod{p}$, and hence $h^{p-1}\not\equiv 1\mod{p^2}$. Thus the order of $h$ mod $p^2$ is $p(p-1)$, so $h$ generates $U(p^2)$.

            So we are done with step $2$. If $g$ is a primitive root mod $p$, then either $g$ or $g + p$ is a primitive root mod $p^2$.
        \item[Step 3.] We wish to show that a primitive root mod $p^2$ is also a primitive root mod $p^e$ $\forall e\geq 2$. We induct on $e$.

            Let $h$ be a primitive root mod $p^e$ for some fixed $e\geq 2$. Let $d$ be the order of $h$ mod $p^{e+1}$. By Lagange, we have that $d\mid \phi(p^{e+1}) = p^e(p-1)$, and from step $2$,
            \[\phi(p^e) = p^{e-1}(p-1)\mid d\]
            Hence $d = p^e(p-1)$ or $p^{e-1}(p-1)$. If it's the former then we are done, so we assume latter.

            We want to show that
            \[h^{p^{e-1}(p-1)}\not\equiv 1\mod{p^{e+1}}\]
            implying that $d = p^e(p-1)$ after all.

            Since $h$ has order $\phi(p^e) = p^{e-1}(p-1)$ in $U(p^e)$, we have \begin{equation}
                h^{p^{e-2}(p-1)}\not\equiv 1\mod{p^e} \label{eqn:5.2-1}\tag{$\star$}
            \end{equation} However,
            \begin{equation}
                h^{p^{e-2}(p-1)}\equiv 1\mod{p^{e-1}} \label{eqn:5.2-2}\tag{$\star\star$}
            \end{equation}
            Combining \cref{eqn:5.2-1} and \cref{eqn:5.2-2} yields
            \[h^{p^{e-2}(p-1)}=1+kp^{e-1}\]
            where $p\nmid k$. Therefore, we have
            \begin{align*}
                h^{p^{e-1}(p-1)} & = (1+kp^{e-1})^p                                    \\
                                 & = 1 + pkp^{e-1} + \binom{p}{2}k^2p^{2e-2} + \cdots
                \intertext{Subsequent terms are all divisible by $p^{3e-3}=(p^{e-1})^3$, and hence divisible by $p^{e+1}$ as $e(e-1)\geq 2+1\ \forall e\geq 2$. Thus
                }
                h^{p^{e-1}(p-1)} & = 1 + kp^e + \frac{1}{2}k^2p^{2e-1}(p-1)\mod{p^e+1}
            \end{align*}
            $p$ is odd, so
            \[\frac{1}{2}k^2p^{2e-1}(p-1)\]
            is divisible by $p^{e+1}$, since $2e-1\geq e+1$. Thus
            \[h^{p^{e-1}(p-1)} \equiv 1 + kp^e \mod{p^e+1}\]
            Since $p\nmid k$, we get that $kp^e\not\equiv 0$ so
            \[h^{p^{e-1}(p-1)} \not\equiv 1 \mod{p^e+1}\]
            This proves that $d = p^e(p-1)$, which is to say that $h$ is a primitive root mod $p^{e+1}$.
    \end{description}
    Altogether, we have that $U(p^e)$ is cyclic.
\end{proof}

\subsubsection{mod powers of 2}
\begin{theorem}
    $U(2^e)$ is cyclic iff $e = 1$ or $e = 2$.
\end{theorem}
\begin{proof}
    Clearly $U(2)$ and $U(4)$ are cyclic\footnote{We don't have much choice since there is only one trivial group and one group of order $2$, both cyclic.}.

    We show that $U(2^e)$ is \emph{not} cyclic for all $e\geq 3$. Notice: it suffices to show that $U(8)$ is not cyclic, since we can find group homomorphisms down powers of $2$.
    \[U(8) = \{\overline{1}, \overline{3}, \overline{5}, \overline{7}\}\]
    and $\overline{1}^2 = \overline{3}^2 = \overline{5}^2 = \overline{7}^2\mod{8}$.
\end{proof}

\subsection{Classification of all cyclic unit groups}
\begin{corollary}\label{cor:cyclicity-of-unit-groups}
    $U(m)$ is cyclic if and only if $m = 1, 2, 4, p^e$ or $2p^e$ for some odd prime $p$.
\end{corollary}
\begin{proof}
    Recall that a product $G$ of finite cyclic groups $G_1$ and $G_2$ is cyclic iff $(|G_1|, |G_2|) = 1$.\footnote{Secretly, Chinese Remainder Theorem.} On the other hand, $\phi(m)$ is even $\forall m\geq 3$. So only one of $G_1$ and $G_2$ needs odd power.

    Combined with our structure theorems on $U(p^e)$ for primes $p$, this proves the corollary since these are the only possibilities.
\end{proof}


\section{Using Python to Model the Earth's Atmosphere}
This stream Simon was not feeling that well and it felt like his brain was not working, so be wary of errors! You have been warned. also the resolutin (size of an individual cell on the latitude
longitude grid) has been decreased to 5 degrees per cell instead of 3 degrees.

\subsection{Interpolating the Air Density}
In order to interpolate (see \autoref{sec:interpolation}) the air density, we need data. However currently we are just guessing the air density at higher levels, instead of taking real values.
So let us change that. For that we are going to use the U.S. Standard Atmosphere, an industry standard measure of the atmosphere on Earth \cite{usatmosp}. This data was provided in a text
(\texttt{TXT}) file which of course needs to be read in order for the data to be used in the model. Here we only care for the density and the temperature at a specific height. So the text file
only contains those two columns of the data (and the height in km of course as that is the index of the row, the property that uniquely identifies a row).

With that in mind, let's get coding and importing the data. We do this in \autoref{alg:usatmosp}. As one can see we do not specify how to open the file or how to split the read line, as this
is language specific and not interesting to describe in detail. I refer you to the internet to search for how to open a text file in the language you are working in. Keep in mind in which
magnitude you are working and in which magnitude the data is. If you work with $km$ for height and the data is in $m$, you need to account for that somewhere by either transforming the imported
data or work in the other magnitude.

\begin{algorithm}
    $data \leftarrow \text{open text file containing the us standard atmosphere data}$ \;
    \ForEach{$line \in data$}{
        Split $line$ into three components, $sh, st$ and $sd$, representing the height, temperature and density respectively \;
        $standardHeight.add(sh)$ \;
        $standardTemperature.add(st)$ \;
        $standardDensity.add(sd)$ \;
    }

    $densityProfile \leftarrow \texttt{interpolate}(heights, standardHeight, standardDensity)$ \;
    $temperatureProfile \leftarrow \texttt{interpolate}(heights, standardHeight, standardTemperature)$ \;

    \For{$alt \in [0, nlevels]$}{
        $\rho[:, :, alt] \leftarrow densityProfile[alt]$ \;
        $T_a[:, :, alt] \leftarrow temperatureProfile[alt]$ \;
    }
    \caption{Loading in the U.S. Standard Atmosphere}
    \label{alg:usatmosp}
\end{algorithm}

Note that the function \texttt{interpolate} takes three arguments, the first one being the data points that we want to have values for, the second one is the data points that we know and the
third one is the values for the data points that we know. This function may or may not exist in your programming language of choice, which might mean that you have to write it yourself.
The formula that we use for interpolation can be found in \autoref{eq:interpolation}, though you still need to figure out what value you need for $\lambda$ (see \autoref{sec:interpolation}).
This is left as an exercise for the reader.

\subsection{Fixing Vertical Motion}
Another attempt was made at fixing the vertical motion. The changes are incorporated in \autoref{alg:advection layer}. Do keep in mind that the low air density in the upper layers messes a lot
with the vertical motion. In other words, it kinda works but not really. Another idea to help fix it, is to introduce a variable called $top$ which indicates the highest point that the
atmosphere may have. This value is initialised as $8 \cdot 10^3$ in meters (so 8 $km$). We then change the definition of $heights$ to: An array of uniform thickness of $\frac{top}{nlevels} m$.
We also added the $\delta z$ to \autoref{alg:temperature layer} as that was something that was still missing.

The current theory why the vertical velocity is not right is that the vertical thermodynamics may be wrong. This will be investigated further and we will report on this in future sections.


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{cite}
\usepackage{amsmath,amssymb,amsfonts}
\usepackage{algorithmic}
\usepackage{graphicx}
\usepackage{import}
\usepackage{textcomp}
\usepackage{xcolor}
\usepackage{balance}
\usepackage{geometry}
\geometry{legalpaper,
portrait,
margin=2in,
lmargin = 2cm,
rmargin = 2cm,
}

\usepackage{multirow} % for tables with complicated header

\usepackage{booktabs,makecell,tabularx}
\renewcommand\theadfont{\small}
\renewcommand\theadgape{}

\def\BibTeX{{\rm B\kern-.05em{\sc i\kern-.025em b}\kern-.08em
    T\kern-.1667em\lower.7ex\hbox{E}\kern-.125emX}}
\begin{document}

\newcommand{\diag}{\operatorname{diag}}
\newcommand{\E}{\operatorname{E}} % expectation
\newcommand{\tr}{\operatorname{tr}} % trace
\newcommand{\iDFT}{\operatorname{IDFT}} %
\newcommand{\DFT}{\operatorname{DFT}} %
\newcommand{\iFFT}{\operatorname{IFFT}} %
\newcommand{\iSFFT}{\operatorname{ISFFT}} %
\newcommand{\SFFT}{\operatorname{SFFT}} %
\newcommand{\FFT}{\operatorname{FFT}} %
\newcommand{\suma}{\operatorname{sum}} %

\title{Full-rate STLC for Four Receive Antennas - Notes }
\author{radim.zedka@vut.cz }
\date{March 2022}
\maketitle

% =================================================
\section{Problem Description}

Formula (6) in \cite{b_FullRate_STLC} is given by
\begin{equation} \label{eq_xi_orig}
    \xi = \rho \frac{\Big( \sum_{p=0}^{M-1} \gamma_{p} \pm 2\mathcal{R}\big\{ \epsilon_p\big\} \Big)^2}{4\sum_{p'=0}^{M-1}\gamma_{p'}},
\end{equation}
% \begin{equation} \label{eq_xi_orig}
%     \xi = \rho \frac{\frac{M^2}{M^2}\Big( \sum_{p=0}^{M-1} \gamma_{p} \pm 2\mathcal{R}\big\{ \epsilon_p\big\} \Big)^2}{4\sum_{p'=0}^{M-1}\gamma_{p'}} = \rho \frac{M^2\Big( \frac{1}{M}\sum_{p=0}^{M-1} \gamma_{p} \pm 2\frac{1}{M} \mathcal{R}\big\{ \epsilon_p\big\} \Big)^2}{4\sum_{p'=0}^{M-1}\gamma_{p'}}\ \overrightarrow{\text{for high }M}\ = \frac{\rho}{4} \sum_{p=0}^{M-1} \gamma_{p},
% \end{equation}
% \begin{equation} \label{eq_xi_orig}
%     \lim_{M\to\infty} \xi = \frac{\rho}{4} \sum_{p=0}^{M-1} \gamma_{p}
% \end{equation}
% \begin{equation} \label{eq_xi_orig}
%     \lim_{M\to\infty} \frac{1}{M} \sum_{p=0}^{M-1} \mathcal{R}\big\{ \epsilon_p\big\} = 0
% \end{equation}
where
\begin{equation} \label{eq_gamma_p}
    \gamma_p = \sum_{q=0}^{3} |h_{p,q}|^2,
\end{equation}
and
\begin{equation} \label{eq_epsilon_p}
    \epsilon_p = h_{p,0}h_{p,2}^* + h_{p,1}h_{p,3}^*.
\end{equation}
Each complex channel gain $h_{p,q}$ is composed of two i.i.d. normal variables $a_{p,q}, b_{p,q} \sim \mathcal{N}(0,1/2)$ which relate to channel gain by $h_{p,q} = a_{p,q} + jb_{p,q}$, where $j = \sqrt{-1}$.
Formula \eqref{eq_gamma_p} then evolves into
\begin{equation} \label{eq_gamma_p_2}
    \gamma_p = \sum_{q=0}^{3} |a_{p,q}|^2 + |b_{p,q}|^2,
\end{equation}
and $\mathcal{R}\big\{ \epsilon_p\big\}$ is expressed as
\begin{equation} \label{eq_epsilon_p_2}
    \mathcal{R}\big\{ \epsilon_p\big\} = a_{p,0}a_{p,2} + b_{p,0}b_{p,2} + a_{p,1}a_{p,3} + b_{p,1}b_{p,3}.
\end{equation}
Formula \eqref{eq_xi_orig} may be expanded into purely real-valued form
\begin{equation} \label{eq_xi_2}
    \xi = \frac{\rho}{4} \frac{\Big( \sum_{p=0}^{M-1} \sum_{q=0}^{3} |a_{p,q}|^2 + |b_{p,q}|^2 \pm 2\mathcal{R}\big\{ \epsilon_p\big\} \Big)^2}{ \sum_{p'=0}^{M-1} \sum_{q=0}^{3} |a_{p',q}|^2 + |b_{p',q}|^2}.
\end{equation}
After Lemma 1 in \cite{b_FullRate_STLC} the receiver SNR is calculated via
\begin{equation} \label{eq_xi_Lemma}
    \xi =  \frac{\rho}{4} \sum_{p=0}^{M-1} \sum_{q=0}^{3} |a_{p,q}|^2 + |b_{p,q}|^2 .
\end{equation}

% =================================================
\section{MATLAB Simulation}

In MATLAB I created a script which generates the normal-distributed variables $a_{p,q}$ and $b_{p,q}$ in vectors of $10^7$ samples each. This way I calculate the histogram of formula \eqref{eq_xi_2} and I attempt to approximate it with Gamma distribution with PDF given by
\begin{equation}\label{eq_Gamma_PDF}
 f_{\Xi}(\xi) =  \frac{\xi^{nM - 1} e^{-\frac{\xi m}{2}}}{ (2/m)^{nM} \cdot \Gamma(nM)} \quad \forall \quad \xi \in \langle 0, \infty).
\end{equation}
where $n$ is related to the shape parameter and $m$ to the Gamma scale parameter. Please note that for PDF approximations I set $\rho = 1$.
As the error metric of the PDF approximation I used mean-squared-error defined as
\begin{equation}\label{eq_PDF_error_metric}
 J = \int \big( f_{\Xi}(\xi)' - f_{\Xi}(\xi) \big)^2 d\xi,
\end{equation}
where $f_{\Xi}(\xi)'$ is the simulated data histogram and $f_{\Xi}(\xi)$ is the approximation.

Diversity gain of systems approximated by Gamma distribution is given by
\begin{equation}\label{eq_DivGain_limit}
\kappa = -\lim_{\rho \to \infty}\frac{d \log_{10}(\bar{\varepsilon}(\rho))}{d \log_{10}(\rho)} = nM,
\end{equation}
where $\bar{\varepsilon}(\rho)$ is the average bit error rate at given $\rho$ and $n$ is the diversity gain coefficient. Table~\ref{table:1} then summarizes PDF approximation parameters $n$, $m$ for several values of $M$.

\begin{table}[t]
\caption{PDF fitting parameters of \eqref{eq_xi_2} at given $M$}
\label{table:1}
\centering
\small
\begin{tabular} {|c|c|c|c|}
\toprule
\thead {$M$} & \thead {$m$ } & \thead {$n$ } & \thead {$J$ [dB] }  \\
    \midrule
$1$ & $1.530445$ & $0.949676$  & $-41.6985$\\
$2$ & $1.574221$ & $0.886598$  & $-46.4452$\\
$4$ & $1.596783$ & $0.847111$  & $-55.8000$\\
$8$ & $1.600477$ & $0.824861$  & $-58.9625$\\
$16$ & $1.606456$ & $0.815408$  & $-60.4560$\\
$32$ & $1.610120$ & $0.811129$  & $-61.2368$\\
$64$ & $1.608200$ & $0.807196$  & $-63.6193$\\
$128$ & $1.613510$ & $0.808271$  & $-62.2914$\\
$256$ & $1.612725$ & $0.807231$  & $-64.0791$\\
$512$ & $1.610609$ & $0.805685$  & $-66.8909$\\
    \bottomrule
    \end{tabular}
\end{table}

Figures \ref{fig_PDF_M1} to \ref{fig_PDF_M512} I present the results for $M = \{1,32,64,256,512\}$ where formula \eqref{eq_xi_2} is fitted by \eqref{eq_Gamma_PDF} with $m$ and $n$ according to Table~\ref{table:1}.

\begin{figure}[!h] \centering
\includegraphics[scale=0.8]{images/PDF_M_1.eps}
\caption{PDF of \eqref{eq_xi_2} approximated by \eqref{eq_Gamma_PDF} for $M=1$. }
\label{fig_PDF_M1}
\end{figure}

\begin{figure}[!h] \centering
\includegraphics[scale=0.8]{images/PDF_M_32.eps}
\caption{PDF of \eqref{eq_xi_2} approximated by \eqref{eq_Gamma_PDF} for $M=32$ . }
\label{fig_PDF_M32}
\end{figure}

\begin{figure}[!h] \centering
\includegraphics[scale=0.8]{images/PDF_M_64.eps}
\caption{PDF of \eqref{eq_xi_2} approximated by \eqref{eq_Gamma_PDF} for $M=64$ . }
\label{fig_PDF_M64}
\end{figure}

\begin{figure}[!h] \centering
\includegraphics[scale=0.8]{images/PDF_M_256.eps}
\caption{PDF of \eqref{eq_xi_2} approximated by \eqref{eq_Gamma_PDF} for $M=256$ . }
\label{fig_PDF_M256}
\end{figure}

\begin{figure}[!h] \centering
\includegraphics[scale=0.8]{images/PDF_M_512.eps}
\caption{PDF of \eqref{eq_xi_2} approximated by \eqref{eq_Gamma_PDF} for $M=512$ . }
\label{fig_PDF_M512}
\end{figure}

% \begin{figure}[!h]
% \begin{tabular}{ll}
% \includegraphics[scale=0.53]{images/PDF_M_32.eps}
% &
% \includegraphics[scale=0.53]{images/PDF_M_64.eps}
% \end{tabular}
% \caption{PDF of \eqref{eq_xi_2} and \eqref{eq_xi_Lemma} with scaled chi-squared PDFs \eqref{eq_ChiSquared_PDF_scaled} for $M=32$ and $M=64$. }
% \label{fig_PDF_1}
% \end{figure}

% \begin{figure}[!h]
% \begin{tabular}{ll}
% \includegraphics[scale=0.53]{images/PDF_M_256.eps}
% &
% \includegraphics[scale=0.53]{images/PDF_M_512.eps}
% \end{tabular}
% \caption{PDF of \eqref{eq_xi_2} and \eqref{eq_xi_Lemma} with scaled chi-squared PDFs \eqref{eq_ChiSquared_PDF_scaled} for $M=256$ and $M=512$. }
% \label{fig_PDF_2}
% \end{figure}

% =================================================
\section{Discussion}

My MATLAB simulation attepts show that \eqref{eq_xi_2} is close to Gamma distribution with $n = 0.8$ and $m = 1.6$. The diversity gain is for Gamma distribution directly given by $nM$. Parameter $n$ is decreasing with increasing value of $M$ and it is approaching $n \sim 0.8$.
I would be very grateful for your advice.

% ======================== References ====================
\begin{thebibliography}{00}

\bibitem{b_FullRate_STLC} S. -chan Lim and J. Joung, “Full-Rate Space–Time Line Code for Four Receive Antennas”, IEEE wireless communications letters, pp. 1-1, 2021.

\end{thebibliography}

\end{document}