\chapter{SIGNAL EXTRACTION AND SYSTEMATICS}

\section{Boosted decision trees method} \label{BDTchaper}
The boosted decision trees(BDT)~\cite{BDTboostPI,TMVAnote} is used in the $\Hmuhad$ analysis. The $\Hmuhad$ result which is referred as BDT fit analysis, with this MVA method, improves a factor of two with respect to the $\mcol$ fit analysis. The BDT algorithm is composed of two parts, the boosting algorithm and the decision tree. The AdaBoost(adaptive boost) is the boosting algorithm used in the $\Hmuhad$ analysis.

The decision tree is a simple two dimensional tree structure. The BDT takes in one signal and one background dataset, which are further divided into training and test samples. A set of selected variables are used the training. At the initial node of the tree structure, events are ordered by the value of variables. The node is split into two branches by a binary selection of the variable that gives the best separation between signal and background in each of the branch. The splitting continues until the training reaches the specified limit set for the number of layers or the number of events in the branch, then the last notes are called leaves. Depending on the main population inside each leaf, the leafs are categorized as signal leaves or background leaves. An example of the tree structure is shown in Figure.~\ref{fig:BDTtreestructure}.  Criteria are needed for the selection of variables at the node or branch. Purity is defined as in Equation.~\ref{Equ.purityBDT}, in which the superscript s stands for signal events, b for background events and W for the event weight.
\begin{align} \label{Equ.purityBDT}
P=\frac{\sum_{s}W_{s}}{\sum_{s}W_{s}+\sum_{b}W_{b}}
\end{align}
Through the definition of purity, a couple of criteria can be configured. Here the one called Gini, defined in Equation.~\ref{Equ.BDTGini}, is selected also it is the default one for the BDT in TMVA package.
\begin{align}\label{Equ.BDTGini}
Gini=(\sum_{i=1}^{n}W_{i})P(1-P)
\end{align}
In each splitting, the variable value that maximize the criterion in Equation.~\ref{Equ.BDTtreecriterion} is selected.

\begin{align}\label{Equ.BDTtreecriterion}
Criterion=Gini_{n-1}-Gini_{n \ left}-Gini_{n \ right}
\end{align}

\begin{figure}[!tbp]
\centering
\includegraphics[width=0.6\textwidth]{chapter7/BDT_tree_show.pdf}
\caption{Tree structure example in BDT}
\label{fig:BDTtreestructure}
\end{figure}

The decision tree method is powerful, but the performance can suffer from fluctuations. The selection of two variables with similar selection quality may be affected by a small change in the training sample. One of the solution is introducing the boosting algorithm. The boosting algorithm not only helps stabilize the decision tree training with respect to the small fluctuations, but also enhances the classification. The AdaBoost is one of the boosting algorithms. In AdaBoost, after the first decision tree training, the misclassified events get higher even weights and are taken as inputs into the second tree. Typically the AdaBoost training 1000 to 2000 trees. The misclassified event weights depend on the training error of each decision tree. The training error is calculated as in Equation.~\ref{Equ.errorBDT},
\begin{equation}\label{Equ.errorBDT}
\textrm{error}_{m}=\frac{\sum_{i=1}^{N}w_{i}I(y_{i}\neq T_{m}(x_{i}))}{\sum_{i=1}^{N}w_{i}}
\end{equation}
in which the superscript m is the tree label and w is the event weight. The $y_{i}$ indicates the i-th event type, 1 for signal and -1 for background. $T_{m}(x_{i})$ indicates the type of the leaf event i with variable x land on, 1 for signal leaf and -1 for background leaf. The variable $I(y_{i}\neq T_{m}(x_{i}))$ constructs on the variables $y_{i}$ and $T_{m}$ equal 1 if $y_{i}\neq T_{m}(x_{i})$ or 0 if $y_{i}= T_{m}(x_{i})$. As shown in Equation.~\ref{Equ.boostingweight}, with the intermediate quantity $\alpha_{m}$ for the tree m, the weight for event i is updated to its new value.
\begin{align}\label{Equ.boostingweight}
\alpha_{m}=&\beta\times \textrm{ln}((1-\textrm{error}_{m})/\textrm{error}_{m})\\
w_{i}\rightarrow &w_{i}\times e^{\alpha I(y_{i}\neq T_{m}(x_{i}))}
%w_{i}\rightarrow &w_{i}/\sum_{i=1}^{N}w_{i}
\end{align}
While $\textrm{error}_{m}$ is required to be less than 0.5 so as the event weights are updated to the right direction. The learning rate parameter $\beta$ can be used to adjust the step size of each re-weighting. The default value of $\beta$ is 1. Event weights in each tree is renormalized to keep the summed weights constant. The final score of each event after the boosting and training processes is obtained with Equation.~\ref{Equ.boostingeventweight}. High score indicates a signal like event while low score indicates a background like event.

\begin{align}\label{Equ.boostingeventweight}
T(x)=\sum_{m=1}^{N_{tree}}\alpha_{m}T_{m}(x)
\end{align}

\section{Statistical methods}

Physics results are extracted with statistical methods. The limit of branching ratio with confident level(CLs) method, significance and best fit branching fraction are mentioned in the analysis.

The probability density function(PDF) and likelihood function are the basic functions used. The PDF of a continuous variable x can be interpreted as the likelihood of x at its different values. The PDF holds the property that it is normalized to unity.
\begin{align*}
\int f(x) dx =1
\end{align*}

Usually the PDF of a variable is accompanied with a set of parameters, thus the PDF is usually written in the form $f(data|\alpha)$, which means the probability density function of the variables in data  given the parameter set $\alpha$. These parameters can from various sources, for example, from theory estimation, detector response and Monte Carlo simulation. All of the parameters in a model or a system, besides the parameters of interests, the other parameters that have an impact on the results are referred as nuisance parameters.

Likelihood function of the same model $L(data|\alpha)$ is a function of parameter set $\alpha$ given data under study. In particle physics, the likelihood function is more often in the form of $L(data|\mu,\theta)$, in which the $\mu$ stands for signal strength modifier and the $\theta$ stands for a full suite of nuisance parameters. The data in a Likelihood function can be experimental data or pseudo-data which is produced to generate sample distribution.
Taking the counting experiment as an example, the PDFs and likelihood function in a binned sample can be estimated with poisson distribution. The  PDF is in the form as
\begin{align*}
f(data|u,\theta)=\prod_{i}\frac{(\mu s_{i}+b_{i})^{n_{i}}}{n_{i}!}e^{-\mu s_{i}-b_{i}}
\end{align*}
The $s_{i}$ and $b_{i}$ stand for the expected number signal and background events respectively and both of them are a function of nuisance parameter $\theta$ as $s_{i}(\theta)$ and $b_{i}(\theta)$. The $n_{i}$ represents the number of events observed in bin i. The likelihood function in the numerical form is similar to the PDF but implement the systematic error PDFs $\rho(\theta|\tilde{\theta})$ if there are systematics considered in the model~\cite{CMS-NOTE-2011-005}. The form of the likelihood function can be expressed as
\begin{align*}
\mathcal{L}=\prod_{i}\frac{(\mu s_{i}+b_{i})^{n_{i}}}{n_{i}!}e^{-\mu s_{i}-b_{i}}\cdot \rho(\tilde{\theta}|\theta)
\end{align*}

Both of the PDF and the likelihood function rely on the knowledge of nuisance parameters. Auxiliary measurements or control regions are often used for the scale estimation of systematic uncertainties. Through the observations in different auxiliary measurements, a PDF $p(\tilde{\theta}|\theta)$ which is referred as the posteriors can be measured. Together with $\pi(\theta)$ which is referred as a prior, the systematic error PDFs can be constructed under the Bayes' theorem as
\begin{align*}
\rho(\theta|\tilde{\theta})\sim p(\tilde{\theta}|\theta)\cdot \pi_{\theta}(\theta)
\end{align*}
The systematic error PDFs can be improved and less affected by the choice of prior $\pi(\theta)$~\cite{statistics:school2016} by the auxiliary measurements.

In the search of lepton flavour violaton Higgs decay, the discovery will be the observation of the Higgs events either decay into $\mu\tau$ pairs in 13 TeV search or $e\tau$ pairs in 8 TeV search. The LFV is forbidden in Standard Model(SM), thus, if taking the SM as a background only model, roughly speaking, a discovery can be claimed if the observation is not compatible with the background only model~\cite{LHCstaticstics}. To test the compatibility of the observed data with respect to the background only model and further calculate limits on the signal strength modifier of the signal+background hypothesis, the quantity test statistics and the PDFs of the two model under tested are needed.

The test statistics is a function which can map a set of data into a number. According to the Neyman-Pearson lemma, the likelihood ratios of two models under test give the most powerful test, which is expressed as
\begin{align*}
\tilde{q}_{\mu}=-2\textrm{ln}\frac{\mathcal{L}(data|\mu,\tilde{\theta}_{\mu})}{\mathcal{L}(data|\hat{\mu},\hat{\theta})}
\end{align*}
The $\mu$ is the given signal strength modifier and $0<\hat{\mu}<\mu$. The $\tilde{\theta}$ is referred as conditional maximum likelihood estimator, given the signal strength modifier $\mu$, while in the denominator, $\hat{\mu}$ and $\hat{\theta}$ are allowed to flow freely to maximize the likelihood. The PDFs of signal+background model and background only model can be calculated in the following way. The observed $\theta^{obs}_{\mu}$ and  $\theta^{obs}_{0}$ for signal+background and background only model are obtained by maximizing the likelihood function respectively, also the observed $\tilde{q}_{\mu}^{obs}$ is calculated give a $\mu$. The PDFs of signal+background model $f(\tilde{q}_{\mu}|\mu,\theta^{obs}_{\mu})$ and background only model $f(\tilde{q}_{\mu}|0,\theta^{obs}_{0})$ are constructed with the MC pseudo-data as shown in Figure.~\ref{fig:teststatistics}.

\begin{figure}[!tbp]
\centering
\includegraphics[width=0.6\textwidth]{chapter7/Test_statistics.png}
\caption{Test statistics distribution of signal+background and background only PDFs and the observed value shown in arrow.}
\label{fig:teststatistics}
\end{figure}

The $\textrm{CL}_{s}$ is defined with two values $p_{\mu}$ and 1-$p_{b}$. In signal+background model, $p_{\mu}$ is defined as
\begin{align*}
p_{\mu}=P(\tilde{q}_{\mu}\geq  \tilde{q}_{\mu}^{obs}  |\textrm{signal+background})=\int^{\infty}_{\tilde{q}_{\mu}^{obs}}f(\tilde{q}_{\mu}|\mu,\hat{\theta}_{\mu}^{obs})d \tilde{q}_{\mu}
\end{align*}
In background only model 1-$p_{b}$ is defined as
\begin{align*}
1-p_{b}=P(\tilde{q}\geq  \tilde{q}_{\mu}^{obs} | \textrm{background only})=\int^{\infty}_{\tilde{q}_{0}^{obs}}f(\tilde{q}_{\mu}|0,\hat{\theta}_{0}^{obs})d \tilde{q}_{\mu}
\end{align*}
The $\textrm{CL}_{s}$ then is expressed as
\begin{align*}
\textrm{CL}_{s}=\frac{p_{\mu}}{1-p_{b}}
\end{align*}

For a given value of signal strength modifier $\mu$, $CL_{s}\leq\alpha$, then the signal+background is said excluded with $(1-\alpha)~ \textrm{CL}_{s}$. Usually $\alpha$ is picked as 95\%.

The expected limit shown in lepton flavour violating Higgs decay is the median 95\% $CL_{s}$ upper limit with the $\pm 1 \sigma$ and $\pm 2 \sigma$ bands in the background only model. By generating a large number of background only pseudo-data, the CLs and $\mu^{95\%}$ of each toy are calculated. An example of $\mu^{95\%}$ distributions is shown in Figure.~\ref{fig:Signal_strength_example}. As shown in the cumulative distribution of $\mu^{95\%}$, the median corresponds to 50\%,while $\pm 1 \sigma$, $\pm 2 \sigma$  correspond to range 16\% to 84\% and 2.5\% to 97.5\% respectively.
\begin{figure}[!tbp]
\centering
\includegraphics[width=0.4\textwidth]{chapter7/Signal_strength_example_1.png}
\includegraphics[width=0.4\textwidth]{chapter7/Signal_strength_example_2.png}
\caption[Signal strength modifier distribution of the MC pseudo-data]{Signal strength modifier $\mu$ at 95\% $CL_{s}$ distribution from the MC pseudo-data. The right plot is the cumulative distribution of $\mu^{95\%}$ with $\pm 1 \sigma$ and $\pm 2 \sigma$ bands.}
\label{fig:Signal_strength_example}
\end{figure}

The estimator p value and significance are used to check if the data is compatible with background only model. The test statistics in the checking of background only model is expressed as
\begin{align*}
\tilde{q}_{\mu}=-2\textrm{ln}\frac{\mathcal{L}(data|0,\tilde{\theta}_{\mu})}{\mathcal{L}(data|\hat{\mu},\hat{\theta})}
\end{align*}
Similar procedure as mentioned above can be used to get the PDF of background ground only model $f(q_{0}|0,\hat{\theta}^{obs}_{0})$ and $q^{obs}_{0}$. The p-value corresponding to the experimental observable is calculated as

\begin{align*}
p_{0}=P(q_{0}\geq q_{0}^{obs})=\int^{\infty}_{q_{0}^{obs}}f(q_{0}|0,\hat{\theta}_{0}^{obs})d q_{0}
\end{align*}
Significance Z of the observable with respect to the background only model test can be derived from p value~\cite{CMS-NOTE-2011-005}.
\begin{align*}
p=\int^{\infty}_{Z}\frac{1}{\sqrt{2\pi}}exp(-x^2/2)dx
\end{align*}
The term used in the discovery as 5$\sigma$ correspond to Z=5 and p=$2.8\times10^{-7}$. The distribution of test statistics tends to be chi-squared distribution in large statistics according to the Wilk's theorem, which gives a quick way of estimating of PDFs in test statistics~\cite{LHCstaticstics}. In the case when the expected number of events is large, expected limit with this asymptotic approximation gives a fairly well performance and saves the computing time for toy samples. The asymptotic approximation can used in the estimation of p value as
\begin{align*}
p^{estimate}=\frac{1}{2}\bigg[1-erf\bigg(\sqrt{q_{0}^{obs}/2}\bigg)\bigg]
\end{align*}

\section{Systematics}
Systematic uncertainties originated from different sources that experimentally or theoretically affect the physics results. These uncertainties are considered by the effects on the normalization and the shapes of the distribution from different processes.

\subsection{Systematics used in $\Hmuhad$}
The systematic uncertainties considered in this analysis are summarized in Table.~\ref{tabsystematicsone} and Table.~\ref{tab:systematicstwo}. The uncertainty on muon trigger, identification and isolation together amounts  to 2\% and the hadronic tau lepton efficiency amounts to 5\%. The uncertainties on lepton selections that include trigger, ID, isolation efficiencies are estimated with tag and probe method with Z boson data sets~\cite{Khachatryan2011,Chatrchyan:2012xi,Khachatryan:2015hwa,Khachatryan:2015dfa,CMS:2016gvn}. The systematic uncertainty of b tagging veto is taken from the uncertainty of b tagging veto efficiency measurement, which adjusts b tagging veto performance in the simulation to match data sample. The exact values used in each category of $t\bar{t}$ and single top samples are summarized in Table.~\ref{tab:btaguncertainty}. The uncertainties on $Z\to\tau\tau$, WW, WZ, ZZ, $t\bar{t}$, single top backgrounds mainly originate from the uncertainties on the cross section measurements. The normalization uncertainty of $\mu$ misidentifying $\tau$ scale factor measurement and the uncertainty on $Z\to \mu\mu$ process together amounts to 25\%. An additional shape uncertainty on muon misidentifying tau is extracted from the measurement of the scale factor and treats independently of different hadronic tau decay modes. The uncertainty of the misidentified lepton background is estimated from the matching of the same sign control region which is defined as region II in Table.~\ref{tab:fakeratediagram}. The 30\% on normalization uncertainty correlated between each category and additional 10\% uncorrelated between each category are taken as a conservative estimation. Only the tau shape uncertainties are considered, which are taken from the variations of the misidentified tau parameters in the fitting. The muon misidentified shape uncertainty is omitted as it would be negligible compared with the bin-by-bin uncertainty applied.  The shape uncertainties of jet energy scale is estimated by varying the fitting parameters of different sources that affect this scale by one $\sigma$. The tau energy scale is treated as shape uncertainties and each of the tau decay modes is considered independently. The unclustered energy refers to the energy deficiency from the jets $\pt<10$ GeV and the PF candidates that are not included in the jet reconstruction. The unclustered energy uncertainty is considered independently for charged particles, photons, neutral hadrons and very forward particles.  The uncertainties on Higgs boson cross section is affected by the renormalization scales, factorization, parton distribution functions(PDF) and the strong coupling constant($\alpha_{s}$). These uncertainties contribution in normalization scale and are taken from the latest recommendation~\cite{YR4}. The uncertainties on Higgs cross section also affect the acceptance and result in the migration of events between categories.  The bin-by-bin uncertainty is applied to account for the statistics uncertainties within each bin. These uncertainties are uncorrected between different bins and categories. The uncertainty of integrated luminosity~\cite{CMS-PAS-LUM-17-001}  amounts to 2.5\%.

\begin{table}[htpb]
\caption{PART ONE OF THE SYSTEMATIC UNCERTAINTIES CONSIDERED IN THE $\Hmuhad$ ANALYSIS}\label{tabsystematicsone}
\centering
\begin{threeparttable}
%\begin{center}
%\caption{Part one of the systematic uncertainties considered in $\Hmuhad$ analysis. Systematic uncertainties in the expected event yields. All uncertainties are treated as correlated between the categories, except those that have two values separated by the $\oplus$ sign. In this case, the first value is the correlated uncertainty and the second value is the  uncorrelated uncertainty for each individual category. Theoretical uncertainties on VBF Higgs boson production~\cite{YR4}  are also applied to VH production. Uncertainties on acceptance lead to migration of events between the categories, and can be correlated or anticorrelated between categories. Ranges of uncertainties for the Higgs boson production indicate the variation in size, from negative (anticorrelated) to positive (correlated)}
%\centering
\begin{tabular}{lc}\hline
Systematic  uncertainty            & $\PH\to\Pgm\tauh$  \\ \hline
Muon  trigger/identification/isolation         &       2\%             \\
Hadronic tau lepton efficiency                  &       5\%              \\
b tagging veto                                          &      2.0--4.5\%     \\
$\PZ\to\Pgt\Pgt$ + jets background         &    10\%$\oplus$5\%   \\
$\PW\PW, \PZ\PZ$ background              &     5\%$\oplus$5\%       \\
\ttbar\  background                                  &     10\%$\oplus$5\%             \\
Single top quark background                 &     5\%$\oplus$5\%  \\
$\Pgm\to\tauh$ background                                      &         25\%         \\
$\text{Jet}\to\tauh, \Pgm $ background             &  30\%$\oplus$10\%   \\
Jet energy scale                                                       &   3--20\% \\
\tauh energy scale                                                    &   1.2\%  \\\hline
\end{tabular}
%\centering
\begin{tablenotes}
\small
\item Note:  All of the uncertainties listed in the systematic tables are correlated between categories besides the ones after the sign $\oplus$. These are the uncertainties correlated within each category but independent between categories. The theoretical uncertainties related to the acceptance and migration of events are listed in a range. The negative or positive values indicate anticorrelated or correlated between categories.
\end{tablenotes}
%\end{center}
\end{threeparttable}
\end{table}

\begin{table}[htpb]
%\begin{threeparttable}
\caption{PART TWO OF THE SYSTEMATIC UNCERTAINTIES CONSIDERED IN $\Hmuhad$ ANALYSIS}
%\caption{Part two of the systematic uncertainties considered in $\Hmuhad$ analysis}
\label{tab:systematicstwo}
%\centering
\begin{center}
\begin{tabular}{l*{2}{c}} \hline
Systematic  uncertainty                                             & $\PH\to\Pgm\tauh$ \\ \hline

$\Pgm \to\tauh$ energy scale                                   &    1.5\%  \\
\Pgm\ energy scale                                                   &        0.2\%      \\
Unclustered energy scale                                         &        $\pm 1 \sigma$  \\
Renorm./fact. scales ({\Pg\Pg}H)   \cite{YR4}          &   \multicolumn{2}{c}{3.9\%}\\
Renorm./fact. scales (VBF and VH) \cite{YR4}         &   \multicolumn{2}{c}{0.4\%}\\
PDF + $\alpha_s$ ({\Pg\Pg}H)    \cite{YR4}             &   \multicolumn{2}{c}{ 3.2\%}\\
PDF + $\alpha_s$ (VBF and VH)   \cite{YR4}          &   \multicolumn{2}{c}{ 2.1\%}\\
Renorm./fact. acceptance ({\Pg\Pg}H)                     &   \multicolumn{2}{c}{$-3.0$\% -- $+2.0$\% } \\
Renorm./fact. acceptance (VBF and VH)                &   \multicolumn{2}{c}{$-0.3$\% -- $+1.0$\% } \\
PDF + $\alpha_s$ acceptance ({\Pg\Pg}H)              &   \multicolumn{2}{c}{ $-1.5$\% --  $+0.5$\%}\\
PDF + $\alpha_s$ acceptance (VBF and VH)          &   \multicolumn{2}{c}{ $-1.5$\% --  $+1.0$\%}\\
Integrated luminosity               &   \multicolumn{2}{c}{ 2.5\%  } \\ \hline
\end{tabular}
\end{center}
%\end{threeparttable}
\end{table}

\begin{table}[htpb]
\caption{B TAGGING VETO SYSTEMATIC UNCERTAINTIES IN EACH CATEOGORY}
%\caption{b tagging veto systematic uncertainty in each category}
\label{tab:btaguncertainty}
\centering
\begin{tabular}{lclclclcl}\hline
                       & 0 jet    &  1 jet       & 2 jets gg-enriched & 2 jets VBF-enriched \\\hline
$t\bar{t}$        &  \NA    &  2.45\%    &4.37\%                   & 2.59\%     \\
$t$   & \NA     &  2.11\%    & 3.07\%                   & 1.98\%   \\\hline
\end{tabular}
\end{table}

\subsection{Systematic uncertainties in $\Hehad$}

The systematic uncertainties affect the $\Hehad$ analysis are summarized in Table.~\ref{tab:systematics_had}, \ref{tab:theory_systematics} and \ref{tab:shape_systematics}, which include the ones affect the normalization scale and the ones affect the shape of $\mcol$ distribution. The uncertainties on the electron measurements, which include the trigger, ID and isolation are estimated with the tag and probe measurement with Z boson datasets \cite{CMS:2011aa,Khachatryan:2015dfa}. The uncertainties on the $\PZ \to \tau \tau$ process is mainly from the uncertainty on the cross section measurement~\cite{Chatrchyan:2014mua} and the $\tau$ identification in the embedded technique. The normalization uncertainty on $\PZ \to ee$ amounts to 30\% which is the from the measurement of cross section and statistics uncertainty in the yields. An extra 5\% shape uncertainty is added due to the mismeasured energy of the electron reconstructed as $\Pgt$ in $\PZ \to \Pe\Pe$ background.  The shift in $\mcol$ distribution is measured by the comparison between data and MC simulation. A 30\% normalization uncertainty is applied to the misidentified lepton background. The extra shape uncertainties to this background are obtained by varying the parameters from fitting the misidentifed ratio in $\pm1$ standard deviation.   The uncertainty from the pileup process is estimated by varying the total inelastic cross section by $\pm5\%$~\cite{Chatrchyan:2012nj}. The uncertainty from the Diboson, single top quark background is taken from the measurement of the cross section. In $t\bar{t}$ background, the uncertainty in 0 jet, 1 jet category are taken from the cross section measurement, in 2  jets category, an extra 33\% uncertainty uncorrelated between categories is added due to the statistical uncertainty. The luminosity uncertainty amounts to 2.6\%.  The jet energy scale(JES) and resolution are measured with $\gamma/\PZ+jets$ and dijet samples~\cite{CMS-JME-10-011}. The uncertainty from JES is applied as a function of $\pt$ and $\eta$. The jet energy resolution(JER) uncertainty is obtained by smearing jets energy as a function of $\pt$ and $\eta$. Both JES and JER affect the shape of $\mcol$ distribution and are taken as shape uncertainties. The energy of the jets below 10 GeV and PF objets that are not clustered are accounted as the unclustered energy scale, which is also taken as shape uncertainties. The uncertainty of $\tau_{h}$ energy scale is obtained by comparing $\PZ \to \tau\tau$ distribution between data and MC samples and is applied as a shape uncertainty. The theoretical uncertainties considered in the $\Hehad$ analysis are listed in Table.~\ref{tab:theory_systematics}. There are several sources that contribute to the theoretical uncertainties and are considered fully correlated between LFV Higgs and SM Higgs production. The uncertainty in Parton distribution function is obtained by counting the yields with different PDFs,  CT10~\cite{Nadolsky:2008zw}, MSTW~\cite{Martin:2009iq}, NNPDF~\cite{Ball:2010de}  from the recommendation in PDF4LHC~\cite{Botje:2011sn}. The uncertainty in renormalization and factorization scales are from scaling up and down by a factor of two to their normal values($\mu_{R}=\mu_{F}=M_{H}/2$). The uncertainty in underlying events and parton shower is estimated by checking with different PYTHIA tunes. The theoretical uncertainties are all correlated or anticorrelated. The anticorrelated ones have a minus superscript and are the result of the event migration.

\begin{table}[hbt]
 \centering
 \begin{threeparttable}
 \caption{THE NORMALIZATION SYSTEMATIC UNCERTAINTIES CONSIDERED IN $\Hehad$ ANALYSIS}
% \caption[The normalization systematic uncertainties considered in $\Hehad$ analysis]{The normalization systematic uncertainties considered in $\Hehad$ analysis. The uncertainties are correlated between categories besides the ones after $\oplus$. These uncertainies are uncorrelated between categories. }
  \label{tab:systematics_had}
\begin{tabular}{lccc} \hline
Systematic uncertainty                  & \multicolumn{3}{c}{\PH$\to \Pe\tauh$}      \\\hline
                                                      & 0-jet       & 1-jet        & 2-jet                 \\ \hline
Electron trigger/ID/isolation           &  1\%           &   1\%         &  2\%                       \\
Efficiency of $\tauh$                      &  6.7\%         &  6.7\%        & 6.7\%                      \\
$\PZ\to \Pgt \Pgt$ background    & 3\%$\oplus$5\% & 3\%$\oplus$5\%& 3\%$\oplus$10\%            \\
$\PZ\to \Pe\Pe$ background                      & 30\%           &  30\%         & 30\%                       \\

Misidentified leptons background                   & 30\%           &  30\%         & 30\%                       \\
Pileup                                           & 4\%           & 4\%          & 2\%                       \\
$\PW\PW,\PW\PZ,\PZ\PZ\mathrm{+jets}$ background                 & 15\%           &  15\%         & 15\%                       \\
$\ttbar$ background                     & 10\%           &  10\%         & 10\%$\oplus$33\%           \\
Single top quark background       & 25\%           &  25\%         & 25\%                      \\
Luminosity                                    & 2.6\%          &  2.6\%        & 2.6\%                      \\ \hline
\end{tabular}
\begin{tablenotes}
\small
\item Note: The uncertainties are correlated between categories besides the ones after $\oplus$. These uncertainies are uncorrelated between categories.
\end{tablenotes}
\end{threeparttable}
\end{table}

\begin{table}[hbtp]
 \centering
 \caption{THE SYSTEMATIC UNCERTAINTIES THAT AFFECT THE SHAPE OF $\mcol$ DISTRIBUTION.}
 %\caption{The systematic uncertainties that affect the shape of $\mcol$ distribution. }
  %\caption{Systematic uncertainties in the shape of the signal and background distributions, expressed in percentage. The systematic uncertainty and its implementation are described in the text.}
  \label{tab:shape_systematics}
  \begin{tabular}{lclc} \hline
Systematic Uncertainty                                 &   $\PH \to \Pe \tauh$                   \\ \hline
$Z \to \Pe\Pe$ bias                                       &   5\%                                         \\
Jet energy scale                                           &    3\%--7\%                                       \\
Jet energy resolution                                    &    1\%--10\%                                       \\
Unclustered energy scale                             &    10\%                                       \\
$\tauh$ energy scale                                    &    3\%                                         \\    \hline
  \end{tabular}
\end{table}

\begin{table}[hbtp]
 \caption{THEORETICAL UNCERTAINTIES CONSIDERED IN $\Hehad$ ANALYSIS}  \label{tab:theory_systematics}
 \centering
 \begin{threeparttable}
 %\caption[Theoretical uncertainties considered in $\Hehad$ analysis]{Theoretical uncertainties that affects the Higgs boson production cross section. These uncertainties are correlated or anticorrelated(with minus sign superscript) between all of the categories.}
 % \caption{Theoretical uncertainties in percentage for the Higgs boson production cross section for each production process and category. All uncertainties are treated as fully correlated between categories except those denoted by a negative superscript which are fully anticorrelated due to the migration of events.}
  \begin{tabular}{l|l|l|l|l|l|l} \hline
Systematic uncertainty                  &  \multicolumn{3}{c|}{Gluon fusion} &  \multicolumn{3}{c}{Vector boson fusion}  \\ \cline{2-7}
                                &    0-jet  & 1-jet  & 2-jet   & 0-jet & 1-jet  & 2-jet  \\ \hline
Parton distribution function         &    $9.7$\%  &  $9.7$\% &   $9.7$\% & $3.6$\%  &   $3.6$\%  &  $3.6$\%  \\
Renormalization/factorization scale           &    $8$\%    &  $10$\%   &  $30^{-}$\%   & $4$\%     &   $1.5$\%  & $2$\%   \\
Underlying event/parton shower  &   $4$\%     & $5^{-}$\%   &  $10^{-}$\%   & $10$\%    &   $<$1\%    & $1^{-}$\%   \\ \hline
  \end{tabular}
  \begin{tablenotes}
  \small
  \item Note: These uncertainties are correlated or anticorrelated(with minus sign superscript) between all of the categories.
  \end{tablenotes}
  \end{threeparttable}
\end{table}


\documentclass{article}
\usepackage[utf8]{inputenc}

\title{MAT257 Notes}
\author{Jad Elkhaleq Ghalayini}
\date{January 14 2019}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{mathtools}
\usepackage{enumitem}
\usepackage{graphicx}
\usepackage{cancel}

\usepackage[margin=1in]{geometry}

\newtheorem{theorem}{Theorem}
\newtheorem{lemma}{Lemma}
\newtheorem{definition}{Definition}
\newtheorem*{corollary}{Corollary}
\newtheorem{exercise}{Exercise}
\newtheorem{claim}{Claim}

\DeclareMathOperator{\Int}{Int}
\DeclareMathOperator{\grad}{grad}
\DeclareMathOperator{\Ker}{Ker}
\DeclareMathOperator{\Ima}{Im}

\newcommand{\reals}[0]{\mathbb{R}}
\newcommand{\nats}[0]{\mathbb{N}}
\newcommand{\ints}[0]{\mathbb{Z}}
\newcommand{\rationals}[0]{\mathbb{Q}}
\newcommand{\brac}[1]{\left(#1\right)}
\newcommand{\sbrac}[1]{\left[#1\right]}
\newcommand{\mc}[1]{\mathcal{#1}}
\newcommand{\eval}[3]{\left.#3\right|_{#1}^{#2}}
\newcommand{\ip}[2]{\left\langle#1,#2\right\rangle}
\newcommand{\prt}[2]{\frac{\partial #1}{\partial #2}}
\newcommand{\mb}[1]{\mathbf{#1}}
\newcommand{\hlfspc}[0]{\mathbb{H}}
\newcommand{\loint}[0]{\operatorname{L}\int}
\newcommand{\hiint}[0]{\operatorname{U}\int}
\newcommand{\indic}[1]{\chi_{#1}}

\begin{document}

\maketitle

\section{Extended Definition of the Integral (``Improper Integral'')}
Suppose that \(A\) is open and \(f\) is bounded and continuous. Then under the current definition, the expression
\begin{equation}
  \int_Af
\end{equation}
may not exist, since the boundary may not be Jordan-measurable. So today we're going to try to rectify that, by attempting to extend the definition of the integral to open subsets \(A \subset \reals^n\) and \textit{locally} bounded functions \(f:A \to \reals\), i.e. for every point in \(A\), there is an open neighborhood around \(A\) such that \(f\) is bounded around \(A\). Furthermore, we assume the set of discontinuities is of measure zero.

Just like the improper integral from first year calculus, it won't always exist. But we'll want to know the value when it does. We'll begin with a useful note: if \(f\) vanishes outside of a compact subset \(C\) of \(A\), then in fact \(f\) \textit{is} integrable on \(C\) (since we can put that compact subset in a big rectangle and multiply by the characteristic function). In fact, we can say that \(f\) is integrable on any bounded open subset \(U\) of \(A\) containing \(C\), for the same reason, and
\begin{equation}
  \int_Uf = \int_Cf
\end{equation}
So it makes sense to say
\begin{equation}
  \int_Af = \int_Cf
  \label{conventioncompact}
\end{equation}
This is going to be our starting point. Now to go further, we're going to talk about a partition of unity. Let \(\mc{O}\) be an open cover of \(A\) such that \(U \subset A\) for all \(U \in \mc{O}\) and let \(\Phi\) be a partitionof unity for \(A\) subordinate to \(\mc{O}\). Note that this is the same as saying let \(\Phi\) be a partition of unity for \(A\) subordinate to \(\{A\}\), since that counts as an open covering of \(A\). As in equation \ref{conventioncompact},
we can define, \(\forall \varphi \in \Phi\),
\begin{equation}
  \int_A\varphi|f|
\end{equation}
We can now extend the definition of the integral as follows:
\begin{definition}
  \(f\) is \underline{integrable} on \(A\) if
  \begin{equation}
    \sum_{\varphi \in \Phi}\int_A\varphi|f|
    \label{absolute}
  \end{equation}
  converges. Then we define
  \begin{equation}
    \int_Af = \sum_{\varphi \in \Phi}\varphi f
    \label{relative}
  \end{equation}
\end{definition}
Note that equation \ref{relative} must converge, and in fact converges absolutely, if equation \ref{absolute} converges, because
\begin{equation}
  \sum_{\varphi \in \Phi}\left|\int_A\varphi f\right| \leq \sum_{\varphi \in \Phi}\int_A|\varphi||f| = \sum_{\varphi \in \Phi}\int_A\varphi f
\end{equation}
since \(\varphi\) is nonnegative.
\begin{theorem}
  Suppose \(A \subset \reals\) is open, \(f: A \to \reals\) is locally bounded and the set of discontinuities of \(f\) has measure 0.
  \begin{enumerate}

    \item Given partitions \(\Phi, \Psi\) as above, not necessarily subordinate to the same open covering, though this doesn't matter since both will be subordinate to \(\{A\}\), then if equation \ref{absolute} converges for \(\Phi\), then it converges for \(\Psi\) and the integral defined using \(\Phi\) and the integral defined using \(\Psi\) are equal, i.e.
    \begin{equation}
      \sum_{\psi \in \Psi}\int_A\psi f = \sum_{\varphi \in \Phi}\int_A\varphi f
      \label{integralequal}
    \end{equation}
    Note that equation \ref{integralequal} would not necessarily be true if equation \ref{absolute} does not converge.

    \item The new definition of the integral \textit{always} exists if both \(A\) and \(f\) are bounded, showing it truly generalizes the old definition. Specificially, if both \(A\) and \(f\) are bounded, whether or not the boundary of \(A\) has measure zero,
    \begin{equation}
      \sum_{\varphi \in \Phi}\int_A\varphi|f|
    \end{equation}
    (equation \ref{absolute}) converges

    \item If \(A\) is Jordan-measurable and \(f\) is bounded, then
    \begin{equation}
      \int_Af = \sum_{\varphi \in \Phi}\int_A\varphi f
    \end{equation}
    where the left is as defined before.

  \end{enumerate}
\end{theorem}
There are other ways of defining the integral, some of which look more like the improper integral from first year calculus. Another thing that you could do is the following: since you can always write an open set as a kind of ``expanding union'' of compact sets \(C_i \subseteq C_{i + 1}\), you can actually do that in such a way such that each \(C_i\) is Jordan-measurable, meaning the integral on each of these \(C_i\)'s exist. We could then define
\begin{equation}
  \int_Af = \lim_{i \to \infty}\int_{C_i}f
\end{equation}
After we finish proving this theorem, either I'll prove this or stick it on the next problem set.
\begin{proof}
  \begin{enumerate}

    \item \(\varphi \cdot f\) vanishes outside a compact set, and only finitely many \(\psi\) are nonzero on this set. So
    \begin{equation}
      \sum_{\psi \in \Psi}\psi = 1
      \implies \sum_{\varphi \in \Phi}\int_A\varphi f
      = \sum_{\varphi \in \Phi}\int_A\left(\sum_{\psi \in \Psi}\psi\right)\varphi f
      = \sum_{\varphi \in \Phi}\sum_{\psi \in \Psi}\int_A\varphi\psi f
    \end{equation}
    A priori, we don't know that we can interchange the order in the double sum of the last equal expression, but we could write down exactly the same stuff with the absolute value in it, which tells us we can. Interchanging them, we get the desired result.

    \item Let \(B\) be a big rectangle containing \(A\), and assume \(|f| \leq M\) on \(A\). For any finite subset \(F\) of \(\Phi\),
    \begin{equation}
      \sum_{\varphi \in F}\int_A\varphi|f| \leq M\int_A\sum_{\varphi \in F}\varphi \leq MV(B)
    \end{equation}
    implying the desired result. We still have to show that in the situation that \(\int_Af\) is defined according to our earlier definition, we get the same result. We'll do so next time.

  \end{enumerate}
\end{proof}

\end{document}


\clearpage
\newpage

\section{New methods}

%% Présenter le problème et la factorisation matricielle
%%
%%

In this section we present two new algorithms for estimating individual admixture coefficients and ancestral genotype frequencies assuming $K$ ancestral populations. In addition to genotypes, the new algorithms require individual geographic coordinates of sampled individuals.

\paragraph{$Q$ and $G$-matrices} Consider a genotypic matrix, {\bf Y}, recording data for $n$ individuals at $L$ polymorphic loci for a $p$-ploid species (common values for $p$ are $p = 1,2$). For autosomal SNPs in a diploid organism, the genotype at locus $\ell$  is an integer number, 0, 1 or 2, corresponding to the number of reference alleles at this locus. In our algorithms, disjunctive forms are used  to encode each genotypic value as the indicator of a heterozygote or a homozygote locus (Frichot et al. 2014). For a diploid organism each genotypic value $,0,1,2$ is encoded as $100$, $010$ and $001$. For $p$-ploid organisms, there are $(p+1)$ possible genotypic values at each locus, and each value corresponds to a unique disjunctive form. While our focus is on SNPs, the algorithms presented in this section extend to multi-allelic loci without loss of generality.
Moreover, the method can be easily extended to genotype likelihoods by using the likelihood to encode each genotypic value~\citep{Korneliussen2014}.

Our algorithms provide statistical estimates for the matrix ${\bf Q} \in \mathbb{R}^{K \times n}$ which contains the admixture coefficients, ${\bf Q}_{i,k}$, for each sampled individual, $i$, and each ancestral population, $k$. The algorithms also provide estimates for the matrix ${\bf G} \in \mathbb{R}^{(p+1)L \times K}$, for which the entries, ${\bf G}_{(p+1)\ell + j, k}$, correspond to the frequency of genotype $j$ at locus $\ell$ in population $k$. Obviously, the $Q$ and $G$-matrices must satisfy the following set of probabilistic constraints

$$
\quad {\bf Q},{\bf G} \geq 0 \, , \quad  \sum_{k=1}^K {\bf Q}_{i,k} = 1 \, , \quad \sum_{j=0}^p {\bf G}_{(p+1)\ell + j, k} = 1 \, , \quad j = 0,1,\dots, p,
$$
for all $i, k$ and $\ell$. Using disjunctive forms and the law of total probability, estimates of {\bf Q} and {\bf G} can be obtained by factorizing the genotypic matrix as follows ${\bf Y}$=${\bf Q}\,{\bf G}^T$~\citep{Frichot2014}. Thus the inference problem can be solved by using constrained nonnegative matrix factorization methods~\citep{Lee1999, Cichocki2009}. In the sequel, we shall use the notations  $\Delta_Q$ and $\Delta_G$ to represent the sets of probabilistic constraints put on the {\bf Q} and {\bf G} matrices respectively.

 \paragraph{Geographic weighting} Geography is introduced in the matrix factorization problem by using weights for each pair of sampled individuals. The weights impose regularity constraints on ancestry estimates over geographic space. The definition of geographic weights is based on the spatial coordinates of the sampling sites, $(x_i)$. Samples close to each other are given more weight than samples that are far apart. The computation of the weights starts with building a complete graph from the sampling sites. Then the weight matrix is defined as follows

$$
w_{ij} = \exp( - {\rm dist}( x_i, x_j )^2/ \sigma^2),
$$
\noindent where dist$( x_i, x_j )$ denotes the geodesic distance between sites $x_i$ and  $x_j$, and $\sigma$ is a range parameter. Values for the range parameter can be investigated by using spatial variograms~\citep{Cressie1993}.
To evaluate variograms, we extend the univariate variogram to genotypic data as follows

\begin{equation}
\gamma(h) = \frac{1}{2 |N(h)|} \sum_{i,j \in N(h)} \frac{1}{L} \sum_{l = 1}^{(p+1)L} |Y_{i,l} - Y_{j,l}|,
\label{eq:gamma}
\end{equation}
\noindent where $N(h)$ is defined as the set of individuals separated by geographic distance $h$.
% The function $\gamma$ can be approximated as follows
% \begin{equation}
% \hat{\gamma}(h) = {C}_0 \frac{1}{2 |N(h)|} \sum_{i,j \in N(h)} \| {\bf Q}_{i,.} - {\bf Q}_{j,.}\|^2 + {C}_1,~~~C_0, C_1 > 0,
% \label{eq:gammahat}
% \end{equation}
% \noindent where $\|  u \|^2$ is the squared norm of the vector $u$, and ${\bf Q}_{i,.}$ is the $i$th row of the admixture matrix ${\bf Q}$. Arguments that justify this approximation are given in Appendix~\ref{app:approx}.
In applications, computing and visualizing the $\gamma$ function  provides useful information on the level of spatial autocorrelation between individuals in the data.

Next, we introduce the {\it Laplacian matrix} associated with the geographic weight matrix, {\bf W}. The Laplacian matrix is defined as ${\bf \Lambda}$ = {\bf D} $-$ {\bf W}  where  {\bf D} is a diagonal matrix with entries
${\bf D}_{i,i} = \sum_{j = 1}^n  {\bf W}_{i,j}$,  for  $i = 1, \dots, n$~\citep{Belkin2003}. Elementary matrix algebra shows that~\citep{DengCai2011}

$$
 {\rm Tr} ({\bf Q}^T {\bf \Lambda} {\bf Q})  = \frac12 \sum_{i,j = 1}^n  w_{ij}  \|   {\bf Q}_{i,.}  - {\bf Q}_{j,.} \|^2 \, .
$$
In our approach, assuming that geographically close individuals are more likely to share ancestry than individuals at distant sites is thus equivalent to minimizing the quadratic form ${\cal C}({\bf Q}) ={\rm Tr} ({\bf Q}^T {\bf \Lambda} {\bf Q})$ while estimating the matrix ${\bf Q}$.

\paragraph{Least-squares optimization problems} Estimating the matrices ${\bf Q}$ and ${\bf G }$ from the observed genotypic matrix ${\bf Y}$ is performed through solving an optimization problem defined as follows~\citep{Caye2016}

\begin{equation}
\begin{aligned}
& \underset{Q, G}{\text{min}}
& & {\rm LS}({\bf Q}, {\bf G}) =   \|  {\bf Y} - {\bf QG}^T \|^2_{\rm F} +  \alpha ' \frac{(p+1)L}{K \lambda_{\max}} {\cal C}({\bf Q}) , \\
& \text{s.t.} & &  {\bf Q} \in \Delta_Q , \\
& & &  {\bf G} \in \Delta_G . \\
\end{aligned}
\label{eq:LS}
\end{equation}
 \noindent The notation $\|  {\bf M}  \|_{\rm F}$ denotes the Frobenius norm of a matrix, {\bf M}. The regularization term is normalized by $(p+1)L/K \lambda_{\max}$, where $\lambda_{\max}$ is the largest eigenvalue of the Laplacian matrix. With this normalization, both terms of the optimization problem~\eqref{eq:LS} are given the same order of magnitude. The regularization parameter $\alpha ' $ controls the regularity of ancestry estimates over geographic space.  Large values of $\alpha ' $ imply that ancestry coefficients have similar values for nearby individuals, whereas small values ignore spatial autocorrelation in observed allele frequencies. In the rest of the article, we will use $\alpha ' = 1$ and $\alpha = (p+1)L/K \lambda_{\max}$. Using the least-squares approach, the number of ancestral populations, $K$, can be chosen after the evaluation of a cross-validation criterion for each $K$~\citep{Alexander2011, Frichot2014, Frichot2015}.

\paragraph{The Alternating Quadratic Programming (AQP) method} Because the poly\-edrons $\Delta_Q$ and  $\Delta_G$ are convex sets and the LS function is convex with
respect to each variable ${\bf Q}$ or ${\bf G}$ when the other one is fixed, the problem~\eqref{eq:LS} is amenable to the application of block coordinate descent~\citep{Bertsekas1995}. The APQ algorithm starts from initial values for the $G$ and $Q$-matrices, and alternates two steps. The first step computes the matrix {\bf G} while  {\bf Q} is kept fixed, and the second step permutates the roles of {\bf G} and {\bf Q}.  Let us assume that {\bf Q} is fixed and write {\bf G} in a vectorial form, $g = {\rm vec({\bf G})} \in \mathbb{R}^{K(p + 1)L}$. The first step of the algorithm actually solves the following quadratic programming subproblem. Find

\begin{equation}
\begin{aligned}
g^\star = \underset{g \in \Delta_G}{\arg \min}  ( -2  v^T_Q \, g + g^T {\bf D}_Q g ) \, ,
\end{aligned}
\label{eq:AQPg}
\end{equation}
\noindent where ${\bf D}_Q = {\bf I}_{(p+1)L} \otimes {\bf Q}^T {\bf Q}$ and $v_Q = {\rm vec}({\bf Q}^T {\bf Y})$. Here, $\otimes$ denotes the Kronecker product and ${\bf I}_d$ is the identity matrix with $d$ dimensions. Note that the block structure of the matrix ${\bf D}_Q$ allows us to decompose the subproblem~\eqref{eq:AQPg} into $L$ independent quadratic programming problems with $K(p + 1)$ variables. Now, consider that {\bf G} is the value obtained after the first step of the algorithm, and write {\bf Q} in a vectorial form, $q = {\rm vec({\bf Q})} \in \mathbb{R}^{nK}$. The second step solves the following quadratic programming subproblem. Find

\begin{equation}
\begin{aligned}
q^\star = \underset{q \in \Delta_Q}{\arg \min} ( -2 v^T_G \, q + q^T {\bf D}_G q ) \,  ,
\end{aligned}
\label{eq:AQPq}
\end{equation}

\noindent where ${\bf D}_G = {\bf I}_{n} \otimes {\bf G}^T {\bf G } + \alpha {\bf \Lambda}  \otimes {\bf I}_K$ and $v_G = {\rm vec}({\bf G}^T{\bf  Y}^T)$. Unlike subproblem~\eqref{eq:AQPg}, subproblem~\eqref{eq:AQPq} can not be decomposed into smaller problems. Thus, the computation of the second step of the AQP algorithm implies to solve a quadratic programming problem with $nK$ variables which can be problematic for large samples ($n$ is the sample size).
The AQP algorithm is described in details in Appendix~\ref{algo:aqp}. For AQP, we have the following convergence result.
\begin{thm}
\label{th}
	The AQP algorithm converges to a critical point of problem~\eqref{eq:LS}.
\end{thm}
\begin{proof}
The quadratic convex functions defined in subproblems~\eqref{eq:AQPg} and~\eqref{eq:AQPq} have finite lower bounds. The convex sets $\Delta_Q$ and $\Delta_G$ are not empty sets, and they are compact sets. Thus the sequence generated by the AQP algorithm is well-defined, and has limit points.
According to Corollary 2 of ~\cite{Grippo2000}, we conclude that
the AQP algorithm converges to a critical point of problem~\eqref{eq:LS}.
\end{proof}

\paragraph{Alternating Projected Least-Squares (APLS)} In this paragraph, we introduce an APLS estimation algorithm which approximates the solution of problem~\eqref{eq:LS}, and reduces the complexity of the AQP algorithm. The APLS algorithm starts from initial values of the $G$ and $Q$-matrices, and alternates two steps. The matrix {\bf G} is computed  while  {\bf Q} is kept fixed, and {\it vice versa}. Assume that the matrix {\bf Q} is known. The first step of the APLS algorithm solves the following optimization problem. Find

\begin{equation}
{\bf G}^\star = \arg \min  \|  {\bf Y} - {\bf QG}^T \|^2_{\rm F} \, .
\end{equation}
This operation can be done by considering $(p+1)L$ (the number of columns of ${\bf Y}$) independent optimization problems running in parallel. The operation is followed by a projection of ${\bf G}^\star$ on the polyedron of constraints, $\Delta_G$. For the second step, assume that {\bf G} is set to the value obtained after the first step is completed. We compute the eigenvectors, {\bf U}, of the Laplacian matrix, and we define the diagonal matrix ${\bf \Delta}$ formed by the eigenvalues of ${\bf \Lambda}$ (The eigenvalues of ${\bf \Lambda}$ are non-negative real numbers). According to the spectral theorem, we have

$$
{\bf \Lambda} = {\bf U}^T {\bf \Delta} {\bf U} \, .
$$
\noindent  After this operation, we project the data matrix {\bf Y} on the basis of eigenvectors as follows

$$
{\rm proj} ({\bf Y}) = {\bf U}{\bf Y} \, ,
$$
\noindent and, for each individual, we solve the following optimization problem

\begin{equation}
q_i^\star = \arg \min  \|  {\rm proj} ({\bf Y})_i  - {\bf G}^Tq \|^2 + \alpha \lambda_i \| q \|^2  \, ,
\label{eq:APSLq}
\end{equation}
\noindent where  proj({\bf Y}$)_i$ is the $i$th row of the projected data matrix, proj({\bf Y}), and $\lambda_i$ is the $i$th eigenvalue of ${\bf \Lambda}$. The solutions, $q_i$, are then concatenated into a matrix, ${\rm conc}(q)$, and ${\bf Q}$ is defined as the projection of the matrix ${\bf U}^T {\rm conc}(q)$ on the polyedron $\Delta_Q$. The complexity of step~\eqref{eq:APSLq} grows linearly with $n$, the number of individuals. While the theoretical convergence properties of AQP algorithms are lost for APLS algorithms, the APLS algorithms are expected to be good approximations of AQP algorithms. The APLS algorithm is described in details in Appendix~\ref{algo:apls}.

\paragraph{Comparison with {\tt tess3}}  The algorithm implemented in a previous version of {\tt tess3} also provides approximation of of solution of~\eqref{eq:LS}. The {\tt tess3} algorithm first computes a Cholesky decomposition of the Laplacian matrix. Then, by a change of variables, the least-squares problem is transformed into a sparse nonnegative matrix factorization problem~\citep{Caye2016}.  Solving the sparse non-negative matrix factorization problem relies on the application of existing methods~\citep{Kim2011, Frichot2014}. The methods implemented in {\tt tess3} have an algorithmic complexity that increases linearly with the number of loci and the number of clusters. They lead to estimates that accurately reproduce those of the Monte Carlo algorithms implemented in the Bayesian method {\tt tess} 2.3~\citep{Caye2016}. Like for the AQP method, the {\tt tess3} previous algorithms have an algorithmic complexity that increases quadratically with the sample size.

\paragraph{Ancestral population differentiation statistics and local adaptation scans} Assuming $K$ ancestral populations, the $Q$ and $G$-matrices  obtained from the AQP and from the APLS algorithms were used to compute single-locus estimates of a population differentiation statistic similar to $F_{\rm ST}$~\citep{Martins2016}, as follows

$$
F^{Q}_{\rm ST} = 1 - \sum_{k=1}^K  q_k \frac{f_k (1-f_k)}{f(1-f)} \, ,
$$

\noindent where $q_k$ is the average of ancestry coefficients over sampled individuals, $q_k = \sum_{i =1}^n q_{ik}/n$, for the cluster $k$, $f_k$ is the ancestral allele frequency in population $k$ at the locus of interest, and $f = \sum_{k = 1}^K q_k f_k$ (Martins et al. 2016). The locus-specific statistics were used to perform statistical tests of neutrality at each locus, by comparing the observed values to their expectations from the genome-wide background. The test was based on the squared $z$-score statistic, $z^2 = (n-K) F^{Q}_{\rm ST}/(1 - F^{Q}_{\rm ST})$, for which a  chi-squared distribution with $K-1$ degrees of freedom was assumed under the null-hypothesis~\citep{Martins2016}. The calibration of the null-hypothesis was achieved by using genomic control to adjust the test statistic for background levels of population structure~\citep{Devlin1999, Francois2016}. After recalibration of the null-hypothesis, the control of the false discovery rate was achieved by using the Benjamini-Hochberg algorithm~\citep{Benjamini1995}.

\paragraph{{\tt R} package} We implemented the AQP and APLS algorithms in the {\tt R} package {\tt tess3r}, available from Github and submitted to the Comprehensive R Archive Network (R Core Team, 2016).

\section{Simulated and real data sets}
\paragraph{Coalescent simulations} We used the computer program {\tt ms} to perform coalescent simulations of neutral and outlier SNPs under spatial models of admixture~\citep{Hudson2002}. Two ancestral populations were created from the simulation of Wright\rq{}s two-island models. The simulated data sets contained admixed genotypes for $n$ individuals for which the admixture proportions varied continuously along a longitudinal gradient~\citep{Durand2009, Francois2010}. In those scenarios, individuals at each extreme of the geographic range were representative of their population of origin, while individuals at the center of the range shared intermediate levels of ancestry in the two ancestral populations~\citep{Caye2016}. For those simulations, the $Q$ matrix, ${\bf Q}_0$, was entirely described by the location of the sampled individuals.

Neutrally evolving ancestral chromosomal segments were generated by simulating DNA sequences with an effective  population size $N_0 = 10^6$ for each ancestral population. The mutation rate per bp and generation was set to $\mu = 0.25 \times 10^{-7}$, the recombination rate per generation was set to $r = 0.25 \times 10^{-8}$, and the parameter $m$ was set to obtained neutral levels of $F_{\rm ST}$ ranging between values of $0.005$ and $0.10$. The number of base pairs for each DNA sequence was varied between 10k to 300k to obtain numbers of polymorphic locus ranging between 1k and 200k after filtering out SNPs with minor allele frequency lower than 5$\%$.  To create SNPs with values in the tail of the empirical distribution of $F_{\rm ST}$,  additional ancestral chromosomal segments were generated by simulating DNA sequences with a migration rate $m_s$ lower than $m$. The simulations reproduced the reduced levels of diversity and the increased levels of differentiation expected under hard selective sweeps occurring at one particular chromosomal segment in ancestral populations~\citep{Martins2016}.  For each simulation, the sample size  was varied in the range $n =$ 50-700.

 We compared the AQP and APLS algorithm estimates with those obtained with the {\tt tess3} algorithm.  Each program was run 5 times.  Using $K = 2$ ancestral populations, we computed the root mean squared error (RMSE) between the estimated and known values of the $Q$-matrix, and between  the estimated and known values of the $G$-matrix.
To evaluate the benefit of spatial algorithms, we compared the statistical errors of APLS algorithms to the errors obtained with {\tt snmf} method that reproduces the outputs of the {\tt structure} program accurately~\citep{Frichot2014,Frichot2015}.  To quantify the performances of neutrality tests as a function of ancestral and observed levels of $F_{\rm ST}$, we used the area under the precision-recall curve (AUC) for several values of the selection rate.  Subsamples from a real data set were used to perform a runtime analysis of the AQP and APLS algorithms ({\it A. thaliana} data, see below). Runtimes were evaluated by using a single computer processor unit Intel Xeon 2.0 GHz.

\paragraph{Application to European ecotypes of {\it Arabidopsis  thaliana}} We used  the APLS algorithm to survey spatial population genetic structure and to investigate the molecular basis of adaptation  by considering SNP data from 1,095  European ecotypes of the plant species {\it A. thaliana} (214k SNPs, \cite{Horton2012}). The cross-validation criterion was used to evaluate the number of clusters in the sample, and a statistical analysis was performed to evaluate the range of the variogram from the data. We used {\tt R} functions of the {\tt tess3r} package to display interpolated admixture coefficients on a geographic map of Europe (R Core team 2016). A gene ontology enrichment analysis using the software AMIGO~\citep{Carbon2009} was performed in order to evaluate which molecular functions and biological processes might be involved in local adaptation in Europe.