% Question  ##################################################################################################################
\section{Question 2}\label{ssec:pt2q2}
\textbf{In this question we will analyse Apple Stock.}
% END Question  ##############################################################################################################

% Question (i) ###############################################################################################################
\subsection{Q2 (i)}\label{sssec:pt2q2i}

\textbf{Download closing prices for Apple for the period 01/01/2013 and 30/11/2017. Apply daily log returns and measure the average daily return.}

\noindent
Code to download the closing prices for Apple can be found in ‘Question 2 (i)’ in the python notebook. The data was downloaded from Yahoo Finance and saved as .csv format in the following path \textit{‘src/data/part\_2’}. The file was saved as \textit{‘AAPL.csv’}. The data was then reloaded from the file and converted into pandas dataframe.

\noindent
After applying the log returns the measure of the average daily log returns was 0.091\%.

% END Question (i) ###########################################################################################################

% Question (ii) ##############################################################################################################

\subsection{Q2 (ii)}\label{sssec:pt2q2ii}
\textbf{We wish to measure volatility. Compare and discuss pros and cons of utilising the Historical method, Exponentially Moving Average method or Garch.}

\noindent
\textit{Historical Method}

\noindent
To measure volatility using the historical method, first the historic data must be obtained. Once the data is obtained the log returns must be calculated and then the standard deviation is calculated on the log returns. In this model when higher frequency data is used the sum of the squared returns is used instead. In this method all past data is given an equal weight (no weight at all).

Pros:
\begin{itemize}
    \item Faster to compute and easy to implement.
    \item Does not require any parameters.
    \item No model risk since no parameters are used.
\end{itemize}

Cons:
\begin{itemize}
    \item Does not assign any weights and all values are given the same importance.
    \item Less accurate since it does not give importance to recent values.
    \item Data is not utilised properly (most of the data is not used).
\end{itemize}

\noindent
\textit{EWMA}

\noindent
Unlike the historical method, the exponentially weighted moving average method does not give equal weights to the data points.  This is done as the recent square returns should give a better indication for volatility. In this method an exponentially decreasing weight is set to the squared returns (starts from the most recent data point).

Pros:
\begin{itemize}
  \item Assigns exponentially decreasing weight to the squared returns. Recent returns better describe volatility.
  \item Less parameters than the GARCH model, thus less Model Risk.
\end{itemize}

Cons:
\begin{itemize}
    \item This model do not handle mean reversion (no term used for mean reversion).
    \item Does not capture trend information.
    \item Does not forecast volatility.
    \item A volatility shock is suddenly dropped from the computation as the trailing window of observations pass.
\end{itemize}

\noindent
\textit{Garch}

\noindent
The Generalized ARCH model is an extension to the ARCH model. This model also uses weights, but the weights are assigned to the following three components; lagged variance, lagged squared return and the long-run variance. The sum of the weights assigned to the first two components is known as persistence. Commonly this model is solved using the maximum likelihood estimation technique.

Pros:
\begin{itemize}
  \item More weight on recent information.
  \item Can output more accurate estimations.
  \item A term is used to handle mean reversion.
  \item Can be used to forecast volatility.
  \item Overcomes Ghosting as a shock in volatility will be immediately impact the estimates (Fades as time passes).
\end{itemize}

Cons:
\begin{itemize}
    \item This models requires more parameters and has a greater Model Risk.
    \item More computationally expensive.
\end{itemize}

\noindent
Comparing the three models, the first model does not assign weights to the log returns, while the EWMA uses exponentially decreasing weights to the squared returns. On the other hand, the GARCH models’ assigns weights to the following components lagged variance, lagged squared returns and long run variance. Both the GARCH and EWMA work with conditional variance but the other model works with realized variance. Also both GARCH and EWMA apply exponential smoothing to their weights. Unlike GARCH the EWMA does not have a term for mean reversion. Both GARCH and EWMA are recursive and parametric.

% END Question (ii) ##########################################################################################################

% Question (iii) #############################################################################################################

\subsection{Q2 (iii)}\label{sssec:pt2q2iii}
\textbf{Use Garch(1,1) to measure volatility based on the data from (i).}

\noindent
The measurement for the annualized volatility for Apple when using Garch(1,1) is 22.46\%.

\noindent
Volatility was computed using the code found in ‘Question 2 (iii)’ in the python notebook and this notebook cell utilises a function called 'arch\_vol' found in the 'fintech' library. Such function uses a third party library called 'arch' \cite{python:arch} which provides different volatility models written in python.

% END Question (iii) #########################################################################################################

% Question (iv) ##############################################################################################################

\subsection{Q2 (iv)}\label{sssec:pt2q2iv}
\textbf{Assume that Apple stock prices follow a Geometric Brownian stochastic process. Utilising the average return and volatility statistics identified in part (i) and (iii), simulate, on a day by day basis, the movement of Apple stock for the next 20 days (i.e. starting from 1st December, as per date range in part (i)). Repeat this process for 100 times, hence creating 100 possible price realisations over the next 20 days. Describe the steps/approach used and present a plot showing the projected 100 realisations.}

\noindent
First the current price was referenced, which is the adjusted close price for the 1st of December. The expected volatility was computed by using the annualized volatility found in (iii) and scaling it to get the expected volatility for the next day. This was done using the following computation (assuming a year has 250 days) $(Annualized \ Volatility \times \sqrt{\frac{1}{250}})$.

\noindent
Now that we have the expected return and expected volatility, an iteration to create different projections was written. The iteration runs for 100 times, since we need to create 100 possible price realisations. In each iteration a new Brownian path is generated using a function which was created in the ‘fintech’ library called ‘brownian\_motion’ as shown in Fig.~\ref{fig:brownpath}. When calling this function, the number of increments is passed as a parameter which in our case is 20 since we are projecting for the next 20 days.

\begin{figure}[H]
\centering
  \includegraphics[scale = .64]{imgs/brownian_path.png}
  \caption{Function to create Brownian path.}
  \label{fig:brownpath}
\end{figure}

\noindent
In this function the Brownian increments $W_i$ are computed by getting $z_i$ which is a random variable selected from a normal distribution with $\mu=0$ and $\sigma=1$ and multiplying it by $\sqrt{t_i}$ where $t_i$ is the time increment, as shown in Eq.~\ref{eq:brownincr}.

\begin{equation} \label{eq:brownincr}
    W_i = z_i \sqrt{\Delta t_i}
\end{equation}

\noindent
After computing the Brownian increments, the Brownian path is computed by getting the cumulative sum of the Brownian increments. This is shown in Eq.~\ref{eq:brownpath}, and the 'brownian\_motion' function returns both the path and the increments.

\begin{equation} \label{eq:brownpath}
    W_n(t) = \sum_{i=1}^{n} W_i(t)
\end{equation}

\noindent
In each iteration when a new Brownian path is generated, the path is used as a parameter in a function called 'geo\_brownian\_motion' which is also found in the 'fintech' library and this function is shown in Fig.~\ref{fig:browngeo}. The returned stock prices and time periods were used to plot the possible price realisations. Each price at $t$ is computed using Eq.~\ref{eq:geobrown}.

\begin{figure}[H]
\centering
  \includegraphics[scale = .64]{imgs/geo_brown_path.png}
  \caption{Function to create Geometric Brownian Motion.}
  \label{fig:browngeo}
\end{figure}

\begin{equation} \label{eq:geobrown}
    S_{t+\Delta t} = S_t \exp[(\mu - \frac{\sigma^2}{2}) + \Delta t + \sigma \epsilon \sqrt{\Delta t}]
\end{equation}

\noindent
These functions were then called in the notebook in 'Question 2 (iv)' as shown in Fig.~\ref{fig:notebrown} and the final output was plotted as shown in Fig.\ref{fig:notebrownplot}.

\begin{figure}[H]
\centering
  \includegraphics[scale = .60]{imgs/simulation100.png}
  \caption{Code for simulation using Geometric Brownian path.}
  \label{fig:notebrown}
\end{figure}

\begin{figure}[H]
\centering
  \includegraphics[scale = .64]{imgs/simulationplot.png}
  \caption{100 price realisations over the next 20 days using Geometric Brownian Motion.}
  \label{fig:notebrownplot}
\end{figure}

% END Question (iv) ##########################################################################################################

% Question (v) ###############################################################################################################

\subsection{Q2 (v)}\label{sssec:pt2q2v}
\textbf{Using the prices obtained at the end of each simulated realisation, i.e. simulated prices at
t+20 days, calculate the 20 day Value at Risk (VaR) at 95\% confidence level. Hint: You can
use the percentile approach on the 20 day returns. }

\noindent
The 20 day VaR(95\%) based on the simulation for question (iv) is \$3.40. Code for this measurement is shown in 'Question 2 (v)' in the notebook.

% END Question (v) ###########################################################################################################


\documentclass[12pt]{article}
\usepackage[utf8]{inputenc}
\usepackage{float}
\usepackage{amsmath}

\usepackage[hmargin=3cm,vmargin=6.0cm]{geometry}
%\topmargin=0cm
\topmargin=-2cm
\addtolength{\textheight}{6.5cm}
\addtolength{\textwidth}{2.0cm}
%\setlength{\leftmargin}{-5cm}
\setlength{\oddsidemargin}{0.0cm}
\setlength{\evensidemargin}{0.0cm}

%misc libraries goes here
%\usepackage{fitch}

\begin{document}

\section*{Student Information }
%Write your full name and id number between the colon and newline
%Put one empty space character after colon and before newline
Full Name : ERTUGRUL AYPEK \\
Id Number : 2171270 \\
\\
% Write your answers below the section tags
\section*{Answer 1}
1-

\begin{table}[H]
	\centering
	\caption{Tautology for 1.1}
	\begin{tabular}{|c|c|c|c|c|c|c|}
	\hline
	\textbf{$p$} & \textbf{$q$} & \textbf{$p \rightarrow q$} & \textbf{$\neg q \vee(p \rightarrow q)$} & \textbf{$\neg q \wedge(p \rightarrow q)$} & \textbf{$\neg p$} & \textbf{$(\neg q \wedge (p \rightarrow q)) \rightarrow \neg p$} \\
	\hline
	T & T & F & T & F & F & T  \\ \hline
	T & F & T & F & F & F & T \\ \hline
	F & T & F & T & F & T & T \\ \hline
	F & F & T & T & T & T & T \\
	\hline

	\end{tabular}

\end{table}

\begin{flushleft}
\textbf{2-}
\end{flushleft}

\begin{table}[H]
	\centering
	\caption{Tautology for 1.2}
	\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
	\hline
	\textbf{$p$} & \textbf{$q$} & \textbf{$r$} & \textbf{$\neg p$} & \textbf{$p \vee q$} & \textbf{$\neg p \vee r$} & \textbf{$(p \vee q) \wedge ( \neg p \vee r)$} & \textbf{$q \vee r$} & \textbf{$((p \vee q) \wedge ( \neg p \vee r)) \rightarrow (q \vee r)$}	\\
	\hline
	T & T & T & F & T & T & T & T & T \\ \hline
	T & T & F & F & T & F & F & T & T \\ \hline
	T & F & T & F & T & T & T & T & T \\ \hline
	T & F & F & F & T & F & F & F & T \\ \hline
	F & T & T & T & T & T & T & T & T \\ \hline
	F & T & F & T & T & T & T & T & T \\ \hline
	F & F & T & T & F & T & F & T & T \\ \hline
	F & F & F & T & F & T & F & F & T \\
	\hline

	\end{tabular}
\end{table}

\section*{Answer 2}
Showing (p $\rightarrow$ q) $\vee$ (p $\rightarrow$ r) and ($\neg$ q $\wedge$ $\neg$ r)$\rightarrow$ $\neg$ p are logically equivalent by deducing the right hand side from left hand side: \\
\\
(p $\rightarrow$ q) $\vee$ (p $\rightarrow$ r) $\equiv$ p $\rightarrow$ (q$\vee$r) .........\textit{from Table 7}
\\
(p $\rightarrow$ q) $\vee$ (p $\rightarrow$ r) $\equiv$ $\neg$(q $\vee$ r) $\rightarrow$ $\neg$ p ......... \textit{from Table 7}
\\
(p $\rightarrow$ q) $\vee$ (p $\rightarrow$ r) $\equiv$ ($\neg$ q $\wedge$ $\neg$ r)$\rightarrow$ $\neg$ p ......... \textit{from Table 6, De Morgan’s laws}

\section*{Answer 3}
1.a) For every cat there exists some dogs such that they are friends. \\
1.b) Some cats are friends of all dogs\\ \\

2.a) $\forall$x$\forall$y($\neg$Customer(x)$\wedge$Meal(y) $\rightarrow$ $\neg$ Eats(x,y)) \\
2.b) $\neg$($\forall$x$\forall$y(Chef(x)$\wedge$Meal(y)$\rightarrow$Cooks(x,y)))\\
2.c)
2.d) $\forall$x$\exists$y$\exists$z(Chef(x)$\rightarrow$Chef(z)$\wedge$(Meal(y)$\rightarrow$($\neg$Cook(x,y)$\wedge$Cooks(z,y)$\wedge$Knows(x,z)))

\section*{Answer 4}

\section*{Answer 5}

\section*{Answer 6}

\end{document}


%% -*- coding:utf-8 -*-
\chapter{Solvability by radicals, Abel's theorem. A few words on
  relation to representations and topology}

We finally arrive to the source of Galois theory, the question which
motivated Galois himself: which equation are solvable by radicals and
which are not? We explain Galois' result: an equation is solvable by
radicals if and only if its Galois group is solvable in the sense of
group theory. In particular we see that the "general" equation of
degree at least 5 is not solvable by radicals. We briefly discuss the
relations to representation theory and to topological coverings.

\section{Extensions solvable by radicals. Solvable groups. Example}

\subsection{Extensions solvable by radicals}

Let $K$ is a field of \mynameref{def:fieldcharacteristic} 0: $char(K) = 0$. It is embedded
into its \mynameref{def:algebraicclosure}.

\begin{definition}[Extension solvable by radicals]
  A finite extension $E$ of $K$ is solvable by radicals if
  $\exists \alpha_1, \dots, \alpha_r$ generating $E$ such that
  $\alpha_i^{n_i} \in K\left(\alpha_1, \dots, \alpha_{i-1}\right)$ for
  some $n_i \in \mathbb{N}$.
  \label{def:solvableextension}
\end{definition}

\begin{example}
  Let $K = \mathbb{Q}$, $E = \mathbb{Q}\left(\sqrt[3]{2 + 3
    \sqrt{7}}, \sqrt[5]{4 + 5 \sqrt{11}}\right)$. We have
    $\alpha_1 = \sqrt{7}, \alpha_2 = \sqrt{11},
    \alpha_3 = \sqrt[3]{2 + 3\sqrt{7}}, \alpha_4 = \sqrt[5]{4 + 5
      \sqrt{11}}$.
\end{example}

\begin{definition}[Polynomial solvable by radicals]
  $P \in K\left[X\right]$ is called solvable by radicals if exists a
  $E$ - \mynameref{def:solvableextension} and containing all roots of
  $P$.
  \label{def:solvablepolynomial}
\end{definition}
So more precisely, it would say that the equation, $P = 0$ is solvable
by radicals.

\begin{property}
  \begin{enumerate}
  \item \mynameref{def:compositeextension} of solvable by radicals is itself
    solvable by radicals
  \item If $L$ extension of $K$ is solvable by radicals (by definition
    $L$ should be finite extension of $K$) then exists a finite
    \mynameref{def:galoisextension} $E$ containing $L$ and solvable by
    radicals.
  \end{enumerate}
  \label{property:solvable}
  \begin{proof}
    For the first property (the proof is missing in the lectures): let
    $L = L_1 L_2$ where $L_1$ and $L_2$ are solvable i.e.
    $L_1 = K\left(\alpha_1, \dots, \alpha_n\right),
    L_2 = K\left(\beta_1, \dots, \beta_m\right)$ with $\alpha_i,
    \beta_j$ which satisfies properties from definition
    \ref{def:solvableextension}. In the case we can assume
    $L = L_1 L_2 = K\left(L_1 \cup L_2\right) =
    K\left(\alpha_1, \dots, \alpha_n, \beta_1, \dots, \beta_m\right)$
    and all properties from definition
    \ref{def:solvableextension} will also be satisfied. Therefore
    the composite extension $L = L_1 L_2$ is also solvable.

    For the second property: Indeed take a composite of all images of $L$ in
    $\bar{K}$ or, in other words, images of $L$ by
    $Gal\left(\bar{K}/K\right)$.
    \footnote{
      Lets prove by induction. For the first step we have that
      $K\left(\alpha\right)$ is solvable therefore
      $\exists d = \min\{j \mid \alpha^j \in K\}$. Let
      $\zeta_d$ is a root of unity i.e. root of $X^d - 1$ then by
      proposition \ref{prop:lec7_1}, the
      $K\left(\zeta_d, \alpha\right) \supset K\left(\alpha\right)$ is
      a Galois extension and it has a finite degree ($\le d \cdot d$).

      Using induction hypothesis we have that if
      $K\left(\alpha_1, \dots, \alpha_i\right)$ is solvable then there
      exists a Galois extension of finite degree
      $F = K\left(\beta_1, \dots, \beta_k \right)$ that is solvable
      and $K\left(\alpha_1, \dots, \alpha_i\right) \subset F$.
      By \mynameref{thm:galoiscorrespondence} (item
      \ref{thm:galoiscorrespondence:item2b}) we have that
      $\forall g \in G = Gal\left(\bar{K}/K\right)$:
      \[
      g\left(F\right) = F.
      \]
      Lets consider $F\left(\alpha_{i+1}\right)$. As we know
      (by induction hypothesis)
      $K\left(\alpha_1, \dots, \alpha_i, \alpha_{i+1}\right)$ is
      solvable i.e.
      \(
      \exists d = \min\{k \mid \alpha_{i+1}^k
      \in K\left(\alpha_1, \dots, \alpha_i\right)\}
      \)
      therefore $\alpha_{i+1}$ is a root of the following irreducible
      polynomial $X^d - a$ where $a = \alpha_{i+1}^d$.
      The Galois group $G = Gal\left(\bar{K}/K\right)$ permutes the
      roots of the polynomial and as result $\forall g \in G$
      \[
      L_g = g\left(F\left(\alpha_{i+1}\right)\right) =
      F\left(\beta_g\right),
      \]
      where $\beta_g = g\left(\alpha_{i+1}\right)$ is a root of $X^d -
      a$. I.e. $L_g$ is solvable. If we take a composite extension
      $L \cup_{g \in G} L_g$ then by first property of
      \ref{property:solvable} it will be solvable but as soon as
      $L$ is the image of all $g \in G$ then
      $\forall g \in G: g\left(L\right) = L$ and by
      \mynameref{thm:galoiscorrespondence} (item
      \ref{thm:galoiscorrespondence:item2b})  the extension $L$ is
      Galois. $L$ is a splitting field of $X^d - a$ (because it
      contains all its roots) but by theorem \ref{thm:lec2_1} it has
      finite degree: $\left[L : F\right] \le d!$.
    }
  \end{proof}
\end{property}

\subsection{Solvable groups}
This shall be a brief reminder since this is not a course on group
theory, you are supposed to know some group theory already. So I
somehow I presume that you are familiar with this definition but I
will recall the definition of basic properties.

\begin{definition}[Solvable group]
  $G$ is called solvable if it has a filtration
  i. e. $G = G_0 \supset G_1 \supset \dots \supset G_{r-1} \supset G_r
  = \left\{e\right\}$, such that $G_i$ is a
  \mynameref{def:normalsubgroup} of
  $G_{i-1}$ and the \mynameref{def:quotientgroup} $G_{i-1}/G_i$ is
  an \mynameref{def:abeliangroup}.
  \label{def:solvablegroup}
\end{definition}

\begin{example}[Group of permutations $S_3$]
  Consider $S_3$ - the group of permutations (see also example
  \ref{ex:s3group}). It's solvable because
  $S_3 \supset A_3 \supset \left\{e\right\}$.

  We have $\left|S_3/A_3\right| = 2$ (see example
  \ref{ex:s3a3quotientgroup}) i.e. $S_3/A_3$ is cyclic of order
  2. $\left|A_3\right| = 3$ i.e. $A_3$ - cyclic of order 3.
  \footnote{
    As it was mentioned at \cite{wiki:finitegroup} there is only one
    group of order 3. We also have (with theorem
    \ref{thm:cyclic_group_is_abelian}) that $A_3$ is
    \mynameref{def:abeliangroup}
  }
  \label{ex:lec8_s3}
\end{example}

\begin{example}[Group of permutations $S_4$]
  Consider $S_4$ - the group of permutations (see also example
  \ref{ex:s3group}). It's solvable because
  $S_4 \supset A_4 \supset K \supset \left\{e\right\}$, where $K$ -
  is a subgroup
  \footnote{
    There is a well known \mynameref{def:v4} $V_4$ -
    the only non-cyclic group of order 4. It also denoted as $K_4$. We
    have already seen it at example \ref{ex:lec7_cyclotomic8}.
  }
  of double transpositions
  (see example
  \ref{ex:permutation} for permutation cycles notation):
  \[
  K = \left\{
  e, (12)(34), (13)(24), (14)(23)
  \right\}.
  \]
  A double transposition is a product of two \mynameref{def:transposition}s with
  distinct support, right, which permute the distinct elements.
  $A_4 \triangleleft S_4$, $\left|S_4/A_4\right| = 2$, i.e.
  $S_4/A_4$ is cyclic of order 2.
  \footnote{
    i.e. theorem
    \ref{thm:cyclic_group_is_abelian} gives us that $S_4/A_4$ is
    \mynameref{def:abeliangroup}
  }

  $K \triangleleft A_4$, $\left|A_4/K\right| = 3$, i.e.
  $A_4/K$ is cyclic of order 3.
  \footnote{
    i.e. theorem
    \ref{thm:cyclic_group_is_abelian} gives us that $A_4/K$ is
    \mynameref{def:abeliangroup}
  }

  $K$ is \mynameref{def:abeliangroup} and
  $K \cong \mathbb{Z}/2\mathbb{Z} \times \mathbb{Z}/2 \mathbb{Z}$.

  So this shows that $S_4$ is solvable.
  \label{ex:lec8_s4}
\end{example}

\section{Properties of solvable groups. Symmetric group}

\begin{property}
  If $G$ is solvable and $H \subset G$ is a subgroup of $G$ then $H$
  is solvable.
  \begin{proof}
    Indeed $G_i \cap H$ gives a filtration with required property.
    \footnote{
      We have $H_i = G_i \cap H$ and as soon as
      $G_i \triangleleft G_{i-1}$ we also get
      $H_i \triangleleft H_{i-1}$. Really $\forall h \in H_{i-1}$ we
      have $h \in G_{i-1} \cap H \subset G_{i-1}$. Thus, using normality
      (see definition \ref{def:normalsubgroup})
      $G_i \triangleleft G_{i-1}$,
      \[
      h G_{i} = G_{i} h.
      \]
      The last equation also holds for any subset of $G_{i-1}$ and
      especially for $H_{i} = G_{i} \cap H \subset G_{i}$. I.e.
      $\forall h \in H_{i-1}: h H_i = H_i h$ or in other words,
      $H_i \triangleleft H_{i-1}$.

      We have that $G_{i-1}/G_i$ is an
      \mynameref{def:abeliangroup} and now we have to prove that the
      \mynameref{def:quotientgroup}
      $H_{i-1}/H_i$ is abelian. $\forall h' \in H_{i-1}/H_i, \exists
      h_{i-1} \in H_{i-1}$ such that $h' = \left\{h_{i-1},
      H_i\right\}$. From other hand $h_{i-1} \in G_{i-1}$ and
      $H_i \subset G_i$ therefore we can associate with $h'$ the
      $g' =  \left\{h_{i-1}, G_i\right\} \in G_{i-1}/G_i$. I.e. we just
      got an injection $f: H_{i-1}/H_i \xrightarrow[h' \to g']{}
      G_{i-1}/G_i$. Obviously we have $\forall h',h'' \in H_{i-1}/H_i$
      the following relation
      \[
      f\left(h' h''\right) = g' g'' =
      f\left(h'\right) f\left(h''\right),
      \]
      where $g'' = \left\{h'', G_i\right\}$ and
      $g' g'' = \left\{h' h'', G_i\right\}$. Therefore $f$ is
      \mynameref{def:homomorphism}. Thus $\forall a,b \in H_{i-1}/H_i$
      we have $f(ab) = f(a) f(b)$, but as soon as $G_{i-1}/G_i$ is
      abelian, $f(a) f(b) = f(b) f(a) = f(ba)$, or, using property of
      homomorphism, one can get that $ab=ba$ i.e. $H_{i-1}/H_i$ is an
      \mynameref{def:abeliangroup} and as result $H$ is solvable.
    }
  \end{proof}
  \label{property:lec8_solvable1}
\end{property}

\begin{property}
  If $G$ is solvable and $H \triangleleft G$ is a normal subgroup of
  $G$ then $G/H$ is solvable.
  \begin{proof}
    Indeed consider a projection map
    \begin{equation}
      \pi: G \to G/H
      \label{eq:lec8_solvable_pi}
    \end{equation}
    then $\pi\left(G_i\right)$ gives a filtration $\left(G/H\right)_i$
    on $G/H$ with required properties.
    \footnote{
      We can associate each element $g \in G$ with
      $\bar{g} \in G/H$ via the following map:
      $\pi: G \xrightarrow[g \to \bar{g}]{} G/H$. I.e.
      $\pi(g) = \bar{g} = gH$. Thus we also have
      $\pi\left(G_i\right) = \bar{G}_i = G_i H$. Lets prove that
      $\bar{G}_i$ forms the required filtration i.e. that
      $\bar{G}_i \triangleleft \bar{G}_{i-1}$
      and $\bar{G}_{i-1}/\bar{G}_i$ is \mynameref{def:abeliangroup}.

      For normality (see definition \ref{def:normalsubgroup}) prove we
      have that
      ${G}_i \triangleleft {G}_{i-1}$ i.e.
      $\forall g \in {G}_{i-1}: g G_i = G_i g$ but
      $\forall \bar{g} \in \bar{G}_{i-1}$ we have
      \[
      \bar{g} \bar{G}_{i} = g H G_i H
      \]
      but $H \triangleleft G$ i.e.
      $\forall g' \in G_i \subset G: g' H = H g'$ therefore
      \[
      \bar{g} \bar{G}_{i} = g G_i H =
      G_i g H = G_i g H H = G_i H g H = \bar{G}_i \bar{g}
      \]
      that finished the normality proof.

      For the abelian quotient proof we have that
      ${G}_{i-1}/{G}_i$ is \mynameref{def:abeliangroup} therefore
      $\forall a,b \in {G}_{i-1}: a b G_i = b a G_i$. Using the fact
      that $H$ is a \mynameref{def:normalsubgroup} one can get
      \begin{eqnarray}
        \bar{a} \bar{b} \bar{G}_i =
        a H b H G_i H = a b G_i H = b a G_i H =
        \nonumber \\
        = b a G_i H H = b a H G_i H = b a H H G_i H =
        b H a H G_i H =
        \nonumber \\
        = \bar{b} \bar{a} \bar{G}_i,
        \nonumber
      \end{eqnarray}
      i.e. $\bar{G}_{i-1}/\bar{G}_i$ is \mynameref{def:abeliangroup}.
    }
  \end{proof}
  \label{property:lec8_solvable2}
\end{property}

\begin{property}
  If $H \triangleleft G$, $H$ and $G/H$ are solvable then $G$ is
  solvable.
  \label{property:lec8_solvable3}
  \begin{proof}
    Put togeter the filtration $H_i$ and
    $\pi^{-1}\left(\left(G/H\right)_j\right)$ (see
    (\ref{eq:lec8_solvable_pi}) for $\pi$ definition).
    \footnote{
      $\forall \bar{g} \in \bar{G} = G/H$ we can consider it as a set
      of elements: $\bar{g} = \{\forall h \in H: h g\}$ where
      $g \in G$.
      If we combine then we will get the following: $\cup_g \bar{g} =
      G$.
      We also have
      \begin{equation}
        G H = \cup_g \bar{g} H =
        \cup_g g H H = \cup_g g H = \cup_g \bar{g} = G.
        \label{eq:lec8_solvable3_note_bar}
      \end{equation}
      Thus from $\bar{G}_i \triangleleft \bar{G}_{i - 1}$ one can
      get $\forall \bar{g} \in \bar{G}_{i - 1}: \bar{g} \bar{G}_i =
      \bar{G}_i \bar{g}$ or in other words
      (with $G_{i - 1} = \cup_{\bar{g} \in \bar{G}_{i - 1}} \bar{g}$)
      $\forall g \in G_{i - 1}$ we have the following
      \[
      g G_{i - 1} = g H G_{i - 1} H =
      \bar{g} \bar{G}_{i-1} =
      \bar{G}_{i-1} \bar{g} =
      G_{i - 1} H g H =
      G_{i - 1} H H g = G_{i - 1} g,
      \]
      i.e.
      ${G}_i \triangleleft {G}_{i - 1}$.
      As result the filtration
      \[
      \{e\} = \bar{H} \triangleleft \dots \triangleleft
      \bar{G}_i \triangleleft \bar{G}_{i - 1} \triangleleft \dots
      \triangleleft \bar{G}_0 = \bar{G}
      \]
      produces the following one
      \begin{equation}
      H \triangleleft \dots \triangleleft
      G_i \triangleleft G_{i - 1} \triangleleft \dots
      \triangleleft G_0 = G.
      \label{eq:lec8_solvable3_note}
      \end{equation}
      If $\bar{G}_{i - 1}/\bar{G}_{i}$ is abelian, i.e.
      $\forall \bar{a},\bar{b} \in \bar{G}_{i - 1}$:
      $\bar{a} \bar{b} \bar{G}_{i} = \bar{b} \bar{a} \bar{G}_{i}$, we
      can get the following
      $\forall a, b \in G_{i-1}$ (as soon as
      (\ref{eq:lec8_solvable3_note_bar}) gives us $G_i = G_i H$)
      \begin{eqnarray}
      a b G_i = a b G_i H = a b G_i H H =
      a b H G_i H = a b H H G_i H = a H b H G_i H =
      \nonumber \\
      =
      \bar{a} \bar{b} \bar{G}_{i} = \bar{b} \bar{a} \bar{G}_{i} =
      b H a H G_i H = b a H G_i H = b a G_i H = b a G_i
      \nonumber
      \end{eqnarray}
      i.e. the \mynameref{def:quotientgroup} $G_{i-1}/G_i$ is an Abelian
      group.

      Finally combining (\ref{eq:lec8_solvable3_note}) with $H$
      solvability one can get
      \[
      \{e\} \triangleleft \dots
      \triangleleft
      H_j \triangleleft H_{j - 1} \triangleleft \dots
      \triangleleft H
      \triangleleft \dots \triangleleft
      G_i \triangleleft G_{i - 1} \triangleleft \dots
      \triangleleft G_0 = G
      \]
      i.e. $G$ is solvable.
    }
  \end{proof}
\end{property}

The following property is not a part of the lectures but it's required
for the property \ref{property:lec8_solvable4} proof
\begin{property}
  If $G$ is finite \mynameref{def:abeliangroup} than $G$ is solvable and
  there exists a finite filtration with cyclic quotients for $G$.
  \begin{proof}
    Accordingly theorem \ref{thm:fgagroup}) each finite
    \mynameref{def:abeliangroup} $G$ can be represented as
    \[
    G = K_1 \oplus K_2 \oplus \dots \oplus K_{n-1} \oplus K_n,
    \]
    where $K_i$ is a \mynameref{def:cyclicgroup}. If we denote
    \[
    G_i = K_1 \oplus K_2 \oplus \dots \oplus K_{i}
    \]
    when, by property \ref{property:directproduct}, we can get
    \[
    \{e\} = G_0 \triangleleft G_1 \triangleleft \dots \triangleleft
    G_n = G.
    \]
    The gotten filtration has cyclic quotients because
    (see property \ref{property:directproductquotient})
    $G_i/G_{i-1} = K_i$, where $K_i$ is a cyclic group.
  \end{proof}
  \label{property:lec8_solvable4add}
\end{property}

\begin{property}
  If $G$ is finite than $G$ is solvable (i.e. has a finite filtration with
  Abelian quotients) if and only if there exists a
  finite filtration with cyclic quotients.
  \label{property:lec8_solvable4}
  \begin{proof}
    This is just because a finite \mynameref{def:abeliangroup} is just a
    product of cyclic groups (see see \mynameref{thm:fgagroup}).
    \footnote{
       If exists a finite filtration of $G$ with cyclic quotient then
      the $G$ is solvable because \mynameref{def:cyclicgroup} is
      abelian (see theorem \ref{thm:cyclic_group_is_abelian}).

      The other direction is not so simple. Lets $G$ is solvable then
      $G_i \triangleleft G_{i - 1}$ and $G_{i-1} / G_{i} = K$ -
      abelian. The property \ref{property:lec8_solvable4add} (that
      follows from \mynameref{thm:fgagroup}) says us that there exists a
      finite filtration with cyclic quotients for $K$ i.e.
      \[
      \{e\} = K^{(n)} \triangleleft \dots \triangleleft
      K^{(j)} \triangleleft K^{(j - 1)} \triangleleft \dots \triangleleft
      K_0 = K
      \]
      where $K^{(j-1)}/K^{(j)}$ is cyclic. The \mynameref{thm:correspondence}
      says us that $\forall K^{(j)}, \exists G_{i}^{(j)} \subset G_{i-1}$
      with the same properties i.e.
      $G_i^{(j)} \triangleleft G_i^{(j-1)}$ and
      $G_i^{(j-1)}/G_i^{(j)}$ is cyclic. As result we can complete
      each $G_i \triangleleft G_{i-1}$ with
      \[
      G_i = G_i^{(n)} \triangleleft \dots
      \triangleleft G_i^{(j)} \triangleleft G_i^{(j-1)} \triangleleft \dots
      \triangleleft G_i^{(0)} = G_{i-1}
      \]
      and the result filtration will have cyclic quotients.
      %% The $K$ is a finite \mynameref{def:abeliangroup}. But by
      %% theorem \ref{thm:simple_subgroup_of_abelian} any finite abelian
      %% group $K$ has a simple (see \mynameref{def:simplegroup}) subgroup
      %% $K_1$. The group $K/K_1$ is again abelian, so it has a simple
      %% subgroup $K_2/K_1$ we can continue until we reach $K$. Thus we
      %% get a set of subgroups
      %% of $G_i$ containing $G_{i-1}$ that we can
      %% insert in the original series; repeat for each $i$ and finally
      %% you get a composition series with abelian factors.
    }
  \end{proof}
\end{property}

Lets also look at another definition of solvable group
\begin{definition}[Solvable group]
  $G$ is called solvable if the following sequence is finite:
  \[
  G
  \supseteq \left[G, G\right] = G^{(1)}
  \supseteq \left[G^{(1)}, G^{(1)}\right] = G^{(2)}
  \supseteq \dots \supseteq
  \left[G^{(n-1)}, G^{(n-1)}\right] = G^{(n)} = \left\{e\right\}
  \]
  where $G^{(i)} = \left[G^{(i-1)}, G^{(i-1)}\right]$ is the
  \mynameref{def:commutatorsubgroup}.
  \label{def:solvablegroupadd}
\end{definition}

\begin{remark}
  Definitions of solvable group \ref{def:solvablegroupadd} and
  \ref{def:solvablegroup} are equivalent.
  \begin{proof}
    Our filtration with \mynameref{def:commutatorsubgroup}s
    $G \supseteq G^{(1)} \supseteq \dots \supseteq G^{(n)} =
    \left\{e\right\}$
    is a filtration with abelian quotient because
    $G/\left[G, G\right]$ as well as
    $G^{(i)}/\left[G^{(i)}, G^{(i)}\right] = G^{(i)}/G^{(i+1)}$ are
    \mynameref{def:abeliangroup}s.
    \footnote{
      See definition \ref{def:abelianization} and theorem
      \ref{thm:about_quotient_group_and_commutatorsubgroup}
    }

    From the other hand if $G/H$ is an \mynameref{def:abeliangroup} then
    $H \supset \left[G, G\right]$.
    \footnote{
      See theorem \ref{thm:about_quotient_group_and_commutatorsubgroup}
    }
    So if a finite filtration with
    abelian quotient exists then the filtration given by $G^{(i)}$ is
    also finite. It must terminate after a finite steps. So, this
    proves the equivalence.
  \end{proof}
  \label{rem:lec8_solvable}
\end{remark}

\begin{theorem}[$S_n$ solvability]
  $S_n$ - the permutation of $n$ elements (see example
  \ref{ex:sngroup}) is not solvable for $n \ge 5$.
  \begin{proof}
    It's easy to use definition \ref{def:solvablegroupadd}.
    Main steps are the following
    \begin{enumerate}
      \item we know that $\left[S_n, S_n\right] = A_n$ - subgroup of
        even permutations (see definition
        \ref{def:alternatinggroup}). It can be seen from the fact that
        any 3-cycle is a \mynameref{def:commutatorsubgroup}
        \footnote{
          Let $n \ge 3$ and $(a,b,c) \in S_n$ is a 3-cycle then
          (see example \ref{ex:transposition_product})
          \begin{eqnarray}
            (a,b,c) = (a,b,c)^2 =
            \left((a,c)(a,b)\right)^2 =
            \nonumber \\
            = (a,c)(a,b)(a,c)(a,b) =
            (c,a)^{-1}(b,a)^{-1}(a,c)(a,b) =
            \nonumber \\
            =
            (a,c)^{-1}(a,b)^{-1}(a,c)(a,b) =
            \nonumber \\
            =
            \left[
              (a,c),(a,b)
              \right] \subset \left[S_n, S_n\right].
            \nonumber
          \end{eqnarray}
        }
        and 3-cycles
        generate $A_n$
        \footnote{
          \label{note:lec8_3cycle}
          Any even permutation can be represented in a form of product
          of transpositions (see theorem
          \ref{thm:permutationrepresent}). The number of
          transpositions should be
          even. We have several cases there:
          \begin{enumerate}
            \item The transpositions with different elements produces
              2 3-cycles
              \[
              (a,b)(c,d) = (a,b,c)(b,c,d).
              \]
              Really
              \[
              (a,b,c)(b,c,d) =
              \begin{array}{c}
                c \to d \\
                d \to b \to c \\
                b \to c \to a \\
                a \to b
              \end{array} =
              (a,b)(c,d)
              \]
            \item The transpositions that have a same element produces
              a 3-cycle (see example \ref{ex:transposition_product})
              \[
              (a,c)(a,b) = (a,b,c)
              \]
          \end{enumerate}
          As soon as we have even number of transposition then we will
          always have a translation to a product of 3-cycles for any
          product of transposition pairs.
        }
      \item If $n \ge 5$ then $\left[A_n, A_n\right] = A_n$ thus the
        filtration generated by commutators will never terminate
        i.e. will never reach the unity ($\left\{e\right\}$) and will
        stabilize on $A_n$. How we can see it? We can remember that
        $\left[A_4, A_4\right] = K$ (see example \ref{ex:lec8_s4}) -
        the subgroup of double transpositions. $A_4 \hookrightarrow
        A_n$ in many different ways. Because you can pick any 4
        elements among our $n$ elements and just consider the
        permutations of
        those 4 elements as a subgroup of permutations of $n$
        elements and then taking the commutators of those $A_4$, we
        see
        that all double transpositions are in the $\left[A_n,
          A_n\right]$ (\mynameref{def:commutatorsubgroup} of $A_n$).
        \footnote{
          The double transposition consists of 4 elements. We can
          consider all permutations of the 4 elements only and ignore
          all others. For instance if $n = 6$ and we are interested in
          all double transpositions of $1,2,4,5$. In the case we have
          to look at the following permutations:
          \[
          \pi_{1,2,4,5} = \begin{pmatrix}
            1 & 2 & 3 & 4 & 5 & 6\\
            i_1 & i_2 & 3 & i_3 & i_4 & 6
          \end{pmatrix}.
          \]
          Note that we consider the even permutations only i.e.
          $\pi_{1,2,4,5} \in A_6$.
          From example \ref{ex:lec8_s4} we know that the commutator of
          such permutations contains all double transpositions for $1,2,4,5$:
          $\left\{e, (12)(45), (14)(25), (15)(24) \right\}$ (the
          elements $3,6$ are not changed and disappeared in the
          commutator). Continue such way we can conclude that for any
          double transposition:
          \[
          (i_1,i_2)(i_3,i_4) \in \left[A_n, A_n\right]
          \]
        }
        But
        if $n \ge 5$, they generate $A_n$.
        \footnote{
          Really if $n \ge 5$ then for any 3 elements $a,b,c$ (which form a
          3-cycle $(a,b,c)$) exist 2 elements $e,d$ such that all
          elements $a,b,c,d,e$ are different. Using the
          $(e,d)(e,d) = id$ one can get that 2 double transpositions
          $(a,b)(d,e)$ and $(d,e)(b,c)$ generate the 3-cycle $(a,b,c)$:
          \[
          (a,b)(d,e)(d,e)(b,c) = (a,b)(b,c) = (a,b,c).
          \]

          I.e. any 3-cycle can be generated by 2 double
          transpositions. Therefore (see note \ref{note:lec8_3cycle})
          the whole $A_n$ is generated by
          the double transpositions.

          Thanks Arnur Nigmetov for the hint.
        }
    \end{enumerate}
  \end{proof}
  \label{thm:lec8_sn_solvability}
\end{theorem}

\section{Galois theorem on solvability by radicals}

The following claim missing in the lectures but it's important for the
theorem \ref{thm:lec8_1} proof.
\begin{claim}
  Let $K \subset L \subset M$ and both $M$ and $L$ are
  \mynameref{def:galoisextension}s over $K$ then
  \[
  \frac{Gal\left(M/K\right)}{Gal\left(M/L\right)}
  \cong Gal\left(L/K\right).
  \]
  \begin{proof}
    Consider the following group \mynameref{def:homomorphism}
    \[
    \phi:Gal\left(M/K\right) \xrightarrow[g \to g\mid_L]{} Gal\left(L/K\right).
    \]
    The \mynameref{thm:galoiscorrespondence} says that $\phi$ is
    a \mynameref{def:surjection} with kernel
    $Gal\left(L/F\right)$. Therefore by
    \mynameref{thm:firstisomorphism} one can get the required
    statement.
  \end{proof}
  \label{claim:galoisquotient}
\end{claim}

\begin{theorem}
  Let $P \in K\left[X\right]$. $P$ is a
  \mynameref{def:solvablepolynomial} if and only if
  $Gal\left(P\right)$ is solvable. There
  $Gal\left(P\right)$ is (by definition) $Gal\left(F/K\right)$ where
  $F$ is a \mynameref{def:splittingfield} of $P$ over $K$.
  \label{thm:lec8_1}
  \begin{proof}
    First of all lets proof that if $Gal\left(P\right)$ is solvable
    then $P$ is solvable. Let $n = \left[F:K\right]$ and consider
    $L = K\left(\zeta_n\right)$ where $\zeta_n$ - $n$-th root of 1.
    Let $M = FL$ is a \mynameref{def:compositeextension}. So this is the
    splitting and field of $P$ of which we have adjoined all the
    $n$-th roots of unity.  Then $M$ is a
    \mynameref{def:galoisextension}
    \footnote{
      Accordingly property \ref{property:lec7_2} the composition of
      two Galois extensions will also be Galois.
    }
    and
    $Gal\left(M/L\right) \hookrightarrow Gal\left(F/K\right)$.
    \footnote{
      See theorem \ref{thm:lec7_3} where $L_1 = F, L_2 = L, L_1 L_2 =
      M = FL$.
    }
    $\forall g \in Gal\left(M/L\right)$ leaves $F$ invariant.
    \footnote{
      $g(F) = F$ see \mynameref{thm:galoiscorrespondence} (point
      \ref{thm:galoiscorrespondence:item2b})
    }
    If $g\mid_F = id$ then $g = id$. Then the image in fact of this
    map is $Gal\left(F/F \cap L\right)$.
    \footnote{
      See theorem \ref{thm:lec7_3} where $L_1 = F, L_2 = L, L_1 L_2 =
      M$.
    }
    So $G = Gal\left(M/L\right)$
    is solvable
    \footnote{
      Using property \ref{property:lec8_solvable1} $G$ is solvable as
      soon as $G \subset Gal\left(F/K\right)$ and
      $Gal\left(F/K\right)$ is solvable
    }
    i.e.
    \[
    G = G_0 \supset G_1 \supset \dots \supset G_r = \left\{e\right\}
    \]
    and $G_i/G_{i+1}$ - cyclic
    \footnote{
      see property \ref{property:lec8_solvable4}
    }
    of order $n_i \mid n$.
    \footnote{
      As soon as $G_i, G_{i+1} \subset G$ then using
      \mynameref{thm:lagrange} one can get that
      $\left|G_i\right| \mid \left|G\right|$ as well as
      $\left|G_{i+1}\right| \mid \left|G\right|$ and therefore
      $\left|G_i/G_{i+1}\right| \mid \left|G\right|$. But
      $G \subset Gal\left(F/K\right)$, thus
      $\left|G\right| \mid \left|Gal\left(F/K\right)\right| = n$ and, as
      result, $\left|G_i/G_{i+1}\right|$ divides $n$.
    }
    And as soon as
    $n_i \mid n$ (very important), all $n$-th roots of 1 are in $M$
    (this is why we adjoin the $L$).

    Let $M_i = M^{G_i}$. We know $M_i \hookrightarrow M_{i+1}$ is a
    cyclic Galois extension of order $n_i \mid n$ and roots of 1 are
    in it, thus there is Kummer extension.
    \footnote{
      We have $M_i = M^{G_i}$ where
      \[
      M_0 = M^{G_0} = M^G = M^{Gal\left(M/L\right)} = L
      \]
      and
      \[
      M_r = M^{G_r} = M^{\{e\}} = M.
      \]
      Thus by \mynameref{thm:galoiscorrespondence},
      as soon as
      $G_r \triangleleft G_{r-1} \triangleleft \dots \triangleleft G_1
      \triangleleft G_0 = G$,
      we have the
      following ``tower'':
      \[
      L = M_0 \subset M_1 \subset \dots \subset M_{r-1} \subset M_r = M.
      \]
      If we consider $M_i \subset M_{i+1}$ then we can get the
      following: $M_i$ contains all $n$-th roots of unity as soon as
      $K\left(\zeta_n\right) = L \subset M_i$; $M_{i+1}$ is
      \mynameref{def:galoisextension} over $M_i$ with
      \mynameref{def:galoisgroup} $G_i/G_{i+1}$
      (we have $G_i = Gal\left(M/M_i\right)$,
      $G_{i+1} = Gal\left(M/M_{i+1}\right)$ and therefore, by claim
      \ref{claim:galoisquotient},
      $G_i/G_{i+1} = Gal\left(M_{i+1}/M_i\right)$)
      which is cyclic of order
      $n_i \mid n$ and as
      result $M_{i+1}$ will be also be cyclic of the same order $n_i
      \mid n$. Therefore $M_{i+1}$ is a Kummer extension
      (see section \ref{sec:kummerextension}) over $M_i$,
    }
    and therefore
    $M_{i+1} = M_i\left(\sqrt[n_i]{a_i}\right)$ (see proposition
    \ref{prop:lec7_2}).
    So $M = K\left(\zeta_n, \alpha_1, \dots, \alpha_r\right)$ where
    $\alpha_i = \sqrt[n_i]{a_i}$. Therefore $M$ is solvable by
    radicals.

    For another direction: if $P$ is solvable then $G$ is
    solvable. Let $E$ is solvable extension containing $F$. We may
    suppose (using property \ref{property:solvable}) that this is Galois.
    Then write
    $E = K\left(\alpha_1, \dots, \alpha_r\right)$ where
    $\alpha_i^{n_i} \in K\left(\alpha_1, \dots,
    \alpha_{i-1}\right)$. Then let $L = K\left(\zeta_n\right)$
    where $n = LCM\left(\left\{n_i\right\}\right)$ so
    $\forall n_i: n_i \mid n$. And
    take $M = LE$. We have
    $K\left(\alpha_1, \dots, \alpha_{i-1}\right) \hookrightarrow
    K\left(\alpha_1, \dots, \alpha_i\right)$ - cyclic extension of
    order $n_i$. We have $Gal\left(M/L\right)$ is solvable by this
    cyclic subgroups. $Gal\left(M/K\right)$ is also solvable since
    \[
    Gal\left(M/L\right) \subset Gal\left(M/K\right)
    \]
    and the quotient (see claim \ref{claim:galoisquotient})
    \[
    \frac{Gal\left(M/K\right)}{Gal\left(M/L\right)}
    \cong Gal\left(L/K\right)
    \]
    which is abelian.
    \footnote{
      as result the property \ref{property:lec8_solvable3} gives us
      the solvability of $Gal\left(M/K\right)$
    }
    $Gal\left(F/K\right)$ is a quotient
    \footnote{
      ??? subset
    }
    of $Gal\left(M/K\right)$ thus
    it is solvable too.
    \footnote{
      by the property \ref{property:lec8_solvable1}
    }
  \end{proof}
\end{theorem}

\section{Examples of equations not solvable by radicals."General
  equation"}
As we can see there exist equations which are not solvable in
radicals.

\begin{example}[Not solvable polynomial of degree 5]
  Let $P \in \mathbb{Q}\left[X\right]$ is an irreducible polynomial
  with rational
  coefficients of degree 5. It has 3 real roots (and 2 complex
  conjugate roots) as it shown on the picture.

  \begin{tikzpicture}
    \draw[->] (-1.5,0) -- (3,0) node[right] {$x$};
    \draw[->] (0,-1.5) -- (0,1.5) node[above] {$y$};
    \draw[scale=0.5,domain=-2.5:4.2,smooth,variable=\x,blue] plot
    ({\x},{0.01*(\x-1)*(\x-1)*(\x-1)*(\x-1)*(\x-1) - 0.5*(\x-1) + 0.01});
  \end{tikzpicture}

  We claim that $Gal\left(P\right) = S_5$. This is because
  \begin{enumerate}
    \item $Gal\left(P\right)$ contains the complex conjugation (we
      have 2 complex conjugated roots but \mynameref{def:galoisgroup} is
      the group of automorphisms which exchange roots and the complex
      conjugation will exchange the 2 complex roots).
      The complex conjugation is the transposition of roots
      \footnote{
        $(1,2) = 1 \to 2 \to 1$
      }
    \item As soon as $P$ is irreducible then $Gal\left(P\right)$
      should act transitively (see definition \ref{def:transitive})
      \index{Transitive group action}
      on
      roots (see remark
      \ref{rem:lec5_onnormalext} item
      \ref{rem:item:lec5_onnormalext_2}). We have an irreducible
      polynomial. We can always send
      one of its roots to another of its roots. We have this
      isomorphism of stem fields which extends to an automorphism of
      the splitting field. But, what is the subgroup of $S_5$, which adds
      transitively?

      $Gal\left(P\right) \subset S_5$ acts transitively. This means
      that $5 \mid \left|Gal\left(P\right)\right|$. That is because
      (see \mynameref{thm:orbitstabilizertheorem})
      $\left|G\right| = \left|Orb\left(x\right)\right| \left|G_x\right|$
      ($Orb\left(x\right)$ - is the \mynameref{def:orbit},
      $G_x$ - \mynameref{def:stabilizersubgroup})
      but the orbit has 5 elements
      \footnote{
        $\left|Orb\left(x\right)\right| = 5$ because there are 5
        different roots that can be permuted by the Galois group.
      }
      and therefore 5 divides
      the cardinality of $G$. This means, by \mynameref{cor:sylow}
      theorems, that our group contains something of order 5. But only
      5-cycle has order 5 (see theorem
      \ref{thm:finite_group_of_prime_order}).
      But a 5-cycle and transposition generate
      $S_5$ (see corollary \ref{cor:sn}).
      So $Gal\left(P\right) = S_5$.
  \end{enumerate}
  In fact, the same argument is valid for $S_p$ with every $p$ - prime.
  I.e. applies to an arbitrary prime number $p$ instead of 5.

  So $Gal\left(P\right) = S_5$ - not solvable and therefore $P$ is not
  solvable by radicals.
  \label{ex:lec8_notsolvable1}
\end{example}

\begin{example}[General equation of degree $n$]
  What's the general equation. It is the following
  \[
  X^n - T_1 X^{n-1} + T_2 X^{n-2} + \dots + \left(-1\right)^n T_n,
  \]
  where $T_i$ is a variable. Where does it come from? Let
  $X_1, \dots, X_n$ are roots of a polynomial of degree $n$ when the
  polynomial itself is
  \begin{eqnarray}
  \left(X - X_1\right) \cdot \dots \cdot \left(X - X_n\right) =
  X^n - \left(\sum_i X_i \right) X^{n-1} +
  \nonumber \\
  +
  \left(\sum_{i,j} X_i X_j \right) X^{n-2} + \dots +
  \left(-1\right)^n \prod_i X_i,
  \nonumber
  \end{eqnarray}
  i.e.
  $T_1 = \sum_i X_i, T_2 = \sum_{i,j} X_i X_j, \dots, T_n = \prod_i
  X_i$.

  One has $K\left[T_1, \dots, T_n\right] \subset K\left[X_1, \dots,
    X_n\right]$ (multi-variable polynomial rings). We have the same
  also for field extensions:
  $K\left(T_1, \dots, T_n\right) \subset K\left(X_1, \dots,
  X_n\right)$.
  The $K\left(X_1, \dots, X_n\right)$ is algebraic and a
  splitting field for our general polynomial. So it has degree at most
  $n!$, i.e.
  $\left[K\left(X_1, \dots, X_n\right):K\right] \le n!$
  (see theorem \ref{thm:lec2_1}). On the other hand
  $K\left(T_1, \dots, T_n\right) \subset K\left(X_1, \dots,
  X_n\right)^{S_n}$
  \footnote{
    This is because $S_n$ permutes the roots, for instance $X_k \to X_l \to
    X_k$, but the equations for $T_i$ are not changed during the
    transformation.
  }
  so degree of the extension is $n!$
  \footnote{
    We have that
    $S_n \subset Gal\left(K\left(X_1, \dots, X_n\right)/
    K\left(T_1, \dots, T_n\right)\right)$ and therefore
    \[
    \left[K\left(X_1, \dots, X_n\right):
      K\left(T_1, \dots, T_n\right)\right] \ge \left|S_n\right| = n!.
    \]
    Thus with $\left[K\left(X_1, \dots, X_n\right):K\right] \le n!$
    one can get that
    $\left[K\left(X_1, \dots, X_n\right):
      K\left(T_1, \dots, T_n\right)\right] = n!$
    and
    $\left[K\left(X_1, \dots, X_n\right):K\right] = n!$.
  }
  and
  \[
  K\left(T_1, \dots, T_n\right) = K\left(X_1, \dots,
  X_n\right)^{S_n}.
  \]
  In particular the Galois group is $S_n$ and our
  general polynomial is not solvable by radicals if $n \ge 5$.
  This is known as Abel theorem
  \label{ex:lec8_generalequation}
\end{example}

\section{Galois action as a representation. Normal base theorem}

Connection with group representations.
\begin{definition}[Group representation]
  Let $G$ is a finite group. $V$ is a \mynameref{def:vectorspace} over
  $K$. Representation of $G$ is a \mynameref{def:homomorphism}
  $\rho: G \to GL\left(V\right)$
  (where $GL\left(V\right)$ is the \mynameref{def:glv} i.e. the group of
  \mynameref{def:automorphism}s of the vector space $V$).
  \label{def:grouprepresentation}
\end{definition}

If $L$ is a finite extension of $K$ we can talk about it as about
$K$-vector space. So we have a representation of $G$ as
\mynameref{def:galoisgroup} $Gal\left(L/K\right)$:
$\rho: G \to GL_K\left(L\right)$ - this is something that we have as
the definition because we define the Galois group as the group of
automorphisms of $L$ over $K$.

We can ask the question: what's kind of representation is the
$\rho$. We claim that $\rho$ is something that is called as
\mynameref{def:regularrepresentation}.
\begin{definition}[Regular representation]
  Let a vector space $V$ has a basis indexed by elements of group $G$:
  $e_g$ where $g \in G$. $\rho_{reg}\left(h\right)$
  \footnote{
    where $h \in G$.
  }
  acts by
  permutations:
  \footnote{
    $\rho_{reg}\left(h\right)$ is an \mynameref{def:automorphism} of $V$
    i.e.
    \[
    \rho_{reg}\left(h\right): V \xrightarrow[e_g \to e_{gh}]{} V.
    \]
  }
  \[
  \rho_{reg}\left(h\right) e_g = e_{hg}.
  \]
  \label{def:regularrepresentation}
\end{definition}

We claim that the representation of Galois group is the regular
representation. We have seen that (see proof of the theorem
\ref{thm:primitiveelement})
\[
L \otimes_K \bar{K} \cong \bar{K}^n.
\]

The $\bar{K}^n$ is a \mynameref{def:directsum} of $n$
($n = \left|G = Gal\left(L/K\right)\right|$)
copies of $\bar{K}$.
The sum is indexed by the embeddings of $L$ into
$\bar{K}$.
\footnote{
  The embeddings are also noticed as the set of
  homomorphisms $Hom_K\left(L, \bar{K}\right)$.
  And $\bar{K}^n = \prod_{i=1}^n\bar{K}$ where
  $n = \left|Hom_K\left(L, \bar{K}\right)\right|$ (accordingly
  (\ref{eq:lec5_AredIsProd})).
}
Pick one $j: L \hookrightarrow \bar{K}$ and all others can
be obtained by group \mynameref{def:action} $j \circ g, g \in G$.
\footnote{
  This is because the \mynameref{def:galoisgroup} acts transitively on
  the set of homomorphisms from $L$ to $\bar{K}$ ($Hom_K\left(L,
  \bar{K}\right)$), see theorem \ref{thm:lec5_3}.
}
So
$\bar{K}^n$ has a basis indexed by $G$ and the \mynameref{def:action} of
$G$ permutes the basis vectors.
\footnote{
  We have $L$ is a \mynameref{def:galoisextension} i.e., via
  \mynameref{thm:primitiveelement}, $\exists \alpha \in L$
  such that $L=K\left(\alpha\right)$, where $\alpha$ is a root of a
  polynomial $P_{min}\left(\alpha, K\right)$ of degree $\left[L:K\right] = n$.
  The consider the set of the following vectors in $\bar{K}^n$:
  \[
  e_i = \left(\phi_1(\alpha^{i-1}), \phi_2(\alpha^{i-1}), \dots,
  \phi_n(\alpha^{i-1})\right),
  \]
  where $\phi_j$ -distinct homomorphisms such that $\phi_j(\alpha) =
  \alpha_j$ - is another root of $P_{min}\left(\alpha, K\right)$.

  The set is linearly independent:
  \begin{eqnarray}
    \sum_{i=1}^{n}c_i e_i =
    \left(\sum_{i=1}^{n} c_i \alpha_1^{i-1},
    \sum_{i=1}^{n} c_i \alpha_2^{i-1}, \dots,
    \sum_{i=1}^{n} c_i \alpha_n^{i-1}\right) =
    \nonumber \\
    = \left(P(\alpha_1), P(\alpha_2), \dots, P(\alpha_n)\right),
    \nonumber
  \end{eqnarray}
  where $P(X) = \sum_{i=1}^{n}c_i X^{i-1}$. It can have $\alpha_1,
  \alpha_2, \dots, \alpha_n$ as roots only if all $c_i = 0$
  because the minimal polynomial $\forall \alpha_i$ has degree $n >
  n-1$.
  As result
  $\{e_i\}$ are linearly independent.

  If we put the vectors into a matrix as rows we can get
  \[
  A =
  \begin{bmatrix}
    \phi_1(1) & \phi_2(1) & \phi_3(1) & \dots &
    \phi_n(1) \\
    \phi_1(\alpha) & \phi_2(\alpha) & \phi_3(\alpha) & \dots &
    \phi_n(\alpha) \\
    \phi_1(\alpha^2) & \phi_2(\alpha^2) & \phi_3(\alpha^2) & \dots &
    \phi_n(\alpha^2) \\
    \vdots & \vdots & \vdots & \ddots & \vdots \\
    \phi_1(\alpha^{n-1}) & \phi_2(\alpha^{n-1}) & \phi_2(\alpha^{n-1})
    & \dots &
    \phi_n(\alpha^{n-1}) \\
  \end{bmatrix}.
  \]
  The columns are also linearly independent (as rows) and therefore
  \begin{equation}
  v_i = \left(\phi_i(1), \phi_i(\alpha), \dots,
  \phi_i(\alpha^{n-1})\right)
  \label{eq:lec8_note_basis}
  \end{equation}
  is a basis of $\bar{K}^n$. If $g \in G$ such that
  $g(\alpha_i) = \alpha_j$ then
  \begin{eqnarray}
    g(v_i) = g\left(
    1,\alpha_i, \dots, \alpha_i^{n-1}
    \right) =
    \nonumber \\
    =
    \left(
    1,\alpha_j, \dots, \alpha_j^{n-1}
    \right)
    =
    \left(\phi_j(1), \phi_j(\alpha), \dots,
    \phi_j(\alpha^{n-1})\right) = v_j
    \nonumber
  \end{eqnarray}
}
So $L \otimes_K \bar{K} \cong
\bar{K}^n \cong R$, where R is a \mynameref{def:regularrepresentation}
of $G$ over $\bar{K}$.
In particular $\exists x \in L \otimes_K \bar{K}$
such that $gx \mid_{g \in G}$
forms a basis of $L \otimes_K \bar{K}$ over $\bar{K}$.
\footnote{
  you can take one basis vector from (\ref{eq:lec8_note_basis}): $x = v_i$
} And, as result, the elements of $G$ are linearly independent in the space of
\mynameref{def:endomorphism}s
$End_{\bar{K}}\left(L \otimes_K \bar{K}\right)$
\footnote{
  See also theorem \ref{thm:dedekind} about linearly independents of
  elements of a group $G$
}

\begin{theorem}[Normal base]
  $\exists x \in L$ such that
  $\left\{gx \mid g \in G\right\}$ is a $K$ basis of $L$.

  $G$ are
  linearly independent in the space of \mynameref{def:endomorphism}s
  $End_{K}\left(L\right)$
  \label{thm:normalbase}
  \begin{proof}
    First of all consider a case when $K$ is infinite. Let pick some
    basis $e_1, \dots, e_n$ - $K$-basis in $L$. $g_1, \dots, g_n \in
    G$. Let $x \in L$ then $g_1\left(x\right), \dots,
    g_n\left(x\right)$ is a basis if and only if matrix formed by
    $g_i\left(x\right)$ in the basis $e_j$ has non zero
    determinant.
    \footnote{
      We have $x = X_1 e_1 + \dots X_n e_n$ where $X_i \in K$.
      $\forall g_l \in G: g_l(x) = \sum_i X_i g_l(e_i)$
      because $g_l(X_i) = X_i$ (remember that $L^G = K$).
      For each $g_l(e_i)$ we can write
      \begin{equation}
        g_l(e_i) = \sum_j k_{ij}^{(l)} e_j
        \label{eq:lec8_note_basis2}
      \end{equation}
      We have $k_{ij}^{(l)}$ are predefined
      (because $\{g_l\}$ and $\{e_i\}$ are predefined)
      elements of $L$ and therefore $X_i$ can only be changed.

      We want $\{g_j(x)\}$ to be linearly independent set of elements
      from $L$. Therefore the equation
      \[
      c_1 g_1(x) + \dots c_n g_n(x) = 0,
      \]
      where $c_i \in K$, should holds only for $c_1 = c_2 = \dots =
      c_n = 0$.
      We also can write
      \begin{eqnarray}
      c_1 \sum_j \left(\sum_i X_i k_{ij}^{(1)} \right) e_j  +
      c_2 \sum_j \left(\sum_i X_i k_{ij}^{(2)} \right) e_j  +
      \dots
      +
      \nonumber \\
      +
      c_n \sum_j \left(\sum_i X_i k_{ij}^{(n)} \right) e_j  =
      \sum_j 0 e_j = 0
      \nonumber
      \end{eqnarray}
      The equation can also be rewritten in the matrix form
      \begin{eqnarray}
        \begin{bmatrix}
          \sum_i X_i k_{i1}^{(1)} & \sum_i X_i k_{i1}^{(2)} & \cdots &
          \sum_i X_i k_{i1}^{(n)} \\
          \sum_i X_i k_{i2}^{(1)} & \sum_i X_i k_{i2}^{(2)} & \cdots &
          \sum_i X_i k_{i2}^{(n)} \\
          \vdots & \vdots & \ddots & \vdots \\
          \sum_i X_i k_{in}^{(1)} & \sum_i X_i k_{in}^{(2)} & \cdots &
          \sum_i X_i k_{in}^{(n)} \\
        \end{bmatrix}
        \begin{bmatrix}
          c_1 \\
          c_2 \\
          \vdots \\
          c_n
        \end{bmatrix}
        =
        \begin{bmatrix}
          0 \\
          0 \\
          \vdots \\
          0
        \end{bmatrix}
        \nonumber
      \end{eqnarray}
      The equation has non-trivial solutions only if
      \begin{equation}
      \det\left(
        \begin{bmatrix}
          \sum_i X_i k_{i1}^{(1)} & \sum_i X_i k_{i1}^{(2)} & \cdots &
          \sum_i X_i k_{i1}^{(n)} \\
          \sum_i X_i k_{i2}^{(1)} & \sum_i X_i k_{i2}^{(2)} & \cdots &
          \sum_i X_i k_{i2}^{(n)} \\
          \vdots & \vdots & \ddots & \vdots \\
          \sum_i X_i k_{in}^{(1)} & \sum_i X_i k_{in}^{(2)} & \cdots &
          \sum_i X_i k_{in}^{(n)} \\
        \end{bmatrix}
        \right) \ne 0.
        \label{eq:lec8_note_det}
      \end{equation}
    }
    But this determinant is a polynomial
    in the
    coefficient, which is not identically zero.
    \footnote{
      The determinant(\ref{eq:lec8_note_det}) can always be written in
      the form of
      multi-variable polynomial
      $P \in L\left[X_1, \dots, X_n\right]$ i.e. the polynomial
      coefficients are from $L$.
    }
    Well, why? Because if
    it was identically zero, it would remain identically zero also
    after the base changed to $\bar{K}$.
    \footnote{
      The element $g \in G$ acts on $l \otimes k \in L \otimes_K
      \bar{K}$ as follows $g\left(l \otimes k\right) = g(l) \otimes k$
      (see proof for the theorem \ref{thm:basechange}). Using
      proposition \ref{prop:lec4_Addon} we
      have $x = X_1 e_1 \otimes 1_{\bar{K}} + \dots X_n e_n \otimes
      1_{\bar{K}}$ and $X_i \in \bar{K}$.
      With equation (\ref{eq:lec8_note_basis2}) we have
      \[
      g_l\left(e_i \otimes 1_{\bar{K}} \right) =
      g_l\left(e_i \right) \otimes 1_{\bar{K}} =
      \sum_j k_{ij}^{(l)} e_j \otimes 1_{\bar{K}}
      \]
      i.e. all $k_{ij}^{(l)}$ are the same as in the
      (\ref{eq:lec8_note_det}) and as result the polynomial
      coefficients will be the same (i.e. the polynomial will be
      identically zero).
    }
    Since it has a $\bar{K}$
    point where it does not vanish.

    There are many $x \in L \otimes_K \bar{K}$ such that
    $g_i\left(x\right)$ form a basis. And over an infinite field, a
    polynomial which is not identically zero cannot vanish
    identically ($P \ne 0$). And over an infinite
    field, only a polynomial which
    is identically 0 can vanish at every point.
    \footnote{
      I.e. polynomial of the following polynomial ring
      $P \in L\left[X_1, \dots, X_n\right]$ can vanish at every point
      only if it is identically zero $P = 0$.
    }

    Let me to clarify the point. $P \in K\left[X\right]$ has at most
    $\deg P$ roots.
    \footnote{
      Therefore it is impossible to have each $k \in K$ as a root of
      the polynomial because in the case the number of roots
      $\left|K\right| = \infty > \deg P$.
    }
    So if $K$ infinite and $P$ has every element of
    $K$ as a root then $P = 0$ ($P$ is zero as an element of
    $K\left[X\right]$).

    By induction we can get the same statement for a polynomial in
    several variables.
    \footnote{
      Let $P_n \in K_n = K\left[X_1, X_2, \dots, X_{n-1}, X_n\right]$ and we
      proved the statement for any $P_{n-1} \in K_{n-1} = K\left[X_1, X_2, \dots,
        X_{n-1}\right]$ i.e. if $\forall k_1, k_2, \dots, k_{n-1}
      \in K$ we have $P_{n-1}\left(k_1, k_2, \dots, k_{n-1}\right) =
      0$ then $P_{n-1} = 0_K$.

      We want to proof that if $P_n$ vanishes for any choice of
      $k_1, \dots, k_n \in K$ then it is identically 0.
      Consider $P_n$ as a polynomial of single
      variable $X_n$ with coefficients from $K_{n-1}$. If the
      polynomial is 0 for every $k_n \in K$ then its coefficients are
      0. But by the induction hypothesis they are identically 0 (as
      soon as they stays zero for any $k_i$ choice). Thus the result
      polynomial is also identically zero: $P_n = 0_K$
    }
    So, our polynomial which is the determinant of
    the matrix,
    is non zero, as a polynomial of several variable because it
    does not have roots over algebraic closure $\bar{K}$.
    And so, it also does not have roots over $K$. So, there
    exists a point $x \in L$ (not anymore in $L \otimes_K \bar{K}$)
    such that $\det(\dots) \ne 0$ at $x$ so $g_i\left(x\right)$ form a
    basis.

    If $K$ is finite then the argument with roots of a polynomial does
    not apply any more. But in the case the \mynameref{def:galoisgroup} is
    cyclic i.e. $G = \left<\sigma\right>$ (see corollary
    \ref{cor:lec3_2}).
    We have $id, \sigma, \dots,
    \sigma^{n-1}$ are linearly independent because they are linearly
    independent  over
    $\bar{K}$.
    \footnote{
      See theorem \ref{thm:dedekind}.
    }
    Then the minimal polynomial of $\sigma$ as an
    \mynameref{def:endomorphism} of $L$ over $K$ is $X^n - 1$.
    This is because a lower degree polynomial cannot vanish at
    $\sigma$
    since, the lower power of $\sigma$ are linearly independent.
    \footnote{
      $\sigma^n = 1$ as soon as $G=\left<\sigma\right>$ (cyclic group
      of order $n$).
      If $\exists m < n$ such that $\sum_{i=0}^m c_i \sigma^i = 0$ and
      $c_i \ne 0$ then the set $1, \sigma, \dots, \sigma^m$ is not
      linearly independent that is in contradiction with
      \mynameref{thm:dedekind}.
      Thus the minimal polynomial for $\sigma$ is
      $X^n-1$.
    }
    Therefore, as a $K\left[X\right]$-module with $X$
    acting by $\sigma$,
    \footnote{
      We consider $L$ as a module over $K\left[X\right]$ where $X$
      acts on $L$ as $\sigma$ i.e. if $f(X) \in K\left[X\right]$ then
      $f(X): L \xrightarrow[\alpha \to f(\sigma)(\alpha)]{} L$
      \cite{bib:KeithConradLinearchar}, for
      instance $f(X) = X^2 + X + 1$ acts on $\alpha \in L$ as follows:
      $\sigma^2(\alpha) + \sigma(\alpha) + \alpha$.
    }
    $L$ is isomorphic to
    \[
    L \cong K\left[X\right]/\left(X^n - 1\right).
    \]
    This is
    a \mynameref{def:cyclicmodule}
    \footnote{
      By claim \ref{claim:cyclicmodule} there is a cyclic
      $K\left[X\right]$-module. I.e. if $x \in L$ is the generator
      then $L = K\left[X\right] x$.
    }
    and any generator $x$ shall do i.e. $x, \sigma x, \dots,
    \sigma^{n-1} x$ form a basis.
    \footnote{
      As soon as $X$ acts as $\sigma$ on $L$ and $\sigma^n - 1 = 0$ then
      any element from $K\left[X\right]$ can be represented as follows $k_0 + k_1 X +
      \dots + k_{n-1} X^{n-1}$, where $k_i \in K$. Let $x \in L$ is
      the generator of $L$
      as $K\left[X\right]$-module thus one can get
      \begin{eqnarray}
      L = \left.\left(k_0 + k_1 X + \dots + k_{n-1}
      X^{n-1}\right)\right|_{X=\sigma}x =
      \nonumber \\
      = k_0 x + k_1 \sigma(x) + \dots + k_{n-1}\sigma^{n-1}(x)
      \nonumber
      \end{eqnarray}
      i.e.
      $x, \sigma x, \dots, \sigma^{n-1} x$ is a $K$ basis of $L$.
    }
  \end{proof}
\end{theorem}

\section{Relation with coverings}

\begin{remark}
  If $L$ is a finite \mynameref{def:galoisextension} of $K$ then
  $ L \otimes_K L$ is a direct sum of fields
  \footnote{
    see also definition \ref{def:directsummodules}
  }
  which are
  isomorphic to $L$. Sums are permuted by $G = Gal\left(L/K\right)$.
  \begin{proof}
  So if $L = K\left(\alpha\right)$ is a splitting field of the
  polynomial $P = \left(X - \alpha_1\right) \cdot \dots \cdot \left(X
  - \alpha_n\right)$ (where $\alpha \in \{\alpha_1, \dots,
  \alpha_n\}$) that is isomorphic to $K\left[X\right]/(P)$. If
  we tensor it to $L$ we will get (see examples \ref{ex:lec4_1} and
  \ref{ex:lec5_1})
  \begin{eqnarray}
    L \otimes_K L \cong
    L \otimes_K K\left[X\right]/\left(X - \alpha_1\right) \cdot \dots \cdot \left(X
    - \alpha_n\right) \cong
    \nonumber \\
    \cong
  L\left[X\right]/\left(X - \alpha_1\right) \cdot \dots \cdot \left(X
  - \alpha_n\right) \cong
  \nonumber \\
  \cong
  L\left[X\right]/\left(X - \alpha_1\right) \times
  \dots \times
  L\left[X\right]/\left(X - \alpha_n\right) \cong
  \nonumber  \\
  \cong
  L \times \dots \times L
  \nonumber
  \end{eqnarray}
  that is a product of copies of $L$ permuted by Galois action
  \end{proof}
  \label{rem:lec8_1}
\end{remark}

In topology one has Galois covering $Y \to X$. $G$ acts on $Y$, $X$ is
a \mynameref{def:quotientofgroupaction}. The covering is characterized
by the property that $Y
\times_X Y = \sqcup_{g \in G} Y_g$ (is a \mynameref{def:disjointunion}),
$Y_g = \{\left(y, gy\right)\}$.
\footnote{
  ??? add an explanation
}


\chapter{An Alternative Means of Regularization for Domain Adaptation Problems}

\section{Introduction}
Previously, we explored the applications of entropic regularization for the Wasserstein distance (Sinkhorn) in domain adaptation. In particular, we derived a new generalization bound for target error with respect to the Sinkhorn divergence. However, for domain adaptation, entropic regularization may not be the ideal route to take.

When we prescribe a divergence measure for domain adaptation, the scenario we seek to penalize against is when the source $S$ and target $T$ distributions are not identical. However, with the entropic regularization, one is penalizing against $S$ and $T$ being independent. In this chapter, we propose an alternative regularization that may be more suitable for domain adaptation problems.

If $S$ and $T$ are identical in distribution, which is the ideal setting in machine learning, then there exists an identity mapping between the two. Thus, it makes sense to use a regularization that penalizes the deviation between the transport map and the identity map.

\newpage

\section{Existence and Uniqueness of an Optimal Transport Map}
The intuition behind our proposed regularization stems from Brenier's theorem, which concerns the existence and uniqueness of optimal maps.

\begin{theorem}{Brenier's Theorem}

	Let $\mu$ and $\nu$ be absolutely continuous probability measures on $\mathbb{R}^d$ with respect to the Lebesgue measure and $\nu$ has bounded support. There exists a convex function $\phi: \mathbb{R}^d\to \mathbb{R}$ such that its gradient, $\nabla \phi$, is the optimal transport map from $\mu$ to $\nu$.

\end{theorem}

We call $\phi$ a Kantorovich potential.

\begin{corollary}{Existence of Solution to Monge Problem}

Under the above assumptions, $\nabla \phi$ uniquely solves the Monge problem.

\[
\int_X \abs{x-\nabla\phi(x)}^2\,d\mu(x) = \int_{T_{\#}\mu=\nu}\abs{x-T(x)}^2\,d\mu(x)
\]

\end{corollary}

\section*{Key Optimization Problem}

We wish to examine the following optimization problem, where $H$ is a reproducing kernel hilbert space and $c(x,y)$ denotes an arbitrary cost function in $x$ and $y$.

$$ \inf_{u,v} \left[ \int u \, ds + \int v \, dt + \lambda (\norm{u}_H^2 + \norm{v}_H^2)\right],\, \frac{1}{2}\norm{x}^2 + \frac{1}{2}\norm{y}^2 - u(x) - v(y)\leq c(x,y). $$

We will circle back to this after discussing some convex analysis preliminaries crucial to analyzing this going forward.

\section{Convex Analysis Prerequisites}

Before we continue, we introduce some concepts from convex analysis that will be needed going forward.

\begin{definition}
	Let $f:\,X\to \mathbb{R}\cup +\infty$ be a lower semicontinuous function. The Fenchel-Legendre transform $f^{*}: X^{*}\to \mathbb{R}\cup +\infty$ is defined by
	$$
	f^{*}(x^{*}) = \sup_{x\in X} [\ang*{x^{*},x}-f(x)]
	$$

	This $f^{*}$ is always convex.
\end{definition}

\begin{definition}
	The subdifferential of a lower semi-continuous convex function $\phi$ at $x\in \textrm{dom} \phi$ is defined by

	$$
	\partial \phi(x) = \{ x^{*}\in X^{*}: \phi(y)-\phi(x)\geq \ang*{x^{*},y-x} \}
	$$
\end{definition}

\begin{corollary}
	Let $f:\,X\to \mathbb{R}\cup \{+\infty\}$ be a convex function. Then for any $x\in \textrm{int dom }f$,

	$$ \partial f(x)\neq \emptyset.$$
\end{corollary}

\begin{theorem}[Fenchel-Young inequality]
	$$f(x)+f^{*}(x^{*})\geq \ang*{x^{*},x}$$
\end{theorem}

\begin{corollary}[Fenchel-Young equality]
	$$f(x)+f^{*}(x^{*})= \ang*{x^{*},x}$$ iff
	$$x^{*}\in \partial f(x).$$
\end{corollary}

\begin{theorem}
	Let $v(y) = \inf_x [f(x)+g(x+y)].$ The Fenchel primal problem is
	$$p = v(0) = \inf_x [f(x)+g(x)].$$

	The dual problem is
	$$d = v^{**}(0)=\sup_{y^{*}} [-f^{*}(y^{*})-g^{*}(-y^{*})].$$
\end{theorem}

\begin{proof}
	Calculate $v^{*}(-y^{*}) = \sup_{x,y}[\ang*{-y^{*},y}-f(x)-g(x+y)].$

	Let $u = x+y$, so we have
	\begin{align*}
	v^{*}(-y^{*}) &= \sup_{x,u} \ang*{-y^{*},u-x} - f(x) - g(u) \\
	&= \sup_x [\ang*{y^{*},x}-f(x)]+\sup_u [\ang*{-y^{*},u}-g(u)] \\
	&= f^{*}(y^{*}) + g^{*}(-y^{*}).
	\end{align*}

	Thus,
	$$d = v^{**}(0) = \sup_{-y^{*}}[0 - v^{*}(-y^{*})] = \sup_{-y^{*}}[-f^{*}(y^{*})-g^{*}(-y^{*})].$$

	Weak duality $p\geq d$ follows immediately. Strong duality $p=d$ requires $\partial v(0)\neq \emptyset$, i.e.
	$$0\in \textrm{int dom }v = \textrm{int}[\textrm{dom } g-\textrm{dom} f].$$
\end{proof}

\begin{proof}[Strong duality criterion]
	Here we prove $0\in \textrm{int dom }v = \textrm{int}[\textrm{dom } g-\textrm{dom} f]$ implies $\partial v(0)\neq \emptyset$. Let $-y^{*}\in \partial v(0)$ we have
	$$
	f(x)+g(x+y)\geq v(y)\geq p-\ang*{y^{*},y}
	$$
	Letting $u=x+y$, we have
	$$p\leq f(x)-\ang*{y^{*},x}+g(u)+\ang*{y^{*},u}
	$$ and taking infimum with respect to $x,u$, we have
	$$
	p\leq -f^{*}(y^{*})-g^{*}(-y^{*})\leq d\leq p.
	$$
	This concludes the proof.
\end{proof}

\section{Derivation of Primal Formulation}

Our goal is to find the primal formulation for the following optimization problem:
$$ \inf_{u,v} \left[ \int u \, ds + \int v \, dt + \lambda (\norm{u}_H^2 + \norm{v}_H^2)\right],\, \frac{1}{2}\norm{x}^2 + \frac{1}{2}\norm{y}^2 - u(x) - v(y)\leq c(x,y). $$ The arguments that follow are similar to those used to prove Kantorovich duality.

Let $$\phi_c = \{ (u,v) \in C(X)\times C(Y): \frac{1}{2}\norm{x}^2 + \frac{1}{2}\norm{y}^2 - u(x) - v(y)\leq c(x,y)\}$$.
%$E^{*} = M(X\times Y)$ is the set of regular Radon measures with total variation norm.

In the next few lines, $z \in E = C(X\times Y)$ is the set of continuous functions on $X\times Y$ with sup. norm and $\pi\in E^{*} = M(X\times Y)$ is the set of regular Radon measures with total variation norm.

$$f(z) = \begin{cases}
\lambda( \left\|u\right\|_H^2 + \left\|v\right\|_H^2) + \int u\,ds + \int v\,dt, \, z(x,y)=u(x)+v(y) \\
+\infty, \, \textrm{ otherwise.}
\end{cases}$$

$$g(z) = \begin{cases}
0, \, \frac{1}{2}\norm{x}^2 + \frac{1}{2}\norm{y}^2 - z(x,y)\leq c(x,y) \\
+\infty, \, \textrm{ otherwise.}
\end{cases}$$

$$f(z)+g(z) = \begin{cases}
\int_X u\,ds + \int_Y v\,dt + \lambda(\left\|u\right\|_H^2 + \left\|v\right\|_H^2), \, \frac{1}{2}\norm{x}^2 + \frac{1}{2}\norm{y}^2 - u(x) - v(y)\leq c(x,y) \\
\infty, \textrm{ otherwise.}
\end{cases}$$

$$\inf_{z\in E} [f(z)+g(z)] = \inf_{(u,v)\in \Phi_c} \left\{ \int_X u\,ds + \int_Y v\,dt + \lambda(\left\|u\right\|_H^2 + \left\|v\right\|_H^2) \right\}$$

\begin{align*}
g^{*}(-\pi) &= \sup_{z\in E} \left[ -\int_{X\times Y} z\,d\pi - g(z)\right] \\
&= \sup_{z\in E} \left[ - \int_{X\times Y} z\,d\pi :\, \frac{1}{2}\norm{x}^2 + \frac{1}{2}\norm{y}^2 - c(x,y)\leq z(x,y) \right] \\
&= \begin{cases}
\int_{X\times Y} \left[c(x,y) - \frac{1}{2}\norm{x}^2 - \frac{1}{2}\norm{y}^2\right] \,d\pi,\, \pi\in M_{+}(X\times Y) \\
+\infty, \, \textrm{otherwise}
\end{cases}
\end{align*}

\begin{align*}
f^{*}(\pi) &= \sup_{z\in E} \left[ \int_{X\times Y} z\,d\pi - f(z)\right] \\
&= \sup_{z\in E} \left[ \int_{X\times Y} z(x,y)\,d\pi(x,y) - \int_X u\,ds - \int_Y v\,dt - \lambda(\left\|u\right\|_H^2 + \left\|v\right\|_H^2) :\, z= u\oplus v \right] \\
&= \begin{cases}
- \lambda(\left\|u\right\|_H^2 + \left\|v\right\|_H^2), \, \pi\in \Pi(s,t) \\
-\inf_u (\int u\,ds - \int u\,d\pi + \lambda \norm{u}_H^2) - \inf_v (\int v\,dt - \int v\,d\pi + \lambda \norm{v}_H^2), \, \textrm{otherwise}
\end{cases}
\end{align*}

When finite, the last line (when $\pi$ is not a coupling) can be rewritten as
$$\frac{\sup_{\norm{u}=1} (\int u\,ds - \int u\,d\pi)^2}{2\lambda} + \frac{\sup_{\norm{v}=1}(\int v\,dt - \int v\,d\pi)^2}{2\lambda}$$ if finite. Otherwise, $f^{*}(\pi)=+\infty$.

The dual problem is $\sup_{\pi\in E^{*}} \left[ -f^{*}(\pi)-g^{*}(-\pi)\right] $, which, from the above, is
$$ \sup_{\pi\in E^{*}} \left\lbrace - \int_{X\times Y} c\,d\pi - \int_X \varphi\,d\mu - \int_Y \psi\,d\nu + \int_{X\times Y} (\varphi(x)+\psi(y))\,d\pi + \lambda(\norm{\varphi}_H^2+\norm{\psi}_H^2)\right\rbrace . $$

\subsection{What does the dual of f look like?}

If the previous claim holds,

$$
f^{*}(\pi) = \frac{1}{4\lambda} \left(Q(s,\pi_1) + Q(t,\pi_2)\right)
$$ where $$Q(s,\pi_1) = \int k(x,y)\,ds(x) ds(y) + \int k(x,y)\,d\pi_1(x) d\pi_1(y) - 2 \int k(x,y)\,ds(x)d\pi_1(y)$$ and similarly for $Q(t,\pi_2)$.

Then our dual problem is
\begin{align*}
\sup_{\pi\in E^{*}}(-f^{*}(\pi) - g^{*}(-\pi)) &= \sup_{\pi\in M_{+}} \left[ -\frac{1}{4\lambda} (Q(s,\pi_1)+Q(t,\pi_2)) - \int \left( c(x,y) - \frac{1}{2}\norm{x}^2 - \frac{1}{2}\norm{y}^2 \right)\,d\pi \right] \\
&= \inf_{\pi\in M_{+}} \left[ \frac{1}{4\lambda} (Q(s,\pi_1)+Q(t,\pi_2)) + \int \left( c(x,y) - \frac{1}{2}\norm{x}^2 - \frac{1}{2}\norm{y}^2 \right)\,d\pi \right]
\end{align*}

From here on, we assume our cost function is the square-loss, i.e. $c(x,y)=\frac{1}{2}\norm{x-y}^2$. It immediately follows that $c(x,y)-\frac{1}{2}\norm{x}^2 - \frac{1}{2}\norm{y}^2 = x^{T}y$

The question now is how to minimize
$$
\frac{1}{4\lambda} (Q(s,\pi_1)+Q(t,\pi_2)) + \int x^{T}y\,d\pi(x,y).
$$

\subsection{Inner product space of signed measures}
Define the inner product with respect to signed measures $m,n$ on $\mathcal{X}$ as $$\ang{m,n} = \int \tilde{k}(u,v)\,dm(u)\,dn(v).$$

and

$$\norm{s\otimes \pi_2 - \pi}^2 = Q(s,\pi_1) = \norm{s-\pi_1}^2.$$

If we're looking at $\mathcal{X}^2$, then we have
$$
\ang{m,n} = \int \tilde{k}((u_1,u_2),(v_1,v_2))\,dm(u_1,u_2)\,dn(v_1,v_2)
$$

and

$$\norm{s\otimes \pi_2 - \pi}^2 = \int \tilde{k}((u_1,u_2),(v_1,v_2))\,d(s(u_1)\pi_2(u_2)-\pi(v_1,v_2))$$

Then our problem reduces to showing $$\inf_m \left\lbrace \ang{m,s-\pi_1} + \lambda \ang{m,m}\right\rbrace  = -\frac{1}{4\lambda} \ang{s-\pi_1,s-\pi_1}.$$

\begin{proof}
	\begin{align*}
	\ang{m,s-\pi_1} + \lambda\ang{m,m} &= \lambda(\ang{m,m}+\ang{m,\frac{s-\pi_1}{\lambda}}) \\
	&= \lambda (\norm{m + \frac{s-\pi_1}{2\lambda}}^2 - \norm{\frac{s-\pi_1}{2\lambda}}^2)
	\end{align*}

	Choosing $m = \frac{\pi_1 - s}{2\lambda}$, we obtain the desired minimum.
\end{proof}

\subsection{Discrete Setting}

If we take $\pi_1$ and $\pi_2$ to be discrete measures, we can represent them by $\pi_1 = \Pi 1$ and $\pi_2^T=1^{T}\Pi$, i.e. taking the row and column marginals of $\Pi$.

Our optimization problem then becomes $$\inf_{\Pi} \frac{1}{4\lambda} (\norm{s-\Pi 1}^2+\norm{t-\Pi^{T} 1}^2) + tr(A^{T}\Pi).$$

\section{An Alternative Optimization Problem}
$f(\Pi,\lambda)=\dfrac{(\Pi\vec{1}-s)^{T}K(\Pi\vec{1}-s)+(\Pi^{T}\vec{1}-t)^{T}K(\Pi^{T}\vec{1}-t)}{4\tau}-Tr[\Pi K]+\lambda(1-\vec{1}^{T}\Pi1)-v_{1}E_{1}^{T}\Pi E_{1}-v_{2}E_{2}^{T}\Pi E_{1}-v_{3}E_{1}^{T}\Pi E_{2}-v_{4}E_{2}^{T}\Pi E_{2}$

where $E_{1}=\left(\begin{array}{c}
1\\
0
\end{array}\right)$ and $E_{2}=\left(\begin{array}{c}
0\\
1
\end{array}\right)$ (assuming this is the 2D setting).

$\frac{df}{d\Pi}=2K(\Pi\vec{1}-s)\vec{1}^{T}+2K(\Pi^{T}\vec{1}-t)\vec{1}^{T}-K-\lambda\vec{1}\vec{1^{T}}=2K[(\Pi+\Pi^{T})\vec{1}-(s+t)]\vec{1}^{T}-K-\lambda11^{T}-v_{1}\left(\begin{array}{cc}
1 & 0\\
0 & 0
\end{array}\right)-v_{2}\left(\begin{array}{cc}
0 & 0\\
1 & 0
\end{array}\right)-v_{3}\left(\begin{array}{cc}
0 & 1\\
0 & 0
\end{array}\right)-v_{4}\left(\begin{array}{cc}
0 & 0\\
0 & 1
\end{array}\right)$

If $K=I$, then by Karsh Kuhn Tucker conditions, we have

$\frac{\pi_{11}-\pi_{22}-(s_{1}+t_{1})+1}{2\tau}-(\lambda+v_{1}+1)=0$

$\frac{\pi_{22}-\pi_{11}-(s_{2}+t_{2})+1}{2\tau}-(\lambda+v_{2})=0$

$\frac{\pi_{11}-\pi_{22}-(s_{1}+t_{1})+1}{2\tau}-(\lambda+v_{3})=0$

$\frac{\pi_{22}-\pi_{11}-(s_{2}+t_{2})+1}{2\tau}-(\lambda+v_{4}+1)=0$

And it follows that

$v_{1}+1=v_{3}$

$v_{4}+1=v_{2}$

and also

$\frac{1}{2\tau}-1=v_{1}+v_{2}+2\lambda=v_{3}+v_{4}+2\lambda$.

If $\pi_{11},\pi_{22}=\frac{s_{1}+t_{1}}{2},\frac{s_{2}+t_{2}}{2}$,
then $\pi_{11}+\pi_{22}=1$ and thus $\pi_{12}=\pi_{21}=0$. Letting
$v_{1}=v_{4}=0$ and $v_{2}=v_{3}=1$, this satisfies slackness constraints.

In the following, we provide some sample code that demonstrates these results empirically.

\subsection*{Example Code}

\begin{lstlisting}
	import cvxpy as cp
	import numpy as np

	n = 2
	#b = 10

	PP = cp.Variable((n,n),"PP")
	KK = [[4,1],[1,4]]
	#s = np.array([[.5, .5]]).T
	#t = np.array([[.2,.8]]).T
	s = np.array([[.3, .7]]).T
	t = np.array([[.2,.8]]).T
	e = np.ones((n,1))
	x = PP.T@e - s
	y = PP@e - t
	for b in range(1,21):
	obj = (1/4/b) * (cp.quad_form(x,KK) + cp.quad_form(y,KK)) - cp.trace(KK@PP)
	prob = cp.Problem(cp.Minimize(obj),[PP>=0,cp.sum(PP)==1])
	obj=prob.solve()
	print("status:",prob.status)
	print("obj:",obj)
	print(PP.value)

	n = 3
	PP = cp.Variable((n,n),"PP")
	KK = [[1,0,0],[0,1,0],[0,0,1]]
	s = np.array([[.1, .4, .5]]).T
	t = np.array([[.4, .2, .4]]).T
	e = np.ones((n,1))
	x = PP.T@e - s
	y = PP@e - t
	for b in range(1,21):
	obj = (1/4/b) * (cp.quad_form(x,KK) + cp.quad_form(y,KK)) - cp.trace(KK@PP)
	prob = cp.Problem(cp.Minimize(obj),[PP>=0,cp.sum(PP)==1])
	obj=prob.solve()
	print("status:",prob.status)
	print("obj:",obj)
	print(PP.value)

	Output after running on Ubuntu machine.

	yannik@yannik-ubuntu:~/OTDA$ python optimization_implementation.py
	status: optimal
	obj: -3.9925
	[[ 2.50000000e-01  1.22249411e-23]
	[-1.23247236e-22  7.50000000e-01]]
	status: optimal
	obj: -3.99625
	[[ 2.50000000e-01 -1.74316142e-22]
	[ 6.32939582e-23  7.50000000e-01]]
	status: optimal
	obj: -3.9975
	[[ 2.50000000e-01 -1.16215745e-22]
	[-2.16851043e-22  7.50000000e-01]]
	status: optimal
	obj: -3.998125
	[[2.50000000e-01 5.50834936e-23]
	[5.59387059e-23 7.50000000e-01]]
	status: optimal
	obj: -3.9985
	[[2.50000000e-01 5.92447828e-23]
	[1.62799830e-22 7.50000000e-01]]
	status: optimal
	obj: -3.99875
	[[ 2.50000000e-01 -1.05815269e-22]
	[-1.16229217e-22  7.50000000e-01]]
	status: optimal
	obj: -3.9989285714285714
	[[ 2.50000000e-01 -1.91865798e-24]
	[ 1.12940857e-22  7.50000000e-01]]
	status: optimal
	obj: -3.9990624999999995
	[[2.50000000e-01 2.21829294e-22]
	[1.11237621e-22 7.50000000e-01]]
	status: optimal
	obj: -3.9991666666666665
	[[2.50000000e-01 1.68413892e-22]
	[5.36304987e-23 7.50000000e-01]]
	status: optimal
	obj: -3.99925
	[[ 2.50000000e-01 -1.11021317e-22]
	[-1.11023280e-22  7.50000000e-01]]
	status: optimal
	obj: -3.999318181818182
	[[ 2.50000000e-01 -1.66641110e-22]
	[-5.54035982e-23  7.50000000e-01]]
	status: optimal
	obj: -3.999375
	[[ 2.50000000e-01 -1.11238505e-22]
	[ 2.16099333e-25  7.50000000e-01]]
	status: optimal
	obj: -3.999423076923077
	[[ 2.50000000e-01 -1.11130073e-22]
	[ 1.07889446e-25  7.50000000e-01]]
	status: optimal
	obj: -3.9994642857142857
	[[ 2.50000000e-01 -5.66878028e-23]
	[ 5.66878108e-23  7.50000000e-01]]
	status: optimal
	obj: -3.9995
	[[ 2.50000000e-01  1.12085206e-22]
	[-1.06311748e-24  7.50000000e-01]]
	status: optimal
	obj: -3.9995312499999995
	[[2.50000000e-01 5.51862768e-23]
	[5.58360336e-23 7.50000000e-01]]
	status: optimal
	obj: -3.9995588235294117
	[[2.50000000e-01 5.46823694e-23]
	[5.63399411e-23 7.50000000e-01]]
	status: optimal
	obj: -3.9995833333333333
	[[ 2.50000000e-01 -1.11130414e-22]
	[-1.10914183e-22  7.50000000e-01]]
	status: optimal
	obj: -3.999605263157895
	[[ 2.50000000e-01 -5.52883039e-23]
	[-1.66756293e-22  7.50000000e-01]]
	status: optimal
	obj: -3.999625
	[[ 2.50000000e-01 -1.10718444e-22]
	[-3.03850898e-25  7.50000000e-01]]
	status: optimal
	obj: -0.9825
	[[ 2.50000000e-01  1.10709851e-22 -1.11209797e-22]
	[ 2.22356962e-22  3.00000000e-01 -1.10897391e-22]
	[-1.10834732e-22 -1.11147135e-22  4.50000000e-01]]
	status: optimal
	obj: -0.99125
	[[ 2.50000000e-01 -1.18086022e-22  5.54360229e-23]
	[-1.03958191e-22  3.00000000e-01 -4.85221266e-23]
	[ 5.55859871e-23 -6.24999931e-23  4.50000000e-01]]
	status: optimal
	obj: -0.9941666666666666
	[[ 2.50000000e-01  1.67542279e-24 -9.02920747e-25]
	[-1.67529689e-24  3.00000000e-01 -2.57830146e-24]
	[ 2.22947625e-22  2.57835866e-24  4.50000000e-01]]
	status: optimal
	obj: -0.995625
	[[ 2.50000000e-01  1.07518204e-22 -6.07356262e-23]
	[ 3.36570089e-22  3.00000000e-01 -5.72319709e-23]
	[ 1.71758402e-22 -5.37898275e-23  4.50000000e-01]]
	status: optimal
	obj: -0.9965
	[[ 2.50000000e-01  3.62041633e-24 -2.23532165e-22]
	[-2.25665827e-22  3.00000000e-01 -1.16130663e-22]
	[-1.09534272e-22 -2.16935737e-22  4.50000000e-01]]
	status: optimal
	obj: -0.9970833333333333
	[[ 2.50000000e-01 -3.04602898e-25  1.08844672e-22]
	[ 2.22349078e-22  3.00000000e-01  2.20171516e-22]
	[ 2.24222382e-22  1.12895546e-22  4.50000000e-01]]
	status: optimal
	obj: -0.9975
	[[ 2.50000000e-01  1.67389178e-22  5.46058619e-23]
	[ 5.46549226e-23  3.00000000e-01 -1.76082074e-24]
	[ 2.78460827e-22  1.76149241e-24  4.50000000e-01]]
	status: optimal
	obj: -0.9978125
	[[ 2.50000000e-01  1.12313616e-22 -1.11893932e-22]
	[ 1.09729020e-22  3.00000000e-01 -2.16476158e-24]
	[-1.10148658e-22  2.16510806e-24  4.50000000e-01]]
	status: optimal
	obj: -0.9980555555555556
	[[ 2.50000000e-01  1.53521320e-24  5.55069310e-23]
	[-1.53498217e-24  3.00000000e-01  5.39719504e-23]
	[ 5.55152244e-23  5.70504391e-23  4.50000000e-01]]
	status: optimal
	obj: -0.99825
	[[ 2.50000000e-01 -5.78584577e-23  2.20132019e-22]
	[-5.31633945e-23  3.00000000e-01  1.66970182e-22]
	[ 1.12932966e-22  5.50760656e-23  4.50000000e-01]]
	status: optimal
	obj: -0.9984090909090909
	[[ 2.50000000e-01 -1.10159194e-22  5.78533683e-23]
	[-1.11884677e-22  3.00000000e-01 -5.40302524e-23]
	[ 5.31672297e-23 -5.69909081e-23  4.50000000e-01]]
	status: optimal
	obj: -0.9985416666666667
	[[ 2.50000000e-01  1.58478915e-24 -2.90879011e-25]
	[-1.58779679e-24  3.00000000e-01 -1.12899214e-22]
	[ 1.11314400e-22 -1.09143652e-22  4.50000000e-01]]
	status: optimal
	obj: -0.9986538461538461
	[[ 2.50000000e-01  1.29692731e-24 -1.09306083e-22]
	[-1.12320800e-22  3.00000000e-01 -1.10603467e-22]
	[-3.34782807e-22 -1.11440161e-22  4.50000000e-01]]
	status: optimal
	obj: -0.99875
	[[ 2.50000000e-01  3.05960762e-25 -1.10047970e-22]
	[-3.07807840e-25  3.00000000e-01  6.67870216e-25]
	[-9.73800980e-25 -1.11688797e-22  4.50000000e-01]]
	status: optimal
	obj: -0.9988333333333334
	[[ 2.50000000e-01 -2.20472007e-22 -1.10265620e-22]
	[-4.45663084e-22  3.00000000e-01 -2.22859007e-22]
	[ 1.10264081e-22 -1.10205137e-22  4.50000000e-01]]
	status: optimal
	obj: -0.99890625
	[[2.50000000e-01 1.67831187e-22 2.23235717e-22]
	[2.76255783e-22 3.00000000e-01 5.54049384e-23]
	[1.09830812e-22 5.56200424e-23 4.50000000e-01]]
	status: optimal
	obj: -0.9989705882352942
	[[ 2.50000000e-01  2.33062237e-24  1.12105844e-22]
	[-2.33398078e-24  3.00000000e-01 -1.24802428e-24]
	[-1.08251257e-24  1.25052514e-24  4.50000000e-01]]
	status: optimal
	obj: -0.9990277777777777
	[[ 2.50000000e-01 -5.37805096e-23  1.66737466e-22]
	[ 5.37758046e-23  3.00000000e-01 -1.52711421e-24]
	[ 5.53077093e-23  1.12553722e-22  4.50000000e-01]]
	status: optimal
	obj: -0.9990789473684211
	[[ 2.50000000e-01 -5.39169018e-23  5.52124567e-23]
	[-5.71093712e-23  3.00000000e-01 -1.12913039e-22]
	[ 5.58078346e-23 -2.20147494e-22  4.50000000e-01]]
	status: optimal
	obj: -0.999125
	[[ 2.50000000e-01  5.63739603e-23  1.67291982e-22]
	[ 5.46420535e-23  3.00000000e-01 -1.05069787e-25]
	[-1.67291107e-22  1.11132958e-22  4.50000000e-01]]

\end{lstlisting}

\section{Error Bounds}

We seek to determine bounds on the difference between our RKHS dual regularized optimal transport problem
$$S(P,\displaystyle \ Q)=\sup_{f\in H,g\in H}\int f(x)\mathrm{d}P(x)+\int g(y)\mathrm{d}Q(y)-\lambda(\left\|f\right\|_H+\left\|g\right\|_H),\quad f\oplus g\leq c $$

and the unregularized OT,

$$OT(P,\displaystyle \ Q)=\sup_{f\in H,g\in H}\int f(x)\mathrm{d}P(x)+\int g(y)\mathrm{d}Q(y),\quad f\oplus g\leq c. $$

We see that the difference between these two quantities is dependent on the norms of the dual potentials for the unregularized and regularized optimal transport problem. The proof here is similar to that of \cite{Sonthalia2020} but here, we generalize from the discrete setting.
\begin{theorem}
	$$
	\lambda(\left\|f^{*}\right\|_H + \left\|g^{*}\right\|_H) \leq OT(P,Q)-S(P,Q)\leq \lambda(\left\|f_0\right\|_H+\left\|g_0\right\|_H)
	$$
\end{theorem}

\begin{proof}
	Let $f_0, g_0$ be dual solutions to $OT(P,Q)$ and $f^{*},g^{*}$ be corresponding solutions to $S(P,Q)$. Since $f^{*},g^{*}$ satisfy the constraint $f^{*}\oplus g^{*}\leq c$, we have

	$$\int f^{*}(x)\mathrm{d}P(x)+\int g^{*}(y)\mathrm{d}Q(y) \leq \int f_0(x)\mathrm{d}P(x)+\int g_0(y)\mathrm{d}Q(y) = OT(P,Q)$$

	Subtracting $\lambda(\left\|f^{*}\right\| + \left\|g^{*}\right\|)$ from both sides gives us

	\begin{align*}
		S(P,Q)&=\int f^{*}(x)\mathrm{d}P(x)+\int g^{*}(y)\mathrm{d}Q(y) - \lambda(\left\|f^{*}\right\|_H + \left\|g^{*}\right\|_H) \\
		&\leq \int f_0(x)\mathrm{d}P(x)+\int g_0(y)\mathrm{d}Q(y) - \lambda(\left\|f^{*}\right\|_H + \left\|g^{*}\right\|_H) \\
		&= OT(P,Q) - \lambda(\left\|f^{*}\right\| + \left\|g^{*}\right\|)
	\end{align*}

	Thus we have $OT(P,Q)-S(P,Q)\geq \lambda(\left\|f^{*}\right\| + \left\|g^{*}\right\|)$.

	Next, note that
	$$
	-\int f_0(x)\mathrm{d}P(x)-\int g_0(y)\mathrm{d}Q(y)+\lambda(\left\|f_0\right\|_H+\left\|g_0\right\|_H) \geq -S(P,Q).
	$$

	Rearranging this gives us $OT(P,Q)-S(P,Q)\leq \lambda(\left\|f_0\right\|_H+\left\|g_0\right\|_H).$

	Combining these two inequalities, we get our desired bound.
\end{proof}

\section{Sample Complexity Bound}

	\subsection*{Introduction}
	Previously, we investigated the dual form of the dual norm regularized optimal transport and examined some of its fundamental properties. Now, we transition to establishing finite sample guarantees. In particular, we show that our convergence rate is faster than that of unregularized OT under certain conditions.

	Our quantity can be expressed as follows, where we regularize by the norms of the dual potentials $f,g$ in an RKHS $H$,
	$$S(P,\displaystyle \ Q)=\sup_{f\in L_{1}(P) \cap H,g\in L_{1}(Q) \cap H}\int f(x)\mathrm{d}P(x)+\int g(y)\mathrm{d}Q(y)-\lambda(\left\|f\right\|_H+\left\|g\right\|_H).$$

	%We focus throughout on subgaussian probability measures. We say that a distribution $P\in \mathcal{P}(\mathbb{R}^{d})$ is $\sigma^{2}$-subgaussian for $\sigma\geq 0$ if $E_{P}e^{\frac{||X\Vert^{2}}{2d\sigma^{2}}}\leq 2$. By Jensen's inequality, if $E_{pe}2d\sigma^{2}\leq C$ for any constant $C\geq 2$, then $P$ is $C\sigma^{2}$-subgaussian. Note that if $P$ is subgaussian, then $ E_{P}e^{v^{\mathrm{T}}X}<\infty$ for all $v\in \mathbb{R}^{d}$. Conversely, standard results (see, e.g., ) imply that our definition is satisfied if $E_{P}e^{u^{\mathrm{T}}X}\leq e^{\Vert u\Vert^{2}\sigma^{2}/2}$ for all $u\in \mathbb{R}^{d}.$

	Throughout this section, we assume $P$ and $Q$ are measures on compact subsets $\mathcal{X}$ and $\mathcal{Y}$ in $\mathbb{R}^d$. And assume the cost function $c$ is continuous in $\mathcal{X}\times \mathcal{Y}$. Let $K=\max_{X\times Y}c(x,y)$.

	\subsection*{Proof Technique}
	 The proof technique here is inspired by work on entropic optimal transport sample complexity \cite{Mena2019}. First, we use a simple argument to remove the regularization terms. Then we examine the empirical process over some set that the optimal potentials belong to. Key to our results are the establishment of uniform bounds of the potential functions. And since these potentials are in an RKHS, we can exploit the structure of the RKHS to establish empirical process bounds. After controlling the potentials, we use a chaining bound.

	\subsection*{Uniform bounds on the optimal potentials}
	\begin{prop}
		The optimal $f,g$, denoted $f^*,g^*$, of $$\int f(x)\mathrm{d}P(x)+\int g(y)\mathrm{d}Q(y)-\lambda(\left\|f\right\|_H+\left\|g\right\|_H)$$ lie in a RKHS ball of radius $W(P,Q)/\lambda$, where
		$$
		W(P,Q)=\min_{\pi\in \Pi(P,Q)} \int c\,d\pi.
		$$
	\end{prop}
	\begin{proof}
		\begin{align*}
			\lambda (\left\|f^*\right\|+\left\|g^*\right\|)&\leq \lambda (\left\|f^*\right\|+\left\|g^*\right\|) + \sup_{f,g s.t. f\oplus g\leq c} \left\{\int f\,dP + \int g\,dQ \right\} - \left(\int f^{*}\,dP + \int g^{*}\,dQ\right) \\
			&\leq \lambda (\left\|f^*\right\|+\left\|g^*\right\|) + W(P,Q) - \left(\int f^{*}\,dP + \int g^{*}\,dQ\right) \\
			&\leq \sup_{f,g\in H} \left\{\lambda (\left\|f\right\|+\left\|g\right\|) + W(P,Q) - \left(\int f\,dP + \int g\,dQ\right)\right\}
		\end{align*}

		Assuming $c(x,y)\geq 0$, we get $$\lambda(\left\|f^*\right\|+\left\|g^*\right\|)\leq K(P,Q)$$ which implies that $f^*$ and $g^*$ lie in an RKHS ball of radius $W(P,Q)/\lambda$.
	\end{proof}

	We denote by $\mathcal{F}$ the set of functions in the RKHS ball with radius $\frac{K}{\lambda}$. The following proposition shows that it suffices to control an empirical process indexed by this set.

	\begin{prop}
		Let $P, Q$, and $P_{n}$ be probability distributions.
		Then
		\begin{center}
			(9)   $|S(P_{n},\displaystyle \ Q)-S(P,\ Q)|\leq 2\sup_{u\in \mathcal{F}}|E_{P}u-E_{P_n}u|.$
		\end{center}
	\end{prop}

	\begin{proof}

		We define the operator $\mathcal{A}^{\alpha,\beta}(u,\ v)$ for the pair of probability measures $(\alpha,\ \beta)$ and functions $(u,\ v)\in L_{1}(\alpha)\otimes L_{1}(\beta)$ as:

		$\displaystyle \mathcal{A}^{\alpha,\beta}(u,\ v)=\int u(x)\mathrm{d}\alpha(x)+\int v(y)\mathrm{d}\beta(y)-\lambda(\left\|u\right\|_H+\left\|v\right\|_H).$

		Denote by $(f_{n},\ g_{n})$ a pair of optimal potentials for $(P_{n},\ Q)$ and $(f,\ g)$ for $(P,\ Q)$, respectively. We can choose smooth optimal potentials $(f,\ g)$ and $(f_{n},\ g_{n})$ to lie in the RKHS balls with radii $W(P,Q)/\lambda$ and $W(P_n,Q)/\lambda$ respectively for all $x, y\in \mathbb{R}^{d}.$ And $W(P,Q)\leq K$ by construction and similarly for $W(P_n,Q)$. Thus $f$, $f_n\in \mathcal{F}$.

		Strong duality implies that $S(P,\ Q)=\mathcal{A}^{P,Q}(f,\ g)$ and $S(P_{n},\ Q)=\mathcal{A}^{P_{n},Q}(f_{n},\ g_{n}).$

		Moreover, by the optimality of $(f,\ g)$ and $(f_{n},\ g_{n})$ for their respective dual problems, we obtain

		$$\mathcal{A}^{P,Q}(f_{n},\ g_{n})-\mathcal{A}^{P_{n},Q}(f_{n},\ g_{n})\leq \mathcal{A}^{P,Q}(f,\ g)-\mathcal{A}^{P_{n},Q}(f_{n},\ g_{n})\leq \mathcal{A}^{P,Q}(f,\ g)-\mathcal{A}^{P_{n},Q}(f,\ g).$$

		From the above bound, we see that

		$|S(P,\ Q)-S(P_{n},\ Q)|=|\mathcal{A}^{P,Q}(f,\ g)-\mathcal{A}^{P_{n},Q}(f_{n},\ g_{n})|$
		$$
		\leq|\mathcal{A}^{P,Q}(f,\ g)-\mathcal{A}^{P_{n},Q}(f,\ g)|+|\mathcal{A}^{P,Q}(f_{n},\ g_{n})-\mathcal{A}^{P_{n},Q}(f_{n},\ g_{n})|.
		$$

		All that is left is bounding the differences $|\mathcal{A}^{P,Q}(f,\ g)-\mathcal{A}^{P_{n},Q}(f,\ g)|$ and $|\mathcal{A}^{P,Q}(f_{n},\ g_{n})- \mathcal{A}^{P_{n},Q}(f_{n},\ g_{n})|.$

		\begin{align*}
			\mathcal{A}^{P,Q}(f,\ g)-\mathcal{A}^{P_{n},Q}(f,\ g) &=\int f(x)(\mathrm{d}P(x)-\mathrm{d}P_{n}(x)) \\
			&\leq\sup_{u\in \mathcal{F}}|\int u(x)(\mathrm{d}P(x)-\mathrm{d}P_{n}(x))|.
		\end{align*}

		and similarly,

		\begin{align*}
			\mathcal{A}^{P,Q}(f_n,\ g_n)-\mathcal{A}^{P_{n},Q}(f_n,\ g_n) &=\int f_n(x)(\mathrm{d}P(x)-\mathrm{d}P_{n}(x)) \\
			&\leq\sup_{u\in \mathcal{F}}|\int u(x)(\mathrm{d}P(x)-\mathrm{d}P_{n}(x))|.
		\end{align*}

	\end{proof}

	\begin{corollary}

		Let $P, Q, P_{n}$, and $Q_{n}$ be probability distributions. Then
		$$|S(P_{n}, Q_{n})-S(P, Q)| \lesssim \sup_{u\in \mathcal{F}}|\int u(x)(\mathrm{d}P(x)-\mathrm{d}P_{n}(x))|
		+\sup_{v\in \mathcal{F}}|\int u(x)(\mathrm{d}Q(x)-\mathrm{d}Q_{n}(x))|
		$$
	\end{corollary}

	PROOF. By the triangle inequality,
	$$|S(P_{n},\ Q_{n})-S(P,\ Q)|\leq|S(P,\ Q)-S(P_{n},\ Q)|+|S(P_{n},\ Q)-S(P_{n},\ Q_{n})|.$$

	almost surely. $\square $
	\medskip

	\subsection*{Bounding the empirical process}
	Denote by $N(\varepsilon,\ \mathcal{F}^{s},\ L_{2}(P_{n}))$ the covering number with respect to the (random) metric $L_{2}(P_{n})$ defined by

	$\displaystyle \Vert f\Vert_{L_{2}(P_{n})}=\left(\frac{1}{n}\sum_{i=1}^{n}f(X_{i})^{2}\right)^{1/2}$

	The empirical process bounds established in this chapter rely on our reproducing kernel hilbert space (RKHS) having a special structure. Particularly, the reason we assumed that the RKHS exhibits a Gaussian radial basis function kernel will soon be clear.

	From here onwards, define $K=\max(K(P,Q),K(P_n,Q))$. Our first preliminary result is the following proposition:

	\begin{prop}
		Let $\mathcal{H}_{\sigma}$ be a Gaussian radial basis function RKHS with the kernel defined as $k(x,y)=e^{-\sigma^2 \left\|x-y\right\|_2^2}$. If $P_{n}$ is an empirical distribution, then, given the sample $X_{1}$, . . . , $X_{n}$, we have the bound
		$$
		\max_{f\in \mathcal{F}} \Vert f\Vert_{L_{2}(P_{n})}^{2}\leq \frac{K^2}{\lambda^2}\ .
		$$
	\end{prop}

	\begin{proof}
		Denote $$f(x)=\langle f(t),K(x,t)\rangle \leq \left\|f\right\|_{\mathcal{H}}\max_{x\in X} \sqrt{k(x,x)}$$ and $k(x,x)=1$ if $k(x,y)=e^{-\sigma^2 \left\|x-y\right\|_2^2}$. Thus $\left|f(x)\right|\leq \left\|f\right\|_{\mathcal{H}}$ for all $x\in X$.
	\end{proof}

	Assuming the RKHS is Gaussian like in the previous proposition, then we have the covering number bound.
	\begin{theorem} \cite{Steinwart2007}
		Let $\sigma\geq 1$, $X\subset \mathbb{R}^d$ be a compact subset with nonempty interior, and $H_{\sigma}(X)$ be the RKHS of the Gaussian RBF kernel $k_{\sigma}$ on $X$. Then for all $0<p\leq 2$ and all $\delta>0$, there exists a constant $C_{p,\delta,d}>0$ independent of $\sigma$ such that for all $\epsilon>0$ we have
		$$
		\sup_{T\in (X\times Y)^n} \log N(\varepsilon,\ \mathcal{F},\ L_{2}(P_{n}))\leq c_{p,\delta,d} \sigma^{(1-p/2)(1+\delta)d}\epsilon^{-p}.
		$$
	\end{theorem}

	Using this result, we then have, by the use of a chaining bound \cite{Gine2016},

	\begin{align*}
		E \displaystyle\Vert P-P_n\Vert_{\mathcal{F}}^{2}&\lesssim \frac{1}{n}E(\int_{0}^{\sqrt{\max_{f\in \mathcal{F}}\Vert f\Vert_{L_{2}(P_{n})}^{2}}}\sqrt{\log 2N(\epsilon,\mathcal{F},L_{2}(P_{n}))}\mathrm{d}\epsilon)^{2} \\
		&\leq \frac{1}{n}E(\int_{0}^{K/\lambda}\sqrt{1+c_{p,\delta,d} \sigma^{(1-p/2)(1+\delta)d}\epsilon^{-p}}\mathrm{d}\epsilon)^{2} \\
		&\leq \frac{c'_{p,\delta,d} \sigma^{(1-p/2)(1+\delta)d}}{n}E(\int_{0}^{K/\lambda} \epsilon^{-p/2} \mathrm{d}\epsilon)^{2} \\
		&= \frac{c'_{p,\delta,d} \sigma^{(1-p/2)(1+\delta)d}}{n(1-p/2)^2} \left(\frac{K}{\lambda}\right)^{2-p}
	\end{align*}

	Here, $c_{p,\delta,d}$ and $c'_{p,\delta,d}$ denote constants with respect to $n$. One can notice here that the upper bound is $O(1/n)$.