\section{Transaction Costs Problem}
\label{sec:83problem}

\minitoc[-6mm]{70mm}{4}

\vspace{-1.5em}

\paragraph{Description}

In the \term{transaction costs problem,}
the individual can invest their money risk-free in bonds
(with a fixed interest rate similar to a bank account)
or in $m_{\vstock} \in \nat$ different risk-affected stocks
\cite{Schober18Solving}.
Every stock transaction,
i.e., buy $\buy_{t,j}$ or sell $\sell_{t,j}$,
inflicts transaction costs $\tac \buysell_{t,j}$ ($\tac \in \nonnegreal$)
proportional to the amount $\buysell_{t,j}$ bought or sold
($j = 1, \dotsc, m_{\vstock}$).
The individual only wants to invest a fixed
amount $\wealth_0$ in stocks, i.e., we omit the individual's income.

\subsection{Unnormalized Problem}
\label{sec:831unnormalized}

\paragraph{Consumption and state transition}

In the following,
$\stock_{t,j}$ denotes the fraction of the total wealth $\wealth_t$
that is invested in the $j$-th stock.
We combine these \term{stock fractions} $\stock_{t,j}$
in a vector $\vstock_t \ceq (\stock_{t,1}, \dotsc, \stock_{t,m_{\vstock}})$;
similarly, $\vbuysell_t \ceq (\buysell_{t,1}, \dotsc, \buysell_{t,m_{\vstock}})$
combines buy and sell amounts.
Then, the consumption can be computed as a residual variable
(i.e., a variable that can be fully computed from $\state$ and $\policy$
and is thus omitted from $\policy$),
which is given by
\begin{equation}
  \consume_t
  \ceq (1 - \sumfcn(\vstock_t)) \wealth_t - \bond_t -
  (1 + \tac) \sumfcn(\vbuy_t) + (1 - \tac) \sumfcn(\vsell_t),
\end{equation}
where $\sumfcn(\*a) \ceq \tr{\*1} \*a$
is the sum of all entries of $\*a$.
The state transition is computed by adding the returns of bonds and stocks:%
\begin{equation}
  \wealth_{t+1}
  \ceq \bond_t \bondreturn_t +
  %\sum_{j=1}^{m_{\vstock}}
  %(\stock_{t,j} \wealth_t + \buy_{t,j} - \sell_{t,j}) \stockreturn_{t,j},\\
  \tr{(\vstock_t \wealth_t + \vbuy_t - \vsell_t)} \vstockreturn_t,\qquad
  \vstock_{t+1}
  \ceq \frac{
    (\vstock_t \wealth_t + \vbuy_t - \vsell_t) \compmult \vstockreturn_t
  }{\wealth_{t+1}},
\end{equation}
where $\bondreturn_t \in \real$ is the bond interest rate,
$
  \vstockreturn_t
  = (\stockreturn_{t,1}, \dotsc, \stockreturn_{t,m_{\vstock}})
  \in \real^{m_{\vstock}}
$
is the vector of (stochastic) stock return rates, and
$\compmult$ is component-wise multiplication.

\subsection{Normalization}
\label{sec:832normalized}

\paragraph{State transition}

The above equations can be normalized with respect to the wealth
$\wealth_t$:
By setting
$\normconsume_t  \ceq \consume_t/\wealth_t$,
$\normbond_t     \ceq \bond_t   /\wealth_t$, and
$\vnormbuysell_t \ceq \vbuysell_t/\wealth_t$, we obtain
\begin{subequations}
  \label{eq:normalizedTCPStateTransition}
  \begin{align}
    \normconsume_t
    &\mathrel{\righthphantom{=}{\ceq}}
    (1 - \sumfcn(\vstock_t)) - \normbond_t -
    (1 + \tac) \sumfcn(\vnormbuy_t) + (1 - \tac) \sumfcn(\vnormsell_t),\\
    \wealthratio_{t+1}
    &\ceq \normbond_t \bondreturn_t +
    \tr{(\vstock_t + \vnormbuy_t - \vnormsell_t)} \vstockreturn_t,
    \qquad(= \wealth_{t+1}/\wealth_t)\\
    \vstock_{t+1}
    &\mathrel{\righthphantom{=}{\ceq}}
    \frac{
      (\vstock_t + \vnormbuy_t - \vnormsell_t) \compmult \vstockreturn_t
    }{\wealthratio_{t+1}},
  \end{align}
\end{subequations}
where $\normconsume_t$ and $\wealthratio_{t+1}$ are residual
variables that specify \term{normalized consumption} and
\term{wealth ratio,} respectively.
All in all, the resulting dynamic portfolio choice model has
the following variables:
\begin{itemize}
  \item
  $\centerhphantom{d}{m_{\stochastic}} = m_{\vstock}$
  state variables $\normstate_t$:
  Stock fractions $\stock_{t,1}, \dotsc, \stock_{t,m_{\vstock}}$

  \item
  $\centerhphantom{m_{\policy}}{m_{\stochastic}} = 2m_{\vstock} + 1$
  policy variables $\normpolicy_t$:
  Normalized bonds $\normbond_t$,
  normalized buy amounts $\normbuy_{t,1}, \dotsc, \normbuy_{t,m_{\vstock}}$ and
  normalized sell amounts $\normsell_{t,1}, \dotsc, \normsell_{t,m_{\vstock}}$

  \item
  $m_{\stochastic} = m_{\vstock}$
  stochastic variables $\stochastic_t$:
  Stock return rates $\stockreturn_{t,1}, \dotsc, \stockreturn_{t,m_{\vstock}}$
\end{itemize}
The state space and policy space constraints are given by
\begin{subequations}
  \label{eq:normalizedTCPConstraints}
  \newcommand*{\centereqline}[1]{%
    \mathclap{\hphantom{\mathrm{(8.99a)}}#1}%
  }%
  \begin{gather}
    \label{eq:normalizedTCPConstraintsShort}
    \centereqline{
      \vstock_t \ge \*0,\qquad
      \sumfcn(\vstock_t) \le 1,\qquad
      \normbond_t \ge 0,\qquad
      \vnormbuysell_t \ge \*0,\qquad
      \vnormsell_t \le \vstock_t,\qquad
      \wealthratio_{t+1} \ge 0,
    }\\
    \label{eq:normalizedTCPConstraintsLong}
    \centereqline{
      \normconsume_{\min} + \normbond_t +
      (1 + \tac) \sumfcn(\vnormbuy_t) - (1 - \tac) \sumfcn(\vnormsell_t)
      \le 1 - \sumfcn(\vstock_t),
    }
  \end{gather}
\end{subequations}
where $\normconsume_{\min} \in \nonnegreal$ is some minimal consumption
that must be maintained.

\paragraph{Bellman equation}

Consequently, the Bellman equation \eqref{eq:gridBellmanCET}
after the certainty-equiva\-lent transformation has to be
normalized as well.
By setting $\normcetvalueintp_t(\state_t^{(k)})
\ceq \cetvalueintp_t(\state_t^{(k)})/\wealth_t$, we obtain
\begin{subequations}
  \begin{align}
    %\normcetvalueintp_t(\state_t^{(k)})
    %&= \frac{\cetvalueintp_t(\state_t^{(k)})}{\wealth_t}\\
    %&= \max_{\policy_t} \left(
    %  \left(
    %    \left(
    %      \frac{\consume_t(\state_t^{(k)}, \policy_t)}{\wealth_t}
    %    \right)^{1-\riskav} +
    %    \patience \expectation[t]{
    %      \left(
    %        \frac{
    %          \cetvalueintp_{t+1}(
    %            \statefcn_t(\state_t^{(k)}, \policy_t, \stochastic_t)
    %          )
    %        }{
    %          \wealth_t
    %        }
    %      \right)^{1-\riskav}
    %    }
    %  \right)^{1/(1-\riskav)}
    %\right)\\
    &\hphantom{=}\hspace{0.6em} \normcetvalueintp_t(\state_t^{(k)})
    = \wealth_t^{-1} \cetvalueintp_t(\state_t^{(k)})\\
    &= \max_{\policy_t} \left(
      \left(
        \left(
          \wealth_t^{-1} \consume_t(\state_t^{(k)}, \policy_t)
        \right)^{1-\riskav} +
        \patience \expectation[t]{
          \left(
            \wealth_t^{-1} \cetvalueintp_{t+1}(
              \statefcn_t(\state_t^{(k)}, \policy_t, \stochastic_t)
            )
          \right)^{1-\riskav}
        }
      \right)^{1/(1-\riskav)}
    \right)\\
    \label{eq:normalizedTCPBellmanEquation}
    &= \max_{\normpolicy_t} \left(
      \left(
        \normconsume_t(\state_t^{(k)}, \normpolicy_t)^{1-\riskav} +
        \patience \expectation[t]{
          \bigl(
            \wealthratio_{t+1} \normcetvalueintp_{t+1}(
              \normstatefcn_t(\state_t^{(k)}, \normpolicy_t, \stochastic_t)
            )
          \bigr)^{1-\riskav}
        }
      \right)^{1/(1-\riskav)}
    \right).
  \end{align}
\end{subequations}
This means that compared with %the unnormalized Bellman equation
\eqref{eq:gridBellmanCET},
the value function in the expectation has to be multiplied by
the wealth ratio $\wealthratio_{t+1}$ introduced above in
\eqref{eq:normalizedTCPStateTransition}.
Since there is no inheritance, the optimal terminal solution
is to sell all stocks and consume everything:
\begin{equation}
  \normcetvalueintp_t(\state_T^{(k)})
  = 1 - \tac \sumfcn(\vstock_T^{(k)}),\quad
  \normbond_T^{\opt}(\state_T^{(k)})
  = 0,\quad
  \vnormbuy[\opt]_T(\state_T^{(k)})
  = \*0,\quad
  \vnormsell[\opt]_T(\state_T^{(k)})
  = \vstock_T^{(k)}.
\end{equation}

\subsection{State Space Cropping}
\label{sec:833cropping}

\paragraph{Sparse grids on non-rectangular domains}

Unfortunately, the constraint $\sumfcn(\vstock_t) \le 1$
from \cref{eq:normalizedTCPConstraints} limits the feasible state space
region to a proper subset (which is the unit simplex)
of the unit hypercube $\clint{\*0, \*1}$,
which impedes the direct application of sparse grids.
There are three possible remedies:
transforming the unit hypercube to the feasible state space,
applying extrapolation techniques as discussed in
\cref{sec:825interpolation}, or
choosing a model-tailored approach to obtain
function values outside the feasible state space.

\paragraph{Virtual selling of stocks}

We choose the third remedy and \term{virtually sell,}
if $\sumfcn(\vstock_t) > 1$,
as many stocks as needed to meet the constraint $\sumfcn(\vstock_t) \le 1$.
We already might need to sell stocks
even if $\sumfcn(\vstock_t)$ is smaller but close to one
in order to satisfy the minimum consumption requirement
\eqref{eq:normalizedTCPConstraintsLong}.
In detail, we replace $\vstock_t$ by $\normcropfactor \vstock_t$
whenever $\normcropfactor < 1$,
where $\normcropfactor \in \posreal$ is a \term{cropping factor}
that is determined by
\begin{equation}
  \label{eq:virtualSelling}
  \Bigl[
    1 - \tac\, \bigl(
      \sumfcn(\vstock_t) - \sumfcn(\normcropfactor\vstock_t)
    \bigr)
  \Bigr]
  \cdot \bigl(1 - \sumfcn(\normcropfactor\vstock_t)\bigr)
  = \normconsume_{\min}.
\end{equation}
Here, $\bigl(\sumfcn(\vstock_t) - \sumfcn(\normcropfactor\vstock_t)\bigr)$
is the amount of virtually sold stocks.
Hence, the term in square brackets is the fraction of wealth
that is still available after deducting the induced transaction costs.
The product of this term with
$\bigl(1 - \sumfcn(\normcropfactor\vstock_t)\bigr)$
is the fraction of wealth that can be consumed after the virtual selling,
which needs to be at least $\normconsume_{\min}$.
Solving \cref{eq:virtualSelling} for $\normcropfactor$ and
choosing the positive solution, we finally obtain
\begin{equation}
  \newcommand*{\sumX}{\sumfcn(\vstock_t)}
  \newcommand*{\cMin}{\normconsume_{\min}}
  \normcropfactor
  \ceq \frac{
    \tac\, \bigl(1 + \sumX\bigr) - 1 +
    \sqrt{
      \tac^2\, \bigl(1 - \sumX\bigr)^2
      - 2 \tac\, \bigl(2 \cMin - 1 + \sumX\bigr) + 1
    }
  }{
    2 \tac \sumX
  }.
\end{equation}

\subsection{Euler Equation Errors}
\label{sec:834eulerErrors}

\paragraph{Motivation}

Due to the curse of dimensionality,
reasonably accurate full grid reference solutions
of the transaction costs problem can only be computed
if the number $m_{\vstock}$ of stocks is small.
Mainly (but not only) in higher-dimensional settings,
a different means of assessing the
quality of sparse grid solutions is desirable.
We use Euler equation errors to measure the deviation in
the first-order optimality conditions.

\paragraph{Derivation}

In the following, we fix the state $\normstate_t \in \clint{\*0, \*1}$
for which we want to compute the Euler equation error.
We abbreviate
the
%normalized and certainty-equivalent-transformed
value function interpolant
$
  \normcetvalueintp_t
  \ceq \normcetvalueintp_t(\normstate_t)
$,
the state transition function
$
  \normstatefcn_t
  \ceq \normstatefcn_t(\normstate_t, \normpolicy_t, \stochastic_t)
$,
the wealth ratio
$
  \wealthratio_{t+1}
  \ceq \wealthratio_{t+1}(
    \normstate_t, \normpolicy_t, \stochastic_t
  )
$, and
the consumption
$
  \normconsume_t
  \ceq \normconsume_t(\normstate_t, \normpolicy_t)
$.
The Lagrangian of the optimization problem corresponding
to the Bellman equation \eqref{eq:normalizedTCPBellmanEquation}
of the normalized transaction costs problem
with respect to the problem's constraints
\eqref{eq:normalizedTCPConstraints} is given by
{%
  \setlength{\abovedisplayskip}{9pt}%
  \setlength{\belowdisplayskip}{9pt}%
  \begin{equation}
    \begin{split}
      \lagrangian_t(\normstate_t, \normpolicy_t, \*\multiplier)
      &\ceq \left(
        (\normconsume_t)^{1-\riskav} +
        \patience \expectation[t]{
          \bigl(
            \wealthratio_{t+1}\;
            \normcetvalueintp_{t+1}(\normstatefcn_t)
          \bigr)^{1-\riskav}
        }
      \right)^{1/(1-\riskav)}\\
      &\hspace*{6mm} {}
      - \multiplier_1 \normbond_t
      - \tr{\*\multiplier_2} \vnormbuy_t
      - \tr{\*\multiplier_3} \vnormsell_t
      + \tr{\*\multiplier_4}\; (\vnormsell_t - \vstock_t)
      + \multiplier_5\; (\normconsume_{\min} - \normconsume_t)
    \end{split}
  \end{equation}%
}%
with $
\*\multiplier \ceq (
  \multiplier_1,
  \*\multiplier_2,
  \*\multiplier_3,
  \*\multiplier_4,
  \multiplier_5
)$,
$\multiplier_1, \multiplier_5 \in \real$, and
$\*\multiplier_2, \*\multiplier_3, \*\multiplier_4 \in \real^{m_{\vstock}}$.
According to the first-order conditions
\term{(Karush--Kuhn--Tucker (KKT) conditions),}
the partial derivative
$
  \partialderiv{\partialdiff \normbond_t}{\lagrangian_t}(
    \normstate_t, \normpolicy_t, \*\multiplier
  )
$
with respect to $\normbond_t$
vanishes in the exact optimum
$\normpolicy_t = \optnormpolicyfcn_t \ceq \optnormpolicyfcn_t(\normstate_t)$,
i.e.,
{%
  \setlength{\abovedisplayskip}{9pt}%
  \setlength{\belowdisplayskip}{9pt}%
  \begin{equation}
    \label{eq:eulerErrorFirstOrderCondition}
    \partialderiv{\partialdiff \normbond_t}{
      \left(
        (\normconsume_t^{\opt})^{1-\riskav} +
        \patience \expectation[t]{
          \bigl(
            \wealthratio_{t+1}^{\opt}\;
            \normcetvalueintp_{t+1}(\normstatefcn[\opt]_t)
          \bigr)^{1-\riskav}
        }
      \right)^{1/(1-\riskav)}
    }
    - \multiplier_1
    - \multiplier_5
    \partialderiv{\partialdiff \normbond_t}{\normconsume_t^{\opt}}
    = 0,
  \end{equation}%
}%
where
$
  \normstatefcn[\opt]_t
  \ceq \normstatefcn_t(\normstate_t, \optnormpolicyfcn_t, \stochastic_t)
$,
$
  \wealthratio_{t+1}^{\opt}
  \ceq \wealthratio_{t+1}(
    \normstate_t, \optnormpolicyfcn_t, \stochastic_t
  )
$, and
$
  \normconsume_t^{\opt}
  \ceq \normconsume_t(\normstate_t, \optnormpolicyfcn_t)
$.
We now neglect binding constraints,
i.e., we assume that $\multiplier_1 = \multiplier_5 = 0$,
otherwise we cannot compute the error.
After calculating the derivatives,
\cref{eq:eulerErrorFirstOrderCondition} becomes
{%
  \setlength{\abovedisplayskip}{9pt}%
  \setlength{\belowdisplayskip}{9pt}%
  \begin{equation}
    \patience \bondreturn_t
    \cdot \expectationsign[t]\Bigl[
      \bigl(
        \normcetvalueintp_t
        - \tr{(\gradient{\normstate_t}{\normcetvalueintp_t})}
        \normstatefcn[\opt]_t
      \bigr) \cdot
      \bigl(
        \wealthratio_{t+1}^{\opt}\; \normcetvalueintp_t
      \bigr)^{-\riskav}
    \Bigr]
    = (\normconsume_t^{\opt})^{-\riskav}.
  \end{equation}%
}%
This equation can be used as an error measure by substituting
$\optnormpolicyfcn_t$ for the interpolated optimum
$\optnormpolicyintp_t = \optnormpolicyintp_t(\normstate_t)$.
By multiplying the resulting equation by
$
  (\normconsume_t^{\opt,\sparse})^{\riskav}
  \ceq (\normconsume_t(\normstate_t, \optnormpolicyintp_t))^{\riskav}
$, we obtain the
\term{unit-free Euler equation errors $\eulererror_t(\normstate_t)$}
with respect to $\normbond_t$:
\begin{equation}
  \eulererror_t(\normstate_t)
  \ceq \Bigl|
    1 - \Bigl(
      \patience \bondreturn_t (\normconsume_t^{\opt,\sparse})^{\riskav}
      \cdot \expectationsign[t]\Bigl[
        \bigl(
          \normcetvalueintp_t
          - \tr{(\gradient{\normstate_t}{\normcetvalueintp_t})}
          \normstatefcn[\opt,\sparse]_t
        \bigr) \cdot
        \bigl(
          \wealthratio_{t+1}^{\opt,\sparse}\;
          \normcetvalueintp_t
        \bigr)^{-\riskav}
      \Bigr]
    \Bigr)^{-1/\riskav}
  \Bigr|
\end{equation}
with
$
  \normstatefcn[\opt,\sparse]_t
  \ceq \normstatefcn_t(\normstate_t, \optnormpolicyintp_t, \stochastic_t)
$ and
$
  \wealthratio_{t+1}^{\opt,\sparse}
  \ceq \wealthratio_{t+1}(
    \normstate_t, \optnormpolicyintp_t, \stochastic_t
  )
$.

%\begin{equation}
%  \begin{split}
%    \patience \bondreturn_t
%    \cdot \expectationsign[t]\Bigl[
%      &\bigl(
%        \normcetvalueintp_t%(\normstate_t)
%        - \tr{
%          (
%            \gradient{\normstate_t}{\normcetvalueintp_t}%(\normstate_t)
%          )
%        }
%        \statefcn_t%(\normstate_t, \optnormpolicyfcn_t, \stochastic_t)
%      \bigr)\\[-1mm]
%      &{} \cdot
%      \bigl(
%        \wealthratio_{t+1}%(
%        %  \normstate_t, \optnormpolicyfcn_t, \stochastic_t
%        %)
%        \;
%        \normcetvalueintp_t%(\normstate_t)
%      \bigr)^{-\riskav}
%    \Bigr]
%    = \normconsume_t%(\normstate_t, \optnormpolicyfcn_t)
%    ^{-\riskav}
%  \end{split}
%\end{equation}
%
%\begin{equation}
%  \begin{split}
%    \eulererror_t(\normstate_t)
%    \ceq \Bigl|
%      1 - \Bigl(
%        \patience \bondreturn_t
%        \cdot \normconsume_t%(\normstate_t, \optnormpolicyfcn_t)
%        ^{\riskav}
%        \cdot \expectationsign[t]\Bigl[
%          &\bigl(
%            \normcetvalueintp_t%(\normstate_t)
%            - \tr{
%              (
%                \gradient{\normstate_t}{\normcetvalueintp_t}%(\normstate_t)
%              )
%            }
%            \statefcn_t%(\normstate_t, \optnormpolicyfcn_t, \stochastic_t)
%          \bigr)\\[-2mm]
%          &{} \cdot
%          \bigl(
%            \wealthratio_{t+1}%(
%            %  \normstate_t, \optnormpolicyfcn_t, \stochastic_t
%            %)
%            \;
%            \normcetvalueintp_t%(\normstate_t)
%          \bigr)^{-\riskav}
%        \Bigr]
%      \Bigr)^{-1/\riskav}
%    \Bigr|,
%  \end{split}
%\end{equation}

\paragraph{Weighted Euler equation errors}

However, the state space cropping as introduced above
distorts Euler equation errors:
The error $\eulererror_t(\normstate_t)$ does not vanish
even for the exact solution and even inside the feasible state space.
This is because the cropping already occurs for large stock holdings
$\sumfcn(\normstate_t)$ that are less than one,
as stocks have to be sold to maintain
minimum consumption $\normconsume_{\min}$.
Numerical experiments show that due to this issue,
the error attains large values in the region near the hyperplane
$\sumfcn(\normstate_t) = 1$.
Economically, this region is not significant
as such large stock fractions are highly unusual,
which is confirmed by Monte Carlo simulations.
We therefore use the \term{weighted Euler equation error}
\begin{equation}
  \weightedeulererror_t(\normstate_t)
  \ceq \bigl(1 - \sumfcn(\normstate_t)\bigr) \cdot
  \eulererror_t(\normstate_t)
\end{equation}
instead of $\eulererror_t$,
although other strategies exist
such as restricting the state domain where the error is computed or
weighting the error with the probability that a given state
occurs in Monte Carlo simulations.


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[letterpaper, margin=1in]{geometry}
\usepackage{amsmath}
\usepackage{tikz}
\usepackage{pgfplots}
\usepackage{amssymb }

\title{CS5780 HW4}
\author{Renhao Lu, NetID: rl839}

\begin{document}
	\maketitle

	\section{Problem: Linear Regression}
	\subsection{Compute the closed form solution for $\textbf{w}$}
	Using the formula in class:
	\[
	\textbf{w} =
		(\textbf{X} \textbf{X}^T) ^{-1}
		\textbf{X}
			\textbf{y}^T
	\]
	where $\textbf{X}=\begin{pmatrix} 1 & 1 & 1 & 1 & 1 \\ -1 & 0 & 1 & 2 & 3\end{pmatrix}$ and  $\textbf{y}=\begin{pmatrix} 4 & 3 & -4& 3 & 7 \end{pmatrix}$\\
	Hence,
	\[\begin{split}
		\textbf{w}&=(\textbf{X}\textbf{X}^T)^{-1}\textbf{X}\textbf{y}^T\\
		&=\begin{pmatrix} 5 & 5 \\ 5 & 15\end{pmatrix}^{-1}\textbf{X}\textbf{y}^T\\
		&=\begin{pmatrix} 0.3 & -0.1 \\ -0.1 & 0.1\end{pmatrix}\textbf{X}\textbf{y}^T\\
		&=\begin{pmatrix} 0.4 & 0.3 & 0.2 & 0.1 & 0 \\ -0.2 & -0.1 & 0 & 0.1 & 0.2\end{pmatrix}\textbf{y}^T\\
		&=\begin{pmatrix} 2 \\ 0.6\end{pmatrix}
	\end{split}\]

	\subsection{Calculate the training loss}
	\[
		\begin{split}
			l(\textbf{w})&=\sum (y_i-\textbf{w}^T\phi(x_i)^2)\\
			&={||\textbf{y}-\textbf{w}^textbf{X}||}_2^2\\
			&={||\begin{pmatrix} 2.6 & 1 & -6.6 & -0.2 & 3.2 \end{pmatrix}||}_2^2\\
			&=61.6
		\end{split}
	\]

	\section{Problem: Linearity of Gaussian Naive Bayes}
	\subsection{Show that the decision rule}
	According to the law of total probability:
	\[
		\begin{split}
			p(\textbf{x}) &= p(y=1)p(\textbf{x}|y=1)+p(y=0)p(\textbf{x}|y=0)\\
			&=p(y=1)\prod_{\alpha=1}^d {p([\textbf{x}]_{\alpha}|y=1)} +p(y=0)\prod_{\alpha=1}^d {p([\textbf{x}]_{\alpha}|y=0)}
		\end{split}
	\]
	Hence,
	\[
		\begin{split}
			p(y=1|\textbf{x}) &=\frac{
				p(y=1)
				\prod_{\alpha=1}^d {
					p([\textbf{x}]_{\alpha}|y=1)
				}
			}{
				p(\textbf{x})
			} \\
			&=\frac{
				p(y=1)\prod_{\alpha=1}^d {
					p([\textbf{x}]_{\alpha}|y=1)
				}
			}{
				p(y=1)\prod_{\alpha=1}^d {
					p([\textbf{x}]_{\alpha}|y=1)
				}
				+p(y=0)\prod_{\alpha=1}^d {
					p([\textbf{x}]_{\alpha}|y=0)
				}
			}
		\end{split}
	\]

	\subsection{Show how to rewrite}
	From the formula we got from 1:
	\[
		\begin{split}
			p(y=1|\textbf{x})
			&=\frac{
				p(y=1)\prod_{\alpha=1}^d {
					p([\textbf{x}]_{\alpha}|y=1)
				}
			}{
				p(y=1)\prod_{\alpha=1}^d {
					p([\textbf{x}]_{\alpha}|y=1)
				}
				+p(y=0)\prod_{\alpha=1}^d {
					p([\textbf{x}]_{\alpha}|y=0)
				}
			}\\
			&=\frac{1}{
				1 + \frac{
					p(y=0)\prod_{\alpha=1}^d {
						p([\textbf{x}]_{\alpha}|y=0)
					}
				}{
					p(y=1)\prod_{\alpha=1}^d {
						p([\textbf{x}]_{\alpha}|y=1)
					}
				}
			}\\
			&=\frac{1}{
				1 + exp(
					-log(
						\frac{
							p(y=1)\prod_{\alpha=1}^d {
								p([\textbf{x}]_{\alpha}|y=1)
							}
						}{
							p(y=0)\prod_{\alpha=1}^d {
								p([\textbf{x}]_{\alpha}|y=0)
							}
						}
					)
				)
			}\\
		\end{split}
	\]
	\subsection{Show that Naive Bayes is a linear model}
	Because
	\[
		p([\textbf{x}]_{\alpha}|y=1)=
		\frac{1}{\sqrt{2\pi[\sigma_{\alpha}]}}
		\exp\left({
			\frac{
			-([\textbf{x}]_{\alpha}-[\mu_1]_{\alpha})^2
			}{
				2[\sigma]_{\alpha}
			}
		}\right)
	\]
	\[
		p([\textbf{x}]_{\alpha}|y=0)=
		\frac{1}{\sqrt{2\pi[\sigma_{\alpha}]}}
		\exp\left({
			\frac{
			-([\textbf{x}]_{\alpha}-[\mu_0]_{\alpha})^2
			}{
				2[\sigma]_{\alpha}
			}
		}\right)
	\]
	Hence,
	\[
		\begin{split}
			p(y=1|\textbf{x})&=\frac{1}{1+\frac{p(y=0)}{p(y=1)}\exp{\left(\sum_{\alpha=1}^d{-\log{\frac{p([\textbf{x}]_\alpha|y=1)}{p([\textbf{x}]_\alpha|y=0)}}}\right)}}\\
			&=\frac{1}{1+\frac{p(y=0)}{p(y=1)}\exp{\left(\sum_{\alpha=1}^d{\frac{([\textbf{x}]_{\alpha}-[\mu_0]_{\alpha})^2-([\textbf{x}]_{\alpha}-[\mu_1]_{\alpha})^2}{2[\sigma]_{\alpha}}}\right)}}\\
			&=\frac{1}{1+\frac{p(y=0)}{p(y=1)}\exp{\left(\sum_{\alpha=1}^d{\frac{2([\mu_1]_{\alpha}- [\mu_0]_{\alpha})[\textbf{x}]_{\alpha}+([\mu_0]_{\alpha}^2-[\mu_1]_{\alpha}^2)}{2[\sigma]_{\alpha}}}\right)}}
		\end{split}
	\]
	We assume:
	\[\textbf{w}=[w_1, w_2,...,w_b], w_\alpha=\frac{[\mu_1]_{\alpha}- [\mu_0]_{\alpha}}{[\sigma]_{\alpha}}\]
	\[b=\sum{b_\alpha}+b_0, b_0=\log{\frac{p(y=0)}{p(y=1)}}, b_\alpha=\frac{[\mu_0]_{\alpha}^2-[\mu_1]_{\alpha}^2}{2[\sigma]_{\alpha}}\]
	Hence,
	\[\begin{split}
		p(y=1|\textbf{x})&=\frac{1}{1+\exp{\left(b_0+\sum_{\alpha=1}^d {(w_\alpha[\textbf{x}]_\alpha+b_\alpha)}\right)}}\\
		&=\frac{1}{1+\exp{\left(\textbf{w}^T\textbf{x}+b\right)}}
	\end{split}\]
	From the formula above, whether $p(y=1|\textbf{x}) > p(y=0|\textbf{x})$ is decided by whether $\textbf{w}^T\textbf{x}+b>0$, hence this Naive Bayes is a linear model.

	\section{Problem: Gradient for Logistic Regression}
	\subsection{Show the sigmoid function property}
	\[\begin{split}
		\sigma(s)+\sigma(-s)&=\frac{1}{1+e^{-s}}+\frac{1}{1+e^{s}}\\
		&=\frac{e^s}{1+e^{s}}+\frac{1}{1+e^{s}}=1
	\end{split}\]
	Hence, $\sigma(-s)=1-\sigma(s)$

	\subsection{Compute the gradient of the log likelihood function}
	\subsubsection{first derivative}
	\[\sigma'(s)=-\frac{1}{(1+e^{-s})^2}(-e^{-s})=\frac{1}{1+e^{-s}}\frac{1}{1+e^s}=\sigma(s)\sigma(-s)=\sigma(s)(1-\sigma(s))\]

	\subsubsection{Show the gradient of the log likelihood function}
	\[
	\begin{split}
		\nabla_w\log(\textbf{y}|X, \textbf{w})&=\nabla_w\sum_{i=1}^n{\log\sigma(y_i(\textbf{w}^T\textbf{x}_i))}\\
		&=\sum_{i=1}^n {[\frac{\partial}{\partial w_1}\log\sigma(y_i(\textbf{w}^T\textbf{x}_i))...\frac{\partial}{\partial w_d}\log\sigma(y_i(\textbf{w}^T\textbf{x}_i))]}
	\end{split}
	\]
	Considering that,
	\[
		\frac{\partial}{\partial w_\alpha}\log\sigma(y_i(\textbf{w}^T\textbf{x}_i)) = \frac{1}{\sigma(y_i(\textbf{w}^T\textbf{x}_i))}\sigma'(y_i(\textbf{w}^T\textbf{x}_i))y_i\frac{\partial}{\partial w_\alpha}\textbf{w}^T\textbf{x}_i=(1-\sigma(y_i(\textbf{w}^T\textbf{x}_i)))y_i\textbf{x}_{i\alpha}
	\]
	Hence,
	\[\nabla_w\log(\textbf{y}|X, \textbf{w})=\sum_{i=1}^n {(1-\sigma(y_i(\textbf{w}^T\textbf{x}_i)))y_i\textbf{x}_{i}}\]

	\subsubsection{the weights are in the span of the feature vectors}
	Assume in $k^{th}$ iteration, the step size is $t_k$\\
	Proof by induction:\par
	Base case: in the $0^{th}$ iteration, $\textbf{w}_0=\textbf{0}=\sum_{i=1}^n0\times\textbf{x}_i$\par
	Induction assumption: assume in $k-1^{th}$ iteration, $\textbf{w}_{k-1}=\sum_{i=1}^nc_{(k-1)i}\textbf{x}_i$, then $\textbf{w}_{k}=\textbf{w}_{k-1}+t_k\nabla_w\log(\textbf{y}|X, \textbf{w})=\textbf{w}_{k-1}+t_k\sum_{i=1}^n {(1-\sigma(y_i(\textbf{w}^T\textbf{x}_i)))y_i\textbf{x}_{i}}$. We note $c_{ki}=c_{(k-1)i}+t_k(1-\sigma(y_i(\textbf{w}^T\textbf{x}_i)))y_i$, then $\textbf{w}_{k}=\sum_{i=1}^nc_{ki}\textbf{x}_i$\par
	Hence, at every iteration of gradient descent, the weights are in the span of the feature vectors.\par

	\section{Problem: Optimization with Gradient Descent}
	\subsection{Perform 3 steps of gradient descent}
	$f'(w)=6(w-3)$, hence $w_k=w_{k-1}-\frac 1 \alpha f'(w_{k-1})=w_{k-1}-\frac 1 2 (w_{k-1}-3)$.
	\[\begin{split}
		&w_0=5, f(w_0)=3\times2^2=12\\
		&w_1=5-\frac 1 2\times 2=4, f(w_1)=3\times1^2=3\\
		&w_2=4-\frac 1 2\times 1=3.5, f(w_2)=3\times0.5^2=0.75\\
		&w_3=3.5-\frac 1 2\times 0.5=3.25, f(w_3)=3\times0.25^2=0.1875
	\end{split}\]
	\begin{center}
		\begin{tikzpicture}
			\begin{axis}[
				xmin = 0, xmax = 6,
				ymin = -2, ymax = 24,
				xtick distance = 1,
				ytick distance = 3,
				grid = both,
				minor tick num = 1,
				major grid style = {lightgray},
				minor grid style = {lightgray!25},
				width =  .5\textwidth,
				height = .5\textwidth,
				xlabel = {$x$},
				ylabel = {$y$},
				legend cell align = {left},
			]
				\addplot[
					domain = 0:6,
					samples = 200,
					smooth,
					thick,
					blue,
				] {3*(x-3)^2};
				\addplot [only marks, mark=o,mark size=2.9pt] table {
					5 12
				};
				\addplot [only marks, mark=halfcircle*,mark size=2.9pt] table {
					4 3
				};
				\addplot [only marks, mark=square,mark size=2.9pt] table {
					3.5 0.75
				};
				\addplot [only marks, mark=x,mark size=2.9pt] table {
					3.25 0.1825
				};
				\legend{$f(w) = 3(w-3)^2$,$w_0$,$w_1$,$w_2$,$w_3$}
			\end{axis}
		\end{tikzpicture}
		\end{center}

	\subsection{Gradient descent can sometimes fail to converge}
	$f'(w)=20(w-11)^3$, hence $w_k=w_{k-1}-\frac 1 \alpha f'(w_{k-1})=w_{k-1}-\frac 1 2 (w_{k-1}-11)^3$.
	\[\begin{split}
		&w_0=13, f(w_0)=5\times2^4=80\\
		&w_1=13-\frac 1 2\times 2^3=9, f(w_1)=5\times2^4=80\\
		&w_2=9-\frac 1 2\times (-2)^3=13, f(w_2)=5\times2^4=80\\
		&w_3=13-\frac 1 2\times 2^3=9, f(w_3)=5\times2^4=80\\
		&...
	\end{split}\]
	The gradient descent is bouncing between $9$ and $13$ and will never converge.
	\begin{center}
		\begin{tikzpicture}
			\begin{axis}[
				xmin = 8, xmax = 14,
				ymin = -1, ymax = 116,
				xtick distance = 1,
				ytick distance = 9,
				grid = both,
				minor tick num = 1,
				major grid style = {lightgray},
				minor grid style = {lightgray!25},
				width =  .5\textwidth,
				height = .5\textwidth,
				xlabel = {$x$},
				ylabel = {$y$},
				legend cell align = {left}
			]
				\addplot[
					domain = 8:14,
					samples = 200,
					smooth,
					thick,
					blue,
				] {5*(x-11)^4};
				\addplot [only marks, mark=o,mark size=2.9pt] table {
					13 80
				};
				\addplot [only marks, mark=halfcircle*,mark size=2.9pt] table {
					9 80
				};
				\legend{$f(w) = 5(w - 11)^4$, ${w_0,w_2,...}$,${w_1,w_3,...}$}
			\end{axis}
		\end{tikzpicture}
		\end{center}

	\section{Problem: Derivation for Hard-margin Linear SVMs}
	\subsection{Prove the optimal solution of formulation A is a feasible solution for formulation B}
	Note the optimal solution of frmulation A is $\textbf{w}_A$. Then $\forall_iy_i(\textbf{w}_A^T\textbf{x}_i+b)\geq 0$ and $min_i|\textbf{w}_A^T\textbf{x}_i+b|=1$. Because $y_i=1or-1$, $min_iy_i(\textbf{w}_A^T\textbf{x}_i+b)=1$. Hence, $\forall_iy_i(\textbf{w}_A^T\textbf{x}_i+b)\geq 1$. $\textbf{w}_A$ is a feasible solution for B.$\blacksquare$
	\subsection{Prove the optimal solution of formulation B is a feasible solution for formulation A}
	Note the optimal solution of frmulation B is $\textbf{w}_B$, then $\forall_i,y_i(\textbf{w}_B^T\textbf{x}_i+b)\geq1$. Then obviously $\forall_iy_i(\textbf{w}_B^T\textbf{x}_i+b)\geq0$. Now we need to consider $min_i|\textbf{w}_B^T\textbf{x}_i+b|$\par
	We can prove $min_i|\textbf{w}_B^T\textbf{x}_i+b|=1$ by contradiction. We assume $min_i|\textbf{w}_B^T\textbf{x}_i+b|=p>1$, then we use a new weight vector $\textbf{w}_B'=\frac{\textbf{w}_B}{p}$. Because $p>1$, $\textbf{w}_B'^T\textbf{w}_B'=\frac{\textbf{w}_B^T\textbf{w}_B}{p^2}<\textbf{w}_B^T\textbf{w}_B$ And $\forall_i,y_i(\textbf{w}_B'^T\textbf{x}_i+b)\geq\frac p p=1$. Then $\textbf{w}_B$ cannot be optimal because $\textbf{w}_B'$ is better. $\blacksquare$

	\subsection{Prove that for the optimal solution in formulation A is the optimal solution for formulation B and vice versa}
	We note $\textbf{w}_A$ as the optimal solution in A, and $\textbf{w}_B$ as the optimal solution in B. From the previous questions, we know that $\textbf{w}_A$ is a feasible solution for B, so $\textbf{w}_A\leq\textbf{w}_B$. Similiarly, $\textbf{w}_B$ is also a feasible solution for A, so $\textbf{w}_B\leq\textbf{w}_A$. Hence $\textbf{w}_B=\textbf{w}_A\blacksquare$
\end{document}


\documentclass[main.tex]{subfiles}
\begin{document}
	\section{Lattices and crystal structure} \label{sec:lattice}
	\subsection{Theory} \label{sec:lattice_theory}
	There are three parts to the description of a crystal structure. First is the lattice. This is the mathematical ``framework'' upon which the physical part of the crystal lies. It can be defined in a number of ways, however the one we will use here is standard and can be found in for example \cite{simon} or \cite{kittel}.

	\textbf{A lattice is defined as the infinite set of points produced by a linear combination of independent \emph{primitive lattice vectors}, with integer coefficients.}

	Throughout this thesis, the primitive lattice vectors are labelled $ \V{a}_i $, and the coefficients $ n_i $, so for a $ d $-dimensional lattice, the lattice points $ \V{R} $ are given by
	\begin{equation}\label{eq:lattice_points}
	R = \sum_{i = 1}^{d} n_i \V{a}_i.
	\end{equation}
	For this thesis we will mainly focus on the cases of $ d = 3 $ (visualization of lattices, families of lattice planes and scattering) and $ d = 2 $ (visualization of the band structure).

	A thing to note is that the choice of primitive lattice vectors is not unique. A new set of primitive lattice vectors can be created by taking a linear combination of the original primitive lattice vectors, with integer coefficients. If the original set is ordered in a matrix $ A = \begin{pmatrix*} \V{a}_1 & \V{a}_2 & \cdots & \V{a}_n \end{pmatrix*}$, and the new set in a matrix $ B = \begin{pmatrix*} \V{b}_1 & \V{b}_2 & \cdots & \V{b}_n \end{pmatrix*} $, then $ B = MA $, where $ M $ is the matrix containing the coefficients. This matrix must have integer entries, and its inverse likewise. The integer entries of the direct coefficient matrix $ M $ makes sure that any lattice point generated with the new set of primitive lattice vectors will also have integer coefficients when expressed in the old set of primitive lattice vectors. The integer entries of the inverse matrix $ M\inverse $ then makes sure that this process will also happen in reverse.

	The second part is the unit cell. This is the building block of the lattice. It is a region of space which, when stacked will completely tile the space. Like with the choice of primitive lattice vectors, the choice of unit cell is not unique. In particular we distinguish between two type of unit cells: the smallest possible unit cell and everything else. The smallest possible unit cell is called a \textit{primitive} unit cell, and must contain only one lattice point (it cannot contain zero lattice points, as then it would not recreate the lattice when tiled). Any unit cell containing more than one lattice point is called a \textit{conventional unit cell}. Usually a conventional unit cell is chosen for ease of calculation (as we will see in the scattering simulation), where the primitive lattice vectors constitute an orthogonal set.

	The third part of the crystal structure is the basis. This is a description of the physical objects that make up the structure, and their positions in relation to the lattice. In our case the objects are atoms. The basis is specified as a list of vectors that are to be added to the lattice points, specifying the position of the atoms in the crystal.

	The user can create any type of crystal they want by specifying any set of primitive lattice vector and supplying any desired basis. However a small selection of crystals will be available as presets. These include the 14 Bravais lattices with a 1 atom basis [at $ (0,0,0) $], named in table \ref{tab:bravais}. Each of these will specify the primitive lattice vectors for a corresponding primitive unit cell. Furthermore five other crystal presets will be available, named in table \ref{tab:presets}. Specifications of all of these presets are available in the appendix \ref{app:lattice}.
	\begin{table}[H]
		\centering
		\begin{tabular}{|l|l|}
			\hline
			Simple cubic & base centred cubic (bcc) \\
			\hline
			face centred cubic (fcc) & tetragonal \\
			\hline
			body centred tetragonal & orthorhombic \\
			\hline
			body centred orthorhombic & face centred orthorhombic \\
			\hline
			base centred orthorhombic & simple monoclinic \\
			\hline
			base centred monoclinic & hexaonal \\
			\hline
			triclinic & rhombohedral \\
			\hline
		\end{tabular}
		\caption{The 14 Bravais lattices}
		\label{tab:bravais}
	\end{table}
	\begin{table}[H]
		\centering
		\begin{tabular}{|l|l|}
			\hline
			fcc, conventional & bcc, conventional \\
			\hline
			zincblende & wurtzite \\
			\hline
			diamond (zincblende with 1 atom) & \\
			\hline
		\end{tabular}
		\caption{Other available crystal presets}
		\label{tab:presets}
	\end{table}

	\subsection{Implementation}
	Mathematically speaking, a lattice is infinite. A physical crystal, of course, is not. However, even though a real crystal is finite, plotting all of the atoms would be infeasible, both due to the number of atoms, and the fact that each atom would be way too small. So for the purposes of this program, only a couple of unit cells will be plotted. A good amount seems to be 8 unit cells: 2 in each of the directions specified by the primitive lattice vectors. This keeps the size of the plot relatively small, whilst still showing the important parts of the crystal structure. As such, in the following we assume that $ n_i \in \{0, 1, 2\} $ for all $ n_i $.

	In creating a program that plots these structures, the thought should always be on how the end product looks. Mainly we want the plot to be as clear and instructive as possible. This constitutes plotting only full unit cells: no unit cell can have missing atoms. There is a caveat to this however. Say we define the unit cell to have atoms on the boundary, then these atoms will be in multiple unit cells. One could say no matter how many unit cells we plot, we would always get some amount of unfinished unit cells. As such we will plot the 8 unit cells along with any atoms on the border of these.
	\begin{wrapfigure}{r}{2in}
		\begin{center}
			\includegraphics[width=\linewidth]{figures/lattice_unfinished_1.pdf}
		\end{center}
		\caption{One conventional unit cell for a bcc lattice. If we naively plot all atoms associated with a lattice point, we will end up with atoms outside the unit cells we want to plot.}
		\label{fig:lattice_unfinished_1}
	\end{wrapfigure}

	Even with the above considerations we need to remember that the screen is still just a two dimensional projection of the actual three dimensional phenomena. As such without some form of depth perception, the crystal will just look like a weird two dimensional pattern. This is fixed by including grid lines. They, along with the ability to interactively rotate the crystal, give the required depth perception to allow the user to comprehend the crystal as a three dimensional structure, and not as a two dimensional sheet of dots.

	For the crystals that can be expressed as a lattice with orthogonal primitive lattice vectors and a basis (cubic, tetragonal and orthorhombic), we usually want to plot orthogonal grid lines, whilst for other crystals plotting grid lines along the lattice vectors will be more useful.

	In the case of plotting primitive unit cells, the ones furthest away from the origin (say with $ n_1 = n_2 = n_3 = 2 $) may not be filled. Say the basis consists of only one atom. Then this atom will just be placed on the lattice points, and there is no problem, as this is the edge of the unit cell. However, if the basis consists of two atoms (or more), where one is placed at $ (0,0,0) $ and the other has some positive coordinates, then this second atom will be in a unit cell which is not supposed to be plotted (it would necessitate plotting the lattice points corresponding to $ n_i = 3 $). An example of this is figure \ref{fig:lattice_unfinished_1}, which shows one unit cell of a simple cubic lattice, with a two atom basis (corresponding to a conventional bcc unit cell). A way to correct for this is to write each plotted atoms position as a linear combination of the primitive lattice vectors with coefficients $ n'_i $ (where the coefficients here are real numbers, as they do not necessarily align with the lattice points). If the coefficients are $ 0 \leq n'_i \leq 2 $ for all $ n'_i $, then we plot the point.

	For plotting conventional unit cells when the user inputs primitive lattice vectors (for cubic, tetragonal and orthorhombic lattices), we need something similar, otherwise a situation like in figure \ref{fig:lattice_unfinished_2} may occur. To fix this we calculate the side lengths of the cuboid plot box that just contains the parallelepiped arising from plotting the lattice points with the specified coefficients ($ n_i \in \{0, 1, 2\}$), and fill this plot box completely with atoms.

	To calculate this, the program creates the 8 possible vectors arising from linear combinations of the primitive lattice vectors, with coefficients from the minimum and maximum coefficients and taking the limits of the plot box as the minimum and maximum values for these 8 vectors. For the specified lattice points, these 8 vectors are:
	\begin{equation}
		\begin{array}{llll}
			\V{v}_1 = \V{0}, & \V{v}_2 = 2\V{a}_1, & \V{v}_3 = 2\V{a}_2, & \V{v}_4 = 2\V{a}_3, \\
			\V{v}_5 = 2(\V{a}_1+\V{a}_2), & \V{v}_6 = 2(\V{a}_1+\V{a}_3), & \V{v}_7 = 2(\V{a}_2+\V{a}_3), &\V{v}_8 = 2(\V{a}_1+\V{a}_2+ \V{a}_3).
		\end{array}
	\end{equation}
	These are the vertices of the aforementioned parallelepiped. The side lengths of the plot box are then taken as the maximum coordinates for these vectors, minus the minimum values (the side length in $ x $ might be $ v_{8,x}-v_{0,x} $ for example). Then we plot atoms for a larger range of coefficients to completely fill out this plot box. If some atoms fall outside of the box we hide them.

	\begin{wrapfigure}{r}{2in}
		\begin{center}
			\includegraphics[width=\linewidth]{figures/lattice_unfinished_2.pdf}
		\end{center}
		\caption{An attempt to plot eight conventional unit cells for an FCC lattice, by using primitive lattice vectors with coefficients in the set $ \{0, 1, 2\} $. This does not fill out all eight unit cells, so we need to plot more lattice points than this. In practise we need $ \{-2, -1, 0, 1, 2, 3, 4\} $}
		\label{fig:lattice_unfinished_2}
	\end{wrapfigure}

	\subsection{Step-by-step}
	When the program is run it will either load the chosen crystal preset or plot the user's manually input crystal. If a crystal preset is chosen, the program loads it in such a way as to make the resulting plot as informative as possible (eg. place lattice vectors along cardinal axes for an orthogonal lattice, to make plotting grid lines easier). If the user manually specifies the lattice and basis, the program attempts to classify the lattice according to the specifications in the appendix \ref{app:lattice} (except for zincblende, diamond and wurtzite). Next the program checks whether or not the crystal should be rotated to make plotting prettier.

	In general the rotation algorithm tries to align one lattice vector with the $ x $-axis. $ \V{a}_1 $ is preferred, but is only chosen to lie along the $ x $-axis if it forms an orthogonal pair with at least one other primitive lattice vector. The second primitive lattice vector of the pair ($\V{a}_2$ being preferred) is then aligned along the $ y $-axis. If the three primitive lattice vectors form an orthogonal set, then the last vector of the set will now be aligned along the $ z $-axis.

	The actual rotation is done by rotating the whole crystal (each primitive lattice vector and all vectors in the basis) around the cross product between the initial vector and the destination vector. Rotating the crystal such that $ \V{a}_1 $ lies along the $ x $-axis is done by rotating along $ \V{a}_1 \times \U{x} $, with an angle of $ \sin \theta = |\V{a}_1 \times \U{x}| / |\V{a}_1| $. However, this might rotate the crystal the wrong way, depending on the orientation between the two vectors. Because of this the program checks whether or not the rotated initial vector and the destination vector are parallel (that is, if the rotated $ \V{a}_1 $ is parallel to $ \U{x} $). If this is not the case, the whole crystal is rotated about the same vector, with an angle $ -2\theta $.

	Five of the Bravais lattices have specialised rotation functions: hexagonal, base centred monoclinic and the three face centred lattices.

	For the hexagonal lattice, the program detects which primitive lattice vectors constitute the triangular lattice and orients them such that one is along the $ x $-axis and the other is in the $ xy $-plane (easily done by rotating the third primitive lattice vector such that it is parallel with the $ z $-axis). The same approach is used for the base centred monoclinic, but here the program always aligns $ \V{a}_1 $ along the $ x $-axis, and $ \V{a}_2 $ in the $ xy $-plane. Here however, the easy option of rotating $ \V{a}_3 $ is not available. Instead the program uses the fact that the vector rejection of $ \V{a}_2 $ with $ \V{a}_1 $ is orthogonal to $ \V{a}_1 $ (the vector rejection of $ \V{a}_2 $ with $ \V{a}_1 $ being $ \V{a}_2 $ minus the projection of $ \V{a}_2 $ along $ \V{a}_1 $):

	\begin{equation}
		\V{a}_{2,rej} = \V{a}_2 - \V{a}_{2, proj} = \V{a}_2 - \frac{\V{a}_2 \D \V{a}_1}{|\V{a}_1|} \V{a}_1.
	\end{equation}
	This vector rejection is then in the $ yz $-plane, and the crystal can be rotated around $ \V{a}_1 $ with the angle the rejection makes with $ \U{y} $: $ \cos \theta = \V{a}_{2,rej} \D \U{y} / |\V{a}_{2,rej}|$.

	For the face centred lattices, the ideal, rotated lattice is created from the magnitudes of the primitive lattice vectors: if $ |\V{a}_1| = a, |\V{a}_2| = b, |\V{a}_3| = c $, then $ \V{a}'_1 = (a/2, b/2, 0), \V{a}'_2 = (a/2, 0, c/2), \V{a}'_3 = (0, b/2, c/2) $. First the crystal is rotated such that $ \V{a}_1 $ aligns with $ \V{a}'_1 $, by using their cross product. Then the crystal is rotated such that the now rotated $ \V{a}_2 $ aligns with $ \V{a}'_2 $, via the vector rejection of $ \V{a}_2 $ with $ \V{a}'_2 $. A thing to note is that these 5 rotation functions necessitates primitive lattice vectors of the form specified in the appendix. If the user specifies a lattice unlike those in the appendix, the program will not be able to identify it, and as such may rotate it ``incorrectly''. This could lead to the program not being able to impose grid lines along the cardinal axes for an fcc lattice, for example.

	With the lattice and basis rotated, the program finds the limits of the plot box as written above, after which the crystal is generated. This is done by looping over the three ranges specified by the minimum and maximum coefficients, creating each lattice point $ \V{R} $ by Eq. \eqref{eq:lattice_points}. For each lattice point the program calculates the positions of the $ n $ atoms in the basis, by adding one of the $ n $ vectors in the basis to the lattice point, i.e. $ \V{r}_{atom, i} = \V{R} + \V{r}_{basis, i} $. Lists of the colours and sizes associated with each atom are also created at this point. After creating all the atoms, the program deletes any that may lie outside the limits of the plot box.

	The only thing missing now is to create the grid lines and plot everything. The program has two ways of creating grid lines: along the primitive lattice vectors and along the cardinal axes.

	Creating grid lines along the primitive lattice vectors works by taking each lattice point $ \V{R}_n $, and finding the lattice point $ \V{R}_m $ the furthest away from it, in the (positive) direction of these lattice vectors, such that $ \V{R}_m = \V{R}_n + \alpha \V{a}_i $, where $ \alpha > 0 $, for all $ i \in \{1,2,3\}$, and creating lines between $ \V{R}_n $ and $ \V{R}_m $. This does create duplicate grid lines, but these do not show on the final plot, since they are all plotted with the same width and colour.

	Creating grid lines along the cardinal axes works by finding the minimum spacing between lattice points on these axes (called $ a_x, a_y$ and $ a_z $), and using these as the spacing between grid lines. The program then finds the maximum and minimum coordinates of lattice points along the cardinal axes (called $x_{min}, x_{max}$, etc.). This is used to create ranges corresponding to each lattice points on the cardinal axes: The range for the $ x $-axis starting at $ x_{min} $  and ending at $ x_{max} $ with steps of $ a_x $.

	These ranges then specify a grid of lattice points on the $ xy$, $ xz $ and $ yz $-planes. The program then creates lines orthogonal to these planes, stretching from $ z_{min} $ to $ z_{max} $ for the points in the $ xy $-plane, and similarly for the other two planes.

	With everything calculated a blank figure is created with limits as calculated above. The atoms are plotted with colours and sizes specified by the user. The grid lines are plotted with a uniform size and colour and lastly the primitive lattice vectors are plotted with corresponding labels.

	\subsection{Examples}
	As an example, we plot a simple cubic lattice with a two atom basis (corresponding to the basis of a bcc lattice with conventional unit cells), where the atoms on the lattice points are grey and the body centred atoms are blue (figure \ref{fig:lattice_demo_1}), we write the following:
\begin{lstlisting}
Lattice(lattice_name="conventional bcc",
		colors=["xkcd:cement", "b"])
\end{lstlisting}
	Or we could plot a hexagonal lattice with a one atom basis (figure \ref{fig:lattice_demo_2}):
\begin{lstlisting}
Lattice(lattice_name="hexagonal")
\end{lstlisting}
	When plotted using the program, these plots are interactive and can be rotated.

	\begin{figure}[h]
		\centering
		\begin{minipage}{.4\textwidth}
			\centering
			\includegraphics[width=2in]{figures/lattice_demo_1.pdf}
			\captionof{figure}{A simple cubic lattice with a two atom basis: One atom on the lattice points and another in the middle of each unit cell.}
			\label{fig:lattice_demo_1}
		\end{minipage}%
		\hfil
		\begin{minipage}{.4\textwidth}
			\centering
			\includegraphics[width=2in]{figures/lattice_demo_2.pdf}
			\captionof{figure}{A hexagonal lattice with a one atom basis. The lattice consists of a series of triangular lattices stacked on top of each other.}
			\label{fig:lattice_demo_2}
		\end{minipage}
	\end{figure}

\end{document}


\documentclass[a4paper,12pt]{article}
% decent example of doing mathematics and proofs in LaTeX.
% An Incredible degree of information can be found at
% http://en.wikibooks.org/wiki/LaTeX/Mathematics

% Use wide margins, but not quite so wide as fullpage.sty
\marginparwidth 0.1in
\oddsidemargin 0.05in
\evensidemargin 0.05in
\marginparsep 0.05in
\topmargin 0.05in
\textwidth 6in \textheight 8 in
% That's about enough definitions

\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}

\makeatletter
\renewenvironment{proof}[1][\proofname] {\par\pushQED{\qed}\normalfont\topsep6\p@\@plus6\p@\relax\trivlist\item[\hskip\labelsep\bfseries#1\@addpunct{.}]\ignorespaces}{\popQED\endtrivlist\@endpefalse}
\makeatother

\newtheoremstyle{break}
  {\topsep}{\topsep}%
  {\itshape}{}%
  {\bfseries}{}%
  {\newline}{}%
\theoremstyle{break}
\newtheorem{theorem}{Theorem}[section]
\newtheorem{corollary}{Corollary}[theorem]
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{definition}[theorem]{Definition}

\begin{document}

\title{Linear Algebra Theorems}
\author{Hyeonsu Lyu, Department of Electrical Engineering, UNIST}
\date{June 29, 2018}
\maketitle

\section{Linear Transformation}
\begin{definition} [Linear Transformation from $\mathbb{R}^n$ to $\mathbb{R}^m$]
    A \textit{linear transformation} $T : \mathbb{R}^n\to\mathbb{R}^m$ is a mapping such that for all scarars $\textit{a}$ and all $\overrightarrow{v}, \overrightarrow{w} \in \mathbb{R}^n,$
\begin{align}
    T(\overrightarrow{v}+\overrightarrow{w}=T(\overrightarrow{v}+\overrightarrow{w})\ and\ T(a\overrightarrow{v})=aT(\overrightarrow{v})
\end{align}
\end{definition}

\begin{theorem} [Matrices and Linear transformations]\label{thm:mlt}
    1. Any $m\times n$ matrix A defines a linear transformation $T:\mathbb{R}^n\to\mathbb{R}^m$ by matrix multiplication:
    \begin{align}
        T(\overrightarrow{v})=A\overrightarrow{v}.
    \end{align}
        2. Every linear transformation $T:\mathbb{R}^n\to\mathbb{R}^m$ is given by multiplication by the $m\times n$ matrix $[T]$:
    \begin{align}
        T(\overrightarrow{v})=[T]\overrightarrow{v}
    \end{align}
        ,where the \textit{i}th column of $[T]$ is $T(\overrightarrow{e_i}).$
\end{theorem}

\begin{proof}[Proof of theorem \ref{thm:mlt}] \hfill \\
1. By definition of \textit{linear transformation}, mapping (2) is \textit{linear}.\\
2. For any vector $\overrightarrow{v}\in \mathbb{R}^n$, $v$ can be represented as basis of $\mathbb{R}^n$, $\{\overrightarrow{e_1},\overrightarrow{e_2}, \dots, \overrightarrow{e_n}\}$.
\begin{align}
    \overrightarrow{v} = v_1\overrightarrow{e_1}+v_2\overrightarrow{e_2}+\dots+v_n\overrightarrow{e_n}
\end{align}
Since the linearity of mapping T,
\begin{align*}
    T(\overrightarrow{v})&=v_1T(\overrightarrow{e_1})+v_2T(\overrightarrow{e_2})+\dots+v_nT(\overrightarrow{e_n})\\
    &=[T(\overrightarrow{e_1})|T(\overrightarrow{e_2})|\dots|T(\overrightarrow{e_n})]\begin{bmatrix}v_1 \\ v_2 \\ \vdots \\ v_n\end{bmatrix}\\
    &= [T]\overrightarrow{v}
\end{align*}
\end{proof}

\begin{definition} [Linear Transformation is onto]
    Let $A$ be $m\times n$ matrix. The followings are logically equivalent: \\
    \indent1. The function $Ax=b$ is onto $\mathbb{R}^m$.\\
    \indent2. For each $b\in \mathbb{R^m}$, the equation Ax=b has a solution.\\
    \indent3. $b\in \mathbb{R^m}$ is a linear combination of the columns of $A$.\\
    \indent4. The columns of $A$ span $\mathbb{R^m}$.\\
    \indent5. $A$ has a pivot position in every row.\\
    \indent6. $A$ has rank m\\
\end{definition}

\begin{definition} [Linear Transformation is 1-1]
    Let $A$ be an $m\times n$ matrix. The followings are logically equivalent:\\
    \indent1. The function $Ax=b$ is $1-1$.\\
    \indent2. The equation $Ax=b$ has at most on solution for every b.\\
    \indent3. The equation $Ax=0$ has only the trivial solution.\\
    \indent4. The columns of A are linearly independent.\\
    \indent5. A has a pivot posiition in every column.\\
    \indent6. A has rank n.\\
\end{definition}

Related theroems : Inverse function theorem, Implicit function theorem, Tangent space on implicit function.

\section{Another Section}

\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS671: Machine Learning
% Copyright 2015 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/beacon
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 2}

Consider the set of points in $\mathbb{R}^2$ shown in Figure \ref{fig1}.
Positive examples are shown as black circles, while negative examples are shown as gray circles.
\begin{enumerate}[label=(\alph*)]
\item Find classifiers for the sample using at least four kernels available in the package of $R$ software.
\item Graph the total time required by these kernels as a function of the error rate obtained on the training set.
\end{enumerate}

\begin{figure}[H]\centering
\tikzstyle{black}=[circle, draw, fill=black]
\tikzstyle{white}=[circle, draw, fill=gray!20]
\begin{tikzpicture}
  \draw [thick, color=gray, step=0.5cm, dashed] (0,0) grid (5,5);
  \draw [<->,thick] (0,5.5) node (yaxis) [left] {$y$} |- (5.5,0) node (xaxis) [below] {$x$};
  \foreach \Point in { (4.5,0.5), (4,1), (3.5,1.5), (3,2), (4,2), (2.5,2.5), (3.5,2.5), (3,3), (4,3), (3.5,3.5), (4,4), (4.5,4.5) } {
	\fill[black] \Point circle (3pt);
  }
  \foreach \Point in {(1,0.5), (1,1.5), (1.5,1.5), (1.5,2.5), (1,3), (2,3.5), (2.5,4), (1,4.5), (3, 2.5)} {
    \fill[white] \Point circle (3pt);
  }
\end{tikzpicture}
\caption{Training Sample}\label{fig1}
\end{figure}

\vfill

\subsection*{Solution}

\begin{table}[H]\centering
\begin{tabular}{r l c c}
Kernel & Function Name & Training Error & Execution Time\\
\hline
Linear & \texttt{vanilladot} & 4.761905 & 0.01\\
Radial Basis & \texttt{rbfdot} & 4.761905 & 0.02\\
Polynomial & \texttt{polydot} & 4.761905 & 0.01\\
ANOVA RBF & \texttt{anovadot} & 4.761905 & 0.02\\
Bessel & \texttt{besseldot} & 4.761905 & 0.00\\
Laplacian & \texttt{laplacedot} & 4.761905 & 0.02\\
Spline & \texttt{splinedot} & 9.52381 & 0.01\\
Hyperbolic Tangent & \texttt{tanhdot} & 19.04762 & 0.00\\
\hline
\end{tabular}
\caption{Performance of Different Kernel Functions on Training Sample given in Fig \ref{fig1}}\label{tab1}
\end{table}

The training dataset shown in Figure \ref{fig1} is constructed in comma-separated value format.
Using R programming-language, the training dataset is imported to workspace and is used to construct classifiers using kernels available in the \texttt{kernlab} package.
Table \ref{tab1} provides the list of kernels used for SVM-based classification as well as their training error and execution time for their construction over given training dataset.

Fig \ref{fig2}, generated by R, presents the training error resulting construction of  classifiers using different kernel functions.
As can be seen, \texttt{splinedot} and \texttt{tanhdot} kernels have failed to classify the training dataset accurately.
The similar training error achieved by other kernel functions can be justified by noting the limited amount of samples in the training dataset as well as the way they are formed  which makes it fairly easy to classify.

\begin{figure}\centering
\includegraphics[width=0.8\textwidth]{\imgDirectory/umb-cs671-2015s-f01-02.png}
\caption{Training Error for Different Kernels}\label{fig2}
\end{figure}

Also, plot of execution time of different kernel functions in constructing classifier for the training dataset is given in Figure \ref{fig3}.

\begin{figure}\centering
\includegraphics[width=0.8\textwidth]{\imgDirectory/umb-cs671-2015s-f01-03.png}
\caption{Execution Time for Different Kernel Functions}\label{fig3}
\end{figure}

Figure \ref{fig4} depicts the execution time based on training error for different kernel functions.

\begin{figure}\centering
\includegraphics[width=0.8\textwidth]{\imgDirectory/umb-cs671-2015s-f01-04.png}
\caption{Execution-Time based on Error-Rate for Different Kernels}\label{fig4}
\end{figure}

It can be inferred that using kernel functions that promise better performance in terms of training error might necessitate higher computational costs which lead to longer execution times.
However, this particular example does not prove this point as can be seen in Fig \ref{fig4}.
In fact, rerunning the same script would give different results each time.
This is due to the limited amount of the training dataset that will make the errors in measurement of the execution time quite considerable.
To prove our claim, the same code in R programming language has been applied to the larger database given for letter recognitions, results of which are included in the files attached to this document.

The code in R data-analysis software to construct and compare classifiers over the training dataset given in \ref{fig1} has been given as appendix to this report.


%
% CMPT 310: Artificial Intelligence - A Course Overview
% Section: Search Algorithms
%
% Author: Jeffrey Leung
%

\section{Search Algorithms}
	\label{sec:search-algorithms}
\subsection{Uninformed Algorithms}
	\label{subsec:uninformed-algorithms}
\begin{easylist}

& \textbf{Breadth-first search:} Utilizes a queue
	&& Time and space complexity: Bounded by the number of nodes the same distance away from the root as the goal node
	&& Complete if branching factor is finite

& \textbf{Uniform-cost search:} Utilizes a priority queue to follow the path with least cost
	&& Time and space complexity: Bounded by the branching factor to the power of the average action cost
	&& Complete if branching factor is finite and step costs $\leq \epsilon$

& \textbf{Depth-first search:} Utilizes a stack
	&& Time complexity: Bounded by the maximum depth of the tree
	&& Space complexity: Bounded by the length of the maximum-length path, and its remaining unexpanded child nodes for each node on the path
	&& Not complete
	&& More space-efficient than breadth-first search

& \textbf{Iterative deepening (depth-first) search:} Depth-first search with maximum limited depth, which increases until the goal is found
	&& Complete if branching factor is finite
	&& Time complexity: Bounded by the number of nodes the same distance away from the root as the goal node
	&& Space complexity: Bounded by the length of the goal path, and its remaining unexpanded child nodes for each node on the path

\end{easylist}
\subsection{Informed Algorithms}
	\label{subsec:informed-algorithms}
\begin{easylist}

& \textbf{Admissible heuristic:} Underestimation of the true cost from a given node to the goal
	&& Notation for a function evaluating a heuristic: $h(n)$
	&& Will provide improved time and space complexity over the worst time
	&& To create an admissible heuristic, relax a rule/constraint and find the shortest solution given
	&& \textbf{Manhattan distance:} Number of unrestricted 4-directional moves from the goal state
	&& \textbf{Dominance:} Characteristic of an admissible heuristic which has a result greater than or equal to another admissible heuristic for all $n$
		&& Given admissible heuristics $h_a(n)$ and $h_b(n)$, $h(n) = \textrm{max}\left(h_a(n), h_b(n)\right)$ is also admissible and dominates $h_a(n)$ and $h_b(n)$

& \textbf{Greedy search:} Best-first search algorithm which chooses subsequent nodes by minimizing direct cost to the goal
	&& Complete: Only if state is checked for repeats
	&& Time and space complexity: At worst, bounded by all nodes

& \textbf{A* search:} Best-first search algorithm which expands only the paths with less cost than the optimal cost to the goal
	&& Evaluation function:

	\end{easylist}
	\begin{align*}
		f(n) & = g(n) + h(n) \\
		\textrm{where }
		& f(n) = \textrm{ estimated total cost of the path through } n \textrm{ to the goal} \\
		& g(n) = \textrm{ cost so far to reach } n \\
		& h(n) = \textrm{ heuristic-estimated cost from } n \textrm{ to the goal}
	\end{align*}
	\begin{easylist}

	&& Complete: Yes (unless there are infinitely many nodes with $f \leq f(G)$)

\end{easylist}
\subsection{Complexity of Algorithms}
	\label{subsec:complexity-of-algorithms}

\begin{figure}[!htb]
	\caption{Complexity of Algorithms}
	\label{fig:algos-complexity}
	\center
	\begin{tabular}{ l | c c p{4cm} }
		& Time & Space & Is the Goal Path Optimal? \\
		\hline
		Breadth-first search & $O(b^{d+1})$ & $O(b^d)$ & Only when $d = 1$ or when paths have no cost \\
		Uniform-cost search & $O(b^{\left\lfloor \frac{C^*}{\epsilon} \right\rfloor})$ & $O(b^{\left\lfloor \frac{C^*}{\epsilon} \right\rfloor})$ & Yes \\
		Depth-first search & $O(b^m)$ & $O(b \cdot m)$ & No \\
		Iterative deepening search & $O(b^d)$ & $O(b \cdot d)$ & Yes \\
		\hline
		Greedy search & $O(b^m)$ & $O(b^m)$ & No \\
		A\textsuperscript{*} search & $O(b^m)$ & $O(b^m)$ & Yes \\
		\hline
		Minimax algorithm & $O(b^m)$ (with $\alpha / \beta$, $O(b^\frac{m}{2})$) & $O(b \cdot m)$ & Yes (with rational adversary) \\
		Backtracking search & & & Yes \\
		Hill climbing search & & & No \\
		Simulated annealing search & & & Yes
	\end{tabular}

	\begin{align*}
		\textrm{where }
		b &= \textrm{ branching factor} \\
		d &= \textrm{ depth of the goal} \\
		C^* &= \textrm{ cost of the optimal solution} \\
		\epsilon &= \textrm{ the minimum cost of an action on the goal path} \\
		m &= \textrm{ maximum depth}
	\end{align*}
\end{figure}

\clearpage


\section{Edge discretization}\label{edge_discretization}
We calculate the location $l$ of the boundary between a significant and nonsignificant portion of an edge analytically.
For example, the parabolic MAT edge generated from the outline vertex $(0,1)$ and an outline segment aligned with the X-axis follows $y(x) = \nicefrac12 x^2$ and $R(x) = y(x)$.
We can determine the significant portion $[-x_\text{bound}, x_\text{bound}]$ by evaluating $\frac{\partial R}{\partial x} > \cos(\alpha_\text{max} / 2)$, which is $ | x_\text{bound} | = (\tan(\alpha_\text{max} / 2))^{-1}$.
Similarly, a MAT edge generated from two vertices at $(0,0)$ and $(0,1)$ follows $y(x) = \nicefrac12$ and $R(x) = \sqrt{\nicefrac14 + x^2}$.
The boundaries of significance are given by $ | x_\text{bound} | = \nicefrac12 (\tan(\alpha_\text{max} / 2))^{-1}$.
From these we can derive the locations $l = (\pm x_\text{bound}, y(x_\text{bound}))$.
These specific cases can easily be transformed into all possible cases using scaling and rotation operations.


\documentclass{article}
\usepackage{amsmath}

\title{Math Proofs}
\author{Andrew Archibald}
\date{July 2017}

\begin{document}

\begin{titlepage}
\maketitle
\end{titlepage}

\section{Pi by the sum of inverse squares}

$$\frac{\pi^2}{6} = \sum_{n=1}^{\infty}\frac{1}{n^2}$$

\section{Pi by inverse tangent Taylor series}

\begin{align*}
\arctan x &= x - \frac{x^3}{3} + \frac{x^5}{5} - \frac{x^7}{7} + \frac{x^9}{9} - \cdots \\
          &= \sum_{n=0}^{\infty} (-1)^n \frac{x^{2n + 1}}{2n + 1}
\end{align*}
$$x \in [-1, 1]$$
$$\frac{\pi}{4} = \arctan 1
= \sum_{n=0}^{\infty} \frac{(-1)^n}{2n + 1}$$

\section{Pi by inscribed and circumscribed polygons}

\subsection{Inscribed}

\subsubsection{Law of sines}

$$\frac{\sin(\frac{\pi}{2}-\frac{\pi}{n})}{r} = \frac{\sin\frac{2\pi}{n}}{l}$$
$$= \frac{\cos\frac{\pi}{n}}{r} = \frac{2\sin\frac{\pi}{n}\cos\frac{\pi}{n}}{l}$$
$$l = 2r\sin\frac{\pi}{n}$$
$$P = ln
= 2rn\sin\frac{\pi}{n}$$

\subsubsection{Law of cosines}

$$l^2 = 2r^2 - 2r^2cos\frac{2\pi}{n}
= 2r^2\left(1 - cos\frac{2\pi}{n}\right)
= 4r^2\left(\frac{1 - cos\frac{2\pi}{n}}{2}\right)
= 4r^2\sin^2{\frac{\pi}{n}}$$
$$l = 2r\sin\frac{\pi}{n} = r\sqrt{2 - 2\cos\frac{2\pi}{n}}$$
$$P = 2n\sin\frac{\pi}{n}$$

\subsubsection{Pythagorean theorem}

$$r^2 = \left(\frac{l}{2}\right)^2 + h^2$$
$$h^2 = r^2 - \frac{1}{4}l^2
= r^2 - r^2\sin^2\frac{\pi}{n}
= r^2\left(1 - \sin^2\frac{\pi}{n}\right)
= r^2\cos^2\frac{\pi}{n}$$
$$h = r\cos\frac{\pi}{n}$$
$$A = \frac{1}{2}lhn
= \frac{1}{2}\left(2r\sin\frac{\pi}{n}\right)\left(rcos\frac{\pi}{n}\right)n
= nr^2\sin\frac{\pi}{n}\cos\frac{\pi}{n}
= \frac{1}{2}nr^2\sin\frac{2\pi}{n}$$

\subsubsection{Results}

Perimeter

$$2\pi{r} = \lim_{n\to\infty} P
= \lim_{n\to\infty} 2rn\sin\frac{\pi}{n}$$

Area

$$\pi{r^2} = \lim_{n\to\infty} \frac{1}{2}nr^2\sin\frac{2\pi}{n}$$
$$\pi = \lim_{n\to\infty} \frac{1}{2}n\sin\frac{2\pi}{n}
= \lim_{n\to\infty} n\sin\frac{\pi}{n}\cos\frac{\pi}{n}$$

\subsection{Circumscribed}

\subsubsection{Law of sines}

$$\frac{\sin\frac{\pi}{n}}{L} = \frac{\sin\left(\pi - \frac{2\pi}{n}\right)}{l}$$
$$\sin\left(\pi - \frac{2\pi}{n}\right) = \sin\left[2\left(\frac{\pi}{2} - \frac{\pi}{n}\right)\right]
= 2\sin\left(\frac{\pi}{2} - \frac{\pi}{n}\right)\cos\left(\frac{\pi}{2} - \frac{\pi}{n}\right)
= 2\sin\frac{\pi}{n}\cos\frac{\pi}{n}$$
$$L = \frac{l\sin\frac{\pi}{n}}{2\sin\frac{\pi}{n}\cos\frac{\pi}{n}}
= \frac{1}{2}l\sec\frac{\pi}{n}
= \frac{1}{2}\left(2r\sin\frac{\pi}{n}\right)\sec\frac{\pi}{n}
= r\tan\frac{\pi}{n}$$

\subsubsection{Law of cosines}

$$l^2 = 2L^2 - 2L^2cos\frac{\pi - 2\pi}{n}
= L^2\left(2 - 2\cos\left(\pi - \frac{2\pi}{n}\right)\right)$$
$$\cos\left(\pi - \frac{2\pi}{n}\right) = \cos\left(2\left(\frac{\pi}{2} - \frac{\pi}{n}\right)\right)
= 1 - 2\sin^2\left(\frac{\pi}{2} - \frac{\pi}{n}\right)
= 1 - 2\cos^2\frac{\pi}{2}$$
$$l^2 = L^2\left[2 - 2\left(1 - 2\cos^2\frac{\pi}{n}\right)\right]
= 4L^2\cos^2\frac{\pi}{n}$$
$$l = 2L\cos\frac{\pi}{n}$$
$$2r\sin\frac{\pi}{n} = 2L\cos{\pi}{n}$$
$$L = r\tan\frac{\pi}{n}$$

\subsubsection{Pythagorean theorem}

$$L^2 = \left(\frac{l}{2}\right)^2 + H^2$$
$$H^2 = r^2tan^2\frac{\pi}{n} - r^2\sin^2\frac{\pi}{n}
= r^2\left(\tan^2\frac{\pi}{n} - \sin^2\frac{\pi}{n}\right)$$
$$tan^2\frac{\pi}{n} - \sin^2\frac{\pi}{n} = tan^2\frac{\pi}{n} - \cos^2\frac{\pi}{2}\tan\frac{\pi}{2}
= \tan^2\frac{\pi}{n}\left(1 - \cos^2\frac{\pi}{n}\right)
= \sin^2\frac{\pi}{n}tan^2\frac{\pi}{n}$$
$$H^2 = R^2sin^2\frac{\pi}{n}\tan^2{\pi}{n}$$
$$H = r\sin\frac{\pi}{n}\tan\frac{\pi}{n}$$

\subsubsection{Results}

Perimeter

$$P = 2nL = 2nr\tan\frac{\pi}{n}$$
$$2\pi{r} = \lim_{n\to\infty} 2nr\tan\frac{\pi}{n}$$
$$\pi = \lim_{n\to\infty} n\tan\frac{\pi}{n}$$

Area

$$A = nH\frac{L}{2} + A_l$$
$$  = n\left(r\sin\frac{\pi}{n}\tan\frac{\pi}{n}\right)\left(r\sin\frac{\pi}{n}\right) + A_l$$
$$  = nr^2\sin^2\frac{\pi}{n}\tan\frac{\pi}{n} + A_l$$
$$  = nr^2\sin^2\frac{\pi}{n}\tan\frac{\pi}{n} + nr^2\sin\frac{\pi}{n}\cos\frac{\pi}{n}$$
$$  = nr^2\sin\frac{\pi}{n}\left(\sin\frac{\pi}{n}\tan\frac{\pi}{n} + \cos\frac{\pi}{n}\right)$$
$$  = nr^2\tan\frac{\pi}{n}\left(\sin^2\frac{\pi}{n} + \cos^2\frac{\pi}{n}\right)$$
$$  = nr^2\tan\frac{\pi}{n}$$
$$\pi{r^2} = \lim_{n\to\infty} nr^2\tan\frac{\pi}{n}$$
$$\pi = \lim_{n\to\infty} n\tan\frac{\pi}{n}$$

\subsection{Simplified results}

$$\pi = \lim_{n\to\infty} n\sin\frac{\pi}{n}$$
$$\pi = \lim_{n\to\infty} n\sin\frac{\pi}{n}\cos\frac{\pi}{n}$$
$$\pi = \lim_{n\to\infty} n\tan\frac{\pi}{n}$$

\section{Multi-dimensional spheres}

$$V_1 = \int\limits_{-R}^{R} dx = 2R$$
$$V_2
= \int\limits_{-R}^{R} \int_{-\sqrt{R^2-x^2}}^{\sqrt{R^2-x^2}} dydx
= \int\limits_{-R}^{R} 2\sqrt{R^2-x^2} dx
= \int\limits_{-R}^{R} 2R\sin\arccos\frac{x}{R} dx$$
$$V_2
= \int\limits_0^{2\pi} \int\limits_0^{R} r drd\theta
= \int\limits_0^{2\pi} \frac{1}{2}R^2 d\theta
= \pi R^2$$

\section{Trigonometric integrals}

\begin{align*}
\int \sin^2{x} dx
  &= \int \frac{1-\cos{2x}}{2} dx \\
  &= \frac{1}{2}(x - \sin{x}\cos{x}) + C
\\\\
\int \cos^2{x} dx
  &= \int \frac{1+\cos{2x}}{2} dx \\
  &= \frac{1}{2}(x + \sin{x}\cos{x}) + C
\end{align*}

\begin{align*}
\int \sin^3{x} dx
  &= \int (1 - \cos^2{x})\sin{x} dx \\
  &= \int (\sin{x} - \cos^2{x}\sin{x}) dx \\
  &= -\cos{x} + \frac{1}{3}\cos^3{x} + C
\\\\
\int \cos^3{x} dx
  &= \int (1 - \sin^2{x})\cos{x} dx \\
  &= \int (\cos{x} - \sin^2{x}\cos{x}) dx \\
  &= \sin{x} - \frac{1}{3}\sin^3{x} + C
\end{align*}

\begin{align*}
\int \sin^4{x} dx
  &= \int \left(\frac{1 -\cos{2x}}{2}\right)^2 dx \\
  &= \int \frac{1}{4}(1 - 2\cos{2x} + \cos^2{2x}) dx \\
  &= \int \frac{1}{4}\left(1 - 2\cos{2x} + \frac{1 + \cos{4x}}{2}\right) dx \\
  &= \int \frac{1}{8}(3 - 4\cos{2x} + \cos{4x}) dx \\
  &= \frac{1}{8}\left(3x - 2\sin{2x} + \frac{1}{4}\sin{4x}\right) + C
\\\\
\int \cos^4{x} dx
  &= \int \left(\frac{1 +\cos{2x}}{2}\right)^2 dx \\
  &= \int \frac{1}{4}(1 + 2\cos{2x} + \cos^2{2x}) dx \\
  &= \int \frac{1}{4}\left(1 + 2\cos{2x} + \frac{1 + \cos{4x}}{2}\right) dx \\
  &= \int \frac{1}{8}(3 + 4\cos{2x} + \cos{4x}) dx \\
  &= \frac{1}{8}\left(3x + 2\sin{2x} + \frac{1}{4}\sin{4x}\right) + C
\end{align*}

\begin{align*}
\int \sin^5{x} dx
  &= \int (1 - \cos^2{x})^2\sin{x} dx \\
  &= \int (\sin{x} - 2\cos^2{x}\sin{x} + \cos^4{x}\sin{x}) dx \\
  &= -\cos{x} + \frac{2}{3}\cos^3{x} - \frac{1}{5}\cos^5{x} + C
\\\\
\int \cos^5{x} dx
  &= \int (1 - \sin^2{x})^2\cos{x} dx \\
  &= \int (\cos{x} - 2\sin^2{x}\cos{x} + \sin^4{x}\cos{x}) dx \\
  &= \sin{x} - \frac{2}{3}\sin^3{x} + \frac{1}{5}\sin^5{x} + C
\end{align*}

\end{document}


% Section with some maths

\section{Ein Abschnitt mit Matheformeln}
\blindmathpaper

\subsection{Formeln inkl. Subequations and Labels}
\label{sec:maths-incl-subeq}
\blindtext
\newcommand{\iu}{{i\mkern1mu}}
\vspace{-.4cm}
\begin{subequations}
  \begin{align}
    \iu \hbar \partial_t \Psi &= - \iu \hbar c \sum_{i=1}^3 \alpha_i \partial_i \Psi + \beta m c^2 \Psi\\ &= \left( c \vec \alpha \cdot \vec p + \beta m c^2 \right) \Psi \\
    \Longrightarrow \quad 0 &= \left( \iu \hbar \gamma^\mu \partial_\mu - mc \right) \Psi
  \end{align}
\end{subequations}
\blindtext
\begin{align}
  F(x,y)=0 ~~\mbox{and}~~
  \left| \begin{array}{ccc}
           F''_{xx} & F''_{xy} &  F'_x \\
           F''_{yx} & F''_{yy} &  F'_y \\
           F'_x     & F'_y     & 0
         \end{array}\right| = 0
\label{eq:1}
\end{align}
\blindtext

% End of file


% !TEX root = ../../main.tex
\subsection{xokker-Planck equation in population genetics}

In the last section after we performed a Taylor expansion of the master
equation we ended up with an equally complicated partial differential equation
of infinite order. So in order to make any progress towards simplifying the
treatment of the equation what we will do is trade accuracy for simplicity.
More concretely if our assumptions on the shape of the transition probability
$\phi_t(x; r)$ being sharply peaked are to be taken seriously, we can then
truncate the Kramers-Moyal expansion to include only up to second order
derivatives. This truncated equation would locally approximate the time
evolution of the probability distribution $P(x, t)$. \eref{eq_km_expansion}
then is of the form
\begin{equation}
  \ddt{P(x, t)} = - {\partial \over \partial x}
  \left[
  a^{(1)}(x, t) P(x, t)
  \right] +
  {1 \over 2}{\partial^2 \over \partial f^2}
  \left[
  a^{(2)}(x, t) P(x, t)
  \right].
  \label{eq_fokker_planck}
\end{equation}
In the physics literature this is known as the Fokker-Planck equation, while in
the mathematics literature is known as the Kolmogorov forward equation. One
intriguing aspect of how to arrive to this equation is the seemingly arbitrary
choice of truncating up to the second moment. The argument that is often thrown
around is that the ``art'' of these truncations is to stop at the first
non-vanishing moment. But for the specific case of the Kramers-Moyal expansion
of the master equation there is a theorem - Pawula theorem - that shows that
for the solutions of the Kramers-Moyal expansion to be interpreted as
probability densities the expansion must either contain one, two or an infinite
number of moments. So two sounds much better than infinite, doesn't it?
\mrm{Need to include appendix with the proof of Pawula theorem.}

\subsubsection{Determination of the Fokker-Planck coefficients}

The two jump moments in \eref{eq_fokker_planck}, $a^{(1)}$ and $a^{(2)}$
defined by \eref{eq_jump_mom} have a specific interpretation in population
genetics. Here is where a little bit of a terminology conflict between physics
and evolutionary biology comes into play. The directional term, i.e. the one
with the first order derivative in \eref{eq_fokker_planck} is known in the
physics literature as the drift term on a diffusion-like equation. This is
confusing because in evolutionary theory the diffusive term, i.e. the one with
the second derivative in \eref{eq_fokker_planck} is known as the genetic drift
term. I will try to be as consistent as possible on these notes using the terms
directional and diffusive to avoid confusion.

Coming back to the directional term, we define $M(x, t)$ to be
\begin{equation}
  M(x, t) \equiv \ee{r(t)} = \int_{-\infty}^{\infty} dr \; \phi_t(x; r) r,
\end{equation}
i.e. the mean of the jump distribution. As we will see in coming sections this
term captures the effect of directional evolutionary forces such as selection,
mutation and migration. For the diffusive term we define $V(x, t)$ as the
second moment of the jump distribution
\begin{equation}
  V(x, t) \equiv \ee{r^2(t)} = \int_{-\infty}^{\infty} dr \; \phi_t(x; r) r^2.
\end{equation}
This term captures the random sampling of alleles, also known as genetic drift.
In all of the population genetics literature I have encounter so far, this term
$V(x, t)$ is treated as the \textbf{variance} rather than the second moment.
This is partly because computing the specific functional form of the variance
for different models of reproduction is much simpler. The variance
$\sigma_r^2(t)$ is defined as
\begin{equation}
  \sigma_r^2(t) = \ee{\left( r - \ee{r} \right)^2} = \ee{r^2} - \ee{r}^2.
\end{equation}
We can therefore work with this more convenient quantity if we assume that
$\ee{r}^2 \approx 0$. This is a reasonable assumption given that for the
Fokker-Planck equation to be accurate we assumed a tight distribution for the
jumpt size $\phi_t(x; r)$. The peaked nature of this distribution must imply
that $\ee{r} \ll 1$, but more importantly, we will assume that
$\ee{r}^2 \ll \ee{r^2}$. So upon using these two definitions we arrive to one
of the main results in population genetics, the Kimura diffusion equation
\begin{equation}
  \ddt{P(x, t)} = - {\partial \over \partial x}
  \left[
  M(x, t) P(x, t)
  \right] +
  {1 \over 2}{\partial^2 \over \partial f^2}
  \left[
  V(x, t) P(x, t)
  \right].
  \label{eq_kimura_diffusion}
\end{equation}
The power of diffusion theory is that in these two terms $M(x, t)$ and $V(x,
t)$ we can include all evolutionary forces acting simultaneously. The
functional forms of these specific terms depend on the reproduction model used.
We will explore that more specifically later on.

\subsubsection{Equilibrium distribution}

In the limit when $t \rightarrow \infty$ we expect the distribution of allele
frequencies to reach a steady-state $P_{ss}(x)$. For the 1D case we have
studied so far, i.e. two alleles with frequencies $x$ and $1 - x$ this steady
state is equivalent to an equilibrium distribution since detailed balance has
to be satisfied. To emphasize this point let us rewrite
\eref{eq_kimura_diffusion} as a statement of conservation of probability. This
is
\begin{equation}
  \ddt{P(x, t)} = - {\partial J(x, t) \over \partial x},
\end{equation}
where $J(x, t)$ is the probability flux at point $x$. If we set the time
derivative to zero there are only two options (in reality there is only one
option for 1D systems):
\begin{enumerate}
  \item ${\partial J \over \partial x} = 0; \; J \neq 0 \Rightarrow$ Steady
  state on a rotating or non-conservative field.
  \item ${\partial J \over \partial x} = 0; \; J = 0 \Rightarrow$ Equilibrium
  distribution that satisfies detailed balance.
\end{enumerate}
For our one-locus two-alleles case the second of these cases must be true.
What this implies is that at steady state the flux $J_{ss}(x)$ takes the form
\begin{equation}
  J_{ss}(x) = - M(x) P_{eq}(x) + {\partial \over \partial x}
  \left[ V(x) P_{eq}(x) \right] = 0,
  \label{eq_flux_eq}
\end{equation}
where we use $P_{eq}(x)$ to define that this is not only a steady-state
distribution, but an equilibrium distribution satisfying detailed balance.
\eref{eq_flux_eq} is a first order homogeneous ordinary differential equation.
We can solve it using the integration factor method. For this we define
$G(x) \equiv V(x)P_{eq}(x)$. Substituting this into \eref{eq_flux_eq} gives
\begin{equation}
  {- M(x) \over V(x)} G(x) + {\partial \over \partial x} G(x) = 0.
  \label{eq_ode_ss}
\end{equation}
In this form we define the integration factor to be
\begin{equation}
  h(x) = \exp \left( \int_0^x -{M(x') \over V(x')} \; dx' \right).
\end{equation}
Notice that we chose the limits of integration to be $[0, x]$. This is because
the fundamental theorem of calculus states that for any function $f(x)$ defined
in $[a, b]$, the antiderivative $F(x)$ is defined as
\begin{equation}
  F(x) = \int_a^x f(x') \; dx',
\end{equation}
regardless of the lower limit of integration. Multiplying both sides of
\eref{eq_ode_ss} by the integration factor $h(x)$ results in
\begin{equation}
  {- M(x) \over V(x)} G(x)
  \exp \left( \int_0^x -{M(x') \over V(x')} \; dx' \right)
  + {\partial \over \partial x} G(x)
  \exp \left( \int_0^x -{M(x') \over V(x')} \; dx' \right)
  = 0.
  \label{eq_ode_int_fact}
\end{equation}
The specific form of the integration factor was chosen such that we could
rewrite \eref{eq_ode_int_fact} as
\begin{equation}
  {d \over dx}
  \left[
  \exp \left(
  - \int_0^x {M(x') \over V(x')}
  \right)G(x) \; dx'
  \right] = 0.
\end{equation}
Written in this form we can simply integrate both sides with respect to $x$ as
\begin{equation}
  \int_0^x {d \over dx''}\left[
  \exp \left(
  - \int_0^{x''} {M(x') \over V(x')}
  \right) G(x'')\; dx'
  \right] =
  \int_0^x 0 \; dx''
\end{equation}
Evaluating these integrals results in
\begin{equation}
  \exp \left(
  - \int_0^{x} {M(x') \over V(x')}
  \right) G(x)\; dx' = C,
\end{equation}
where $C$ is an integration constant. Notice that for a specific interval
$[a, b] \in \mathbb{R}$ the integral of zero is
\begin{equation}
  \int_a^b 0 dt = 0,
\end{equation}
but when we set the upper integration limit as an independent variable, what we
are asking for is the antiderivative of zero which is a constant $C$.
Substituting the definition of $G(x) = V(x) P_{eq}(x)$ gives
\begin{equation}
  \exp \left(
  - \int_0^x dx' \; {M(x') \over V(x')}\right)
  V(x) P_{eq}(x) = C.
\end{equation}
We can then solve for the equilibrium allele distribution $P_{eq}(x)$ obtaining
the result we were aiming for
\begin{equation}
  P_{eq}(x) = {C \over V(x)} \exp \left(
  \int_0^x dx' \; {M(x) \over V(x)}
  \right).
\end{equation}
This is a Boltzmann-like distribution! The analogy with statistical mechanics
becomes even more clear when we substitute specific functional forms for the
directional term $M(x)$ and the diffusive term $V(x)$.

In the next section we will explore how to obtain the coefficients for our
Fokker-Planck equation given the Langevin dynamics that we defined in
\secref{sec_langevin_intro}.


\subsection{Endomorphisms as group actions}

We can view each member of the group \(g\) as a homomorphim on \(s\).

Where \(s\) is a vector space \(V\), the representation on each group member is an invertible square matrix.

If the set we use is the vector space \(V\), then we can represent each group element with a square matrix acting on \(V\).

Faithful means \(a\ne b\) holds for repesentation too.

Representation theory. groups defined by \(ab=c\). if we can match each eleemnt to amatrix where this holds we have represented the matrix.


\input{../header_function}

%---------- start document ---------- %
 \section{cubic\_root -- cubic root, residue, and so on}\linkedzero{cubic\_root}
%
  \subsection{c\_root\_p -- cubic root mod p}\linkedone{cubic\_root}{c\_root\_p}
   \func{c\_root\_p}{\hiki{a}{integer},\ \hiki{p}{integer}}{\out{list}}\\
   \spacing
   % document of basic document
   \quad Return the cubic root of \param{a} modulo prime \param{p}. (i.e. solutions of the equation $x^3 = \param{a} \pmod{\param{p}}$).\\
   \spacing
   % added document
   %\spacing
   % input, output document
   \quad \param{p} must be a prime integer.\\
   This function returns the list of all cubic roots of \param{a}.\\
%
  \subsection{c\_residue -- cubic residue mod p}\linkedone{cubic\_root}{c\_residue}
   \func{c\_residue}{\hiki{a}{integer},\ \hiki{p}{integer}}{\out{integer}}\\
   \spacing
   % document of basic document
   \quad Check whether the rational integer \param{a} is cubic residue modulo prime \param{p}.\\
   \spacing
   % added document
   \quad If $\param{p}\ |\ \param{a}$, then this function returns $0$,
   elif \param{a} is cubic residue modulo \param{p}, then it returns $1$,
   otherwise (i.e. cubic non-residue), it returns $-1$.\\
   \spacing
   % input, output document
   \quad \param{p} must be a prime integer.\\
%
  \subsection{c\_symbol -- cubic residue symbol for Eisenstein-integers}\linkedone{cubic\_root}{c\_symbol}
   \func{c\_symbol}{\hiki{a1}{integer},\ \hiki{a2}{integer},\ \hiki{b1}{integer},\ \hiki{b2}{integer}}{\out{integer}}\\
   \spacing
   % document of basic document
   \quad Return the (Jacobi) cubic residue symbol of two Eisenstein-integers $\left(\frac{\param{a1}+\param{a2}\omega}{\param{b1}+\param{b2}\omega}\right)_3$,
where $\omega$ is a primitive cubic root of unity.\\
   \spacing
   % added document
   If $\param{b1}+\param{b2}\omega$ is a prime in $\mathbb{Z}[\omega]$, it shows $\param{a1}+\param{a2}\omega$ is cubic residue or not.\\
   \spacing
   % input, output document
   \quad We assume that $\param{b1}+\param{b2}\omega$ is not divisible $1-\omega$.\\
%
  \subsection{decomposite\_p -- decomposition to Eisenstein-integers}\linkedone{cubic\_root}{decomposite\_p}
   \func{decomposite\_p}{\hiki{p}{integer}}{(\out{integer},\ \out{integer})}\\
   \spacing
   % document of basic document
   \quad Return one of prime factors of \param{p} in $\mathbb{Z}[\omega]$.\\
   \spacing
   % added document
   \quad If the output is (\param{a},\ \param{b}), then $\frac{\param{p}}{\param{a}+\param{b}\omega}$ is a prime in $\mathbb{Z}[\omega]$.
   In other words, \param{p} decomposes into two prime factors $\param{a}+\param{b}\omega$ and $\param{p}/(\param{a}+\param{b}\omega)$ in $\mathbb{Z}[\omega]$.\\
   \spacing
   % input, output document
   \quad \param{p} must be a prime rational integer.
   We assume that $\param{p}\equiv 1 \pmod 3$.\\
%
  \subsection{cornacchia -- solve $x^2+dy^2=p$}\linkedone{cubic\_root}{cornacchia}
   \func{cornacchia}{\hiki{d}{integer},\ \hiki{p}{integer}}{(\out{integer},\ \out{integer})}\\
   \spacing
   % document of basic document
   \quad Return the solution of $x^2 + \param{d}y^2 = \param{p}$.\\
   \spacing
   % added document
   \quad This function uses Cornacchia's algorithm. See \cite{Cohen1}.\\
   \spacing
   % input, output document
   \quad \param{p} must be prime rational integer.
   \param{d} must be satisfied with the condition $0<\param{d}<\param{p}$.
   This function returns (\param{x},\ \param{y}) as one of solutions of the equation $x^2 + \param{d} y^2 = \param{p}$.\\
%
\begin{ex}
>>> cubic_root.c_root_p(1, 13)
[1, 3, 9]
>>> cubic_root.c_residue(2, 7)
-1
>>> cubic_root.c_symbol(3, 6, 5, 6)
1
>>> cubic_root.decomposite_p(19)
(2, 5)
>>> cubic_root.cornacchia(5, 29)
(3, 2)
\end{ex}%Don't indent!(indent causes an error.)
\C

%---------- end document ---------- %

\input{../footer}