\subsection{Finding the angle between the sun and the line perpendicular to the solar panels}\label{subsec:findingTheAngleBetweenTheSunAndTheLinePerpendicularToTheSolarPanels}
To find the angle $\alpha$ between the sun and the line perpendicular to the solar panel, we determined both lines in spherical coordinates (with the same distance to the origin) and then calculated the angle between the two.

For the sun this would be
\[
    \vvec{x \\ y \\ z} =
    \vvec{\cos \gamma_s \sin \left(\frac{\pi}{2} - \theta_s\right) \\
    \sin \gamma_s \sin \left(\frac{\pi}{2} - \theta_s\right) \\
    \cos \left(\frac{\pi}{2} - \theta_s\right)}
\]
where $ \gamma_s $ is the azimuth, and $ \theta_s $ the altitude of the sun.

For the line perpendicular to the solar panels this would be
\[
    \vvec{x \\ y \\ z} =
    \vvec{\cos \gamma_p \sin \left(\frac{\pi}{2} - \theta_p\right) \\
    \sin \gamma_p \sin \left(\frac{\pi}{2} - \theta_p\right) \\
    \cos \left(\frac{\pi}{2} - \theta_p\right)}
\]
where $ \gamma_p $ is the azimuth (direction of the solar panels in respect to the South), and $ \theta_p $ the altitude of the solar panels.

Then, to find the angle $\alpha$ between these two lines, we take the $ \arccos $ of the dot product.
This gives us

\[
    \alpha = \arccos (\cos (\gamma_p - \gamma_s) \cos \theta_s \sin \theta_p + \sin \theta_s \cos \theta_p)\,.
\]

\subsection{Implementation of calculations to find optimal angles}

\subsubsection{Mathematica calculations}\label{subsec:mathematicaCalculations}

The Mathematica package (imported with \verb|<< SolArduino`|, be sure to place it in your
\url{\%AppData\%\\Mathematica\\Applications} folder) can calculate the optimal angle for a given day.
To do that, it calculates for each angle between $0$ and $90$ the total of the insolation (power received by the sun) at each half hour of that day.
Then it finds the angle for which that value is maximal.
To find the insolation at a given hour, the function \verb|angle| calculates the misalignment with the sun using the formula from the previous subsection, and then calculates the insolation using a formula from \href{http://www.powerfromthesun.net/Book/chapter02/chapter02.html#ZEqnNum929295 }{www.powerfromthesun.net}, where parameters for urban haze compared a lot better with real life values (17/8) than clear day parameters.

It can therefore make graphs for optimal angles for a month, and averaging the values for a month, also for a year, and lots more as seen in the demonstration notebook.
When plotting real data, for example days like 13/5, 19/7 and 17/8 are all cloudless days with the solar panels at around 25 degrees.

It is important to note that the functions \verb|angle|, \verb|directPower| and more do not take the hour of the day as input, but the index of the \verb|sunPositions| table which contains the azimuth and altitude of the sun over the day.
Therefore, \textit{before you call functions which take an index as parameter you need to make sure you have }\verb|sunPositions| \textit{initialised} at the right day, done by calling \verb|calculatesunPos[DateObject[{2016,7,18}]]| with whatever day you want in the table.

Exporting the angles for ten times a day for two years, like

\verb|exportPeriod[DateObject[{2016, 9, 7}], DateObject[{2018, 9, 15}], 10]| took only ten minutes (Lenovo W541 laptop on high performance).

\subsubsection{Haskell calculations}

Because Mathematica is proprietary, we decided to also provide a Haskell implementation.
It uses the \href{https://hackage.haskell.org/package/astro}{Astro package} to find the position of the sun.


\color{black}
\subsection*{Sampling a Markov model}

Given a given Markov chain $\lambda$ over some state space $\set{S}$, the following procedure generates a sequence of states $\vec{s} = s[0] s[1] s[2] \cdots$
\begin{itemize}
\item[] manually initialize $s[0] \in \set{S}$
\item[] if $s[t-1] = s_i$, \emph{draw} $s[t] = s_j$ according to $\cprob{s_j}{s_i}$, that is select
\begin{equation*}
s_j \sim \bigl( \mat{P}_\lambda \bigr)_{ji}
\end{equation*}
\end{itemize}
where $\mat{P}_\lambda$ is the state transition probability matrix of the process $\lambda$. \\[1ex]

Using matrix $\mat{P}_{\lambda_1}$, generate a sequence $\vec{s}$ of $200$ states where $s[0] = A$. That is, replace the dots in the following expression by $199$ symbols sampled according to the above algorithm.
\color{blue}
%%%%%
%%%%% enter your answer after the letter 'A'
%%%%%
\\\\
$ \vec{s} = $ A A B B A A B A B B C A A C A A A C C B B A C C A B C A B A B A B C A B C A B A C A A A B C C B C A B C A A A A A B C C A B B B C A B A B C C C C A C C A B C A B C A B B C C A A B B C C B C C A B C B A B C A C C A B C C B C B B C A A B C B C B B A A B C A A B C A C C A B B C C A B B C A B B A C C C B C A C A B C A B C A C B A B A B B C B A B C C B C B C A A B B C A A A B C A B C C A C A A A B A B
\\
%%%%%
%%%%%
%%%%%
\color{black}

Using matrix $\mat{P}_{\lambda_1}$, generate $n=1000$ sequences $\vec{s}_l$ of length $200$ where $s_l[0] = A$ for all $l$. Compute their average log-likelihood
\color{blue}
%%%%%
%%%%% enter your answer after the '=' sign
%%%%%
\begin{equation*}
\frac{1}{n} \sum_{l=1}^n  \mathcal{L}\bigl( \vec{s}_l \bigr) = -205.06
\end{equation*}
%%%%%
%%%%%
%%%%%
\color{black}


\vspace{1em}
%\pagebreak
\section*{APPENDIX}
\subsubsection*{Part A}
In this part, we develop the details of the approximate solution
of the recursive equation defined in Section~\ref{sec-analyses}.

\[ f(N) = \frac{N}{2} + 2f(\frac{N}{2}) \]

Developing the second term of this equation one step, we obtain:

\[ f(N) = \frac{N}{2} + 2f(\frac{N}{2}) = \]
\[ = \frac{N}{2} + 2(\frac{N}{4} + 2f(\frac{N}{4})) =\]
\[ = \frac{N}{2} + \frac{N}{2} + 4f(\frac{N}{4}) =\]
\[ = 2\frac{N}{2} + 4f(\frac{N}{4})\]

Developing the second term of this equation one more step, we obtain:

\[ f(N) = 2\frac{N}{2} + 4f(\frac{N}{4}) = \]
\[ = 2\frac{N}{2} + 4(\frac{N}{8} + 2f(\frac{N}{8})) =\]
\[ = 2\frac{N}{2} + \frac{N}{2} + 8f(\frac{N}{8}) =\]
\[ = 3\frac{N}{2} + 8f(\frac{N}{8}) \]

After developing the second term $p-1$ times, we obtain:

\[ f(N) = p\frac{N}{2} + 2^pf(\frac{N}{2^p})\]

When $p = n = \mathsf{lb}~N$, this equation turns into:

\[ f(N) = \frac{N}{2}\mathsf{lb}~N + Nf(1) = \frac{N}{2}\mathsf{lb}~N\]

\subsubsection*{Part B}
In this part, we develop the details of the approximate solution
of the recursive equation defined in Section~\ref{sec-analyses}.

\[ f(N) = \left\{ \begin{array}{ll}
    N-1 & \mbox{if $N \le K$} \\
    \frac{N}{2} + 2f(\frac{N}{2}) &\mbox{otherwise}
  \end{array} \right. \]

Since the second equation is the same as for the basic technique,
developing the second equation $p-1$ times, we again obtain:

\[ f(N) = p\frac{N}{2} + 2^pf(\frac{2^n}{2^p})\]

When $p = n - k = \mathsf{lb}~\frac{N}{K}$ we get:

\[ f(N) = \frac{N}{2}\mathsf{lb}~\frac{N}{K} + \frac{N}{K}f(K)\]

Substituting $K-1$ for $f(K)$ and factoring out $N$, we obtain:

\[ f(N) = N(\frac{1}{2}\mathsf{lb}~\frac{N}{K} + \frac{K - 1}{K})\]

Or:

\[ f(N) = N(1 - \frac{1}{K} + \frac{1}{2}\mathsf{lb}~\frac{N}{K})\]

Clearly, the term $\frac{1}{K}$ can be ignored, giving:

\[ f(N) \approx N(1 + \frac{1}{2}\mathsf{lb}~\frac{N}{K})\]

\subsubsection*{Part C}
In this part, we develop the details of the approximate solution
of the recursive equation defined in Section~\ref{sec-analyses}.

\[ f(N) = \left\{ \begin{array}{ll}
                    0 & \mbox{if $N = 1$} \\
                    N - \frac{N}{M} + Mf(\frac{N}{M}) &\mbox{otherwise}
                  \end{array} \right. \]

Solving as before, after developing the last term once, we obtain:

\[ f(N) = N - \frac{N}{M} + Mf(\frac{N}{M}) = \]
\[ = N - \frac{N}{M} + M(\frac{N}{M} - \frac{N}{M^2} + Mf(\frac{N}{M^2})) = \]
\[ = N - \frac{N}{M} + N - \frac{N}{M} + M^2f(\frac{N}{M^2}) = \]
\[ = 2(N - \frac{N}{M}) + M^2f(\frac{N}{M^2}) \]

Developing the last term a second time, we obtain:

\[ f(N) = 2(N - \frac{N}{M}) + M^2f(\frac{N}{M^2}) = \]
\[ = 2(N - \frac{N}{M}) + M^2(\frac{N}{M^2} - \frac{N}{M^3} + Mf(\frac{N}{M^3})) = \]
\[ = 2(N - \frac{N}{M}) + N - \frac{N}{M} + M^3f(\frac{N}{M^3}) = \]
\[ = 3(N - \frac{N}{M}) + M^3f(\frac{N}{M^3}) \]

After developing the last term $p-1$ times, we obtain:

\[ f(N) = p(N - \frac{N}{M}) + M^pf(\frac{N}{M^p}) = \]

Setting $p = \frac{n}{m} = \frac{\mathsf{lb}~N}{\mathsf{lb}~M}$ so
that $M^p = N$, we get:

\[ f(N) = \frac{\mathsf{lb}~N}{\mathsf{lb}~M}(N - \frac{N}{M}) + Nf(1)
   = \frac{\mathsf{lb}~N}{\mathsf{lb}~M}(N - \frac{N}{M}) \]

Factoring out $N$, we obtain:

\[ f(N) = N(1 - \frac{1}{M})\frac{\mathsf{lb}~N}{\mathsf{lb}~M} \]

and thus:

\[ F(N) = N(1 + (1 - \frac{1}{M})\frac{\mathsf{lb}~N}{\mathsf{lb}~M}) \]

and again:

\[ F(N) \approx N(1 + \frac{\mathsf{lb}~N}{\mathsf{lb}~M}) \]


\subsection{Classical principal component analysis}

\subsubsection{Introduction}

Principal component analysis takes a dataset \(X\) with \(m\) variables and returns a principal component matrix \(A\) with size \(m\times k\).

Each new dimension is a linear function of the existing data. \(Z=XA\).

Each dimension in uncorrelated, and ordered, in order of descending explanation of variability.

The problem of principal component analysis is to find these weightings \(A\).

\subsubsection{Classical PCA}

We take the first \(k\) eigenvectors of the covariance matrix, ordered by eigenvalue.

\subsubsection{Getting the eigenvectors using SVD}

We can decompose \(X=U\Sigma A^T\).

We can take the eigenvectors from \(A\).

\subsubsection{Choosing the number of dimension}

We can choose \(k\) such that a certain percentage of the variance is retained.


\subsection{Real-valued functions}\label{subsec:real_valued_functions}

\begin{definition}\label{def:epigraph}
  Let \( X \) be an arbitrary set. The \term{epigraph} of the function \( f: X \to \BbbR \) is defined as
  \begin{equation*}
    \epi f \coloneqq \{ (x, r) \in X \times \BbbR \colon r \geq f(x) \},
  \end{equation*}
\end{definition}


\section*{Chapter 11: Permutation Algorithms}

\paragraph{Exercise 11.1}
Prove \textsl{Cayley's theorem}: Any group $G$ is isomorphic to a subgroup of
the symmetric group on $G$, \Sym{G}.

\begin{proof}

For any $a \in G$, consider the following function $F_a: G \rightarrow G$:
$$F_a(x) = a x$$
\begin{itemize}
    \item $F_a$ is one-to-one, since $F_a(x) = a x = a y = F_a(y)$ implies
    that $x = y$ (left multiplying by \Inv{a}).
    \item $F_a$ is onto: for a given $y \in G$, $F_a(\Inv{a} y) = a (\Inv{a} y)
    = (a \Inv{a}) y = y$.
\end{itemize}
Thus, $F_a$ is a bijection on $G$, which in turn means that $F_a$ is a permutation
of the elements in $G$. Thus, $S = \{ F_a \, / \, a \in G \}$ is a subset of \Sym{G}.
Moreover, $S$ is a subgroup of \Sym{G}:
\begin{itemize}
    \item $S$ contains the identity permutation: $F_{\Id{G}}(x) = \Id{G} \, x = x$.
    \item $S$ is closed by composition: $(F_a \circ F_b)(x) = F_a(F_b(x)) = a (b x)
    = (a b) x = F_{ab}(x)$.
    \item $S$ is closed by inverses: $\Inv{F_a} = F_{\Inv{a}}$, since
    $(F_a \circ F_{\Inv{a}})(x) = a (\Inv{a} x) = x$.
\end{itemize}
Let $\mathcal{F} : G \rightarrow S$  be a function defined as follows:
$$\mathcal{F}(a) = F_a$$
Then, $\mathcal{F}$ is a group isomorphism from $G$ to $S$:
\begin{itemize}
    \item $\mathcal{F}(ab) = F_{ab} = F_a \circ F_b =
    \mathcal{F}(a) \circ \mathcal{F}(b)$, using that $S$ is closed by composition.
    \item $\mathcal{F}$ is one-to-one, since $\mathcal{F}(a) = \mathcal{F}(b)$
    implies that $F_a(x) = a x = b x = F_b(x)$, and so $a = b$ after right
    multiplying by \Inv{x}.
    \item $\mathcal{F}$ is onto, since for a given $F_a \in S$, $\mathcal{F}(a) = F_a$.
\end{itemize}
\end{proof}

\paragraph{Exercise 11.2}
What is the order of $S_n$?

\begin{proof}[Answer]
$S_n$ contains every permutation of $\{1,\dots,n\}$. Since there are $n!$ of
them, the order of $S_n$ is $n!$.
\end{proof}

\paragraph{Exercise 11.3}
Prove that, if $n > 2$, $S_n$ is not abelian.

\begin{proof}
Let $P_1 = (2 \, 1 \, 3 \, \dots \, n)$ and let
$P_2 = (3 \, 1 \, 2 \, \dots \, n)$. Then,
\begin{itemize}
    \item $P_1 \circ P_2 = (1 \, 3 \, 2 \, \dots \, n)$, and
    \item $P_2 \circ P_1 = (3 \, 2 \, 1 \, \dots \, n)$.
\end{itemize}
Since $P_1 \circ P_2 \neq P_2 \circ P_1$, $S_n$ is not commutative.
\end{proof}

\paragraph{Exercise 11.9}
Prove that if a rotation of $n$ elements has a trivial cycle, then it has $n$
trivial cycles.

\begin{proof}
Let $\rho$ be an $n$ by $k$ rotation. That is,
$$\rho = (k \MOD{n}, k+1 \MOD{n}, \dots, k+n-1 \MOD{n})$$
Suppose that $\rho$ has a trivial cycle. This means that there is an
$0 \leq i < n$ such that $i = k + i \MOD{n} \Rightarrow
k = 0 \MOD{n}$. Then,
\begin{eqnarray*}
    \rho &=& (0 \MOD{n}, 1 \MOD{n}, \dots, n-1 \MOD{n}) \\
         &=& (0, 1, \dots, n-1)
\end{eqnarray*}
which means that $\rho$ does not move any element.
\end{proof}

\paragraph{Exercise 11.11}
How many assignments does 3-reverse rotate perform?

\begin{proof}[Answer]
Call $\alpha_r(f,m,l)$ the number of assignments we seek.
First, let $\sigma(f,l)$ be the number of swaps performed by \Rev{f}{l}. Since this
function swaps element $f$ with element $l-1$, element $f+1$ with element
$l-2$, and so on, we have that
$$\sigma(f, l) = \floor{\frac{l-f}{2}}$$
Then, if we note the number of swaps performed by 3-reverse rotate by
$\sigma_r(f,m,l)$,
\begin{eqnarray*}
\sigma_r(f,m,l) &=& \floor{\frac{m-f}{2}} + \floor{\frac{l-m}{2}} +
                    \floor{\frac{l-f}{2}} \\
             &\leq& \frac{m-f}{2} + \frac{l-m}{2} + \frac{l-f}{2} \\
                &=& \frac{(m-f) + (l-m) + (l-f)}{2} \\
                &=& \frac{2l - 2f}{2} \\
                &=& l - f \\
                &=& n
\end{eqnarray*}
being $n$ the number of elements being rotated. Thus, at most $n$ swaps are
performed by 3-reverse rotate (and at least $n-1$, following a similar
approach). Since each swap takes three assignments,
$$3(n-1) \leq \alpha_r(f,m,l) \leq 3n$$
\end{proof}


%!TEX root=ClassNotes.tex

\section{Completeness of the Real Numbers}
Our next goal is to prove the IVT for continuous functions.

\begin{theorem}[Intermediate Value Theorem]
	If $f$ is a continuous function on $[a,b]$ and $k$ is a real number lying between $f(a)$ and $f(b)$ then there exists a real number $ c $ lying between $a$ and $b$ such that $f(c) = k$.
\end{theorem}

The proof of this trivial looking theorem relies on a defining axiom of the system of real numbers called {\it completeness} which we'll now try to understand. For this we'll need to rigorously define the simple notions of {\it min} and {\it max}.\\

For a non-empty set $S$ of real numbers, it's maximum element can be defined as follows.
\begin{definition}
	\label{def:maximum}
	$\max S$ is the element $a \in S$ such that $a \ge x$ for all elements $x \in S$.
\end{definition}
This definition exposes a subtle shortcoming of $\max S$. It doesn't exist even for {\it nice} sets.
\begin{exercise}
	What is $\max \: (0,1)$? Make sure that your answer agrees with Definition \ref{def:maximum}.
\end{exercise}

There is a more intricately defined number, called the {\bf supremum}, that we can associate to a set of real numbers which also captures the {\it largest-element} property. For sets containing infinitely many elements, the supremum, rather than the maximum gives us the correct {\it largest-element}.

\begin{definition}
	An {\bf upper bound} of a set $S$ is any real number $a$ satisfying $a \ge x$ for all elements $x \in S$. (Note that $a$ does not have to be in $S$.)
\end{definition}

\begin{exercise}
	\label{q:upper_bounds}
	Upper bounds are not unique. Find all the possible upper bounds on the following sets. (No proofs needed. Make sure you've found {\it all} the upper bounds.)
	\begin{enumerate}
		\item $[0,1]$
		\item $(0,1)$
		\item $\R$
		\item $\{ 1/n :$ where $n$ is a positive integer$\}$
		\item The set of rational numbers $x$ satisfying $x^2 < 2$
		\item The set of irrational numbers in interval $[0,1]$.
	\end{enumerate}
\end{exercise}

\begin{definition}
	The {\bf supremum} or the {\bf least upper bound} of a set $S$ of real numbers, denoted $\sup S$, is the smallest real number $a$ such that $a$ is an upper bound of $S$.
\end{definition}

\begin{exercise}
	Find the supremum of the sets in Exercise \ref{q:upper_bounds}.
\end{exercise}

\begin{definition}
	We say that a set $S$ is {\bf bounded from above} if it has at least one upper bound i.e. there exists a real number $a$ such that $a \ge x$ for all elements $x \in S$.
\end{definition}

\begin{exercise}
	Which of the sets in Exercise \ref{q:upper_bounds} are bounded from above?\\
\end{exercise}

We can now state the {\bf Completeness of the Real Numbers}:
\begin{theorem}
	If a non-empty set $S$ is bounded from above then $S$ has a supremum.
\end{theorem}

Depending on exactly how the real numbers are constructed, this is either a theorem or an axiom. This theorem/axiom is the precise way of saying that there are no holes in the real line. (Read the Wikipedia article titled {\it Completeness of the real numbers}.) We'll next restate the property of being a supremum using our (favorite) language of $\epsilon$'s and $\delta$'s.

\begin{exercise}
	\label{q:supremum_closeness}
	Let $S$ be a non-empty set that is bounded from above and let $L = \sup S$.
	\begin{enumerate}
		\item Show that for every $ \delta > 0 $, there exists an element $x \in S$ such that $L - x < \delta$. \hint{Proof by Contradiction.}
		\item Let $a$ be an upper bound of $S$. Prove that, if for every $ \delta > 0 $, there exists an element $x \in S$ such that $a - x < \delta$, then $a = L$. \hint{Prove by Contrapositive.}
	\end{enumerate}
\end{exercise}
Thus we're saying that elements of $S$ get infinitely close to the supremum.

\subsection*{Optional Problems}

Similarly to the supremum we can define the {\bf infimum} of a set $S$, denoted $S$, which corresponds to $\min S$.

\begin{exercise}$ $
	\begin{enumerate}
		\item Define $\inf S$.
		\item Find the infimum of all the sets in Exercise \ref{q:upper_bounds}.
		\item Show that \begin{align*}
			      \inf S = - \sup (-S)
		      \end{align*} (if they exist), where $-S$ is the set containing elements of the form $-x$ where $x \in S$.
		\item State the analogue of Exercise \ref{q:supremum_closeness} for $\inf S$.
	\end{enumerate}
\end{exercise}

\newpage
\subsection{Intermediate Value Theorem}

We'll now prove the Intermediate Value Theorem. This is our first encounter with a complicated proof which requires multiple steps. You probably won't see the full picture until you completely write down the proof yourself. It'll help you a lot to draw pictures to keep a track of all the variables. \\\\
\begin{tabular}{|p{\textwidth}|}
  \hline \\{\it Once you have solved all the exercises in the proof of Theorem \ref{theorem:IVT}, submit your final solution as one single logically coherent proof of the IVT, also include all the text that is in between the exercises, so that you yourself see how all the pieces fit together. This will also teach you to write complex proofs.}\\\\
  \hline
\end{tabular}

\vspace{1em}
We'll need the following proposition from the previous section, which says that the elements of a set $S$ are infinitely close to it's supremum.
\begin{prop}
  \label{thm:sup_lemma}
  For a set $S$, if $L = \sup S$, then
  for every $ \delta > 0 $, there exists an element $x \in S$ such that $L - x < \delta$.
\end{prop}

\begin{theorem}[Intermediate Value Theorem]
	\label{theorem:IVT}
	Let $f$ be a continuous function on $[a,b]$ and let $k$ be a real number satisfying
	\begin{align*}
		f(a) < k < f(b).
	\end{align*}
	Then, there exists a real number $c$ satisfying $a < c < b$ such that
	\begin{align*}
		f(c) = k.
	\end{align*}
\end{theorem}

\begin{proof}[Proof of \ref{theorem:IVT}]

	We'll first prove the following slightly easier version.
	\begin{prop}
		\label{theorem:easy_IVT}
		If $g$ is a continuous function on $[a,b]$ satisfying
		\begin{align*}
			g(a) < 0< g(b)
		\end{align*}
		then, there exists a real number $c$ satisfying $a < c < b$ such that
		\begin{align*}
			g(c) = 0.
		\end{align*}
	\end{prop}
	\begin{proof}[Proof of \ref{theorem:easy_IVT}]
		Let $g,a,b$ be as in the statement of the Proposition. We'll construct a number $c$ that satisfies $a < c < b$ and $g(c) = 0$.

		Let $S$ be a set defined as follows,
		\begin{align*}
			S = \{ x \mid a \le x \le b \mbox{ and } g(x) \le 0\}.
		\end{align*}
		\begin{exercise}
			Argue that $S$ is non-empty and is bounded from above.
		\end{exercise}
		Hence, $S$ has a supremum by the {\it Completeness Property of Real Numbers}. Denote this supremum by $c$. We'll prove that\\

    \begin{indentPara}
      {\bf Claim: } $g(c) = 0$.\\
    \end{indentPara}

		We'll prove this claim by contradiction. Suppose on the contrary that $g(c) \neq 0$. There are two possible cases:\\
		\begin{indentPara}
      \begin{description}
  			\item[Case 1:] $g(c) < 0$
  			\item[Case 2:] $g(c) > 0$\\
  		\end{description}
    \end{indentPara}
		Suppose {\bf Case 1} is true i.e. $g(c) < 0$.
		\begin{exercise}
			\begin{enumerate}
				\item Use the formal definition of continuity (from right) of $g$ to prove that there exists an $x > c$ such that $f(x) < 0$.\hint{Use $\epsilon = -g(c)/2$.}
				\item Argue that this contradicts one of your assumptions, hence $g(c)$ cannot be less than 0.
			\end{enumerate}
		\end{exercise}
		\noindent Suppose {\bf Case 2} is true i.e. $g(c)> 0$.
		\begin{exercise}
			\begin{enumerate}
				\item Use the formal definition of continuity (from left) of $g$ to prove that there exists a $\delta$ such that, for every $x$, if $c - x < \delta$ then $g(x) > 0$.\hint{Use $\epsilon = g(c)/2$.}
  				\item For this particular $\delta$, prove that, for every $x$ in $S$, $c - x > \delta$.
				\item Argue that this contradicts Proposition \ref{thm:sup_lemma}, hence $g(c)$ cannot be greater than 0.
			\end{enumerate}
		\end{exercise}
		Thus we've contradicted both the possible cases, which completes the proof by contraction and proves the Claim that $g(c) = 0$.

    By construction $c$ satisfies $ a < c < b$. So we've constructed a $c$ such that $a < c < b$ and $g(c) = 0$ which completes the proof of Proposition \ref{theorem:easy_IVT}.
	\end{proof}
	\begin{exercise}
		Going back to $f$, use the function $g(x) = f(x) - k$ in Proposition \ref{theorem:easy_IVT} to complete the proof of Theorem \ref{theorem:IVT}.
	\end{exercise}
\end{proof}

Both the hypotheses are necessary for the IVT to be true.
This means that we must be crucially using these hypotheses in our proof somewhere and the proof should fail if we do not have these assumptions. It is clear where we are using continuity, the other one is more subtle.
\begin{exercise}$ $
	\begin{enumerate}
		\item Find the {\it exact} argument in your proof of Proposition \ref{theorem:easy_IVT} which fails if we assume $0 < g(a) < g(b)$.
		\item Find the {\it exact} argument in your proof of Proposition \ref{theorem:easy_IVT} which fails if we assume $ g(a) < g(b) < 0$.
	\end{enumerate}
\end{exercise}


\section{Roots of complex numbers}

\begin{outcome}
  \begin{enumerate}
  \item Understand De Moivre's theorem and be able to use it to find
    the roots of a complex number.
  \end{enumerate}
\end{outcome}

A fundamental identity is the
\index{De Moivre's theorem} formula of De Moivre with which we begin this section.

\begin{theorem}{De Moivre's theorem}{de-moivre-theorem}
For any positive integer $n$, we have
\begin{equation*}
(e^{i \theta})^n = e^{i n \theta}
\end{equation*}
Thus for any real number $r>0$ and any positive integer $n$, we have:
\begin{equation*}
(r(\cos \theta+i\sin \theta)) ^{n}=r^{n}(\cos n \theta +i\sin
n\theta)
\end{equation*}
\end{theorem}

\begin{proof}
 The proof is by induction on $n$. It is clear the formula holds if $n=1$. Suppose it is true
for $n$. Then, consider $n+1$.
\begin{equation*}
(r(\cos \theta+i\sin \theta)) ^{n+1}=(r(\cos
\theta+i\sin \theta)) ^{n}(r(\cos \theta+i\sin \theta))
\end{equation*}
which by induction equals
\begin{eqnarray*}
&=&r^{n+1}(\cos n\theta+i\sin n\theta) (\cos \theta+i\sin \theta) \\
&=& r^{n+1}((\cos n\theta\cos \theta-\sin n\theta\sin \theta) +i(\sin
n\theta\cos \theta+\cos n\theta\sin \theta))\\
&=&r^{n+1}(\cos (n+1) \theta+i\sin (n+1) \theta)
\end{eqnarray*}
by the formulas for the cosine and sine of the sum of two angles.
\end{proof}

The process used in the previous proof, called {\em mathematical
induction} is very powerful in Mathematics and Computer Science
and explored in more detail in the Appendix.

Now, consider a corollary of Theorem~\ref{thm:de-moivre-theorem}.

\begin{corollary}{Roots of complex numbers}{roots-complex-numbers}
Let $z$ be a non-zero complex number.
\index{complex numbers!roots}Then there are always exactly $k$ many  $k\th$
roots of $z$ in $\C$.
\end{corollary}

\begin{proof}
Let $z=a+bi$ and let $z=\abs{z}(\cos
\theta+i\sin \theta) $ be the polar form of the complex number. By De Moivre's
theorem, a complex number
\begin{equation*}
w= r e^{i \alpha} = r(\cos \alpha +i\sin \alpha)
\end{equation*}
is a $k\th$ root of $z$ if and only if
\begin{equation*}
w^k = (r e^{i \alpha})^k = r^k e^{ik\alpha} = r^{k}(\cos k\alpha +i\sin k\alpha) =\abs{z}(\cos \theta+i\sin \theta)
\end{equation*}
This requires $r^{k}=\abs{z}$ and so $r=\abs{z}^{1/k}$. Also, both $\cos (k\alpha) =\cos \theta$ and
$\sin (k\alpha) =\sin \theta$. This can only happen if
\begin{equation*}
k\alpha =\theta+2 \ell \pi
\end{equation*}
for $\ell$ an integer. Thus
\begin{equation*}
\alpha =
\frac{\theta+2 \ell \pi }{k},\; \ell = 0, 1, 2,\ldots, k-1
\end{equation*}
and so the $k\th$ roots of $z$ are of the form
\begin{equation*}
\abs{z}^{1/k}\paren{\cos \paren{\frac{\theta+2 \ell \pi }{k}}
+i\sin \paren{\frac{\theta+2 \ell \pi }{k}}} ,\;\ell = 0, 1, 2,\ldots, k-1
\end{equation*}
Since the cosine and sine are periodic of period $2\pi$, there are exactly $
k$ distinct numbers which result from this formula.
\end{proof}

The procedure for finding the $k$ $k\th$ roots of $z \in \C$ is as follows.

\begin{procedure}{Finding roots of a complex number}{finding-kth-roots}
Let $w$ be a complex number. We wish to find the $n\th$ roots of $w$, that is all $z$ such that $z^n = w$.

There are $n$ distinct $n\th$ roots and they can be found as follows:.

\begin{enumerate}
\item Express both $z$ and $w$ in polar form $z=re^{i\theta}, w=se^{i\phi}$. Then $z^n = w$ becomes:
\[
(re^{i\theta})^n = r^n e^{i n \theta} = se^{i\phi}
\]
We need to solve for $r$ and $\theta$.
\item Solve the following two equations:
\begin{eqnarray*}
r^n &=& s
\end{eqnarray*}
\begin{eqnarray}
e^{i n \theta} &=& e^{i \phi}
\label{roots-eqns}
\end{eqnarray}
\item The solutions to $r^n = s$ are given by $r = \sqrt[n]{s}$.

\item The solutions to $e^{i n \theta} = e^{i \phi}$ are given by:
\[
n\theta = \phi + 2\pi \ell,  \; \mbox{for} \; \ell = 0,1,2,\ldots, n-1
\]
or
\[
\theta = \frac{\phi}{n} + \frac{2}{n} \pi \ell, \; \mbox{for} \; \ell = 0,1,2,\ldots, n-1
\]
\item
Using the solutions $r, \theta$ to the equations given in (\ref{roots-eqns})
construct the $n\th$ roots of the form $z = re^{i\theta}$.
\end{enumerate}
\end{procedure}

Notice that once the roots are obtained in the final step, they can then be converted to standard form if necessary. Let's consider an example of this concept. Note that according to Corollary~\ref{cor:roots-complex-numbers},
there are exactly $3$ cube roots of a complex number.

\begin{example}{Finding cube roots}{cube-roots}
Find the three cube roots of $i$. In other words find all $z$ such that $z^3 = i$.
\end{example}

\begin{solution}
First, convert each number to polar form: $z = re^{i\theta}$ and $i = 1 e^{i \pi/2}$. The equation now becomes
\[
(re^{i\theta})^3 = r^3 e^{3i\theta} = 1 e^{i \pi/2}
\]
Therefore, the two equations that we need to solve are $r^3 = 1$ and $3i\theta = i \pi/2$. Given that $r \in \R$ and $r^3 = 1$ it follows that $r=1$.

Solving the second equation is as follows. First divide by $i$. Then, since the argument of $i$ is not unique we write $3\theta = \pi/2 + 2\pi\ell$ for $\ell = 0,1,2$.
\begin{eqnarray*}
3\theta &=& \pi/2 + 2\pi\ell \; \mbox{for} \; \ell = 0,1,2 \\
\theta &=& \pi/6 + \frac{2}{3} \pi\ell \; \mbox{for} \; \ell = 0,1,2
\end{eqnarray*}

For $\ell = 0$:
\[
\theta = \pi/6 + \frac{2}{3} \pi (0) = \pi/6
\]

For $\ell = 1$:
\[
\theta = \pi/6 + \frac{2}{3} \pi(1) = \frac{5}{6} \pi
\]

For $\ell = 2$:
\[
\theta = \pi/6 + \frac{2}{3} \pi(2) = \frac{3}{2} \pi
\]

Therefore, the three roots are given by \[
1e^{i \pi/6}, 1e^{i \frac{5}{6}\pi}, 1e^{i \frac{3}{2}\pi}
\]

Written in standard form, these roots are, respectively,
\[
\frac{\sqrt{3}}{2} + i \frac{1}{2}, -\frac{\sqrt{3}}{2} + i \frac{1}{2}, -i
\]

\end{solution}

The ability to find $k\th$ roots can also be used to factor some
polynomials.

\begin{example}{Solving a polynomial equation}{solving-polynomial}
Factor the polynomial $x^{3}-27$.
\end{example}

\begin{solution}
First find the cube roots of 27. By the above procedure
\index{polynomials!factoring}, these cube roots
are $3,3\paren{\displaystyle
\frac{-1}{2}+i\displaystyle\frac{\sqrt{3}}{2}}$, and $3\paren{
\displaystyle\frac{-1}{2}-i\displaystyle\frac{\sqrt{3}}{2}}$. You may wish to verify
this using the above steps.

Therefore, $x^{3}-27 =$
\begin{equation*}
 (x-3) \paren{x-3\paren{\frac{-1}{2}+i\frac{\sqrt{3}}{2}}
} \paren{x-3\paren{\frac{-1}{2}-i\frac{\sqrt{3}}{2}}}
\end{equation*}
Note also $\paren{x-3\paren{\frac{-1}{2}+i\frac{\sqrt{3}}{2}}}
\paren{x-3\paren{\frac{-1}{2}-i\frac{\sqrt{3}}{2}}}
=\allowbreak x^{2}+3x+9$ and so
\begin{equation*}
x^{3}-27=(x-3) (x^{2}+3x+9)
\end{equation*}
where the quadratic polynomial $x^{2}+3x+9$ cannot be factored without using
complex numbers.
\end{solution}

Note that even though the polynomial $x^{3}-27$ has all real coefficients,
it has some complex zeros, $3\paren{\displaystyle
\frac{-1}{2}+i\displaystyle\frac{\sqrt{3}}{2}}$, and $3\paren{
\displaystyle\frac{-1}{2}-i\displaystyle\frac{\sqrt{3}}{2}}$.
These zeros are complex conjugates of each other. It is always the case that if a polynomial has real
coefficients and a complex root, it will also
have a root equal to the complex conjugate.


\section{Induction}

Induciton is an important tool in computer science, as it provides us one with
an procedure to show correctness of our algorithms, and two can be directly
translated to recursion.

\subsection{Definition and Examples}

\begin{definition*}

\textbf{Induction} is a mathematical proof technique. Formally, suppose we have
some program $P$, which has input $n$. Furthermore, suppose we can show that:
\begin{enumerate}[1)]
\item $[$Base Case$]$ $P(0)$ is provably correct.
\item $[$Inductive Step$]$ Suppose that $P(0), \dots, P(k)$ is correct for some
fixed $k \in \N$.  Then we can show that $P(k+1)$ is provably correct.
\end{enumerate}
Then it must be true that $\forall n, P(n)$ is correct. (Note that we don't need
for \textit{zero} that $P(0)$ is provably correct, but rather that $P(n_0)$ is
provably correct, for some fixed $n_0$. We will typically take $n_0 = 0, 1$.)

Conceptually you can envision this as a staircase. Suppose that you can stand at
the bottom of the staircase, and that if you're on step $k$, you can move to
step $k+1$. Then we know that $\forall n$, we can stand on step $n$.

\end{definition*}

Take the following as applications of induction.

\begin{itemize}

\item Prove that $1 + 2 + \dots + n = \frac{n(n+1)}{2}$.

\begin{proof}

Clearly for $n = 1$, we have that $1 = \frac{1(1+1)}{2} = 1$. Now suppose that
$1 + 2 + \dots + n = \frac{n(n+1)}{2}$. We want to show that $1 + \dots + n+1 =
\frac{(n+1)(n+2)}{2}$. Notice that:
$$
\sum_{i=1}^{n+1} i = n+1 + \sum_{i=1}^{n}i = n+1 + \frac{n(n+1)}{2}
$$
The last equality being true by our assumption. Now notice we can write
$n(n+1)/2 = (n^2 + n)/2$, and we can write $n+1 = \frac{2n + 2}{2}$. Therefore
adding the two expresions together results in $(n^2 + 3n + 2)/2 =
(n+1)(n+2)/2$.

\end{proof}

\item We call a \textbf{perfect binary tree} a tree structure such that each
non-leaf vertex has exactly two children. So for example, a perfect binary tree
of height $2$ would be:

\Tree [.a [.b d e ] [.c f g ] ]

Prove that a perfect binary tree of height $n$ has exactly $2^n - 1$ vertices.

\begin{proof}

First suppose $n = 1$. Clearly a tree of height $1$ has only one vertex, and
indeed $2^1 - 1 = 1$.

Now suppose a tree of height $n$ has $2^n - 1$ vertices. We want to show that a
tree of height $n+1$ must have $2^{n+1} - 1$ vertices. Consider the root of such
a tree (in the example above it's vertex $a$). Notice that it's left subchild is
a perfect binary tree of height $n$, and so is it's right subtree. Therefore,
counting the verteces in each of those subtrees using our supposition that a
tree of height $n$ has $2^n - 1$ vertices gives us a count of $2(2^n - 1)$ for
the vertices under the root. Therefore a tree of height $n+1$ has $2(2^n - 1) +
1$ vertices ($+1$ for the root). Notice that:
$$
2(2^n - 1) + 1 = 2^{n+1} -2  + 1 = 2^{n+1} - 1
$$
and therefore we've shown our inductive step.

Thus, a perfect binary tree of height $n$ has $2^n - 1$ vertices.

\end{proof}

\end{itemize}

\subsection{Towers of Hanoi and thinking recursively}

The reason why we're reviewing induction, is that induction is very (read:
\textit{\textbf{VERY}}) useful for proving correctness for recursive functions.
You will use it in literally every single part of the rest of the course so it
pays to get it down well. A classical example is Towers of Hanoi.

\begin{definition*}

Towers of Hanoi is a mathematical puzzle. Suppose we have three rods, and a
number of disks of different sizes $1 \to n$. which can slide onto any rod. The
puzzle starts with the disks all arranged on the first rod in descending
order such that $n$ is at the bottom, and $1$ is at the top. For example, for $n
= 3$, the beginning configuration is:

$$
\begin{matrix}
-   & | & | \\
--  & | & | \\
--- & | & | \\
A & B & C
\end{matrix}
$$

The goal of this game is to move the entire stack of rings from pole $A$ to pole
$B$, while obeying the constraints:

\begin{itemize}

\item You may move one disk at a time with the command $Move(arg1, arg2)$ where
this moves the disk on top of pole $arg1$ to the top of pole $arg2$.

\item No larger disk may be placed on top of a smaller disk.

\end{itemize}

\end{definition*}

We want to write an algorithm to solve this puzzle, but to also do so with a
minimal amount of $Move$ calls.


\subsection{Sample variance}

We previously defined the population variance as \(\sigma^2=E[(X-\mu)^2]\).

We define the sample variance as \(\sigma^2=\dfrac{1}{n}\sum_i(x_i-\bar x)^2\).

We can calculate this using matrices:

\(M=X-\bar x\)

\(\sigma^2=\dfrac{1}{n}M^TM\).

\subsubsection{Centred variance}

If \(\bar x =0\) then:

\(\sigma^2=\dfrac{1}{n}X^TX\).


%!TEX root =  ../main.tex

\subsection{Properties of Derivatives}

\objective{Calculate derivates using the Power, Product, and Quotient Rules.}

\index{Derivative!properties}
The derivative of a sum of two functions is the sum of the derivatives of each function.

The derivative of a constant times a function is the constant times the derivative of the function.

The derivative of a constant is 0.

\subsubsection{Product Rule (inductive)}
\index{Product Rule}\index{Derivative!of a product}
The following proof is very arbitrary and does not feel like something anyone would try
unprompted (especially in the second line: who would ever think of adding and subtracting the same
term in just that way?!).  We will use the following \textbf{Product Rule} some over the coming chapters,
but a full proof that you are responsible for reproducing does not come until §8.3

\begin{align*}
  (fg)'(x) & = \lim_{h\rightarrow0}\frac{f(x+h)g(x+h) - f(x)g(x)}{h} \\
  	&= \lim_{h\rightarrow0}\frac{f(x+h)g(x+h) - f(x)g(x+h) + f(x)g(x+h) + f(x)g(x)}{h} \\
	&= \lim_{h\rightarrow0}\frac{f(x+h)g(x+h) - f(x)g(x+h)}{h} + \lim_{h\rightarrow0}\frac{f(x)g(x+h)-f(x)g(x)}{h} \\
	&= \lim_{h\rightarrow0}\left[\frac{f(x+h) - f(x)}{h} \cdot g(x+h)\right] + \lim_{h\rightarrow0}\left[\frac{g(x+h)-g(x)}{h} \cdot f(x)\right] \\
	&= \lim_{h\rightarrow0}\left[\frac{f(x+h) - f(x)}{h}\right] \cdot g(x) + \lim_{h\rightarrow0}\left[\frac{g(x+h)-g(x)}{h}\right] \cdot f(x)\\
	&=f'(x)g(x) + f(x)g'(x)
\end{align*}

\subsubsection{Quotient Rule}
\index{Quotient Rule}\index{Derivative!of a quotient}
The \textbf{Quotient Rule} is presented here with a very similar arbitrary trick.  Wait to memorize the proof until chapter 8.

\begin{align*}
\frac{f}{g}' & = \lim_{h\rightarrow0}\cfrac{\frac{f(x+h)}{g(x+h)}-\frac{f(x)}{g(x)}}{h} \\
	&=  \lim_{h\rightarrow0}\frac{g(x)f(x+h)-f(x)g(x+h)}{g(x)g(x+h)h} \\
	&=  \lim_{h\rightarrow0}\frac{g(x)f(x+h)-f(x)g(x)+f(x)g(x)-f(x)g(x+h)}{g(x)g(x+h)h} \\
	&=  g(x)\left[\lim_{h\rightarrow0}\frac{1}{g(x)g(x+h)}\cdot{}\frac{f(x+h)-f(x)}{h}\right]-f(x)\left[\lim_{h\rightarrow0}\frac{1}{g(x)g(x+h)}\cdot{}\frac{g(x+h)-g(x)}{h}\right] \\
	&= \frac{g(x)f'(x) - f(x)g'(x)}{[g(x)]^2}
\end{align*}

\subsection{Power Rule}
\index{Power Rule}\index{Derivative!of a power}
The algebraic derivative of $x^n$ is $n\cdot x^{n-1}$.  While we will furnish a systematic proof
later (also in section 8.3), we have seen enough inductive
examples of the \textbf{Power Rule} in this chapter
that you are formally responsible for knowing it and using it in all cases.


\subsubsection{Isothermal Processes}
Isothermal processes are compression/expansion processes in which the temperature of the gas remains constant ($\Delta T = 0$). These processes occur very slowly. As an example, we consider a scenario where I have some gas in a cylinder (closed off at the top by a piston), where I push down on the piston very very slowly. Doing so, the gas inside the piston remains at thermal equilibrium with its surroundings. Hence, I as I do work on the gas, an equal amount of energy leaves the gas in the form of heat, and the gas inside the piston stays at the same temperature. \\
\noindent
One immediate consequence that we obtain from $\Delta E = nc_v\Delta T$ is that:
\begin{equation}
    \Delta E = 0
\end{equation}
And by the first law of thermodynamics, we have that:
\begin{equation}
    W = -Q
\end{equation}
What exactly is the amount of work done in an isothermal process? We once again return to the definition of work:
\[ W = -\int_{V_1}^{V_2}P(V)dV \]
We have to be very careful here though; in this situation, the pressure is not constant as we vary/integrate over the volume. So we have to be a bit clever, and think of a way in which we can express the pressure as a function of volume so we can carry out the integral. If your intution was "ideal gas law", then you'd be exactly right! Let us make the substition:
\[ P(V) = \frac{nRT}{V} \]
Which turns the integral into:
\[ W = -\int_{V_1}^{V_2}nRT\frac{dV}{V}\]
Throughout this process $T$ is held constant, and the amount of gas does not change, so we can take most of the terms out of the integral as a constant:
\[ W = -nRT\int_{V_1}^{V_2}\frac{dV}{V}\]
The remaining integral is straightforward:
\[ W = -nRT \left. \ln(V) \right|_{V_1}^{V_2} = -nRT\left(\ln(V_2)-\ln(V_1)\right) = nRT\left(\ln(V_1)-\ln(V_2)\right) \]
So applying some laws of logarithms, we hence obtain our expression for the work done on the gas in an isothermal process:
\begin{equation}
W = nRT\ln\left(\frac{V_1}{V_2}\right)
\end{equation}
As we found earlier, the heat flow during an isothermal process is simply the negative of this, so:
\begin{equation}
    Q = -nRT\ln\left(\frac{V_1}{V_2}\right) = nRT\ln\left(\frac{V_2}{V_1}\right)
\end{equation}
\noindent
Finally, we consider how isothermal curves show up on PV diagrams. From how the expression for the work looks (with the $\ln$ term), you may suspect that we might get a "curved" curve rather than a straight diagonal line, and you would be right. Pictured here is an isothermal expansion from initial state $P_1,V_1$ to final state $P_2,V_2$. The temperature stays constant at every point on the curve.

\begin{center}
    \begin{tikzpicture}
    \begin{axis}[
        axis x line=bottom,
        axis y line=left,
        xmin=0, xmax=10,
        ymin=0, ymax=10,
%        % (made labels more common)
%        % (because of the "sketch" type of the plot these should not be needed)
%        xlabel={Volume $(\mathrm{m}^3)$},
%        ylabel={Pressure (Pa)},
        % (changed ticks + labels to normal ticks instead of extra ticks)
        xtick={3,6},
        xticklabels={$V_1$,$V_2$},
        ytick={1.5,6},
        yticklabels={$P_2$,$P_1$},  % <-- (changed order of entries)
    ]
        % fill the area below the curve
        % (draw it first, so it is below everything else)

        % draw the dashed lines
        % (using two different approaches)
        \addplot [dashed,domain=0:3,samples=2] {6};
        \addplot [dashed,domain=0:6,samples=2] {1.5};

        \draw [dashed,thin] (axis cs:6,1.5) -- (axis cs:6,0);
        \draw [dashed,thin] (axis cs:3,6)   -- (axis cs:3,0);

        % now draw the curve
        \draw [
            fleche={0.6:black},              % <-- added
        ] (axis cs:3,6) to [bend right=30]
            % store start and end coordinates
            coordinate [pos=1] (start)
            coordinate [pos=0] (end)
        (axis cs:6,1.5);

        % draw start and end point
        \fill [radius=2pt]
            (start) circle[]
            (end)   circle[];
    \end{axis}
        \node[below] at (6.5,0) {$V$};
        \node[left] at (0,5.5) {$P$};
        \node[right] at (2.2,3.4) {$T_1$};
        \node[right] at (4.25,0.9) {$T_1$};
\end{tikzpicture}
\end{center}


\subsection{Method} \label{subsec:method_linear_regression}

Consider that we have a data set of inputs and their corresponding output. Regression is a process of arriving at a relation between the independent input variables and the dependent output variables. In the field of statistical learning, regression is considered to be a supervised learning method. This is because the model parameters generated in regression are learned from labeled data which is data that comprises both inputs and their corresponding output, i.e the labels. In this report, we assume that there are multiple input variables and a single output variable for each member of the data set.
\newline \newline
The dataset with $n$ members is defined as $\mathcal{D} := [({x}_1, y_1 ), ({x}_2, y_2 ), ..., ({x}_n, y_n )]$. Where ${x}_i$ and $y_i$  are the $i^{th}$ input and output respectively. Let, $m$ be the number of components of the input such that ${x}_i := (x_{i1}, x_{i2}, ..., x_{im})$. In linear regression we assume that the data is of the form
\begin{equation}
    {y}_i = 1 \cdot\beta_0 + x_{i1}\beta_1 + x_{i2}\beta_2 + ... + x_{im}\beta_m + \epsilon_i \label{eq:lin_reg_data}
\end{equation}
where $\beta_0, \beta_1, \dots, \beta_m$ are unknown fixed regression parameters and $\epsilon_i$ is a random error or noise process consisting of independent identically (iid) normally distributed variables with mean zero and variance $\sigma^2$ \cite{shumway_time_2017}. Equation \ref{eq:lin_reg_data} can be rewritten as $y=\beta \phi(x) + \epsilon$ with
\begin{equation}
    \phi(x):=[1, x_{i,1}, x_{i,2}, \dots, x_{i,m}].
\end{equation}
Moreover, it can be shown that $\mathbb{E}[y] = \beta \phi(x)$ and $var(y) = \sigma^2$. Therefore, $y \sim \mathcal{N}(\beta \phi(x),\,\sigma^{2})$ \cite{murphy_machine_2012}.
\newline \newline
In linear regression, we seek to fit a best-fit line between input, or some transformation of the input, and the output. This best-fit line is chosen such that it minimizes an error or a cost function. The best fit line is a function of input variables and its value is called the predicted value or the fit. We denote this with $\hat{y}$. The linear regression model has the form:
\begin{equation}
    \hat{y}_i = 1 \cdot\beta_0 + x_{i1}\beta_1 + x_{i2}\beta_2 + ... + x_{im}\beta_m
\end{equation}
The $\beta$'s are chosen such that they minimize a cost function. If this cost function is chosen to be the mean squared error (MSE) then we call the regression \textit{least squares} regression. The mean squared error is defined as
\begin{equation}
    \mathcal{C} = \frac{1}{n}\sum_{i=1}^{n}(y_i-\hat{y}_i)^2 \label{eq:cost_function_sum}
\end{equation}

By extending the input such that ${x}_i = (1_i, x_{i1}, x_{i2}, ..., x_{im})$, i.e. the first entry of $x_i$ is a vector which only contains 1, and denoting the parameter vector by ${\beta} = (\beta_0, \beta_1, ..., \beta_m)$, we can express the predicted value as an inner product such that
\begin{equation*}
    \hat{y}_i = {x_i}\cdot {\beta} = {x_i}^T{\beta}.
\end{equation*}
Moreover, we can express the set of predicted outputs as a vector which allows us to write the previous equation as a matrix-vector product:
\begin{equation}
    \hat{y} = {X}{\beta} \label{eq:lin_reg_matrix_model}
\end{equation}
where, $\hat{y}$, ${X}$ and ${\beta}$ have dimensions $n \times 1$, $n \times (m+1)$ and $(m+1) \times 1$. $X$ is called design matrix. However, such a description is only useful if the output is a linear or near-linear function of the input variables.  We can make the model more "complex" by allowing higher powers of input variables and their products to be the features. One possibility is to consider polynomial functions of the input variables. For simplicity, we consider input vectors where $m=2$ as they are the only ones relevant for this project. Then, we can express the predicted output $\hat{y}$ as a polynomial function of order $p$ such that
\begin{equation}
    \hat{y}_i = \beta_0 + \sum_{k=1}^{k=p}\sum_{l=0}^{l=p}\beta_{\frac{k(k+1)}{2}+l}x_{i1}^{p-l}x_{i2}^{l}
\end{equation}
which is equivalent to
\begin{equation}
\resizebox{.9\hsize}{!}{$
    \underbrace{\begin{bmatrix}y_1 \\ y_2 \\ \vdots \\ y_n\end{bmatrix}}_{n \times 1} = \underbrace{\begin{bmatrix}
    1 & x_{11} & x_{12} & x_{11}^2 & \dots & x_{11}^p & x_{11}^{p-1}x_{12} & \dots & x_{11}x_{12}^{p-1} & x_{12}^p
    \\ 1 & x_{21} & x_{22} & x_{21}^2 & \dots & x_{21}^p & x_{21}^{p-1}x_{22} & \dots & x_{21}x_{22}^{p-1} & x_{22}^p
    \\ \vdots &&&&& \ddots &&&&\vdots \\
    1 & x_{n1} & x_{n2} & x_{n1}^2 & \dots & x_{n1}^p & x_{n1}^{p-1}x_{n2} & ... & x_{n1}x_{n2}^{p-1} & x_{n2}^p
    \end{bmatrix}}_{n \times \frac{p^2+5p}{2}}\underbrace{\begin{bmatrix}\beta_0 \\ \beta_1 \\ \vdots \\ \beta_{\frac{p^2+3p}{2}}\end{bmatrix}.}_{\frac{p^2+5p}{2} \times 1}$}
\end{equation}
For clarity if $p=2$
\begin{equation*}
    \hat{y}_i = \beta_0+\beta_1 x_{i1} + \beta_2 x_{i2}.
\end{equation*}
We see that the output vector is linear in the vector of parameters even though it is nonlinear in the features. Note that $p$ determines the model complexity and therefore the number of columns in the design matrix and the number of parameters. The output vector though depends only on the number of inputs. By changing $p$ we are changing the basis with which we describe $y_i$.
\newline \newline
As mentioned before our aim is to find $\beta$ such that the cost function is minimized. We start by rewriting the equation \ref{eq:cost_function_sum}:
\begin{equation}
    \mathcal{C}(\beta) = \frac{1}{n}\sum_{i=1}^{n}(y_i-\hat{y}_i(\beta))^2 = \frac{1}{n}( y -  {\hat y})^T( y -  {\hat y}) = \frac{1}{n}( y -  {X\beta})^T( y -  {X\beta}).
\end{equation}

The cost function is at its minimum when its gradient with respect to all parameters is 0, i.e.
\begin{equation*}
    \nabla_{\beta}\mathcal{C}(\beta) = X^T(y-X\beta) = 0.
\end{equation*}

Assuming that $X$ has full column rank, and hence $X^T X$ is positive definite, solving the above equation provides us with the optimal parameters as follows
\begin{equation}
    \beta = (X^TX)^{-1}X^Ty.
\end{equation}
The cost function for a linear regression problem is convex. Hence, there is only one global extremum. This is not proven in this report.
\newline \newline
In the case where $X^T X$ is near singular or singular calculating the pseudoinverse (specifically the
Moore-Penrose inverse) is a very useful and well-known method. It calculates the inverse by using the Singular Value Decomposition algorithm. Moreover, this approach is more efficient than computing the regular inverse \cite{geron_hands-machine_2019}.
\newline \newline
An important step when calculating the best suited model for given input and output data is the splitting of the dataset $\mathcal{D}$ into a training dataset $\mathcal{D}_{train} = \{\boldsymbol{x}_{train}, y_{train}\}$ and a test dataset $\mathcal{D}_{test} = \{\boldsymbol{x}_{test}, y_{test}\}$. The ratio of the number of data points in the $\mathcal{D}_{test}$ to the number of data points in the $\mathcal{D}_{train}$ is called the test ratio $r$. The model parameters are learned on the training dataset in the training phase. While, the performance of the model is assessed by various indicators that compare the output of fitting the model, on the input of the test set, with the output of the test set. \newline

Let the mean squared error(MSE) be
\begin{equation}
    MSE(y, \hat{y}) = \frac{1}{n}\sum_{i=1}^{n}(y_i - \hat{y_i})^2
\end{equation}
the squaring is important so that the positive and negative errors do not cancel each other. Certainly, the lower the value of MSE the closer is our prediction($\hat{y}$) to the ground truth($y$). The training MSE is given as $MSE_{train} = MSE(y_{train}, \hat{y}_{train})$ and the testing MSE as $MSE_{test} = MSE(y_{test}, \hat{y}_{test})$.

The MSE can be decomposed into bias and variance as
\begin{equation}
    MSE = \underbrace{\mathbb{E}[(y - \mathbb{E}[\hat{y}])^2]}_{\text{bias}} + \underbrace{\mathbb{E}[(\hat{y} - \mathbb{E}[\hat{y}])^2]}_{\text{variance}} + \sigma^2
\end{equation}

Let the $R^2$ score be
\begin{equation}
    R^2(y, \hat{y}) = 1 - \frac{\sum_{i=1}^{n}(y_i-\hat{y})^2}{\sum_{i=1}^{n}(y_i-\mathbb{E}[\hat{y}])^2}.
\end{equation}
It measures the ability of the model to capture the true variance relative to the actual variance. When the model fits the data perfectly $R^2=1$. While, the least possible value of $R^2_{train}$ is 0, but $R^2_{test}$ can be negative.

The difference in the MSE values  for the training and testing set determines the degree of generalization of the model from trained data to new unseen data. The average generalization error is \cite{mehta2019high}
\begin{equation*}
    |MSE_{train} - MSE_{test}| = 2\sigma^2\frac{m}{n}.
\end{equation*}
If $m >> n$ then the model is not generalising, i.e. learning. In addition, the error can be large if the intrinsic noise $\sigma^2$ in the data is large. To counter this problem, regularisation is performed. In regularisation, the parameter values are penalized. In ridge regression, the $L^2$ norm of the parameter vector is penalized and in lasso regression, the $L^1$ norm of the parameter vector is penalized. This is done by modifying the cost functions as follows
\begin{align}
    \mathcal{C}_{ridge}(\beta) &=  \frac{1}{n}( y -  {X\beta})^T( y -  {X\beta}) + \lambda||\beta||_2^2, \\
    \mathcal{C}_{lasso}(\beta) &=  \frac{1}{n}( y -  {X\beta})^T( y -  {X\beta}) + \lambda||\beta||_1.
\end{align}

\subsubsection{Feature Scaling}\label{subsubsec:scaling}
In general, machine learning algorithms do not perform well if the scales of the data vary a lot. There exist different scaling methods. We focus on min-max scaling, also known as min-max normalization. In this approach, the values are scaled and shifted such that they range from 0 to 1, i.e.
\begin{equation*}
    x^{'} = \frac{x - min(x)}{max(x) - min(x)}.
\end{equation*}

\subsubsection{Need for resampling}
Typically, data contains outliers. These outliers can be erroneously obtained data. They can also be the true data that sometimes deviates from the established patterns of the system due to intrinsic noise. The presence of outliers in both the training and the testing set can either influence the fitting of parameters or the statistics. Therefore, it is essential to subdue the effects of outliers using methods that usually involve multiple iterations of training and testing on different subsets of the data. Moreover, the statistics obtained for a single fit are not necessarily the true indicator of the model performance. There is intrinsic variance in those statistics. To ameliorate the effect of variance we utilize resampling techniques. In resampling, we repeatedly learn the model with a different training set and find their statistical performance on the different test sets. In the end, the mean of the statistics of individual iterations is taken to be the final statistic. \newline \newline In the following sections we discuss two of the most commonly used methods: the bootstrap algorithm and cross-validation.
\subsubsection{Bootstrap Algorithm}\label{subsubsec:bootstrap}
The bootstrap algorithm is a resampling technique that uses sampling with replacement. There are multiple variants of the bootstrap method. We use one that is distinct from the lecture notes. \newline
The steps of the algorithm are as follows:
\begin{enumerate}
    \item Let the number of bootstrap iterations on given data be: $n_{boots}$
    \item For each bootstrap iteration $n_{bi}$:
    \begin{enumerate}
        \item Randomly divide the input data consisting of $n$ samples into two categories. The two categories are the training and test set. The training set has $(1-r)n$  number of samples which are randomly selected with replacement. The rest of the samples then make up the testing set. Therefore the number of samples in the testing set is greater than or equal to $rn$. Here, $r$ is the testing ratio.
        \item Train parameters for the training set and fit them to the testing set.
        \item Compute statistics on the training set: $MSE_i, R^2_i$
    \end{enumerate}
    \item The final statistics of the bootstrap algorithm are the mean of the statistics of all the bootstrap iterations.
\end{enumerate}

\begin{figure}[H]
    \centering
    \includegraphics[width=1\linewidth]{Images/bootstrap_diagram.png}
    \caption{Flowchart of the bootstrap algorithm.}
    \label{fig:bootstrap_algorithm}
\end{figure}

\subsubsection{K-fold Cross-Validation}\label{subsubsec:cross_validation}
K-fold cross-validation is an ideal method to circumvent the problem that data is often scarce and there is usually not enough data to set aside a validation set. To avoid this problem the data into $k$ parts of the same size \cite{friedman2001elements}. \newline
The steps of the algorithm are as follows:
\begin{enumerate}
    \item The data is divided into a finite number $k_{folds}$ of equal-sized sets called folds.
    \item For each fold $k_{fi}$:
    \begin{enumerate}
        \item Take the fold as the testing set while the rest of the folds comprise the training set.
        \item Train parameters for the training set and fit them to the testing set.
        \item Compute statistics on the training set: $MSE_i, R^2_i$
    \end{enumerate}
    \item The final statistics of the cross-validation algorithm are the mean of the statistics obtained by considering each fold as the testing set.
\end{enumerate}
\begin{figure}[H]
    \centering
    \includegraphics[width=1\linewidth]{Images/cross_validation_diagram.png}
    \caption{Flowchart of the k-fold cross-validation algorithm with $k_{folds}=3$ and $n=9$.}
    \label{fig:cross_val_algorithm}
\end{figure}

%\subsubsection{Stochastic Gradient Descent}\label{subsubsec:SGD}
%SGD is not a machine learning technique but an optimization technique. It is useful in the optimization %requirement for logistic regression.

%SGD comes in various variants. The most common variant due to its good empirical performance is a mix of epoch and mini-batches. Mini-batches are a subset of data of size m obtained by sampling the whole training data without replacement. This is where randomness and stochasticity come from.


\subsection{Combining Two Functions}
Let $f$ and $g$ be two functions.
Then we can form new functions by adding, subtracting, multiplying, or dividing.
These new functions, $f+g$, $f-g$, $fg$ and $f/g$, are defined in the usual way.

\begin{formulabox}[Operations on Functions]
$$(f+g)(x)=f(x)+g(x)\qquad \qquad (f-g)(x)=f(x)-g(x)$$
$$(fg)(x)=f(x)g(x)\qquad \qquad \left(\frac{f}{g}\right)(x)=\frac{f(x)}{g(x)}$$
\end{formulabox}

Suppose $D_f$ is the domain of $f$ and $D_g$ is the domain of $g$.
Then the domains of $f+g$, $f-g$ and $fg$ are the same and are equal to the intersection $D_f\cap D_g$ (that is, everything that is in \ifont{common} to both the domain of $f$ and the domain of $g$).
Since division by zero is \ifont{not allowed}, the domain of $f/g$ is $\{x\in D_f\cap D_g:g(x)\neq 0\}$.

Another way to combine two functions $f$ and $g$ together is a procedure called composition.

\begin{formulabox}[Function Composition]
Given two functions $f$ and $g$, the \ffont{composition} of $f$ and $g$, denoted by $f\circ g$, is defined as:
$$(f\circ g)(x)=f(g(x)).$$
\end{formulabox}

The domain of $f\circ g$ is $\{x\in D_g:g(x)\in D_f\}$, that is, it contains all values $x$ in the domain of $g$ such that $g(x)$ is in the domain of $f$.

\begin{example}{Domain of a Composition}{DomainofaComposition}
Let $f(x)=x^2$ and $g(x)=\sqrt x$.
Find the domain of $f\circ g$.
\end{example}

\begin{solution}
The domain of $f$ is $D_f=\{x\in\R\}$.
The domain of $g$ is $D_g=\{x\in\R:x\geq 0\}$.
The function $(f\circ g)(x)=f(g(x))$ is:
$$f(g(x))=\left(\sqrt x\right)^2=x.$$
Typically, $h(x)=x$ would have a domain of $\{x\in\R\}$, but since it came from a {\bf composed function}, we must consider $g(x)$ when looking at the domain of $f(g(x))$.
Thus, the domain of $f\circ g$ is $\{x\in\R:x\geq 0\}$.
\end{solution}

\begin{example}{Combining Two Functions}{CombiningTwoFunctions}
Let $f(x)=x^2+3$ and $g(x)=x-2$.
Find $f+g$, $f-g$, $fg$, $f/g$, $f\circ g$ and $g\circ f$.
Also, determine the domains of these new functions.
\end{example}

\begin{solution}
For $f+g$ we have:
$$(f+g)(x)=f(x)+g(x)=(x^2+3)+(x-2)=x^2+x+1.$$
For $f-g$ we have:
$$(f-g)(x)=f(x)-g(x)=(x^2+3)-(x-2)=x^2+3-x+2=x^2-x+5.$$
For $fg$ we have:
$$(fg)(x)=f(x)\cdot g(x)=(x^2+3)(x-2)=x^3-2x^2+3x-6.$$
For $f/g$ we have:
$$\left(\frac{f}{g}\right)(x)=\frac{f(x)}{g(x)}=\frac{x^2+3}{x-2}.$$
For $f\circ g$ we have:
$$(f\circ g)(x)=f(g(x))=f(x-2)=(x-2)^2+3=x^2-4x+7.$$
For $g\circ f$ we have:
$$(g\circ f)(x)=g(f(x))=g(x^2+3)=(x^2+3)-2=x^2+1.$$
The domains of $f+g$, $f-g$, $fg$, $f\circ g$ and $g\circ f$ is $\{x\in\mathbb{R}\}$, while the domain of $f/g$ is $\{x\in\mathbb{R}\,:\,x\neq 2\}$.
\end{solution}

As in the above problem, $f\circ g$ and $g\circ f$ are generally different functions.


\section{Trace and Determinant}
\begin{definition}
    Let $A\in M_n(F)=M_{n,n}(F)$ be a square $n\times n$ matrix.
    The trace of $A$ is defined to be
    $$\operatorname{tr}A=\sum_{i=1}^nA_{ii}$$
\end{definition}
\begin{remark}
    The map sending a matrix to its trace is a linear form.
\end{remark}
\begin{lemma}
    $\operatorname{tr}(AB)=\operatorname{tr}(BA)$.
\end{lemma}
\begin{proof}
    Write stuff out.
\end{proof}
\begin{corollary}
    Similar matrices have the same trace.
\end{corollary}
\begin{proof}
    $\operatorname{tr}(P^{-1}AP)=\operatorname{tr}(APP^{-1})=\operatorname{tr}(A)$.
\end{proof}
\begin{definition}
    If $\alpha:V\to V$ is linear, then $\operatorname{tr}\alpha=\operatorname{tr}[\alpha]_B$ for any choice of basis $B$ of $V$.
    It is well-defined by the preceding corollary.
\end{definition}
\begin{lemma}
    Let $\alpha:V\to V$ be linear and $\alpha^\ast:V^\ast\to V^\ast$ be the dual map, then $\operatorname{tr}\alpha=\operatorname{tr}\alpha^\ast$.
\end{lemma}
\begin{proof}
    Choose any basis $B$ of $V$, then
    $$\operatorname{tr}\alpha=\operatorname{tr}[\alpha]_B=\operatorname{tr}[\alpha]_B^\top=\operatorname{tr}[\alpha^\ast]_{B^\ast}=\operatorname{tr}\alpha^\ast$$
    as desired.
\end{proof}
Recall that we can decompose any permutation $\sigma\in S_n$ into a product of transpositions.
\begin{definition}
    The signature of a permutation is the (necessarily unique) homomorphism $\epsilon:S_n\to\{1,-1\}$ that sends any transposition to $-1$.
\end{definition}
This map $\epsilon$ is well-defined as we know that the parity of the number of transpositions that builds up a permutation is fixed.
\begin{definition}
    Let $A=(a_{ij})\in M_n(F)$.
    We define the determinant of $A$ as
    $$\det A=\sum_{\sigma\in S_n}\epsilon(\sigma)a_{\sigma(1)1}a_{\sigma(2)2}\cdots a_{\sigma(n)n}$$
\end{definition}
\begin{example}
    For $n=2$, we have
    $$\det\begin{pmatrix}
        a_{11}&a_{12}\\
        a_{21}&a_{22}
    \end{pmatrix}=a_{11}a_{22}-a_{12}a_{21}$$
\end{example}
\begin{lemma}
    If $A=(a_{ij})$ is an upper (resp. lower) triangular matrix, i.e. $a_{ij}=0$ for $i>j$ (resp. $i<j$), then $\det A=0$.
\end{lemma}
\begin{proof}
    The only permutation $\sigma$ such that $\sigma(j)\le j$ (resp. $\sigma(j)\ge j$) for all $j$ is the identity.
\end{proof}
\begin{lemma}
    $\det A=\det A^\top$.
\end{lemma}
\begin{proof}
    For any $\sigma\in S_n$ we know that $\epsilon(\sigma)=\epsilon(\sigma^{-1})$.
\end{proof}
\begin{definition}
    A volumn form $d$ in $F^n$ is a function $(F^n)^n\to F$ such that:\\
    1. It is multilinear:
    For any $i\in\{1,\ldots,n\}$ and $v_1,\ldots,v_{i-1},v_{i+1},\ldots,v_n\in F^n$, the map
    $$v\mapsto d(v_1,\ldots,v_{i-1},v,v_{i+1},\ldots,v_n\in F^n)$$
    is linear.\\
    2. It is an alternating form:
    If $v_i=v_j$ for some $i\neq j$, then $d(v_1,\ldots,v_n)=0$.
\end{definition}
What we want to prove that there is only one volumn form (up to multiplicative constant).
If this is true, then it necessarily equals $\det$ in the following way:
\begin{lemma}
    $\det$ is a volumn form via the obvious identification $M_n(F)=(F^n)^n$ by grouping the $n$ column vectors as a tuple.
\end{lemma}
\begin{proof}
    $\det$ is linear as it is linear in any entry.
    It is an alternating form as $\epsilon$ sends any transposition to $-1$.
\end{proof}
\begin{lemma}
    Let $d$ be a volumn form, then swapping two entries changes the sign.
\end{lemma}
\begin{proof}
    For any $i\neq j$, $d(v_1,\ldots,v_i,\ldots,v_j,\ldots,v_n)+ d(v_1,\ldots,v_j,\ldots,v_i,\ldots,v_n)= d(v_1,\ldots,v_i+v_j,\ldots,v_i+v_j,\ldots,v_n)=0$.
\end{proof}
\begin{corollary}
    For any $\sigma\in S_n$ and volume form $d$,
    $$d(v_{\sigma_1},\ldots,v_{\sigma_n})=\epsilon(\sigma)d(v_1,\ldots,v_n)$$
\end{corollary}
\begin{proof}
    Just decompose $\sigma$ into transpositions.
\end{proof}
\begin{theorem}
    Let $A\in M_n(F)$ and let $A^{(i)}$ be the $i^{th}$ column of $A$.
    For any volumn form $d$, we have
    $$d(A^{(1)},\ldots,A^{(n)})=\det(A)d(e_1,\ldots,e_n)$$
    where $(e_i)_j=\delta_{ij}$.
\end{theorem}
This is what we wanted.
\begin{proof}
    Just expand using linearity and the preceding corollary.
\end{proof}
\begin{corollary}
    $\det$ is the unique volumn form that maps $(e_1,\ldots,e_n)$ to $1$.
\end{corollary}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage[legalpaper, portrait, margin=1in]{geometry}

\title{Mathemaddict Arctan Integral}
\author{Shreenabh Agrawal}
\date{\today}

\begin{document}

\maketitle

\section{Question}
Evaluate: $$\int\limits_{0}^{\infty} \frac{\tan ^{-1}\left(\frac{3}{2 x}\right)-\tan ^{-1}\left(\frac{1}{x}\right)}{x} \: d x$$
\section{Solution}

Taking Substitution,
$$\begin{aligned}
\frac{1}{x} &=t \\
\frac{-1}{x^{2}} \: d x &=d t
\end{aligned}$$
Hence, our Integral becomes,
$$I=\int\limits_{0}^{\infty} \frac{\tan ^{-1}\left(\frac{3 t}{2}\right)-\tan ^{-1}(t)}{t} \: dt$$
Let's derive a general formula for:-
$$I_{(a, b)}=\int_{0}^{\infty} \frac{\tan ^{-1}(a x)-\tan ^{-1}(b x)}{x} \: dx$$
Applying Leibniz Rule w.r.t Parameter $a$,
$$\frac{\partial I_{(a, b)}}{\partial a}=\int_{0}^{\infty} \frac{1}{1+a^{2} x^{2}}\:  d x=\frac{\pi}{2 a}$$
Integrating it back,
$$\therefore \quad I_{(a, b)}=\frac{\pi}{2} \ln (a)+c$$
If we evaluate the special case for $a=b$,
we can calculate the value of $c$ as,
$$c=-\frac{\pi}{2} \ln b$$
Hence,
$$\begin{aligned}
I_{(a, b)}
&=\frac{\pi}{2} \ln a-\frac{\pi}{2} \ln b \\
I_{(a, b)}
&=\frac{\pi}{2} \ln \left(\frac{a}{b}\right)
\end{aligned}$$
Now substituting:-
$$a=\frac{3}{2}, b=1$$
Thus,
$$\boxed{I_{\left(\frac{3}{2}, 1\right)}=\frac{\pi}{2} \ln \frac{3}{2}}$$
\end{document}


\section*{Exercises}

\begin{ex}
  Let $T:\Poly_2 \to \R$ be a linear transformation such that
  \begin{equation*}
    T(x^2)=1,
    \quad
    T(x^2+x)=5,
    \quad\mbox{and}\quad
    T(x^2+x+1)=-1.
  \end{equation*}
  Find $T(ax^2+bx+c)$.
  \begin{sol}
    By linearity we have
    $T(x^2)=1$, $T(x) = T(x^2+x - x^2)= T(x^2+x) - T(x^2)= 5-1=4$, and
    $T(1) = T((x^2+x+1)-(x^2+x))=T(x^2+x+1)-T(x^2+x)= -1-5=-6$.
    Thus $T(ax^2+bx+c) = aT(x^2) + bT(x) + cT(1) = a+4b-6c$.
  \end{sol}
\end{ex}

\begin{ex}
  Let vectors $\vect{v}_1,\ldots,\vect{v}_n\in\R^n$ and
  $\vect{w}_1,\ldots,\vect{w}_n\in\R^m$ be given. Let $A$ be the
  matrix whose columns are $\vect{v}_1,\ldots,\vect{v}_n$, and assume
  that $A^{-1}$ exists. Show that there exists a linear transformation
  $T$ such that $T(\vect{v}_i)=\vect{w}_i$ for $i=1,\ldots,n$.
  \begin{sol}
    The matrix $A$ is invertible if and only if its rank is $n$, which
    means that $\vect{v}_1,\ldots,\vect{v}_n$ are linearly independent
    and therefore a basis of $\R^n$. The existence of $T$ then follows
    from Theorem~\ref{thm:transformation-basis}.
  \end{sol}
\end{ex}


\subsection{Goal}

Let's prove that:

\(\sum_{i=0}^n i= \dfrac{n(n+1)}{2}\)

\subsection{Proof by induction}

We use the inference rules Modus Ponens, which says that if \(X\) is true, and \(X\rightarrow Y\) is true, then \(Y\) is true.

\subsection{True for \(n=0\)}

We know this is true for \(n=0\):

\(0=\dfrac{0(0+1)}{2}\)

\(0=0\)

\subsection{If it's true for \(n\), it's true for \(n+1\)}

We can also prove that if it true for \(n\), it is true for \(n+1\).

\(\sum_{i=0}^{n+1} i= \dfrac{(n+1)(n+2)}{2}\)

\((n+1)+\sum_{i=0}^{n} i= \dfrac{n^2 +3n +2}{2}\)

If it is true for \(n\), then:

\((n+1)+\dfrac{n(n+1)}{2}= \dfrac{n^2 +3n +2}{2}\)

\(\dfrac{n^2+3n+2}{2}= \dfrac{n^2 +3n +2}{2}\)

\(1=1\)

\subsection{Result}

So we know that it is true for \(n=0\), and if it is true for \(n\), then it is true for \(n+1\). As a result it is true for all natural numbers.


\subsection{Odds}

Given a set of outcomes for a variable, the odds of the outcome are defined as:

\(o_f=\dfrac{P(E)}{P(E^C)}\)

For example, the odds of rolling a \(6\) are \(\dfrac{1}{5}\).


\section{Some proofs}

In this section, we prove Theorems~\ref{thm:well-defined-determinant}
and {\ref{thm:determinant-row-operations}}. This section can be
skipped by readers who are not interested in the proofs.

% ----------------------------------------------------------------------
\subsection{Proof of Theorem~\ref{thm:well-defined-determinant}}

\begin{lemma}{Expanding along any column}{well-defined-column-cofactor}
  Let $A$ be an $n\times n$-matrix. Expanding along any two columns
  gives the same answer.
\end{lemma}

\begin{proof}
  The result is proved by induction on $n$. This means, when we prove
  the result for $n\times n$-matrices, we already assume it is true
  for $(n-1)\times(n-1)$-matrices. We only sketch the proof in the
  case where $n=4$. We also only show that expanding along the first
  and second columns gives the same result. The proof for general $n$
  and for two arbitrary columns $i$ and $j$ is very similar.

  Expanding $A$ along the first column, we get
  \begin{equation}\label{eqn:well-defined-column-cofactor1}
    a_{11}\begin{absmatrix}{ccc}
      a_{22} & a_{23} & a_{24} \\
      a_{32} & a_{33} & a_{34} \\
      a_{42} & a_{43} & a_{44} \\
    \end{absmatrix}
    - a_{21}\begin{absmatrix}{ccc}
      a_{12} & a_{13} & a_{14} \\
      a_{32} & a_{33} & a_{34} \\
      a_{42} & a_{43} & a_{44} \\
    \end{absmatrix}
    + a_{31}\begin{absmatrix}{ccc}
      a_{12} & a_{13} & a_{14} \\
      a_{22} & a_{23} & a_{24} \\
      a_{42} & a_{43} & a_{44} \\
    \end{absmatrix}
    - a_{41}\begin{absmatrix}{ccc}
      a_{12} & a_{13} & a_{14} \\
      a_{22} & a_{23} & a_{24} \\
      a_{32} & a_{33} & a_{34} \\
    \end{absmatrix}.
  \end{equation}
  Expanding $A$ along the second column, we get
  \begin{equation}\label{eqn:well-defined-column-cofactor2}
    - a_{12}\begin{absmatrix}{ccc}
      a_{21} & a_{23} & a_{24} \\
      a_{31} & a_{33} & a_{34} \\
      a_{41} & a_{43} & a_{44} \\
    \end{absmatrix}
    + a_{22}\begin{absmatrix}{ccc}
      a_{11} & a_{13} & a_{14} \\
      a_{31} & a_{33} & a_{34} \\
      a_{41} & a_{43} & a_{44} \\
    \end{absmatrix}
    - a_{32}\begin{absmatrix}{ccc}
      a_{11} & a_{13} & a_{14} \\
      a_{21} & a_{23} & a_{24} \\
      a_{41} & a_{43} & a_{44} \\
    \end{absmatrix}
    + a_{42}\begin{absmatrix}{ccc}
      a_{11} & a_{13} & a_{14} \\
      a_{21} & a_{23} & a_{24} \\
      a_{31} & a_{33} & a_{34} \\
    \end{absmatrix}.
  \end{equation}
  To show these two expressions are equal, we expand each of the
  $3\times 3$-matrices along the first column. Expanding the matrices
  in {\eqref{eqn:well-defined-column-cofactor1}}, we get
  \begin{eqnarray*}
    &&
       a_{11}\paren{
       a_{22} \begin{absmatrix}{cc}
         a_{33} & a_{34} \\
         a_{43} & a_{44} \\
       \end{absmatrix}
    -a_{32} \begin{absmatrix}{cc}
      a_{23} & a_{24} \\
      a_{43} & a_{44} \\
    \end{absmatrix}
    +a_{42} \begin{absmatrix}{cc}
      a_{23} & a_{24} \\
      a_{33} & a_{34} \\
    \end{absmatrix}}
    \\
    &&
       -a_{21}\paren{
       a_{12} \begin{absmatrix}{cc}
         a_{33} & a_{34} \\
         a_{43} & a_{44} \\
       \end{absmatrix}
    -a_{32} \begin{absmatrix}{cc}
      a_{13} & a_{14} \\
      a_{43} & a_{44} \\
    \end{absmatrix}
    +a_{42} \begin{absmatrix}{cc}
      a_{13} & a_{14} \\
      a_{33} & a_{34} \\
    \end{absmatrix}}
    \\
    &&
       +a_{31}\paren{
       a_{12} \begin{absmatrix}{cc}
         a_{23} & a_{24} \\
         a_{43} & a_{44} \\
       \end{absmatrix}
    -a_{22} \begin{absmatrix}{cc}
      a_{13} & a_{14} \\
      a_{43} & a_{44} \\
    \end{absmatrix}
    +a_{42} \begin{absmatrix}{cc}
      a_{13} & a_{14} \\
      a_{23} & a_{24} \\
    \end{absmatrix}}
    \\
    &&
       -a_{41}\paren{
       a_{12} \begin{absmatrix}{cc}
         a_{23} & a_{24} \\
         a_{33} & a_{34} \\
       \end{absmatrix}
    -a_{22} \begin{absmatrix}{cc}
      a_{13} & a_{14} \\
      a_{33} & a_{34} \\
    \end{absmatrix}
    +a_{32} \begin{absmatrix}{cc}
      a_{13} & a_{14} \\
      a_{23} & a_{24} \\
    \end{absmatrix}}.
  \end{eqnarray*}
  Expanding the matrices in
  {\eqref{eqn:well-defined-column-cofactor2}}, we get
  \begin{eqnarray*}
    &&
     -a_{12}\paren{
       a_{21} \begin{absmatrix}{cc}
         a_{33} & a_{34} \\
         a_{43} & a_{44} \\
       \end{absmatrix}
    -a_{31} \begin{absmatrix}{cc}
      a_{23} & a_{24} \\
      a_{43} & a_{44} \\
    \end{absmatrix}
    +a_{41} \begin{absmatrix}{cc}
      a_{23} & a_{24} \\
      a_{33} & a_{34} \\
    \end{absmatrix}}
    \\
    &&
       +a_{22}\paren{
       a_{11} \begin{absmatrix}{cc}
         a_{33} & a_{34} \\
         a_{43} & a_{44} \\
       \end{absmatrix}
    -a_{31} \begin{absmatrix}{cc}
      a_{13} & a_{14} \\
      a_{43} & a_{44} \\
    \end{absmatrix}
    +a_{41} \begin{absmatrix}{cc}
      a_{13} & a_{14} \\
      a_{33} & a_{34} \\
    \end{absmatrix}}
    \\
    &&
       -a_{32}\paren{
       a_{11} \begin{absmatrix}{cc}
         a_{23} & a_{24} \\
         a_{43} & a_{44} \\
       \end{absmatrix}
    -a_{21} \begin{absmatrix}{cc}
      a_{13} & a_{14} \\
      a_{43} & a_{44} \\
    \end{absmatrix}
    +a_{41} \begin{absmatrix}{cc}
      a_{13} & a_{14} \\
      a_{23} & a_{24} \\
    \end{absmatrix}}
    \\
    &&
       +a_{42}\paren{
       a_{11} \begin{absmatrix}{cc}
         a_{23} & a_{24} \\
         a_{33} & a_{34} \\
       \end{absmatrix}
    -a_{21} \begin{absmatrix}{cc}
      a_{13} & a_{14} \\
      a_{33} & a_{34} \\
    \end{absmatrix}
    +a_{31} \begin{absmatrix}{cc}
      a_{13} & a_{14} \\
      a_{23} & a_{24} \\
    \end{absmatrix}}.
  \end{eqnarray*}
  Comparing the two expressions term by term, we see that they are
  equal.
\end{proof}

\begin{lemma}{Expanding along any row}{well-defined-row-cofactor}
  Let $A$ be an $n\times n$-matrix. Expanding along any two rows gives
  the same answer.
\end{lemma}

\begin{proof}
  The proof is analogous to that of
  Lemma~\ref{lem:well-defined-column-cofactor}, expanding along rows instead
  of columns.
\end{proof}

\begin{lemma}{Expanding along a column and row}{well-defined-column-row-cofactor}
  Let $A$ be an $n\times n$-matrix. Expanding along the first column
  gives the same answer as expanding along the first row.
\end{lemma}

\begin{proof}
  Again, we only illustrate the proof in case of a $4\times 4$-matrix,
  but the general proof is very similar. Expanding $\det(A)$ along the
  first column, we get:
  Expanding $A$ along the first column, we get
  \begin{equation}\label{eqn:well-defined-row-column-cofactor1}
    a_{11}\begin{absmatrix}{ccc}
      a_{22} & a_{23} & a_{24} \\
      a_{32} & a_{33} & a_{34} \\
      a_{42} & a_{43} & a_{44} \\
    \end{absmatrix}
    - a_{21}\begin{absmatrix}{ccc}
      a_{12} & a_{13} & a_{14} \\
      a_{32} & a_{33} & a_{34} \\
      a_{42} & a_{43} & a_{44} \\
    \end{absmatrix}
    + a_{31}\begin{absmatrix}{ccc}
      a_{12} & a_{13} & a_{14} \\
      a_{22} & a_{23} & a_{24} \\
      a_{42} & a_{43} & a_{44} \\
    \end{absmatrix}
    - a_{41}\begin{absmatrix}{ccc}
      a_{12} & a_{13} & a_{14} \\
      a_{22} & a_{23} & a_{24} \\
      a_{32} & a_{33} & a_{34} \\
    \end{absmatrix}.
  \end{equation}
  Expanding $A$ along the first row, we get:
  \begin{equation}\label{eqn:well-defined-row-column-cofactor2}
    a_{11}\begin{absmatrix}{ccc}
      a_{22} & a_{23} & a_{24} \\
      a_{32} & a_{33} & a_{34} \\
      a_{42} & a_{43} & a_{44} \\
    \end{absmatrix}
    - a_{12}\begin{absmatrix}{ccc}
      a_{21} & a_{23} & a_{24} \\
      a_{31} & a_{33} & a_{34} \\
      a_{41} & a_{43} & a_{44} \\
    \end{absmatrix}
    + a_{13}\begin{absmatrix}{ccc}
      a_{21} & a_{22} & a_{24} \\
      a_{31} & a_{32} & a_{34} \\
      a_{41} & a_{42} & a_{44} \\
    \end{absmatrix}
    - a_{14}\begin{absmatrix}{ccc}
      a_{21} & a_{22} & a_{23} \\
      a_{31} & a_{32} & a_{33} \\
      a_{41} & a_{42} & a_{43} \\
    \end{absmatrix}.
  \end{equation}
  To show these two expressions are equal, let us expand each of the
  $3\times 3$-matrices in
  {\eqref{eqn:well-defined-row-column-cofactor1}} along the first row,
  except for the first matrix. This yields:
  \begin{eqnarray*}
    &&
       a_{11}\begin{absmatrix}{ccc}
         a_{22} & a_{23} & a_{24} \\
         a_{32} & a_{33} & a_{34} \\
         a_{42} & a_{43} & a_{44} \\
       \end{absmatrix}
    \\
    &&
    - a_{21}\paren{
    a_{12}\begin{absmatrix}{ccc}
      a_{33} & a_{34} \\
      a_{43} & a_{44} \\
    \end{absmatrix}
    -a_{13}\begin{absmatrix}{ccc}
      a_{32} & a_{34} \\
      a_{42} & a_{44} \\
    \end{absmatrix}
    +a_{14}\begin{absmatrix}{ccc}
      a_{32} & a_{33} \\
      a_{42} & a_{43} \\
    \end{absmatrix}
    }
    \\
    &&
    + a_{31}\paren{
    a_{12}\begin{absmatrix}{ccc}
      a_{23} & a_{24} \\
      a_{43} & a_{44} \\
    \end{absmatrix}
    -a_{13}\begin{absmatrix}{ccc}
      a_{22} & a_{24} \\
      a_{42} & a_{44} \\
    \end{absmatrix}
    +a_{14}\begin{absmatrix}{ccc}
      a_{22} & a_{23} \\
      a_{42} & a_{43} \\
    \end{absmatrix}
    }
    \\
    &&
    - a_{41}\paren{
    a_{12}\begin{absmatrix}{ccc}
      a_{23} & a_{24} \\
      a_{33} & a_{34} \\
    \end{absmatrix}
    -a_{13}\begin{absmatrix}{ccc}
      a_{22} & a_{24} \\
      a_{32} & a_{34} \\
    \end{absmatrix}
    +a_{14}\begin{absmatrix}{ccc}
      a_{22} & a_{23} \\
      a_{32} & a_{33} \\
    \end{absmatrix}
    }.
  \end{eqnarray*}
  On the other hand, let us expand each of the
  $3\times 3$-matrices in
  {\eqref{eqn:well-defined-row-column-cofactor2}} along the first
  column, except for the first matrix. This yields:
  \begin{eqnarray*}
    &&
       a_{11}\begin{absmatrix}{ccc}
         a_{22} & a_{23} & a_{24} \\
         a_{32} & a_{33} & a_{34} \\
         a_{42} & a_{43} & a_{44} \\
       \end{absmatrix}
    \\
    &&
    - a_{12}\paren{
    a_{21}\begin{absmatrix}{ccc}
      a_{33} & a_{34} \\
      a_{43} & a_{44} \\
    \end{absmatrix}
    -a_{31}\begin{absmatrix}{ccc}
      a_{23} & a_{24} \\
      a_{43} & a_{44} \\
    \end{absmatrix}
    +a_{41}\begin{absmatrix}{ccc}
      a_{23} & a_{24} \\
      a_{33} & a_{34} \\
    \end{absmatrix}
    }
    \\
    &&
    + a_{13}\paren{
    a_{21}\begin{absmatrix}{ccc}
      a_{32} & a_{34} \\
      a_{42} & a_{44} \\
    \end{absmatrix}
    -a_{31}\begin{absmatrix}{ccc}
      a_{22} & a_{24} \\
      a_{42} & a_{44} \\
    \end{absmatrix}
    +a_{41}\begin{absmatrix}{ccc}
      a_{22} & a_{24} \\
      a_{32} & a_{34} \\
    \end{absmatrix}
    }
    \\
    &&
    - a_{14}\paren{
    a_{21}\begin{absmatrix}{ccc}
      a_{32} & a_{33} \\
      a_{42} & a_{43} \\
    \end{absmatrix}
    -a_{31}\begin{absmatrix}{ccc}
      a_{22} & a_{23} \\
      a_{42} & a_{43} \\
    \end{absmatrix}
    +a_{41}\begin{absmatrix}{ccc}
      a_{22} & a_{23} \\
      a_{32} & a_{33} \\
    \end{absmatrix}
    }.
  \end{eqnarray*}
  The two expressions can be seen to be equal by comparing them term
  by term.
\end{proof}

Together, Lemmas~\ref{lem:well-defined-column-cofactor},
{\ref{lem:well-defined-row-cofactor}}, and
{\ref{lem:well-defined-column-row-cofactor}} imply
Theorem~\ref{thm:well-defined-determinant}.

% ----------------------------------------------------------------------
\subsection{Proof of Theorem~\ref{thm:determinant-row-operations}}

\begin{lemma}{}{L2}
  Assume $A$, $B$ and $C$ are $n\times n$-matrices and
  $i\in\set{1,\ldots,n}$ is an index such that the following holds.
  \begin{enumerate}
  \item The $k$th rows of all three matrices are identical, for $k\neq i$.

  \item Each entry in the $i$th row of $A$ is the sum of the
    corresponding entries in $i$th rows of $B$ and $C$, i.e.,
    $a_{ij}=b_{ij}+c_{ij}$, for all $j=1,\ldots,n$.
  \end{enumerate}
  Then $\det A=\det B+\det C$.
\end{lemma}

\begin{proof}
  Notice that all three matrices have the same cofactors
  $\cofactor{A}{i1},\ldots,\cofactor{A}{in}$, since these cofactors
  involve deleting the $i\th$ row (and the matrices $A$, $B$, and $C$
  only differ in the $i\th$ row). Expanding all determinants along the
  $i\th$ row, we have
  \begin{eqnarray*}
    \det(A)
    &=& a_{i1}\cofactor{A}{i1} + \ldots + a_{in}\cofactor{A}{in} \\
    &=& (b_{i1}+c_{i1})\cofactor{A}{i1} + \ldots + (b_{in}+c_{in})\cofactor{A}{in} \\
    &=& (b_{i1}\cofactor{B}{i1} + \ldots + b_{in}\cofactor{B}{in})
        + (c_{i1}\cofactor{C}{i1} + \ldots + c_{in}\cofactor{C}{in}) \\
    &=& \det(B) + \det(C).
  \end{eqnarray*}
\end{proof}

% ======================================================================
\subsection{CONTINUE HERE...}

First we recall the definition of a determinant. If $A=\mat{a_{ij} }$
is an $n\times n$-matrix, then $\det A$ is defined by computing the
expansion along the first row:
\begin{equation}
  \label{E1}
  \det A=\sum_{i=1}^n a_{1,i} \cofactor{A}{1,i}.
\end{equation}
If $n=1$ then $\det A=a_{1,1}$.

The following example is straightforward and strongly recommended as a
means for getting used to definitions.

\begin{example}{}{EX1}
  (1) Let $E_{ij}$ be the elementary matrix obtained by interchanging
  $i$th and $j$th rows of $I$.  Then $\det E_{ij}=-1$.

  (2) Let $E_{ik}$ be the elementary matrix obtained by multiplying
  the $i$th row of $I$ by $k$.  Then $\det E_{ik}=k$.

  (3) Let $E_{ijk}$ be the elementary matrix obtained by multiplying
  $i$th row of $I$ by $k$ and adding it to its $j$th row. Then
  $\det E_{ijk}=1$.

  (4) If $C$ and $B$ are such that $CB$ is defined and the $i$th row
  of $C$ consists of zeros, then the $i$th row of $CB$ consists of
  zeros.

  (5) If $E$ is an elementary matrix, then $\det E=\det E^T$.
\end{example}

Many of the proofs in section use the Principle of Mathematical
Induction. This concept is discussed in
Appendix~\ref{well-ordering-section} and is reviewed here for
convenience.  First we check that the assertion is true for $n=2$ (the
case $n=1$ is either completely trivial or meaningless).

Next, we assume that the assertion is true for $n-1$ (where $n\geq 3$)
and prove it for $n$.  Once this is accomplished, by the Principle of
Mathematical Induction we can conclude that the statement is true for
all $n\times n$-matrices for every $n\geq 2$.

If $A$ is an $n\times n$-matrix and $1\leq j \leq n$, then the matrix
obtained by removing $1$st column and $j$th row from $A$ is an
$(n-1)\times (n-1)$-matrix (we shall denote this matrix by $A(j)$
below). Since these matrices are used in computation of cofactors
$\cofactor{A}{1,i}$, for $1\leq i\neq n$, the inductive assumption applies
to these matrices.

Consider the following lemma.

\begin{lemma}{}{L1}
  If $A$ is an $n\times n$-matrix such that one of its rows consists
  of zeros, then $\det A=0$.
\end{lemma}

\begin{proof}
  We will prove this lemma using Mathematical Induction.

  If $n=2$ this is easy (check!).

  Let $n\geq 3$ be such that every matrix of size $(n-1)\times(n-1)$
  with a row consisting of zeros has determinant equal to zero.  Let
  $i$ be such that the $i$th row of $A$ consists of zeros.  Then we
  have $a_{ij}=0$ for $1\leq j\leq n$.

  Fix $j\in \set{1,2, \dots ,n}$ such that $j\neq i$. Then matrix
  $A(j)$ used in computation of $\cofactor{A}{1,j}$ has a row consisting of
  zeros, and by our inductive assumption $\cofactor{A}{1,j}=0$.

  On the other hand, if $j=i$ then $a_{1,j}=0$.  Therefore
  $a_{1,j}\cofactor{A}{1,j}=0$ for all $j$ and by \eqref{E1} we have
  \begin{equation*}
    \det A=\sum_{j=1}^n a_{1,j} \cofactor{A}{1,j}=0
  \end{equation*}
  as each of the summands is equal to 0.
\end{proof}

\begin{theorem}{} {T1}
  Let $A$ and $B$ be $n\times n$-matrices.
  \begin{enumerate}
  \item If $A$ is obtained by interchanging $i$th and $j$th rows of
    $B$ (with $i\neq j$), then $\det A=-\det B$.
  \item If $A$ is obtained by multiplying $i$th row of $B$ by $k$ then
    $\det A=k\det B$.
  \item If two rows of $A$ are identical then $\det A=0$.
  \item If $A$ is obtained by multiplying $i$th row of $B$ by $k$ and
    adding it to $j$th row of $B$ ($i\neq j$) then $\det A=\det B$.
  \end{enumerate}
\end{theorem}

\begin{proof}
  We prove all statements by induction. The case $n=2$ is easily
  checked directly (and it is strongly suggested that you do check
  it).

  We assume $n\geq 3$ and (1)--(4) are true for all matrices of size
  $(n-1)\times (n-1)$.

  (1) We prove the case when $j=i+1$, i.e., we are interchanging two
  consecutive rows.

  Let $l\in \set{1, \dots, n}\setminus \set{i,j}$.  Then $A(l)$ is
  obtained from $B(l)$ by interchanging two of its rows (draw a
  picture) and by our assumption
  \begin{equation}
    \label{E2}
    \cofactor{A}{1,l}=-\cofactor{B}{1,l}.
  \end{equation}

  Now consider $a_{1,i} \cofactor{A}{1,l}$. We have that $a_{1,i}=b_{1,j}$
  and also that $A(i)=B(j)$. Since $j=i+1$, we have
  \begin{equation*}
    (-1)^{1+j}=(-1)^{1+i+1}=-(-1)^{1+i}
  \end{equation*}
  and therefore $a_{1i}\cofactor{A}{1i}=-b_{1j} \cofactor{B}{1j}$ and
  $a_{1j}\cofactor{A}{1j}=-b_{1i} \cofactor{B}{1i}$.  Putting this together with
  \eqref{E2} into \eqref{E1} we see that if in the formula for
  $\det A$ we change the sign of each of the summands we obtain the
  formula for $\det B$.
  \begin{equation*}
    \det A=\sum_{l=1}^n a_{1l}\cofactor{A}{1l}
    =-\sum_{l=1}^n b_{1l} B_{1l}
    =\det B.
  \end{equation*}
  We have therefore proved the case of (1) when $j=i+1$. In order to
  prove the general case, one needs the following fact. If $i<j$, then
  in order to interchange $i$th and $j$th row one can proceed by
  interchanging two adjacent rows $2(j-i)+1$ times: First swap $i$th
  and $i+1$st, then $i+1$st and $i+2$nd, and so on.  After one
  interchanges $j-1$st and $j$th row, we have $i$th row in position of
  $j$th and $l$th row in position of $l-1$st for $i+1\leq l\leq
  j$. Then proceed backwards swapping adjacent rows until everything
  is in place.

  Since $2(j-i)+1$ is an odd number $(-1)^{2(j-i)+1}=-1$ and we have
  that $\det A=-\det B$.

  (2) This is like (1)\dots{} but much easier.  Assume that (2) is
  true for all $(n-1)\times (n-1)$-matrices.  We have that
  $a_{ji}=k b_{ji}$ for $1\leq j\leq n$.  In particular
  $a_{1i}=kb_{1i}$, and for $l\neq i$-matrix $A(l)$ is obtained from
  $B(l)$ by multiplying one of its rows by $k$.  Therefore
  $\cofactor{A}{1l}=k\cofactor{B}{1l}$ for $l\neq i$, and for all $l$ we have
  $a_{1l} \cofactor{A}{1l}=k b_{1l}\cofactor{B}{1l}$.  By \eqref{E1}, we have
  $\det A=k\det B$.

  (3) This is a consequence of (1). If two rows of $A$ are identical,
  then $A$ is equal to the matrix obtained by interchanging those two
  rows and therefore by (1) $\det A=-\det A$. This implies $\det A=0$.

  (4) Assume (4) is true for all $(n-1)\times (n-1)$-matrices and fix
  $A$ and $B$ such that $A$ is obtained by multiplying $i$th row of
  $B$ by $k$ and adding it to $j$th row of $B$ ($i\neq j$) then
  $\det A=\det B$.  If $k=0$ then $A=B$ and there is nothing to prove,
  so we may assume $k\neq 0$.

  Let $C$ be the matrix obtained by replacing the $j$th row of $B$ by
  the $i$th row of $B$ multiplied by $k$.  By Lemma~\ref{lem:L2}, we
  have that
  \begin{equation*}
    \det A=\det B+\det C
  \end{equation*}
  and we `only' need to show that $\det C=0$. But $i$th and $j$th rows
  of $C$ are proportional. If $D$ is obtained by multiplying the $j$th
  row of $C$ by $\frac 1k$ then by (2) we have $\det C=\frac 1k\det D$
  (recall that $k\neq 0$!).  But $i$th and $j$th rows of $D$ are
  identical, hence by (3) we have $\det D=0$ and therefore $\det C=0$.
\end{proof}

\begin{theorem}{}{T2}
  Let $A$ and $B$ be two $n\times
  n$-matrices. Then\index{determinant!product}
  \begin{equation*}
    \det (AB) =\det (A) \det (B)
  \end{equation*}
\end{theorem}

\begin{proof}
  If $A$ is an elementary matrix of either type, then multiplying by
  $A$ on the left has the same effect as performing the corresponding
  elementary row operation. Therefore the equality
  $\det (AB) =\det A\det B$ in this case follows by
  Example~\ref{exa:EX1} and Theorem~\ref{thm:T1}.

  If $C$ is the {\rref} of $A$ then we can write
  $A=E_1\cdot E_2\cdot\dots\cdot E_m\cdot C$ for some elementary
  matrices $E_1,\dots, E_m$.

  Now we consider two cases.

  Assume first that $C=I$. Then $A=E_1\cdot E_2\cdot \dots\cdot E_m$
  and $AB= E_1\cdot E_2\cdot \dots\cdot E_m B$.  By applying the above
  equality $m$ times, and then $m-1$ times, we have that
  \begin{align*}
    \det AB&=\det E_1\det E_2\cdot \det E_m\cdot \det B\\
           &=\det (E_1\cdot E_2\cdot\dots\cdot E_m) \det B\\
           &=\det A\det B.
  \end{align*}
  Now assume $C\neq I$. Since it is in {\rref}, its last row consists
  of zeros and by (4) of Example~\ref{exa:EX1} the last row of $CB$
  consists of zeros.  By Lemma~\ref{lem:L1} we have
  $\det C=\det (CB)=0$ and therefore
  \begin{equation*}
    \det A=\det (E_1\cdot E_2\cdot  E_m)\cdot  \det (C)
    =
    \det (E_1\cdot E_2\cdot  E_m)\cdot 0=0
  \end{equation*}
  and also
  \begin{equation*}
    \det AB=\det (E_1\cdot E_2\cdot  E_m)\cdot  \det (C B)
    =\det (E_1\cdot E_2\cdot\dots\cdot E_m) 0
    =0
  \end{equation*}
  hence $\det AB=0=\det A \det B$.
\end{proof}

The same `machine' used in the previous proof will be used again.

\begin{theorem}{}{T.T}
  Let $A$ be a matrix where $A^T$ is the transpose of $A$. Then,
  \begin{equation*}
    \det(A^T) = \det (A)
  \end{equation*}
\end{theorem}

\begin{proof}
  Note first that the conclusion is true if $A$ is elementary by (5)
  of Example~\ref{exa:EX1}.

  Let $C$ be the {\rref} of $A$. Then we can write
  $A= E_1\cdot E_2\cdot \dots\cdot E_m C$.  Then
  $A^T=C^T\cdot E_m^T\cdot \dots \cdot E_2^T\cdot E_1$.  By
  Theorem~\ref{thm:T2} we have
  \begin{equation*}
    \det (A^T)=\det (C^T)\cdot \det (E_m^T)\cdot \dots \cdot \det (E_2^T)\cdot \det(E_1).
  \end{equation*}
  By (5) of Example~\ref{exa:EX1} we have that $\det E_j=\det E_j^T$
  for all $j$.  Also, $\det C$ is either 0 or 1 (depending on whether
  $C=I$ or not) and in either case $\det C=\det C^T$. Therefore
  $\det A=\det A^T$.
\end{proof}

The above discussions allow us to now prove
Theorem~\ref{thm:well-defined-determinant}. It is restated below.

\begin{theorem}{}{}
  Expanding an $n\times n$-matrix along any row or column always gives
  the same result, which is the determinant.
\end{theorem}

\begin{proof}
  We first show that the determinant can be computed along any
  row. The case $n=1$ does not apply and thus let $n \geq 2$.

  % Assume the theorem is true for all $(n-1)\times (n-1)$-matrices.

  Let $A$be an $n\times n$-matrix and fix $j>1$. We need to prove that
  \begin{equation*}
    \det A=\sum_{i=1}^n a_{j,i} \cofactor{A}{j,i}.
  \end{equation*}
  Let us prove the case when $j=2$.

  Let $B$ be the matrix obtained from $A$ by interchanging its $1$st
  and $2$nd rows.  Then by Theorem~\ref{thm:T1} we have
  \begin{equation*}
    \det A=-\det B.
  \end{equation*}
  Now we have
  \begin{equation*}
    \det B=\sum_{i=1}^n b_{1,i} \cofactor{B}{1,i}.
  \end{equation*}
  Since $B$ is obtained by interchanging the $1$st and $2$nd rows of
  $A$ we have that $b_{1,i}=a_{2,i}$ for all $i$ and one can see that
  $\minor{B}{1,i}=\minor{A}{2,i}$.

  Further,
  \begin{equation*}
    \cofactor{B}{1,i}=(-1)^{1+i} minor B_{1,i}=- (-1)^{2+i} \minor{A}{2,i} = - \cofactor{A}{2,i}
  \end{equation*}
  hence $\det B=-\sum_{i=1}^n a_{2,i} \cofactor{A}{2,i}$, and therefore
  $\det A=-\det B= \sum_{i=1}^n a_{2,i} \cofactor{A}{2,i}$ as desired.

  The case when $j>2$ is very similar; we still have
  $\minor{B}{1,i}=\minor{A}{j,i}$ but checking that
  $\det B=-\sum_{i=1}^n a_{j,i} \cofactor{A}{j,i}$ is slightly more
  involved.

  Now the cofactor expansion along column $j$ of $A$ is equal to the
  cofactor expansion along row $j$ of $A^T$, which is by the above
  result just proved equal to the cofactor expansion along row 1 of
  $A^T$, which is equal to the cofactor expansion along column $1$ of
  $A$. Thus the cofactor cofactor along any column yields the same
  result.

  Finally, since $\det A=\det A^T$ by Theorem~\ref{thm:T.T}, we
  conclude that the cofactor expansion along row $1$ of $A$ is equal
  to the cofactor expansion along row $1$ of $A^T$, which is equal to
  the cofactor expansion along column $1$ of $A$. Thus the proof is
  complete.
\end{proof}


\section{Eigenvalues and Eigenvectors}
If $\bfA$ is a square matrix, the \emph{characteristic polynomial} of $\bfA$ is defined by $p(\lambda)=\det(\bfA-\lambda I)$, the zeros of $p$ are \emph{eigenvalues}, or characteristic values, of the matrix $\bfA$.

\subsection{Spectral Radius}
\begin{defn}[Spectral Radius]
The \emph{spectral radius} $\rho(\bfA)$ of a matrix $\bfA$ is defined by
\[
\rho(\bfA)=\max\abs{\lambda},\quad\text{where $\lambda$ is an eigen value of $\bfA$.}
\]
(For complex $\lambda=\alpha+\beta i$, we define $\abs{\lambda}=(\alpha^2+\beta^2)^{1/2}.$)
\end{defn}

\begin{theo}
If $\bfA$ is an $n\times n$ matrix, then
\begin{enumerate}[(i)]
    \item $\norms{\bfA}_1=\max_j\sum_{i=1}^{n}\abs{a_{ij}}$,(列和最大)
    \item $\norms{\bfA}_2=\left[\rho(A^tA)\right]^{1/2}$,
    \item $\norms{\bfA}_\infty=\max_i\sum_{j=1}^{n}\abs{a_{ij}}$,(行和最大)
    \item $\rho(\bfA)\leq\norms{\bfA}$, for any natural norm $\norms{\cdot}$.
\end{enumerate}
\end{theo}

\subsection{Convergent Matrices}
\begin{defn}[Convergent]
We call an $n\times n$ matrix $\bfA$  convergent if
\[
\lim_{k\to\infty}(\bfA^k)_{ij}=0,\quad\text{for each $i=1,2,\ldots,n$ and $j=1,2,\ldots,n$.}
\]
\end{defn}

\begin{theo}
The following statements are equivalent
\begin{enumerate}[(i)]
    \item $\bfA$ is a convergent matrix.
    \item $\lim_{n\to\infty}\norms{\bfA^n}=0$, for some natural norm.
    \item $\lim_{n\to\infty}\norms{\bfA^n}=0$, for all natural norms.
    \item $\rho(\bfA)<1$.
    \item $\lim_{n\to\infty}\bfA^n\bfx=\bf 0$, for every $\bfx$.
\end{enumerate}
\end{theo}


\subsection{Fungi Interaction Model}
The \textit{Logistic model}~\cite{logisticmodel} can precisely analyze the growth phenomenon of animals, human and plants who naturally confront obstacles of survival attribute to the excess of the total population and the lack of food resources, so does microorganism, fungi. Therefore, it is suitable to use the \textbf{Logistic model} to study the interactions of different kinds of fungi.
\par
When a single fungal population lives, its number changes obey the \textbf{Single-group Logistic model}.
\begin{equation}
  \label{12eq}
  \frac{1}{N(t)}\frac{dN(t)}{dt} = r(1-\frac{N(t)}{N_{max}})
\end{equation}
where $N(t)$ represents current population size, and $r$ represents the growth rate of population.
When considering two fungal populations living together, due to competition for habitat and food, the interaction between them would hinder each other's growth and reproduction. Therefore, we introduce the \textbf{Relative Growth Blocking Index} (RGB) on the basis of Single-group Logistic model, and establish \textbf{Multi-groups Logistic Model}~\cite{multigroups} as follows.
\begin{equation}
  \begin{cases}
    \frac{1}{N_x(t)}\frac{dN_x(t)}{dt} = r_x(1-\frac{N_x(t)}{{N_x}_{max}} - RGB_x\frac{N_y(t)}{{N_y}_{max}}) \\ \\
    \frac{1}{N_y(t)}\frac{dN_y(t)}{dt} = r_y(1-\frac{N_y(t)}{{N_y}_{max}} - RGB_y\frac{N_x(t)}{{N_x}_{max}})
  \end{cases}
\end{equation}
where $N_x(t)$ and $N_y(t)$ represents the numbers of the two populations $x$ and $y$, $r_x$ and $r_y$ represents the inherent growth rate of populations, ${N_x}_{max}$ and ${N_y}_{max}$ are maximum capacities of populations, $RGB_x$ means for resources that support $x$, the consumption of $y$ is $RGB_x$ times the consumption of $x$ per unit quantity, and $RGB_y$ has the similar definition to $RGB_x$.
\par
Taking fungal species $F_A$ and $F_B$ proposed in the previous article as examples, analyze and predict the interaction between them. Calculate the values of parameters and list them as follows.
\begin{table}[H]
  \centering
  \caption{Values of parameters.}
  \label{valueofparameters}
  \begin{tabular*}{\hsize}{@{\extracolsep{\fill}}ccccc}
    \toprule
    \textbf{Parameter} & $RGB_A$ & $RGB_B$ & $r_A$ & $r_B$ \\
    \midrule
    \textbf{Value} & $0.39$ & $0.60$ & $1$ & $1$ \\
    \bottomrule
  \end{tabular*}
\end{table}
Draw the change curves of $N_A(t)$ and $N_B(t)$ as well as the phase trajectory of these two function variables as follows.
\par
\begin{figure}[H]
  \centering
  \subfigure[Curves of $N_A(t)$ and $N_B(t)$.]{
    \includegraphics[width=0.48\textwidth]{figures/FA&FBN(t).png}
  }
  \subfigure[Phase trajectory of $N_A(t)$ and $N_B(t)$.]{
    \includegraphics[width=0.48\textwidth]{figures/FA&FBline.png}
  }
  \caption{The interaction of $F_A\ \&\ F_B$.}
\end{figure}
\par
The dynamics of the interaction between Fungi A and Fungi B are characterized and described in the above figures. \textbf{In the short-term period} (about $1\sim 7$ days), $F_A$ and $F_B$ are in a period of rapid growth and reproduction, the curves of which maintain exponential growth. \textbf{In the long term} (after $8$ days), the growths of $F_A$ and $F_B$ would be both restricted by environmental resources. Meanwhile, because $F_A$ has a stronger ability to decompose, it has a stronger blocking effect on $F_B$. The final relative population numbers tend to $796$ and $522$ respectively and remain stable.
\par
In the same way, we calculate and analyze the interaction of Fungi C and Fungi E, and obtain the $N_C(t)$ and $N_E(t)$ function curves as well as the phase trajectory of the two function variables as follows.
\par
\begin{figure}[H]
  \centering
  \subfigure[Curves of $N_C(t)$ and $N_E(t)$.]{
    \includegraphics[width=0.48\textwidth]{figures/FC&FEN(t).png}
  }
  \subfigure[Phase trajectory of $N_C(t)$ and $N_E(t)$.]{
    \includegraphics[width=0.48\textwidth]{figures/FC&FEline.png}
  }
  \caption{The interaction between $F_C\ \&\ F_E$.}
\end{figure}
\par
\textbf{In a short period of time} (about $1\sim 5$ days), the numbers of $F_C$ and $F_E$ grow exponentially. \textbf{From a long-term perspective} (after $6$ days), since the discrepancy of decomposition ability between $F_E$ and $F_C$ is greater than that between $F_A$ and $F_B$, the blocking effect of $F_E$ on $F_C$ is extremely obvious. The final relative populations numbers of $F_C$ and $F_E$ tend to $365$ and $894$ respectively and remain stable.