\subsection{Fleiss' Kappa}
\label{chp:fundamentals:sec:inter_rater_agreement:subsec:fleiss_kappa}
Cohen's kappa and Scott's $\pi$ focus only on the agreement between two raters.
To overcome this limitation \textcite{Fleiss:1971} introduced another kappa statistics as a generalization of Scott's $\pi$ \parencite{Scott:1955}.
Consider $N$ datapoints, each assigned $n$ times to one of $C$ classes by different raters.
How many raters have assigned the $i$th datapoint to the $j$th class is indicated by $n_{ij}$.
$p_j$ is the proportion of all assignments which were made to the $j$th class and is calculated as:

\begin{equation}\label{eq:fleiss_pj}
    p_j = \frac{1}{Nn}\sum_{i=1}^N n_{ij}
\end{equation}

The agreement of $n$ raters on the $i$th datapoint, indicated by $P_i$, is the proportion of agreeing rater pairs out of the $n(n-1)$ possible pairs and given by the following:

\begin{equation}\label{eq:fleiss_Pi}
    \begin{aligned}
        P_i &= \frac{1}{n(n-1)} \sum_{j=1}^C n_{ij} (n_{ij}-1) \\
        &= \frac{1}{n(n-1)} \left[\left(\sum_{j=1}^C n_{ij}^2 \right) - n\right]\\
    \end{aligned}
\end{equation}

The mean of all $P_i$s then forms the overall agreement.

\begin{equation}\label{eq:fleiss_P_bar}
    \bar{P} = \frac{1}{N} \sum_{i=1}^N P_i
\end{equation}

The agreement achieved merely by chance $\bar{P_e}$ is similar to \textcite{Scott:1955} indicated by

\begin{equation}\label{eq:fleiss_P_e}
    \bar{P_e} = \sum_{j=1}^C p_j^2
\end{equation}

According to \textcite{Fleiss:1971}, the term $1-\bar{P_e}$ measures the agreement which can be achieved in extent to the agreement which is obtained by chance.
The actual agreement including the agreement by chance is represented by $\bar{P} - \bar{P_e}$.
Then the normalized kappa statistics is similar to \textcite{Cohen:1960} given by:

\begin{equation}\label{eq:fleiss_kappa}
    \kappa_{fleiss} = \frac{\bar{P}-\bar{P_e}}{1-\bar{P_e}}
\end{equation}

When applying Fleiss' kappa to the sample data of \cref{tab:cohens_kappa_sample_data} one can easily see that all $P_i$s are either $1$ when the raters agree or $0$ if one rater votes for "Good" and one for "Bad".
Considering this, \cref{eq:fleiss_P_bar} simplifies to $\bar{P} = \frac{20+15}{50} = 0.7$.
Since the expected agreement purely achieved by chance is calculated in the same way as for Scott's $\pi$, one immediately notices that for this test data the two metrics are equal $\kappa_{fleiss} = \pi = 0.\overline{39}$ like one would expect.


\section*{Calculations}
\label{sec:calc}
% CHANGE GB TO 1073741824 BYTES!!!

\subsection{Storage}

Storage requirements for 24 hours of 720p video data (.mp4):
\begin{equation*}
\begin{split}
Continual & = \left(\frac{24 \ hr}{1 \ day}\right)\left(\frac{60 \ min}{1 \ hr}\right)\left(\frac{60 \ sec}{ 1 \ min}\right)\left(\frac{17 \ mbit}{1 \ sec}\right)\left(\frac{1 \ byte}{8 \ bit}\right)\left(\frac{1 \ GB}{1737441824 \ byte}\right) \\
& = 170.99 \ \frac{GB}{day}
\end{split}
\end{equation*}

By limiting the active recording to moments where objects are detected, this storage requirement can be significantly lowered. Taking a theoretical family of 4 and a single dog, we can calculate for a minimum duration that the RPi2 is expected to record in 24 hours:

\begin{table}[H]
\centering
\caption{Known \glspl{sighting} during an average 24 hour day for a front door camera.}
\label{floor_table}
\begin{tabular}{@{}lcccc@{}}
\toprule
Type     & Qty & Duration (min) & Sighting/24 hr & Total  \\ \midrule
Person   & 4   & 1              & 2               & 8      \\
Dog      & 1   & 1              & 2               & 2      \\
Delivery & 2   & 1              & 1               & 2      \\ \midrule
         &     &                &                 & 12 min
\end{tabular}
\end{table}

With a minimum recording duration of 12 minutes based on the expected sightings (detected objects), the floor data requirements per 24 hours becomes:

\begin{equation*}
\begin{split}
Floor \ & = \left(\frac{12 \ min}{1 \ day}\right)\left(\frac{60 \ sec}{ 1 \ min}\right)\left(\frac{17 \ Mbit}{1 \ sec}\right)\left(\frac{1 \ byte}{8 \ bit}\right)\left(\frac{1 \ GB}{1737441824 \ byte}\right) \\
& = 0.000014254 \ \frac{GB}{day}\Rightarrow 0.014254 \ \frac{MB}{day} \Rightarrow 14.254 \ \frac{KB}{day}
\end{split}
\end{equation*}

\subsection{Analog Light Sensor}

\begin{equation*}
\begin{split}
    \log\left(I_{O}\right) &= mx + b \\
    I_{O} &= m\log\left(x\right) + b \\
    I_{O} &= 10\log\left(E_{v}\right)\left[l_{x}\right]
\end{split}
\end{equation*}

The sensor has an output current range of $0\mu A$--$50\mu A$ between $1$ lux--$100000$ lux. The breakout board converts this current into a voltage using a $68K\Omega$ resistor, giving a voltage output of $0V$--$3.4V$. Given the use-case of the light sensor, the $3.4V$ can be approximated to $3.3V$, which is a suitable voltage reference for the RPi2. With the output current range approximated to a voltage range of $0V$--$3.3V$ between $1$ lux--$100000$ lux,

\begin{equation*}
\begin{split}
    log_{i} &= \log\left(l_{x \ MAX} - l_{x \ MIN}\right) \\
    &= \log\left(100000\right) = 5 \\
\end{split}
\end{equation*}

To convert the analog value to a digital value, an 8-bit ADC is used:
\begin{equation*}
    ADC = \frac{log_{i}}{1024 \ steps} = 0.00488
\end{equation*}

Finally, the formula to obtain the final lux output, $l_{xo}$, on the RPi2:

\begin{equation*}
%\begin{split}
    l_{xo} = 10^{raw_{i} \times ADC} = 10^{raw_{i} \times 0.00488}
%\end{split}
\end{equation*}

where $raw_{i}$ is the raw value received by the ADC connected to the RPI2's GPIO.

%\newpage

%\begin{multicols}{2}
%\begin{onehalfspacing}
%\printglossary
%\end{onehalfspacing}
%\end{multicols}


\section{Hyperbolic Functions}{}{}\label{sec:hyperbolic functions}

The \textbf{hyperbolic functions} are a set of functions that have many applications to mathematics, physics, and engineering. Among many other applications, they are used to describe the formation of satellite rings around planets, to describe the shape of a rope hanging from two points, and have application to the theory of special relativity. This section defines the hyperbolic functions and describes many of their properties, especially their usefulness to calculus.

These functions are sometimes referred to as the ``hyperbolic trigonometric functions'' as there are many, many connections between them and the standard trigonometric functions. Figure \ref{fig:geometric def trigh} demonstrates one such connection. Just as cosine and sine are used to define points on the circle defined by $x^2+y^2=1$, the functions \textbf{hyperbolic cosine} and \textbf{hyperbolic sine} are used to define points on the hyperbola $x^2-y^2=1$. This is a bit surprising given our initial definitions.

\begin{definition}{Hyperbolic Functions}{Hyperbolic Sine and Cosine}
\label{def:hyperbolic_functions}
%The \dfont{hyperbolic cosine} is the function
%$$\cosh x ={e^x +e^{-x }\over2},$$
%and the \dfont{hyperbolic sine} is the function
%$$\sinh x ={e^x -e^{-x}\over 2}.$$
\begin{minipage}{.5\textwidth}
\begin{enumerate}
\item		$\ds \cosh x = \frac{e^x+e^{-x}}2$\index{hyperbolic function!definition}
\item		$\ds \sinh x = \frac{e^x-e^{-x}}2$
\item		$\ds \tanh x = \frac{\sinh x}{\cosh x}$
\end{enumerate}
\end{minipage}
\begin{minipage}{.5\textwidth}
\begin{enumerate}\addtocounter{enumi}{3}
\item		$\ds \sech x = \frac{1}{\cosh x}$
\item		$\ds \csch x = \frac{1}{\sinh x}$
\item		$\ds \coth x = \frac{\cosh x}{\sinh x}$
\end{enumerate}
\end{minipage}
\end{definition}

These hyperbolic functions are graphed in Figure \ref{fig:hyperbolic}. In the graphs of $\cosh x$ and $\sinh x$, graphs of $e^x/2$ and $e^{-x}/2$ are included with dashed lines. As $x$ gets ``large,'' $\cosh x$ and $\sinh x$ each act like $e^x/2$; when $x$ is a large negative number, $\cosh x$ acts like $e^{-x}/2$ whereas $\sinh x$ acts like $-e^{-x}/2$.

Notice the domains of $\tanh x$ and $\sech x$ are $(-\infty,\infty)$, whereas both $\coth x$ and $\csch x$ have vertical asymptotes at $x=0$. Also note the ranges of these functions, especially $\tanh x$: as $x\to\infty$, both $\sinh x$ and $\cosh x$ approach $e^{-x}/2$, hence $\tanh x$ approaches $1$.

%It is no coincidence that these functions share a name similar to the trigonometric functions.
The following example explores some of the properties of these functions that bear remarkable resemblance to the properties of their trigonometric counterparts.\\

{\textbf{Pronunciation Note:} \par
``cosh'' rhymes with ``gosh,'' \par ``sinh'' rhymes with ``pinch,'' or alternatively is pronounced``shine" and \par ``tanh'' rhymes with ``ranch.'', or alternatively is pronounced``tank"}

Notice that $\cosh$ is even (that is, $\cosh(-x)=\cosh(x)$)
while $\sinh$ is odd ($\sinh(-x)=-\sinh(x)$), and
$\ds\cosh x + \sinh x = e^x$. Also, for all $x$,
$\cosh x >0$, while $\sinh x=0$ if and only if $\ds e^x -e^{-x }=0$,
which is true precisely when $x=0$.

\begin{theorem}{Range of Hyperbolic Cosine}{Range of Hyperbolic Cosine}
The range of $\cosh x$ is $[1,\infty)$.
\end{theorem}

\begin{proof}
Let $y= \cosh x$. We solve for $x$:
\begin{eqnarray*}
y&=&{e^x +e^{-x }\over 2}\cr
2y &=& e^x + e^{-x }\cr
2ye^x &=& e^{2x} + 1\cr
 0 &=& e^{2x}-2ye^x +1\cr
 e^{x} &=& {2y \pm \sqrt{4y^2 -4}\over 2}\cr
 e^{x} &=& y\pm \sqrt{y^2 -1}
\end{eqnarray*}
From the last equation, we see $\ds y^2 \geq 1$, and since
$y\geq 0$, it follows that $y\geq 1$.

Now suppose $y\geq 1$, so $\ds y\pm \sqrt{y^2 -1}>0$. Then
$\ds x = \ln(y\pm \sqrt{y^2 -1})$ is a real number, and
$y =\cosh x$, so $y$ is in the range of $\cosh(x)$.
\end{proof}

%\begin{definition}{Hyperbolic Functions}{Hyperbolic Functions}
%We can also define hyperbolic functions for the other trigonometric functions as you would expect:
%$$\tanh x=\frac{\sinh x}{\cosh x}\qquad \csch x=\frac{1}{\sinh x}\qquad \sech x=\frac{1}{\cosh x}\qquad \coth x=\frac{1}{\tanh x}$$
%\end{definition}

The graphs can be generated based on our knowledge of the exponential function.

%The graph of $\sinh x$ is shown below:
%$$\includegraphics[width=2.5in]{images2/sinh}$$
%The graph of $\cosh x$ is shown below:
%$$\includegraphics[width=2.5in]{images2/cosh}$$

\begin{figure}
\begin{subfigure}{.5\textwidth}
  \begin{tikzpicture}
\begin{axis}[tick label style={font=\scriptsize},
    xmin=-2.5, xmax=2.5,
    ymin=-2.5, ymax=2.5,
    axis lines=center,
    axis on top=true,
    domain=-2.5:2.5,
    ylabel=$y$,
    xlabel=$x$,
    ]

    \addplot [mark=none,draw=red,ultra thick] {sinh(\x)};
    \addplot [mark=none,draw=blue,dashed, thick] {(1/2)*exp(\x)};
    \addplot [mark=none,draw=blue, dashed,thick] {(-1/2)*exp(-\x)};
    \node [right, red] at (axis cs: 1,0.7) {\scriptsize $y = \sinh x$};
    \node [right, blue] at (axis cs: -2,0.7) {\scriptsize $\ds y = \frac{e^x}{2}$};
    \node [right, blue] at (axis cs: 0.5,-0.7) {\scriptsize $\ds y = -\frac{e^{-x}}{2}$};

    %% Add the asymptotes
   %\draw [blue, dotted, thick] (axis cs:-2.5,-1)-- (axis cs:0,-1);
  %  \draw [blue, dotted, thick] (axis cs:+2.5,+1)-- (axis cs:0,+1);
\end{axis}
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}{.5\textwidth}
\begin{tikzpicture}
\begin{axis}[tick label style={font=\scriptsize},
    xmin=-3, xmax=3,
    ymin=-0.5, ymax=4.5,
    axis lines=center,
    axis on top=true,
    domain=-2.5:2.5,
    ylabel=$y$,
    xlabel=$x$,
    ]

    \addplot [mark=none,draw=red,ultra thick] {cosh(\x)};
    \addplot [mark=none,draw=blue,dashed, thick] {(1/2)*exp(\x)};
    \addplot [mark=none,draw=blue,dashed, thick] {(1/2)*exp(-\x)};
    \node [right, red] at (axis cs: 0.1,3.5) {\scriptsize $y = \cosh x$};
    \node [right, blue] at (axis cs: 1.5,1.3) {\scriptsize $\ds y = \frac{e^x}{2}$};
    \node [right, blue] at (axis cs: -3,1.3) {\scriptsize $\ds y = \frac{e^{-x}}{2}$};

    %% Add the asymptotes
    %\draw [blue, dotted, thick] (axis cs:-2.5,-1)-- (axis cs:0,-1);
    %\draw [blue, dotted, thick] (axis cs:+2.5,+1)-- (axis cs:0,+1);
\end{axis}
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}{.5\textwidth}
\begin{tikzpicture}
\begin{axis}[width=\textwidth,%
tick label style={font=\scriptsize},axis y line=middle,axis x line=middle,name=myplot,axis on top,%
			%x=.37\marginparwidth,
			%y=.37\marginparwidth,
%			xtick=\empty,%
%			extra x ticks={.5,3},
%			extra x tick labels={$a$,$b$},
			ytick={-2,2},
%			yticklabels={$-0.002$,$0.002$,$0.004$},
			%minor y tick num=1,
%			extra y ticks={0.001},%
%			minor x tick num=4,
			ymin=-3.5,ymax=3.5,%
			xmin=-3.5,xmax=3.5,%
			scaled ticks=false
]

\addplot [{\colorone},thick,smooth] coordinates {(-3.,-0.995)(-2.9,-0.994)(-2.8,-0.993)(-2.7,-0.991)(-2.6,-0.989)(-2.5,-0.987)(-2.4,-0.984)(-2.3,-0.98)(-2.2,-0.976)(-2.1,-0.97)(-2.,-0.964)(-1.9,-0.956)(-1.8,-0.947)(-1.7,-0.935)(-1.6,-0.922)(-1.5,-0.905)(-1.4,-0.885)(-1.3,-0.862)(-1.2,-0.834)(-1.1,-0.8)(-1.,-0.762)(-0.9,-0.716)(-0.8,-0.664)(-0.7,-0.604)(-0.6,-0.537)(-0.5,-0.462)(-0.4,-0.38)(-0.3,-0.291)(-0.2,-0.197)(-0.1,-0.0997)(0,0)(0.1,0.0997)(0.2,0.197)(0.3,0.291)(0.4,0.38)(0.5,0.462)(0.6,0.537)(0.7,0.604)(0.8,0.664)(0.9,0.716)(1.,0.762)(1.1,0.8)(1.2,0.834)(1.3,0.862)(1.4,0.885)(1.5,0.905)(1.6,0.922)(1.7,0.935)(1.8,0.947)(1.9,0.956)(2.,0.964)(2.1,0.97)(2.2,0.976)(2.3,0.98)(2.4,0.984)(2.5,0.987)(2.6,0.989)(2.7,0.991)(2.8,0.993)(2.9,0.994)(3.,0.995)};

\draw (axis cs:1.7,-1.5) node {\scriptsize $f(x)=\tanh x$};
\draw (axis cs:2.2,2) node {\scriptsize $f(x)=\coth x$};
\draw [->,>=stealth] (axis cs:1,-1) -- (axis cs:.6,.45);

\addplot [{\colortwo},smooth,thick] coordinates {(-3.,-1.)(-2.9,-1.01)(-2.8,-1.01)(-2.7,-1.01)(-2.6,-1.01)(-2.5,-1.01)(-2.4,-1.02)(-2.3,-1.02)(-2.2,-1.02)(-2.1,-1.03)(-2.,-1.04)(-1.9,-1.05)(-1.8,-1.06)(-1.7,-1.07)(-1.6,-1.08)(-1.5,-1.1)(-1.4,-1.13)(-1.3,-1.16)(-1.2,-1.2)(-1.1,-1.25)(-1.,-1.31)(-0.9,-1.4)(-0.8,-1.51)(-0.7,-1.65)(-0.6,-1.86)(-0.5,-2.16)(-0.4,-2.63)(-0.3,-3.43)(-0.2,-5.07)(-0.1,-10.)};

\addplot [{\colortwo},smooth,thick] coordinates {(0.1,10.)(0.2,5.07)(0.3,3.43)(0.4,2.63)(0.5,2.16)(0.6,1.86)(0.7,1.65)(0.8,1.51)(0.9,1.4)(1.,1.31)(1.1,1.25)(1.2,1.2)(1.3,1.16)(1.4,1.13)(1.5,1.1)(1.6,1.08)(1.7,1.07)(1.8,1.06)(1.9,1.05)(2.,1.04)(2.1,1.03)(2.2,1.02)(2.3,1.02)(2.4,1.02)(2.5,1.01)(2.6,1.01)(2.7,1.01)(2.8,1.01)(2.9,1.01)(3.,1.)};
%\draw (axis cs:2.4,-0.002) node {\scriptsize $f(x)$};
\draw [loosely dashed] (axis cs:-3,1) -- (axis cs:3,1);
\draw [loosely dashed] (axis cs:-3,-1) -- (axis cs:3,-1);

\end{axis}

\node [right] at (myplot.right of origin) {\scriptsize $x$};
\node [above] at (myplot.above origin) {\scriptsize $y$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}{.5\textwidth}
\begin{tikzpicture}
\begin{axis}[width=\textwidth,%
tick label style={font=\scriptsize},axis y line=middle,axis x line=middle,name=myplot,axis on top,%
			%x=.37\marginparwidth,
			%y=.37\marginparwidth,
%			xtick=\empty,%
%			extra x ticks={.5,3},
%			extra x tick labels={$a$,$b$},
			ytick={-3,-2,-1,1,2,3},
%			yticklabels={$-0.002$,$0.002$,$0.004$},
			%minor y tick num=1,
%			extra y ticks={0.001},%
%			minor x tick num=4,
			ymin=-3.5,ymax=3.5,%
			xmin=-3.5,xmax=3.5,%
			scaled ticks=false
]

\addplot [{\colorone},thick,smooth] coordinates {(-3.,0.0993)(-2.9,0.11)(-2.8,0.121)(-2.7,0.134)(-2.6,0.148)(-2.5,0.163)(-2.4,0.18)(-2.3,0.199)(-2.2,0.219)(-2.1,0.241)(-2.,0.266)(-1.9,0.293)(-1.8,0.322)(-1.7,0.354)(-1.6,0.388)(-1.5,0.425)(-1.4,0.465)(-1.3,0.507)(-1.2,0.552)(-1.1,0.599)(-1.,0.648)(-0.9,0.698)(-0.8,0.748)(-0.7,0.797)(-0.6,0.844)(-0.5,0.887)(-0.4,0.925)(-0.3,0.957)(-0.2,0.98)(-0.1,0.995)(0,1.)(0.1,0.995)(0.2,0.98)(0.3,0.957)(0.4,0.925)(0.5,0.887)(0.6,0.844)(0.7,0.797)(0.8,0.748)(0.9,0.698)(1.,0.648)(1.1,0.599)(1.2,0.552)(1.3,0.507)(1.4,0.465)(1.5,0.425)(1.6,0.388)(1.7,0.354)(1.8,0.322)(1.9,0.293)(2.,0.266)(2.1,0.241)(2.2,0.219)(2.3,0.199)(2.4,0.18)(2.5,0.163)(2.6,0.148)(2.7,0.134)(2.8,0.121)(2.9,0.11)(3.,0.0993)};

\draw (axis cs:-2,1.5) node {\scriptsize $f(x)=\sech x$};
\draw (axis cs:2.2,2) node {\scriptsize $f(x)=\csch x$};
%\draw [->,>=latex] (axis cs:1,-1) -- (axis cs:.6,.45);

\addplot [{\colortwo},smooth,thick] coordinates {(-3.,-0.0998)(-2.9,-0.11)(-2.8,-0.122)(-2.7,-0.135)(-2.6,-0.149)(-2.5,-0.165)(-2.4,-0.183)(-2.3,-0.203)(-2.2,-0.224)(-2.1,-0.249)(-2.,-0.276)(-1.9,-0.306)(-1.8,-0.34)(-1.7,-0.378)(-1.6,-0.421)(-1.5,-0.47)(-1.4,-0.525)(-1.3,-0.589)(-1.2,-0.662)(-1.1,-0.749)(-1.,-0.851)(-0.9,-0.974)(-0.8,-1.13)(-0.7,-1.32)(-0.6,-1.57)(-0.5,-1.92)(-0.4,-2.43)(-0.3,-3.28)(-0.2,-4.97)(-0.1,-9.98)};

\addplot [{\colortwo},smooth,thick] coordinates {(0.1,9.98)(0.2,4.97)(0.3,3.28)(0.4,2.43)(0.5,1.92)(0.6,1.57)(0.7,1.32)(0.8,1.13)(0.9,0.974)(1.,0.851)(1.1,0.749)(1.2,0.662)(1.3,0.589)(1.4,0.525)(1.5,0.47)(1.6,0.421)(1.7,0.378)(1.8,0.34)(1.9,0.306)(2.,0.276)(2.1,0.249)(2.2,0.224)(2.3,0.203)(2.4,0.183)(2.5,0.165)(2.6,0.149)(2.7,0.135)(2.8,0.122)(2.9,0.11)(3.,0.0998)};
%\draw (axis cs:2.4,-0.002) node {\scriptsize $f(x)$};
\end{axis}

\node [right] at (myplot.right of origin) {\scriptsize $x$};
\node [above] at (myplot.above origin) {\scriptsize $y$};
\end{tikzpicture}
\end{subfigure}
\caption{Graphs of the Hyperbolic Functions\label{fig:hyperbolic}}
\end{figure}

Certainly the hyperbolic functions do not closely resemble the
trigonometric functions graphically. But they do have analogous
properties, beginning with the following identity.

\begin{theorem}{Hyperbolic Identity}{Hyperbolic Identity}
\label{hyp identity}
For all $x$ in $\R$, $\ds \cosh ^2 x -\sinh ^2 x = 1$.
\end{theorem}

\begin{proof}
The proof is a straightforward computation:
$$\cosh ^2 x -\sinh ^2 x =
 {(e^x +e^{-x} )^2\over 4} -{(e^x -e^{-x} )^2\over 4}=
 {e^{2x} + 2 + e^{-2x } - e^{2x } + 2 - e^{-2x}\over 4}=
 {4\over 4} = 1.
$$
\end{proof}

This immediately gives two additional identities:
$$1-\tanh^2 x =\sech^2 x\qquad\hbox{and}\qquad
\coth^2 x - 1  =\csch^2 x.$$

The identity of the theorem also helps to provide a geometric
motivation. Recall that the graph of $\ds x^2 -y^2 =1$ is a hyperbola
with asymptotes $x=\pm y$ whose $x$-intercepts are $\pm 1$. If
$(x,y)$ is a point on the right half of the hyperbola, and if
we let $x=\cosh \theta$, then
$\ds y=\pm\sqrt{x^2-1}=\pm\sqrt{\cosh^2x-1}=\pm\sinh \theta$. So for some
suitable $\theta$, $\cosh \theta$ and $\sinh \theta$ are the coordinates of a typical
point on the hyperbola. In fact, it turns out that $\theta$ is twice the
area shown in the first graph of  figure~\ref{fig:geometric def trigh}.  Even
this is analogous to trigonometry; $\cos \theta$ and $\sin \theta$ are the
coordinates of a typical point on the unit circle, and $\theta$ is twice
the area shown in the second graph of Figure~\ref{fig:geometric def
trigh}.

\mtable{.7}{Using trigonometric functions to define points on a circle and hyperbolic functions to define points on a hyperbola. The area of the shaded regions are included in them. \label{fig:geometric def trigh}}{fig:hfcircle}{\begin{tikzpicture}
\begin{axis}[width=.45\textwidth,%
axis equal,
tick label style={font=\scriptsize},axis y line=middle,axis x line=middle,name=myplot,axis on top,%
			%x=.37\marginparwidth,
			%y=.37\marginparwidth,
			xtick={-1,1},%
%			extra x ticks={.5,3},
%			extra x tick labels={$a$,$b$},
			ytick={-1,1},
			%minor y tick num=1,%extra y ticks={-5,-3,...,7},%
%			minor x tick num=4,
			ymin=-1.1,ymax=1.1,%
			xmin=-1.1,xmax=1.1%
]

\addplot [{\coloronefill},fill={\coloronefill},domain=0:45] ({cos(x)},{sin(x)}) -- (axis cs:0,0)--cycle;

\addplot [{\colorone},domain=0:360,thick,smooth,samples=100] ({cos(x)},{sin(x)});

\filldraw (axis cs:.707,.707) circle (2pt) node [shift={(13pt,8pt)}] {\scriptsize ($\cos \theta$,$\sin \theta$)};

\draw (axis cs:.6,.25) node {\scriptsize $\ds\frac{\theta}{2}$};
\draw (axis cs:-.75,1) node {\scriptsize $x^2+y^2=1$};

\end{axis}

\node [right] at (myplot.right of origin) {\scriptsize $x$};
\node [above] at (myplot.above origin) {\scriptsize $y$};
\end{tikzpicture}\hspace*{25pt}\begin{tikzpicture}
\begin{axis}[width=.45\textwidth,%
tick label style={font=\scriptsize},axis y line=middle,axis x line=middle,name=myplot,axis on top,%
			%x=.37\marginparwidth,
			%y=.37\marginparwidth,
%			xtick={.25,.5,.75,1},%
%			extra x ticks={.5,3},
%			extra x tick labels={$a$,$b$},
%			ytick=\empty,
			%minor y tick num=1,%extra y ticks={-5,-3,...,7},%
%			minor x tick num=4,
			ymin=-3.1,ymax=3.1,%
			xmin=-3.1,xmax=3.1%
]

\addplot [{\coloronefill},fill={\coloronefill},domain=0:1.6] ({cosh(x)},{sinh(x)}) -- (axis cs:0,0)--cycle;

\addplot [{\colorone},domain=-2:2,thick,smooth] ({cosh(x)},{sinh(x)});
\addplot [{\colorone},domain=-2:2,thick,smooth] ({-cosh(x)},{sinh(x)});

\filldraw (axis cs:2.577,2.376) circle (2pt) node [left] {\scriptsize ($\cosh \theta$,$\sinh \theta$)};

\draw (axis cs:.73,.32) node {\scriptsize $\frac{\theta}{2}$};
\draw (axis cs:-1.75,2.75) node {\scriptsize $x^2-y^2=1$};

\end{axis}

\node [right] at (myplot.right of origin) {\scriptsize $x$};
\node [above] at (myplot.above origin) {\scriptsize $y$};
\end{tikzpicture}}

%\begin{figure}[!ht]
%$$\includegraphics[width=5in]{images2/sinh-and-cosh-motivation}$$
%\caption{\label{fig:geometric def trigh}
%Geometric definitions. Here, $t$ is twice the shaded area in each figure.}
%\end{figure}

\begin{example}{Computing Hyperbolic Tangent}{Computing Hyperbolic Tangent}
 Use the definition of the hyperbolic functions to rewrite the following expressions.
\begin{enumerate}
%\item		$\cosh^2 x-\sinh^2x$
\item		$\tanh^2 x+\sech^2 x$
\item		$2\cosh x\sinh x$
\item		$\tanh(\ln 2)$.
\end{enumerate}
\end{example}

\begin{solution}
\begin{enumerate}
%\item		%\vskip-\baselineskip%
%\hfill$\begin{aligned}[t]
% \cosh^2x-\sinh^2x &= \left(\frac{e^x+e^{-x}}2\right)^2 -\left(\frac{e^x-e^{-x}}2\right)^2\\
% 						&= \frac{e^{2x}+2e^xe^{-x} + e^{-2x}}4 - \frac{e^{2x}-2e^xe^{-x} + e^{-2x}}4\\
% 						&= \frac44=1.
%\end{aligned}$\hfill
%So $\cosh^2 x-\sinh^2x=1$.
\item $\begin{aligned}[t]
\tanh^2 x+\sech^2 x  =\frac{\sinh^2x}{\cosh^2 x} + \frac{1}{\cosh^2 x}
					&= \frac{\sinh^2x+1}{\cosh^2 x}\qquad \text{\small Now use Theorem \ref{hyp identity}.}\\
					&= \frac{\cosh^2 x}{\cosh^2 x} = 1.\\
\end{aligned}$

So $\tanh^2 x+\sech^2 x=1$.
\item $\begin{aligned}[t]
	2\cosh x\sinh x &= 2\left(\frac{e^x+e^{-x}}2\right)\left(\frac{e^x-e^{-x}}2\right) \\
					&= 2 \cdot\frac{e^{2x} - e^{-2x}}4\\
					&= \frac{e^{2x} - e^{-2x}}2 = \sinh (2x).\\
			\end{aligned}$

Thus $2\cosh x\sinh x = \sinh (2x)$.
\item $\begin{aligned}[t]
	\tanh(\ln 2) & =\frac{\sinh(\ln 2)}{\cosh(\ln 2)}\\
	             & = \frac{\ds\frac{e^{\ln 2}-e^{-\ln 2}}{2}}{\ds\frac{e^{\ln 2}+e^{-\ln 2}}{2}}\\
	%= \frac{\ds\frac{e^{\ln 2}-e^{\ln(1/2)}}{2}}{\ds\frac{e^{\ln 2}+e^{\ln(1/2)}}{2}}
	&=  \frac{\ds\frac{2-(1/2)}{2}}{\ds\frac{2+(1/2)}{2}}
	 =\frac{2-(1/2)}{2+(1/2)}
	%=  \frac{3/2}{5/2}
	= \frac{3}{5}\\
			\end{aligned}$

Thus $\tanh(\ln 2) = \frac35$.
\end{enumerate}
\end{solution}

The following summarizes some of the important identities relating to hyperbolic functions. Each can be verified by referring back to Definition \ref{def:hyperbolic_functions}.

\textbf{Basic Identities}\par
\begin{enumerate}
\item $\cosh^2x-\sinh^2x=1$%
\index{hyperbolic function!identities}\index{hyperbolic function!derivatives}\index{hyperbolic function!integrals}\index{derivative!hyperbolic funct.}\index{integration!hyperbolic funct.}%
\item	$\tanh^2x+\sech^2x=1$
\item	$\coth^2x-\csch^2x = 1$
\item	$\cosh 2x=\cosh^2x+\sinh^2x$
\item	$\sinh 2x = 2\sinh x\cosh x$
\item	$\ds\cosh^2x = \frac{\cosh 2x+1}{2}$
\item $\ds \sinh^2x=\frac{\cosh 2x-1}{2}$
\end{enumerate}

%\figure
%\texonly
%\hbox to \hsize{\hfill
%\tikzpicture[domain=0:3,x=6mm,y=6mm,baseline=0]
%\draw (0,0) -- (3,0) ;
%\draw (0,-3) -- (0,3) ;
%\foreach \x in {1,2,3} \draw (\x,0) -- (\x,-2pt) node[anchor=north] {\eightpoint $\x$};
%\foreach \y in {-3,-2,-1,0,1,2,3} \draw (0,\y) -- (-2pt,\y) node[anchor=east]
%{\eightpoint $\y$};
%\node[anchor=west] at (2,1.3) {\ninepoint $(\cosh t,\sinh t)$};
%\global\advance\gpnum by 1
%\draw[color=black] plot[parametric,id=\the\gpnum,domain=-1:1]
%function{1+2*t**2,2*t*sqrt(t**2+1)};
%\draw (0,0) -- (2,1.732);
%\gpad
%\fill[opacity=0.5,fill=red!20]
%(0,0) -- (1,0)
%plot[parametric,id=\the\gpnum,domain=0:.707]
%function{1+2*t**2,2*t*sqrt(t**2+1);} -- (0,0);
%\node at  (2,1.732) {\fivepoint$\bullet$};
%\endtikzpicture
%\hskip2cm
%\tikzpicture[domain=-1.2:1.2,baseline=0]
%\draw (-1.2,0) -- (1.2,0) ;
%\draw (0,-1.2) -- (0,1.2) ;
%\foreach \x in {1} \draw (\x,0) -- (\x,-2pt) node[anchor=north west]
         %{\ninepoint $\x$};
%\node[anchor=south west] at (0.5,0.886) {\ninepoint $(\cos t,\sin t)$};
%\draw[color=black] (0,0) circle (1);
%\draw (0,0) -- (0.5,0.886);
%\fill[opacity=0.5,fill=red!20]
%(0,0) -- (1,0) arc (0:60:1) -- (0,0);
%\node[anchor=mid] at  (0.5,0.886) {\fivepoint$\bullet$};
%\endtikzpicture\hfill}
%\endtexonly
%\figrdef{fig:geometric def trigh}
%\htmlfigure{Transcendental-GeomDefs.html}
%\begincaption
%Geometric definitions of sin, cos, sinh, cosh: $t$ is twice the shaded
%area in each figure.
%\endcaption
%\endfigure

Since $\cosh x > 0$, $\sinh x$ is increasing and hence one-to-one, so
$\sinh x$ has an inverse, ${\rm arcsinh} x$. Also, $\sinh x > 0$ when
$x>0$, so $\cosh x$ is injective on $[0,\infty)$ and has a (partial)
inverse, ${\rm arccosh} x$. The other hyperbolic functions have inverses
as well, though ${\rm arcsech} x$ is only a partial inverse.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Opensolutionfile{solutions}[ex]
\section*{Exercises for \ref{sec:hyperbolic functions}}

\begin{enumialphparenastyle}

%%%%%%%%%%

\begin{ex}
Verify the given identity using Definition \ref{def:hyperbolic_functions}, as done in Example \ref{exa:Computing Hyperbolic Tangent}.
\begin{enumerate}
\item $\coth^2 x-\csch^2x=1$.
\item {$\cosh 2x = \cosh^2x+\sinh^2x$}

\item {$\ds\cosh^2x = \frac{\cosh2x+1}{2}$}

\item {$\ds\sinh^2x = \frac{\cosh2x-1}{2}$}

\end{enumerate}
	\begin{sol}
		\begin{enumerate}
		\item {\hfill$\begin{aligned}[t]
				\coth^2x-\csch^2x &= \left(\frac{e^x+e^{-x}}{e^x-e^{-x}}\right)^2 - \left(\frac{2}{e^x-e^{-x}}\right)^2 \\
													&= \frac{(e^{2x} + 2 + e^{-2x}) - (4)}{e^{2x} - 2 + e^{-2x}}\\
													&= \frac{e^{2x} - 2 + e^{-2x}}{e^{2x} - 2 + e^{-2x}}\\
													&= 1
		\end{aligned}$\hfill\null}
		\item {\hfill$\begin{aligned}[t]
				\cosh^2x+\sinh^2x &= \left(\frac{e^x+e^{-x}}{2}\right)^2 + \left(\frac{e^x-e^{-x}}{2}\right)^2 \\
													&= \frac{e^{2x} + 2 + e^{-2x}}{4} + \frac{e^{2x} - 2 + e^{-2x}}{4}\\
													&= \frac{2e^{2x}  + 2e^{-2x}}{4}\\
													&= \frac{e^{2x} + e^{-2x}}{2} \\
													&= \cosh 2x.
		\end{aligned}$\hfill\null}
		\item {\hfill$\begin{aligned}[t]
				\cosh^2x &= \left(\frac{e^x+e^{-x}}{2}\right)^2  \\
													&= \frac{e^{2x} + 2 + e^{-2x}}{4} \\
													&= \frac12\frac{(e^{2x}  + e^{-2x})+2}{2}\\
													&= \frac12\left(\frac{e^{2x}  + e^{-2x}}{2}+1\right)\\
													&= \frac{\cosh2x+1}{2}.
		\end{aligned}$\hfill\null}
		\item {\hfill$\begin{aligned}[t]
				\sinh^2x &= \left(\frac{e^x-e^{-x}}{2}\right)^2  \\
													&= \frac{e^{2x} - 2 + e^{-2x}}{4} \\
													&= \frac12\frac{(e^{2x}  + e^{-2x})-2}{2}\\
													&= \frac12\left(\frac{e^{2x}  + e^{-2x}}{2}-1\right)\\
													&= \frac{\cosh2x-1}{2}.
		\end{aligned}$\hfill\null}
		\end{enumerate}
	\end{sol}
\end{ex}

\begin{ex}
Show that the range of $\sinh x$ is all real
numbers. (Hint: show that if $y=\sinh x$ then
$\ds x =\ln (y+\sqrt{y^2+1})$.)
\end{ex}

\begin{ex}
Show that the range of $\tanh x$ is $(-1,1)$. What
are the ranges of $\coth$, $\sech$, and $\csch$?
(Use the fact that they are reciprocal functions.)
\end{ex}

\begin{ex}
Prove that for every $x,y\in\R$, $\sinh (x+y)
=\sinh x \cosh y + \cosh x \sinh y$. Obtain a similar identity for
$\sinh(x-y)$.
\end{ex}

\begin{ex}
Prove that for every $x,y\in\R$, $\cosh (x+y) =\cosh x \cosh y + \sinh x
  \sinh y$. Obtain a similar identity for $\cosh(x-y)$.
\end{ex}

\begin{ex}
Show that $\sinh(2x)=2\sinh x \cosh x$ and $\ds \cosh(2x)=\cosh^2 x
+\sinh^2 x$ for every $x$.  Conclude also that $\ds (\cosh (2x) -1)/2 = \sinh
^2 x$.
\end{ex}

%\begin{ex}
%What are the domains of the six inverse hyperbolic functions?
%\end{ex}
%
%\begin{ex}
%Sketch the graphs of all six inverse hyperbolic
%  functions.
%\end{ex}

\end{enumialphparenastyle}


\documentclass[dvipdfmx]{article}
\usepackage[dvipdfmx]{graphicx}
\usepackage[dvipdfmx]{hyperref}
\begin{document}

\section*{Transpose}
The arbitrary matrix $A$ is transposed to matrix $A^T$.\\
The example is shown as
\[
  A = \left(
    \begin{array}{ccc}
      1 & 2 & 3 \\
      4 & 5 & 6 \\
      7 & 8 & 9
    \end{array}
  \right)
  , \ \ \
  A^T = \left(
    \begin{array}{ccc}
      1 & 4 & 7 \\
      2 & 5 & 8 \\
      3 & 6 & 9
    \end{array}
  \right) .
\]

\section*{Trace}
The trace of a square matrix A (${\rm{tr}}A$) is defined to be the sum of elements on the main diagonal of A.
The example is shown as
\[
  A =
  \left(
    \begin{array}{ccc}
      a_{11} & a_{12} & a_{13} \\
      a_{21} & a_{22} & a_{23} \\
      a_{31} & a_{32} & a_{33}
    \end{array}
  \right)  =
  \left(
    \begin{array}{ccc}
      1 & 2 & 3 \\
      4 & 5 & 6 \\
      7 & 8 & 9
    \end{array}
  \right)
\]
then \
${\rm{tr}}A =  {\sum_i}a_{ii} = 15$.

\section*{Determinant}
For a 3 × 3 matrix A, its determinant is \\
\[
  |A| =
    \left|
    \begin{array}{ccc}
      a_{11} & a_{12} & a_{13} \\
      a_{21} & a_{22} & a_{23} \\
      a_{31} & a_{32} & a_{33}
    \end{array}
  \right| =
   a_{11}
  \left|
    \begin{array}{ccc}
      \times & \times & \times \\
      \times & a_{22} & a_{23} \\
      \times & a_{32} & a_{33}
    \end{array}
  \right|
  - a_{12}
  \left|
    \begin{array}{ccc}
      \times & \times & \times \\
      a_{21} & \times & a_{23} \\
      a_{31} & \times & a_{33}
    \end{array}
  \right|
  + a_{13}
    \left|
    \begin{array}{ccc}
      \times & \times & \times \\
      a_{21} & a_{22} & \times \\
      a_{31} & a_{32} & \times
    \end{array}
  \right|
\]
\[
   = a_{11}
     \left|
    \begin{array}{ccc}
      a_{22} & a_{23} \\
      a_{32} & a_{33}
    \end{array}
  \right|
  - a_{12}
  \left|
    \begin{array}{ccc}
      a_{21} & a_{23} \\
      a_{31} & a_{33}
    \end{array}
  \right|
  + a_{13}
    \left|
    \begin{array}{ccc}
      a_{21} & a_{22}  \\
      a_{31} & a_{32}
    \end{array}
  \right|
\]
\[
  \ \ \ \ \ \ = a_{11} a_{22}
     \left|
    \begin{array}{ccc}
      \times & \times \\
      \times & a_{33}
    \end{array}
  \right|
  - a_{11} a_{23}
     \left|
    \begin{array}{ccc}
      \times & \times \\
      a_{32} & \times
    \end{array}
  \right|
  - a_{12} a_{21}
  \left|
    \begin{array}{ccc}
      \times & \times \\
      \times & a_{33}
    \end{array}
  \right|
   \]
 \[
   \ \ \ \ \ \ + a_{12} a_{23}
  \left|
    \begin{array}{ccc}
      \times & \times \\
      a_{31} & \times
    \end{array}
  \right|
  + a_{13} a_{21}
    \left|
    \begin{array}{ccc}
      \times & \times \\
      \times & a_{32}
    \end{array}
  \right|
    - a_{13} a_{22}
    \left|
    \begin{array}{ccc}
      \times & \times \\
      a_{31} & \times
    \end{array}
  \right|
\]
\ \ \ \ \ = $a_{11}a_{22}a_{33} - a_{11}a_{23}a_{32} - a_{12}a_{21}a_{33} + a_{12}a_{23}a_{31} + a_{13}a_{22}a_{31}$

\section*{Cramer's rule}
In a system of $n$ linear equations, represented in matrix multiplication form
$A{\bf{x}} = {\bf{b}}$ \\
where $A$ is the $n {\times} n$ matrix and {\bf{x}} and {\bf{b}} are the $n$-th column vectors.
${\bf{x}} = (x_1, {\cdots}, x_n)^T$ , ${\bf{b}} = (b_1, {\cdots}, b_n)^T$. \\
Then, if $|A| {\neq} 0$,  \\
 \[
   x_i = |A_i| / |A|, \ \ \
   A_i =
   \left(
   \begin{array}{ccccc}
   a_{11} & \cdots & b_{1i} & \cdots & a_{1n} \\
   \vdots & \ddots  & \vdots & 	     &  \vdots \\
   a_{k1} &             & b_{ki} &           & a_{kn} \\
   \vdots &            & \vdots & \ddots  &  \vdots \\
   a_{n1} & \cdots & b_{ni} & \cdots &  a_{nn}
   \end{array}
   \right)
   \]
This is Cramer's rule.

\section*{LU decomposition}
$A = LU$ where
 \[
   A =
   \left(
   \begin{array}{ccc}
   a_{11} & a_{12} & a_{13} \\
   a_{21} & a_{22} & a_{23} \\
   a_{31} & a_{32} & a_{33}
   \end{array}
   \right) ,
    L =
   \left(
   \begin{array}{ccc}
   1 & 0 & 0 \\
   l_{21} & 1 & 0 \\
   l_{31} & l_{32} & 1
   \end{array}
   \right) ,
     U =
   \left(
   \begin{array}{ccc}
   u_{11} & u_{12} & u_{13} \\
   0         & u_{22} & u_{23} \\
   0         & 0         & u_{33}
   \end{array}
   \right)
  \]

\section*{Direct method by LU decomposition}
In linear equation $A{\bf{x}} = {\bf{b}}$,
$LU{\bf{x}} = {\bf{b}}$
by using LU decomposition $A = LU$.

Here, we consider $L{\bf{y}} = {\bf{b}}$ and $U{\bf{x}} = {\bf{y}}$.

In forward substitution,
\begin{eqnarray}
y_1 &&= b_1 \nonumber \\
y_2 &&= b_2 - l_{21}y_1 \nonumber \\
{\vdots} \nonumber \\
y_n &&= b_n - {\sum_{j=1}^{n-1}} l_{nj}y_j \nonumber
\end{eqnarray}

In backforward substitution,
\begin{eqnarray}
x_n &&= y_n / u_{nn} \nonumber \\
x_{n-1} &&= (y_{n-1} - u_{n-1, n}x_n) / u_{n-1, n-1} \nonumber \\
{\vdots} \nonumber \\
x_1 &&= (y_1 - {\sum_{j=2}^{n}} u_{1, j}x_j) / u_{11} \nonumber
\end{eqnarray} .

\section*{Constant multiple}
 \[
   c
   \left(
   \begin{array}{ccc}
   a_{11} & \cdots & a_{1n} \\
   \vdots & \ddots  & \vdots \\
   a_{n1} & \cdots & a_{nn}
   \end{array}
   \right)
   =
   \left(
   \begin{array}{ccc}
   ca_{11} & \cdots & ca_{1n} \\
   \vdots & \ddots  & \vdots \\
   ca_{n1} & \cdots & ca_{nn}
   \end{array}
   \right)
   \]
where $c$ is the scalar constant.

\section*{Inverse matrix}
$AB = BA = I$ \\
where $A$ and $B$ is the $n$ × $n$ matrices and $I$ is the $n$ × $n$ unit matrix.
In the case, the matrix $B$ is uniquely determined by $A$ and is called the inverse matrix of $A$.
The inverse matrix of $A$ is denoted by $A^{-1}$.

\section*{Product}
The elements of the matrix product $C = AB$ is that
$c_{ij} = [AB]_{ij} = {\sum_k}a_{ik}b_{kj}$
where $A$ is an $n \times m$ matrix and $B$ is an $m \times l$  matrix.

\section*{Addition and Subtraction}
 \[
   A=
   \left(
   \begin{array}{ccc}
   a_{11} & a_{12} & a_{13} \\
   a_{21} & a_{22}  & a_{23} \\
   a_{31} & a_{32} & a_{33}
   \end{array}
   \right) ,
   B =
   \left(
   \begin{array}{ccc}
   b_{11} & b_{12} & b_{13} \\
   b_{21} & b_{22}  & b_{23} \\
   b_{31} & b_{32} & b_{33}
   \end{array}
   \right) ,
 \]
then the addition/subtraction is that
 \[
   A {\pm} B=
   \left(
   \begin{array}{ccc}
   a_{11}  {\pm} b_{11} & a_{12}  {\pm} b_{12}  & a_{13}  {\pm} b_{13} \\
   a_{21}  {\pm} b_{21} & a_{22}  {\pm} b_{22} & a_{23}  {\pm} b_{23} \\
   a_{31}  {\pm} b_{31} & a_{32}  {\pm} b_{32} & a_{33}  {\pm} b_{33}
   \end{array}
   \right)
 \]

\section*{Hadamard product}
 \[
   A=
   \left(
   \begin{array}{ccc}
   a_{11} & a_{12} & a_{13} \\
   a_{21} & a_{22}  & a_{23} \\
   a_{31} & a_{32} & a_{33}
   \end{array}
   \right) ,
   B =
   \left(
   \begin{array}{ccc}
   b_{11} & b_{12} & b_{13} \\
   b_{21} & b_{22}  & b_{23} \\
   b_{31} & b_{32} & b_{33}
   \end{array}
   \right) ,
 \]
then the Hadamard product is that
 \[
   A {\circ} B=
   \left(
   \begin{array}{ccc}
   a_{11} b_{11} & a_{12} b_{12}  & a_{13} b_{13} \\
   a_{21} b_{21} & a_{22} b_{22} & a_{23} b_{23} \\
   a_{31} b_{31} & a_{32} b_{32} & a_{33} b_{33}
   \end{array}
   \right)
 \]

\section*{Hadamard division}
 \[
   A=
   \left(
   \begin{array}{ccc}
   a_{11} & a_{12} & a_{13} \\
   a_{21} & a_{22}  & a_{23} \\
   a_{31} & a_{32} & a_{33}
   \end{array}
   \right) ,
   B =
   \left(
   \begin{array}{ccc}
   b_{11} & b_{12} & b_{13} \\
   b_{21} & b_{22}  & b_{23} \\
   b_{31} & b_{32} & b_{33}
   \end{array}
   \right) ,
 \]
then the Hadamard division is that
 \[
   A / B=
   \left(
   \begin{array}{ccc}
   a_{11} / b_{11} & a_{12} / b_{12}  & a_{13} / b_{13} \\
   a_{21} / b_{21} & a_{22} / b_{22} & a_{23} / b_{23} \\
   a_{31} / b_{31} & a_{32} / b_{32} & a_{33} / b_{33}
   \end{array}
   \right)
 \]

\section*{Hadamard power}
 \[
   A^{(n)}=
   \left(
   \begin{array}{ccc}
   a_{11}^n & a_{12}^n & a_{13}^n \\
   a_{21}^n & a_{22}^n  & a_{23}^n \\
   a_{31}^n & a_{32}^n & a_{33}^n
   \end{array}
   \right)
 \]
 where $n$ is scalar.

\section*{Tensor product}
 \[
   A=
   \left(
   \begin{array}{ccc}
   a_{11} & a_{12} & a_{13} \\
   a_{21} & a_{22}  & a_{23} \\
   a_{31} & a_{32} & a_{33}
   \end{array}
   \right) ,
   B =
   \left(
   \begin{array}{ccc}
   b_{11} & b_{12} & b_{13} \\
   b_{21} & b_{22}  & b_{23} \\
   b_{31} & b_{32} & b_{33}
   \end{array}
   \right) ,
 \]
then the tensor product is that
 \[
   A {\otimes} B =
   \left(
   \begin{array}{ccc}
   a_{11} B & a_{12} B  & a_{13} B \\
   a_{21} B & a_{22} B & a_{23} B \\
   a_{31} B & a_{32} B & a_{33} B
   \end{array}
  \right)
\]
\[
    =
   \left(
   \begin{array}{ccccccccc}
   a_{11} b_{11} & a_{11} b_{12} & a_{11} b_{13} & a_{12} b_{11} & a_{12} b_{12} & a_{12} b_{13} & a_{13} b_{11} & a_{13} b_{12} & a_{13} b_{13} \\
   a_{11} b_{21} & a_{11} b_{22} & a_{11} b_{23} & a_{12} b_{21} & a_{12} b_{22} & a_{12} b_{23} & a_{13} b_{21} & a_{13} b_{22} & a_{13} b_{23} \\
   a_{11} b_{31} & a_{11} b_{32} & a_{11} b_{33} & a_{12} b_{31} & a_{12} b_{32} & a_{12} b_{33} & a_{13} b_{31} & a_{13} b_{32} & a_{13} b_{33} \\
   a_{21} b_{11} & a_{21} b_{12} & a_{21} b_{13} & a_{22} b_{11} & a_{22} b_{12} & a_{22} b_{13} & a_{23} b_{11} & a_{23} b_{12} & a_{23} b_{13} \\
   a_{21} b_{21} & a_{21} b_{22} & a_{21} b_{23} & a_{22} b_{21} & a_{22} b_{22} & a_{22} b_{23} & a_{23} b_{21} & a_{23} b_{22} & a_{23} b_{23} \\
   a_{21} b_{31} & a_{21} b_{32} & a_{21} b_{33} & a_{22} b_{31} & a_{22} b_{32} & a_{22} b_{33} & a_{23} b_{31} & a_{23} b_{32} & a_{23} b_{33} \\
   a_{31} b_{11} & a_{31} b_{12} & a_{31} b_{13} & a_{32} b_{11} & a_{32} b_{12} & a_{32} b_{13} & a_{33} b_{11} & a_{33} b_{12} & a_{33} b_{13} \\
   a_{31} b_{21} & a_{31} b_{22} & a_{31} b_{23} & a_{32} b_{21} & a_{32} b_{22} & a_{32} b_{23} & a_{33} b_{21} & a_{33} b_{22} & a_{33} b_{23} \\
   a_{31} b_{31} & a_{31} b_{32} & a_{31} b_{33} & a_{32} b_{31} & a_{32} b_{32} & a_{32} b_{33} & a_{33} b_{31} & a_{33} b_{32} & a_{33} b_{33}
   \end{array}
   \right)
 \]

\section*{Eigenvalue (Algebraic method)}
An eigen equation is written as
$A{\bf{u}} = {\lambda}{\bf{u}}$
where ${\lambda}$ is scalar and ${\bf{u}}$ is vector, known as the eigenvalue and eigenvector.

By rearranging above equation, we obtain:  $A{\bf{u}} = {\lambda}{\bf{u}}$, $(A - {\lambda}I){\bf{u}} = {\bf{0}}$.
If this equation has a nontrivial solution (${\bf{u}} {\neq} 0$),
the determinant $|A - {\lambda}I| = 0$.

\begin{flushleft}
[$2 {\times} 2$ matrix case]
\end{flushleft}
When the matrix $A$ is written as
 \[
   A=
   \left(
   \begin{array}{cc}
   a_{11} & a_{12}  \\
   a_{21} & a_{22} \\
   \end{array}
   \right) ,
  \]
the quadratic equation ${\lambda}^2 -(a_{11} - a_{22}){\lambda} + a_{11}a_{22} - a_{12}a_{21}$ is obtained.
By using quadratic formula,
\begin{eqnarray}
{\lambda} =  \frac{ a_{11} -a_{22} {\pm} \sqrt{(a_{11}-a_{22})^2 - 4(a_{11}a_{22} - a_{12}a_{21})} }{2}. \nonumber
\end{eqnarray} \\

\begin{flushleft}
[$3 {\times} 3$ matrix case]
\end{flushleft}
When the matrix $A$ is written as
 \[
   A=
   \left(
   \begin{array}{ccc}
   a_{11} & a_{12} & a_{13} \\
   a_{21} & a_{22} & a_{23} \\
   a_{31} & a_{32} & a_{33} \\
   \end{array}
   \right) ,
  \]
we obtain the cubic equation $a{\lambda}^3 + b{\lambda}^2 + c{\lambda} + d = 0$  where \\
$a = -1$, \\
$b = a_{11} + a_{22} + a_{33}$, \\
$c = a_{21}a_{12} + a_{13}a_{31} + a_{32}a_{23} - a_{11}a_{22} - a_{11}a_{33} - a_{22}a_{33}$, \\
$d =  a_{11}a_{22}a_{33} + a_{12}a_{23}a_{31} + a_{13}a_{32}a_{21} - a_{11}a_{32}a_{23} - a_{22}a_{31}a_{13} - a_{33}a_{21}a_{12}$ . \\

Therefore,
we can solve the eigen equation in the case of the $3{\times}3$ matrix $A$ by substituting above $a$, $b$, $c$ and $d$ for the cubic formula.
The cubic formula is that
\begin{eqnarray}
{\lambda}_1 &&= -\frac{b}{3a} \nonumber \\
       &&- \frac{1}{3a} \sqrt[3]{ \frac{1}{2} (2b^3 -9abc + 27a^2d + \sqrt{(ab^3) - 9abc + 27a^2d)^2 - 4(b^2 - 3ac)^3} ) } \nonumber \\
       &&- \frac{1}{3a} \sqrt[3]{ \frac{1}{2} (2b^3 -9abc + 27a^2d - \sqrt{(ab^3) - 9abc + 27a^2d)^2 - 4(b^2 - 3ac)^3} ) } \ \ \ , \nonumber \\
\nonumber \\
{\lambda}_2 &&= -\frac{b}{3a} \nonumber \\
       &&- \frac{1 + i \sqrt{3}}{6a} \sqrt[3]{ \frac{1}{2} (2b^3 -9abc + 27a^2d + \sqrt{(ab^3) - 9abc + 27a^2d)^2 - 4(b^2 - 3ac)^3} ) } \nonumber \\
       &&- \frac{1 - i \sqrt{3}}{6a} \sqrt[3]{ \frac{1}{2} (2b^3 -9abc + 27a^2d - \sqrt{(ab^3) - 9abc + 27a^2d)^2 - 4(b^2 - 3ac)^3} ) } \ \ \ , \nonumber \\
\nonumber \\
{\lambda}_3 &&= -\frac{b}{3a} \nonumber \\
       &&- \frac{1 - i \sqrt{3}}{6a} \sqrt[3]{ \frac{1}{2} (2b^3 -9abc + 27a^2d + \sqrt{(ab^3) - 9abc + 27a^2d)^2 - 4(b^2 - 3ac)^3} ) } \nonumber \\
       &&- \frac{1 + i \sqrt{3}}{6a} \sqrt[3]{ \frac{1}{2} (2b^3 -9abc + 27a^2d - \sqrt{(ab^3) - 9abc + 27a^2d)^2 - 4(b^2 - 3ac)^3} ) } \ \ \ . \nonumber
\end{eqnarray}

In this Elixir library, the complex numbers in the above equations are calculated as Gaussian plane.
\begin{center}
  \includegraphics[width=10cm]{gaussian_plane.jpg}
\end{center}

The real part and imaginary part are calculated by using arctangent's integral formula, written as
 \begin{eqnarray}
 \arctan{x} = {\int_0^x} \frac{1}{z^2 + 1} dz. \nonumber
 \end{eqnarray}
 This formula is treated as the numerical integration.

\section*{Eigenvalue and eigenvector (Power iteration method to solve maximum eigenvalue and eigenvector of $n$-th eigen equation)}
An arbitrary (initial) vector ${\bf{b}}^0$ is written by the linear combination of eigenvectors ${\sum_i} c_i{\bf{u}}_i$
because eigenvectors are linearly independent.
\begin{eqnarray}
{\bf{b}}^k {\equiv} A^k {\bf{b}}^0 = A^k {\sum_i} c_i{\bf{u}}_i = {\sum_{i=1}} c_i {\lambda}^k_i {\bf{u}}_i \nonumber \\
= {\lambda}_1^k (c_1{\bf{u}}_1 + {\sum_{i=2}} c_i \frac{{\lambda}^k_i}{{\lambda}^k_1} {\bf{u}}_i)  \nonumber
\end{eqnarray}
where ${\lambda}_1$ is maximum value of the eigenvalue so that $|\frac{{\lambda}^k_i}{{\lambda}^k_1}| < 1$.

If $k$ is a large enough number,
we can write the eigenvector of the maximum eigenvalue, shown as
\begin{eqnarray}
{\bf{b}}^k {\simeq} {\lambda}^k_1c_1{\bf{u}}_1. \nonumber
\end{eqnarray}
Moreover,  we can write the maximum eigenvalue
\begin{eqnarray}
{\lambda}_1 = \frac{({\bf{b}}^k)^TA{\bf{b}}^k}{({\bf{b}}^k)^T{\bf{b}}^k}. \nonumber
\end{eqnarray}

\section*{Eigenvalue and eigenvector (Jacobi method)}
The Jacobi method is an iterative method for the numerical calculation of the eigenvalues and eigenvectors of a real $n$-th symmetric matrix.
(cf. \url{https://en.wikipedia.org/wiki/Jacobi_eigenvalue_algorithm} )

\section*{Eigenvalue (iterative method using QR decomposition)}
The iterative method using QR decomposition is used to calculate eigenvalues of a $n$-th real square matrix.
The QR decomposition is the decomposition of matrix $A$ into orthogonal matrix $Q$ and upper-triangular matrix $R$ as shown below.
$$
A = QR
$$
In this section, QR decomposition is performed by using Householder transformation.
Householder transformation corresponds to matrix $H$ below.
\begin{eqnarray}
{\bf{x}} = H {\bf{y}} \nonumber \\
{\bf{y}} = H {\bf{x}} \nonumber
\end{eqnarray}
\begin{eqnarray}
H = I - \frac{2({\bf{x}} - {\bf{y}})({\bf{x}} - {\bf{y}})^T}{|{\bf{x}} - {\bf{y}}|^2} \nonumber
\end{eqnarray}
$H$ is orthogonal matrix where $HH^T=I$.
Here,
\[
  A = A^{(1)} =
  \left(
  \begin{array}{ccc}
  a_{11} & \cdots & a_{1n} \\
  \vdots & \ddots & \vdots \\
  a_{n1} & \cdots & a_{nn}
  \end{array}
  \right),
\]
$$
A^{(2)}=H^{(1)}A^{(1)}, \\
A^{(2)} =
\left(
\begin{array}{cccc}
a_{11} & a_{12} & \cdots & a_{1n} \\
0      & \vdots & \vdots & \vdots \\
\vdots & \vdots & \vdots & \vdots \\
0      & a_{n2} & \cdots & a_{nn}
\end{array}
\right).
$$
$H^{(1)}$ to Householder transform the first column into a finite vector with only the top element.

$$
A^{(3)}=H^{(2)}A^{(2)}, \\
A^{(3)} =
\left(
\begin{array}{ccccc}
a_{11} & a_{12} & a_{12} & \cdots & a_{1n} \\
0      & a_{22} & \vdots & \vdots & \vdots \\
0      & 0      & \vdots & \vdots & \vdots \\
\vdots & \vdots & \vdots & \vdots & \vdots \\
0      & 0      & a_{n2} & \cdots & a_{nn}
\end{array}
\right).
$$
$H^{(2)}$ to Householder transform the second column into a finite vector with the top 2 elements.

$$
H^{(n-1)} {\cdots} H^{(1)} A^{(1)} = R,
$$
$$
A = A^{(1)} = H^{(n-1)T} {\cdots} H^{(1)T} R = QR.
$$
The product of orthogonal matricies $Q_1$ and $Q_2$ is also orthogonal matrix $Q_3(=Q_1Q_2)$.

Thus,
$$
A = A_1 = Q_1R_1, A_2 = R_1Q_1,
$$
$$
A_{k+1} = (Q_k^{-1}{\cdots}Q_1^{-1}) A_1 (Q_1{\cdots}Q_k) = {\tilde{Q}}_k^{-1}A_1{\tilde{Q}}_k.
$$

On the other hand,
$$
A_1^k = Q_1R_1 {\cdots} Q_1R_1 = Q_1(R_1Q_1){\cdots}(R_1Q_1)R_1 = Q_1{\cdots}Q_k R_k{\cdots}R_1 = {\tilde{Q}}_k {\tilde{R}}_k.
$$

The eigen equations are written as follows.
$$
AX=X{\Lambda}.
$$
Then,
$$
A^k = A_1^k = X{\Lambda}X^{-1} {\cdots} X{\Lambda}X^{-1} = X{\Lambda}^kX^{-1}.
$$

$QR$ decomposition of $X$ and $LU$ decomposition of $X^{-1}$.
$$
X = Q_XR_X, X^{^-1} = LU.
$$
$$
A_1^k = X{\Lambda}X^{-1} = Q_XR_X {\Lambda}^k LU = Q_XR_X ({\Lambda}^k L {\Lambda}^{-k})({\Lambda}^kU).
$$
Since it is
$$
{\Lambda}^k L {\Lambda}^{-k} =
\left(
\begin{array}{cccc}
1                                  & 0                                 & 0      & 0      \\
l_{21}({\lambda}_2/{\lambda}_1)^k  & 1                                 & 0      & 0      \\
l_{31}({\lambda}_3/{\lambda}_1)^k  & l_{32}({\lambda}_3/{\lambda}_2)^k & 1      & 0      \\
\vdots                             & \vdots                            & \ddots & \vdots \\
l_{n1}({\lambda}_3/{\lambda}_1)^k  & l_{n2}({\lambda}_3/{\lambda}_1)^k & \cdots & 1
\end{array}
\right),
$$
$$
{\lim_{k{\rightarrow}{\infty}}} A_1^k
= {\lim_{k{\rightarrow}{\infty}}} Q_XR_X ({\Lambda}^k L {\Lambda}^{-k})({\Lambda}^kU)
= Q_X(R_X {\lim_{k{\rightarrow}{\infty}}} {\Lambda}^k U)
$$
On the other hand,
$$
{\lim_{k{\rightarrow}{\infty}}} A_1^k = {\lim_{k{\rightarrow}{\infty}}} {\tilde{Q}}_k {\tilde{R}}_k.
$$
$$
Q_X = {\lim_{k{\rightarrow}{\infty}}} {\tilde{Q}}_k
$$
$$
{\lim_{k{\rightarrow}{\infty}}} Q_k
= {\lim_{k{\rightarrow}{\infty}}} (Q_{k-1}^{-1}{\cdots}Q_{1}^{-1}) (Q_{1}{\cdots}Q_{k-1}) Q_k
= {\lim_{k{\rightarrow}{\infty}}} {\tilde{Q}}_{k-1}^{-1} {\lim_{k{\rightarrow}{\infty}}} {\tilde{Q}}_{k}
= Q_X^{-1}Q_X
= I,
$$
$$
{\lim_{k{\rightarrow}{\infty}}} R_k
= {\lim_{k{\rightarrow}{\infty}}} A_{k+1}Q_k^{-1}
= {\lim_{k{\rightarrow}{\infty}}} ({\tilde{Q}}_k^{-1}A_1{\tilde{Q}}_k)Q_k^{-1}
= {\lim_{k{\rightarrow}{\infty}}} {\tilde{Q}}_k^{-1}A_1(Q_1{\cdots}Q_k)Q_k^{-1}
= Q_X^{-1}A_1Q_X.
$$
Since $X=Q_XR_X$,
$$
{\lim_{k{\rightarrow}{\infty}}} R_k = Q_X^{-1}A_1Q_X = (R_XX^{-1})A_1(XR_X^{-1}) = R_X{\Lambda}R_X^{-1},
$$
Therefore,
$$
{\lim_{k{\rightarrow}{\infty}}} A_k =
{\lim_{k{\rightarrow}{\infty}}} Q_kR_k =
R_X{\Lambda}R_X^{-1} =
\left(
\begin{array}{cccc}
{\lambda_1} & *           & *        & *           \\
0           & {\lambda_2} & *        & *           \\
\vdots      & {\ddots}    & {\ddots} & *           \\
0           & {\cdots}    & 0        & {\lambda_n}
\end{array}
\right).
$$
Also, by finding the eigenvalues above, we can use inverse power iteration and eigenvalue shift to find the eigenvectors.

\section*{Rank}
Since the rank of matrix is equal to the number of nontrivial nonzero eigenvalues,
it is calculated from the eigenvalues obtained by the Jacobi method or iterative method using QR decomposition.
In this library, we use iterative method using QR decomposition, which has a faster processing speed.

\section*{Singular Value Decomposition}
Singular value decomposition (SVD) states:
$$
A = U {\Sigma} V^T
$$
where $A$ and ${\Sigma}$ is ${n{\times}m}$ matrix, $U$ is ${n{\times}n}$ orthogonal matrix, $U$ is ${m{\times}m}$ orthogonal matrix.
In the case $m > n$,
 \[
   {\Sigma} =
   \left(
   \begin{array}{ccc|c}
   {\sigma}_{11} & \            & O                    & \ \\
   \                      & {\ddots} & \                     & O \\
   O                    & \            & {\sigma}_{nn} & \
   \end{array}
   \right)
  \]
  where ${\sigma}$ is singular value.

It can be replaced by an eigenvalue problem from the following relation.
$$
AA^T = U {\Sigma} V^T  (U {\Sigma} V^T)^T = U{\Sigma}^2U^T ,
$$
 \[
   {\Sigma}^2 =
   \left(
   \begin{array}{ccc}
   {\sigma}_{11}^2 & \            & O \\
   \                         & {\ddots} & \   \\
   O                       & \            & {\sigma}_{nn}^2
   \end{array}
   \right) =
   \left(
   \begin{array}{ccc}
   {\lambda}_{11} & \            & O \\
   \                       & {\ddots} & \   \\
   O                     & \            & {\lambda}_{nn}
   \end{array}
   \right)
  \]
  where $\lambda$ is eigenvalue of $AA^T$.

\section*{Diagonalization}
An $n{\times}n$ matrix $A$ is diagonalizable when $A$ has $n$ eigenvectors that are linear independent of each other.
We consider the matrix $P$ that is written as $P=[{\bf{x}}_1, {\bf{x}}_2, {\cdots}, {\bf{x}}_n]$ where ${\bf{x}}_i, i=1,{\cdots},n$ linear independent eigenvector of $A$.

\begin{eqnarray}
AP = A[{\bf{x}}_1, {\bf{x}}_2, {\cdots}, {\bf{x}}_n]
     = [{\lambda}_1{\bf{x}}_1, {\lambda}_2{\bf{x}}_2, {\cdots}, {\lambda}_n{\bf{x}}_n] \nonumber
\end{eqnarray}
where ${\lambda}_i, i=1,{\cdots},n$ eigenvalue of $A$.
Since ${\bf{x}}_1, {\bf{x}}_2, {\cdots}, {\bf{x}}_n$ are linear independent,
\[
  P^{-1}AP =
  \left(
  \begin{array}{ccc}
  {\lambda}_{1} & \ & O \\
  \          & \ddots  & \    \\
  O        &   \          & {\lambda}_{n}
  \end{array}
  \right)
\] .
This matrix is the diagonal matrix of $A$.

\section*{Jordan normal form}
Since $(A - {\lambda}E){\bf{u}} = {\bf{x}}$ and $(A - {\lambda}E){\bf{x}} = {\bf{0}}$,
\begin{eqnarray}
\left\{ \begin{array}{ll}
A{\bf{u}}= {\bf{x}} + {\lambda}{\bf{u}} \\
A{\bf{x}}={\lambda}{\bf{x}} \\
\end{array} \right.
\end{eqnarray} .

Therefore,
 \[
   A
   \left(
   \begin{array}{cc}
   {\bf{x}} & {\bf{u}}
   \end{array}
   \right)
   =
   \left(
   \begin{array}{cc}
   {\bf{x}} & {\bf{u}}
   \end{array}
   \right)
   \left(
   \begin{array}{cc}
   {\lambda} & 1 \\
   0 & {\lambda}
   \end{array}
   \right)
  \] .

$$P^{-1}AP = J$$ where
\[
   P =
   \left(
   \begin{array}{cc}
   {\bf{x}} & {\bf{u}}
   \end{array}
   \right) ,
   J =
   \left(
   \begin{array}{cc}
   {\lambda} & 1 \\
   0 & {\lambda}
   \end{array}
   \right).
  \] .

\section*{Matrix norms}
$A$ is $n{\times}m$ matrix.

Frobenius norm:
$$
||A||_F = \sqrt{{\sum_i^n}{\sum_j^m} |a_{ij}|^2} \ \ .
$$

$L_1$ norm:
$$
||A||_1 = \max_{j}  {\sum_i^n} |a_{ij}| \ \ .
$$

Max norm:
$$
||A||_{\infty} = \max_{i}  {\sum_j^n} |a_{ij}| \ \ .
$$

$L_2$ norm:
$$
||A||_2 = \max_{ij}  {\sigma}_{ij}
$$
where $\sigma$ is singular value of $A$.

\section*{Variance covariance matrix}
A variance covariance matrix can be defined as
 \[
   S =
   \left(
   \begin{array}{cc}
   s_{xx} & s_{xy} \\
   s_{yx} & s_{yy}
   \end{array}
   \right)
  \]
where $s_{xx}$ is variance value and $s_{xy}$ is covariance value.
$s_{xy} = \frac{1}{n}({\bf{x}} - \bar{\bf{x}})({\bf{y}} - \bar{\bf{y}})$, $\bar{\bf{x}} = {\sum_{i=1}^n} x_i /n$.

By the way, we can consider the Principal Component Analysis (PCA) by this variance covariance matrix with above power Iteration library.

\end{document}


\textbf{ Using finite differences on the given grid and the data in $u$ and $v$ from \texttt{velocitydata.mat}, solve the equation of temperature in a fluid flow}
\begin{align*}
T_t + uT_x + vT_y = \alpha(T_{xx}+T_{yy}),~~(x,y)\in (0,1) \times (0,1),~~t>0,
\end{align*}
\textbf{where $\alpha = 0.0005$. Assume that $T=0$ at $t=0$ and that $T(t,x,y) = xy\tanh(10t)^2$ on the boundary. use \texttt{contourf} to plot the level curves to the temperature at times $t = 10,50,150,300$.}

We can appreciate that the equation to solve has the same form than the vorticity equation using the frozen values for the velocity given as data. Just adjusting the given code we obtain the following results.

\begin{figure}[H]
\centering     %%% not \center
\hspace*{\fill}
\subfigure[$t=1$ s.]{\includegraphics[scale=0.55]{T_10.eps}}
\hfill
\subfigure[$t=2$ s.]{\includegraphics[scale=0.55]{T_50.eps}}
\hspace*{\fill}

\hspace*{\fill}
\subfigure[$t=3$ s.]{\includegraphics[scale=0.55]{T_150.eps}}
\hfill
\subfigure[$t=5$ s.]{\includegraphics[scale=0.55]{T_300.eps}}
\hspace*{\fill}
\caption{Temperature Contours at different values of time.}
\end{figure}

\subsection*{Matlab code for this part}
\begin{verbatim}
clear variables; close all; clc
DATA=load('velocitydataPlatte.mat');
u=DATA.u;
v=DATA.v;
xx=DATA.xx;
yy=DATA.yy;
fluidflowFDTemperature(xx,yy,u,v)

% Fluid flow in a cavity (Navier-Stokes equations)
% Stream-function and vorticity formulation
%
% Rodrigo Platte, Arizona State University, April 2013.

function fluidflowFDTemperature(xx,yy,u,v)
% Parameters
path='Latex/FIGURES/';
N = 150;
alpha = 0.0005;
dt = min(0.2/(alpha*N^2),1e-3);

h = 1/N; % N+1 points in each direction

% Initial condition & pre-allocate memory (start with zeros at t = 0)
T = 0*xx;

% boundary condition
ii = 2:N; jj = 2:N;  % index for interior nodes
Ti = zeros(N-1,N-1); % interior values of vorticity

count = 0;
time = 0;
outputTime=[10 50 150 300];
endtime=outputTime(end);
n=1;
% main loop
while time<endtime
	if ( time < outputTime(n) && time+dt >= outputTime(n) ) %#ok<ALIGN>
        dt=outputTime(n)-time;
        n=n+1;
    else
        dt = min(0.2/(alpha*N^2),1e-3);
    end
   % advance Temperature with forward Euler
   Ti = Ti + dt*Trhs(u,v,T,alpha);
   T(ii,jj)=Ti;

   % update Temperature at boundary
   T = BCT(xx,yy,time,T);

   time = time+dt;
   count = count + 1;

   % Plot results every now and then
   if ismember(time,outputTime);
       plotresults(time,T);
   end

end
    % Right-hand side of vorticity equation
    function rhs = Trhs(u,v,T,alpha)
        Tx = (T(ii,jj+1)-T(ii,jj-1))/(2*h);
        Ty = (T(ii+1,jj)-T(ii-1,jj))/(2*h);
        LT = (T(ii,jj+1)+T(ii,jj-1)+T(ii+1,jj)+T(ii-1,jj)-4*T(ii,jj))/h^2;
        rhs = -u(ii,jj).*Tx - v(ii,jj).*Ty + alpha*LT;
    end

    % Enforce boundary condition (temperature)
    function T = BCT(xx,yy,t,T)
        T(1,:)=xx(1,:).*yy(1,:).*tanh(10*t).^2;
        T(N+1,:)=xx(N+1,:).*yy(N+1,:).*tanh(10*t).^2;
        T(:,1)=xx(:,1).*yy(:,1).*tanh(10*t).^2;
        T(:,N+1)=xx(:,N+1).*yy(:,N+1).*tanh(10*t).^2;
    end

    % Plot results
    function plotresults(time,T)
        figure();set(gcf,'Visible', 'off');
        pcolor(xx,yy,T)
        shading interp
        H = colorbar; set(H,'fontsize',16)
        caxis([0 1])
        colorbar
        axis square
        drawnow
        xlabel('$x$','Interpreter','latex')
        ylabel('$y$','Interpreter','latex')
        set(get(gca,'ylabel'),'rotation',0)
        txt=[path,'T_' num2str(time)];
        saveas(gcf,txt,'epsc')
    end

end

\end{verbatim}


\section{Motivation}
The history of variational principles can be dated back to a pretty concrete classical problem known as the Brachistochrone problem:
\begin{example}[The Brachistochrone Problem]
    Consider a particle moving under the influence of gravity on a wire joining points $A$ and $B$.
    What shape of the curve gives the shortest travel time of the particle given that it starts from rest?\\
    Mathematically, we want to minimise the quantity
    $$\tau=\int_A^B\mathrm dt=\int_A^B\frac{\mathrm dL}{v(x,y)}$$
    This problem was first created by Johann Bernoulli in 1696, who posted the problem in a journal as a challenge to the world's mathematcians at the time.
    And the answer given by Newton starts the study of calculus of variations.
    We are not going to fully solve the problem in this section, but instead just give a taste of the whole picture.\\
    Assuming $A$ is the origin and $B=(x_2,y_2)$
    Using energy conservation,
    $$\frac{1}{2}mv^2+mgy=0\implies v=\sqrt{-2gy}$$
    Take $y$ as a function of $x$, then we are aiming at finding the minima of the functional
    $$\tau[y]=\frac{1}{\sqrt{2g}}\int_0^{x_2}\frac{\sqrt{1+(y^\prime)^2}}{\sqrt{-y}}\,\mathrm dx$$
    subject to
    $$\begin{cases}
        y(0)=0\\
        y(x_2)=y_2
    \end{cases}$$
\end{example}
Another very famous and very useful example is geodesics.
\begin{example}[Geodesics]
    On a surface $\Sigma$, a geodesics $\gamma$ is the path of least length joining two given points.
    If $\Sigma$ is the Euclidean plane, it is well-known that $\gamma$ has to be a straight line.
    Let $D[y]$ denote the length of the curve given by $y$ and assume that the curve traverse as a function $y(x)$ from $x_1,x_2$, then minimising the following integral
    $$D[y]=\int_{x_1}^{x_2}\sqrt{1+(y^\prime)^2}\,\mathrm dx$$
    subject to $y(x_1)=y_1,y(x_2)=y_2$ would be our aim.
\end{example}
In general, the calculus of variations focus on solving optimisation problems of the a functional (i.e. functions from a space of functions to the reals)
$$F[y]=\int_{x_1}^{x_2}f(x,y(x),y^\prime(x))\,\mathrm dx$$
among all (sufficiently smooth) functions $y$ subject to initial conditions.
\begin{example}[Examples of Functionals]
    1. For a function $y(x)$, we can define the functional calculating the area under the curve by setting $f(x,y,y^\prime)=y$.\\
    2. We can also define the functional calculating length by
    $$f(x,y,y^\prime)=\sqrt{1+(y^\prime)^2}$$
\end{example}
Notationally, we write $C(\mathbb R)$ as the space of continuous functions from $\mathbb R$ to $\mathbb R$.
We write $C^k(\mathbb R)$ to denote the space of $k$-time differentiable functions with continuous $k^{th}$ derivative.
And $C^k_{\alpha,\beta}(\mathbb R)$ is the space of functions $[\alpha,\beta]\to\mathbb R$ that are $C^k$ on $[\alpha,\beta]$ and vanishes at $\alpha,\beta$.
One should know that these are these are all (infinite-dimensional) vector spaces over $\mathbb R$, the detail of analysis of which will be covered in functional analysis contexts.\\
The reason why the course is called ``variational principles'' instead of ``calculus of variations'' is because what we are studying are principles in nature and laws of physics that follow from extremising functionals.
\begin{example}[Fermat's Principle]
    Light that travels between two points follow the path that extremise the travel time.
\end{example}
\begin{example}[Principle of Least Action]
    Let $T,V$ be the kinetic and potential energies.
    We define the functional on a path
    $$S[\gamma]=\int_{t_1}^{t_2}(T-V)\,\mathrm dt$$
    then the path that a particle travels from $t=t_1$ to $t=t_2$ is the one that extremises $S$.
    Once we have developed our theory further, one can show that Newton's Second Law follows from this principle.\\
    Leibniz's take on this principle is that we live in ``the best of all possible worlds''.
    In some sense, we go from science to something like the field of theology.
    Of course, this is not in the scope of this course.\\
    Richard Feynman's take on this principle is that it is ``wrong''.
    Indeed, in quantum physics, nothing takes a definitive path, but every paths are possible with some probability.
    The principle under this framework, instead, is that the stationary path of a particle is, in fact, the interference along all paths.
\end{example}
What we shall show in this course are the follows:
First, we will see some necessary conditions for a function to be an extrema, like the Euler-Lagrange equations.
Secondly, we will bring this principle on problems in geometry, physics, and problem with constraints (like the isoperimetric inequality).
We will also talk about what is called the ``second variation'' as an analog of the second derivative.


\section*{Exercises}

\begin{ex}
  Solve the following system of second order linear differential equations:
  \begin{equation*}
    \begin{array}{c@{~}c@{~}r@{~}r@{~}r}
      y'' &=& -5y &-& 6z, \\
      z'' &=& 3y &+& 4z. \\
    \end{array}
  \end{equation*}
\end{ex}

\begin{ex}
  Solve the following system of second order linear differential equations:
  \begin{equation*}
    \begin{array}{c@{~}c@{~}r@{~}r@{~}r}
      y'' &=& 4y &-& 3z, \\
      z'' &=& 2y &-& z. \\
    \end{array}
  \end{equation*}
\end{ex}

\begin{ex}
  Solve the following system of second order linear differential equations:
  \begin{equation*}
    \begin{array}{c@{~}c@{~}l}
      x'' &=& -2y + 2z, \\
      y'' &=& x - z, \\
      z'' &=& x - 2y + z. \\
    \end{array}
  \end{equation*}
\end{ex}

\begin{ex}
  Solve the following system of first order linear differential equations.
  \begin{equation*}
    \begin{array}{c@{~}c@{~}r@{~}r@{~}r}
      y' &=& 4y &+& 6z, \\
      z' &=& -3y &-& 5z. \\
    \end{array}
  \end{equation*}
  Hint: The method is similar to that of second-order equations. Use
  the fact, known from calculus, that the equation $f'=kf$ has basic
  solution $f(x) = e^{kx}$. Here $k$ is any constant (positive,
  negative, or zero).
\end{ex}

\begin{ex}
  Consider three coupled train cars as in
  Example~\ref{exa:coupled-train-cars}, except that all three cars
  have mass $1\kg$ and both spring have spring constant
  $k=1\frac{\SI{N}}{\m}$. Find and solve the equations of motion.
\end{ex}


\subsection{Fundamental Theorem of Arithmetic}

\subsubsection{Statement}

Each natural number is a prime or unique product of primes.

\subsubsection{Proof: existance of each number as a product of primes}

If \(n\) is prime, no more is needed.

If \(n\) is not prime, then \(n=ab\), \(a,b\in \mathbb{N} \).

If \(a\) and \(b\) are prime, this is complete. Otherwise we can iterate to find:

$n=\prod_{i=1} p_i$

\subsubsection{Proof: this product of primes is unique}

Consider two different series of primes for the same number:

$s=\prod_{i=1}^n p_i = \prod_{i=1}^m q_i$

We need to show that \(n=m\) and \({p}={q}\).

We know that \(p_i\) divides \(s\). We also know that through Euclid's lemma that if a prime number divides a non-prime number, then it must also divide one of its components. As a result \(p_i\) must divide one of \({q}\).

But as all of \({q}\) are prime then \(p_i\)=\(q_j\).

We can repeat this process to to show that \({p}={q}\) and therefore \(n=m\).