\documentclass[12pt]{article}

\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{indentfirst}
\usepackage{tikz-cd}
\usepackage{mathtools}
\usepackage[shortlabels]{enumitem}
\usepackage{xcolor}
\usepackage{hyperref}
\usepackage{multicol}
\usepackage[margin=0.75in, top=1in, a4paper]{geometry}

\theoremstyle{definition}
\newtheorem{defn}{Definition}[section]
\newtheorem{ex}{Exercise}[section]
\newtheorem{oex}[ex]{*Exercise}

\newcommand{\mo}[1]{\lvert #1 \rvert}
\newcommand{\mos}[1]{\lvert #1 \rvert^2}
\newcommand{\RR}{\mathbb{R}}
\newcommand{\QQ}{\mathbb{Q}}
\newcommand{\NN}{\mathbb{N}}
\newcommand{\T}{\text{true}}
\newcommand{\F}{\text{false}}
\newcommand{\mov}[1]{\lvert \vec{#1} \rvert}
\newcommand{\CC}{\mathbb{C}}
\newcommand{\p}{\partial}
\newcommand{\iv}[1]{\langle #1 \rangle}
\newcommand{\adj}{\text{adj}}
\newcommand{\dom}{\text{dom}}
\newcommand{\st}{\text{s.t. }}
\newcommand{\ltc}[1]{\colorbox{lightgray}{\textbackslash #1}}
\newcommand{\ltcc}[1]{\colorbox{lightgray}{#1}}
\newcommand{\bras}[1]{\lbrace #1 \rbrace}
\newcommand{\braks}[1]{\lbrack #1 \rbrack}
\newcommand{\id}[1]{\text{id}_{#1}}

\definecolorset{gray/rgb/hsb/cmyk}{}{}%
 {black,0/0,0,0/0,0,0/0,0,0,1;%
  darkgray,.25/.25,.25,.25/0,0,.25/0,0,0,.75;%
  gray,.5/.5,.5,.5/0,0,.5/0,0,0,.5;%
  lightgray,.85/.85,.85,.85/0,0,.85/0,0,0,.15;%
  white,1/1,1,1/0,0,1/0,0,0,0}

\title{\vspace{-2.0cm}Mathematics Notation Cheatsheet}
\author{David Ma}

\begin{document}
	\maketitle

	\section{Sets}

	A set is a collection of objects, which are referred to as the $members$ or $elements$ of the set.

	\begin{itemize}
		\item $a \in A$: $a$ is an element of the set $A$.
		\item $|A|$: The cardinality (i.e. size) of the set $A$, e.g. the set $\bras{1, 3, 5}$ has cardinality $3$.
	\end{itemize}

	\subsection{Common Sets}
	\begin{itemize}
		\item $\varnothing$: The empty set with no elements.
		\item $\NN$: The set of natural numbers, e.g. $2$, $6969$, $42$.
		\item $\mathbb{Z}$: The set of integers, e.g. $-2, 0, 9$.
		\item $\mathbb{Q}$: The set of rational numbers, e.g. $2.34$, $\frac{6}{7}$.
		\item $\RR$: The set of real numbers, e.g. $12$, $\frac{3}{8}$, $-\sqrt{2}$.
		\item $\RR_+$: The set of non-negative real numbers, e.g. $0$, $\pi$, $1$. This notation works for other number sets (e.g. $Z_+$ denotes non-negative integers).
		\item $\RR_{++}$: The set of strictly positive real numbers (not including $0$).
		\item $\CC$: The set of complex numbers, e.g. $2 + 3i$, $5i$, $12$.
		\item $\RR^n$: The set of vectors of length (dimension) $n$.
		\item $\RR^{m \times n}$: The set of matrices of size $(m, n)$.
	\end{itemize}

	\subsection{Construction of Sets}

	An arbitrary set $S$ can be constructed in numerous ways:
	\begin{itemize}
		\item $S = \bras{a, b, c}$: Set $S$ has elements $a$, $b$ and $c$.
		\item $S = \bras{x \mid x \in A}$: Set $S$ comprise of elements from set $A$.
		\item $S = \bras{x \mid x \in A,\ C_1(x), \dots, C_n(x)}$: Set $S$ comprises of all elements from set $A$ that satisfy all assertions $C_1, \dots, C_n$ of the element, e.g. $\bras{x \mid x \in \NN,\ x > 5}$ is the set of all natural numbers that are larger than $5$.
		\item $S = \bras{f(a, b) \mid a \in A,\ b \in B}$: Set $S$ contains the results of applying function $f$ to all combinations of elements from set $A$ and $B$.
	\end{itemize}

	Apart from the first notation, the rest are referred to as \emph{set comprehensions}. They can be used in conjunction, i.e. a set comprehension can contain multiple assertion statements and multiple element definition statements.

	\subsection{Set Operations}

	By operating on sets, new sets can be formed from existing sets.

	\begin{itemize}
		\item $\overline{A}$: The complementary set of $A$. The definition of this is sensitive to the definition of the universal set in the current context.
		\item $A \setminus B$: The difference of $A$ and $B$, i.e. all members of $A$ that are not contained in $B$.
		\item $A \cap B$: The intersection of $A$ and $B$, i.e. the set whose elements are all elements contained in \emph{both} $A$ and $B$.
		\item $A \cup B$: The union of $A$ and $B$, i.e. the set whose elements are all elements contained in \emph{either} $A$ or $B$.
		\item $A \sqcup B$: The disjoint union of $A$ and $B$. This is similar to a union, except that each element in the resulting union has semantics indicating which source set (of $A$ and $B$) this element came from. One representation of this idea is treating $\bras{(a, A) \mid a \in A} \cup \bras{(b, B) \mid b \in B}$ as the disjoint union of $A$ and $B$.
		\item $A \times B$: The cartesian product of $A$ and $B$. This is the set of tuples/pairs defined as $\bras{(a, b) \mid a \in A,\ b \in B}$.
	\end{itemize}

	It is clear that the cardinality of some composed set can be inferred from its constituent sets. This is useful when proving theorems via the Howard-Curry isomorphism.
	\begin{itemize}
		\item $A \cap B = \varnothing \text{ ($A$ and $B$ are disjoint)} \iff |A \cap B| = |A| + |B|$
		\item $|A \sqcup B| = |A| + |B|$
		\item $|A \times B| = |A| * |B|$
	\end{itemize}

	In addition, note that there are also compact forms for some of the above operations:
	\begin{align*}
		\bigcap^n_{i=1} a_i &= a_1 \cap a_2 \cap \dots \cap a_n\\
		\bigcup^n_{i=1} a_i &= a_1 \cup a_2 \cup \dots \cup a_n
	\end{align*}

	\subsection{Set Relations}

	The relationship between sets can also be described with symbols:
	\begin{itemize}
		\item $A \subseteq B$: $A$ is a subset of $B$, i.e. all members of $A$ are also members of $B$.
		\item $A \not\subseteq B$: $A$ is not a subset of $B$, i.e. there exists at least one member of $A$ that is not a member of $B$.
		\item $A \subset B$: $A$ is a strict ($A \neq B$) subset of $B$.
		\item $A \not\subset B$: $A$ is not a strict subset of $B$.
 	\end{itemize}

 	\section{Functions}

 	The \emph{signature} of a function denotes what type of function it is, i.e. what are the parameters and return types of the function.

 	To illustrate, the function signature of the addition function $+$ over the set of natural numbers can be written in the forms of either:
 	\begin{itemize}
 		\item $+ : \NN \to \NN \to \NN$
 		\item $+ : \NN \times \NN \to \NN$
 	\end{itemize}

 	The above signatures state that "$+$ is a function that accepts two parameters from $\NN$ and returns a value of type $\NN$".

 	Note that the two signatures listed above are equivalent up to isomorphism (over the currying operation). From this observation, it is evident that the arrow $\to$ in function signatures is right-associative.

 	Some additional notations of functions:
 	\begin{itemize}
 		\item $\id{A}: A \to A$ is the identity function that simply returns the given parameter.
 		\item In addition to the common function definition, e.g. $f(x, y) = x^2 + y^2$, a function can also be defined in-line with the mapping arrow $\mapsto$, e.g. $(x, y) \mapsto x^2 + y^2$.
 		\item Given $f : A \to B$ and $g : B \to C$, the function composition $g \circ f : A \to C$ denotes the chaining of $g$ after $f$, i.e. $x \mapsto g(f(x))$.
 	\end{itemize}

 	\subsection{Function Descriptions}

 	A function $f : A \to B$ can be described according to its behavior on how it maps elements from $A$ to $B$. Specifically, $f$ can be:
 	\begin{enumerate}
 		\item \textbf{Injective}: $f(x) \neq f(y)$ for all $x, y \in A$, $x \neq y$.
 		\item \textbf{Surjective}: For all $b \in B$, there exists an $a \in A$ such that $f(a) = b$.
 		\item \textbf{Bijective}: A bijective function is one that is both injective and surjective.
 	\end{enumerate}

 	Sets can also be described in relation to a function. Consider the function $f : X \to Y$, then a set in relation to $f$ can be described as:
 	\begin{itemize}
 		\item \textbf{Domain}: The domain of $f$ is $X$.
 		\item \textbf{Codomain}: The codomain of $f$ is $Y$.
 		\item \textbf{Image} (range): The image $M$ of $f$ is the set of actual values $f(x)$ for all $x \in X$. If $f$ is surjective, then $M = Y$.
 		\item \textbf{Preimage}: The preimage of $P \subseteq Y$ over $f$ is the set of $x \in X$ such that $f(x) \in P$.
 	\end{itemize}

 	For example, the notions of image and preimage are commonly used when describing linear mapping of vector spaces, as singular matrices (i.e. whose determinant is $0$) maps a vector space to a strict subset of itself, thereby distinguishing its image from its codomain.

	\section{Relations}

	In academic writing, custom relationships between arbitrary objects are often defined for better elaboration. Such an abstraction is especially prevalent in proofs. This section aims to clarify some common relation definitions and terminologies.

	\begin{defn}
		A \emph{preorder} relation on set $A$ is a binary relation that denotes an order over the elements of $A$. A preorder relationship between two objects is denoted as $a \preceq b$ or with a similar operator.

		A preorder relation $\preceq$ on set $A$ must satisfy the following axioms:
		\begin{enumerate}
			\item \textbf{Identity}: $a \preceq a$ for all $a \in A$
			\item \textbf{Transitivity}: $a \preceq b \land b \preceq c \implies a \preceq c$
			\item \textbf{Anti-symmetry}: $a \preceq b \land b \preceq a \implies a \cong b$
		\end{enumerate}
		where $\cong$ is some notion of equivalence (see below).
 	\end{defn}

 	For example, a preorder can be formed on the set of restaurants near YKPS based on their distance to the school: $a \preceq b$ if the distance to $a$ is less than or equal to that of $b$.

 	\begin{defn}
 		An \emph{equivalence} relation $a \cong b$ states that $a$ and $b$ are equal in some sense. A definition for equivalence on set $A$ must satisfy the following constraints:
 		\begin{enumerate}
 			\item \textbf{Reflexivity}: $a \cong a$ for all $a \in A$
 			\item \textbf{Symmetry}: $a \cong b \iff b \cong a$
 			\item \textbf{Transitivity}: $a \cong b \land b \cong c \implies a \cong c$
 		\end{enumerate}
 	\end{defn}

 	The symbol for equivalence can also be a tilde $\sim$.

 	For example, for a purchasing problem, the equivalence of purchasing strategies can be defined as having the same cost/price.

 	\begin{defn}
 		An \emph{isomorphism} between structures (structures are more generalized than sets) $A$ and $B$ states that "having either $A$ or $B$ is as good as having the other". Formally, structures $A$ and $B$ are isomorphic if there exists a pair of functions $f : A \to B$ and $g : B \to A$ such that:
 		\begin{enumerate}
 			\item $g \circ f = \id{A}$
 			\item $f \circ g = \id{B}$
 		\end{enumerate}
 	\end{defn}

 	For example, a bijection is an isomorphism in the category of sets.

 	\section{Formal Logic}

 	Logic statements can shorten your descriptions with short notations. Such notations are ubiquitous in academic writings, so it is important to get them right!

 	Loosely speaking, a statement is an observation or deduction, e.g. $x > 5$ states that $x$ is larger than $5$.

 	Some common operations on statements:
 	\begin{itemize}
 		\item $\lnot A$: Negates the statement $A$.
 		\item $A \lor B$: Either $A$ or $B$, e.g. $x > 10 \lor x < 0$ means $x$ can be \emph{either} greater than $10$ or less than $0$.
 		\item $A \land B$: Both $A$ and $B$, e.g. $x \in X \land x > 5$ means $x$ must be \emph{both} in set $X$ and greater than $5$.
 	\end{itemize}

 	\subsection{Logical Deduction Notations}

 	Arrows are commonly used in logical deductions; however, note that different arrows have \textbf{completely different} meanings, and misusing arrows can cause confusions for the reader. For example, $\to$ and $\implies$ are completely different, and should never be used interchangeably.

 	\begin{defn}
 		Statement $A$ \emph{implies} statement $B$ means that if $A$ is true, then $B$ is true. Such a relation is denoted $A \implies B$.
  	\end{defn}

  	For example:
  	\begin{center}
  		Thomas is at Oxford $\implies$ Thomas is not in China
  	\end{center}

  	\begin{defn}
  		An \emph{if and only if} relation on statement $A$ and $B$ denotes that $A$ and $B$ should either both be true, or  both be false, as the state of either statement can guarantee the state of the other. Denoted as $A \iff B$, the "if and only if" relation can be relaxed into two relations:
  		\begin{itemize}
  			\item \textbf{Necessary}: $A$ is needed for $B$ to be true.
  			\item \textbf{Sufficient}: Knowing that $A$ is true is enough for $B$ to be true.
  		\end{itemize}
  		If both requirements above are satisfied, then $A \iff B$.
  	\end{defn}

  	For example:
  	\begin{center}
  		S is the empty set $\iff$ $|S| = 0$
  	\end{center}

  	An alternative way of defining $A \iff B$ is:
  	\begin{gather*}
  		(A \implies B) \land (B \implies A)
  	\end{gather*}

  	\subsection{Quantifiers}

  	Quantifiers are symbols that define a variable with certain given semantics, i.e. information regarding the value of the variable. This is useful when formulating a predicate or statement.

  	\begin{itemize}
  		\item $\forall a \in A$ (Universal Quantifier): For all $a$ in set $A$, i.e. $a$ can be set to any value in $A$.
  		\item $\exists a \in A$ (Existential Quantifier): There exists an $a$ i $A$.
  	\end{itemize}

  	For example, the statement "all integers have an additive inverse" can be rewritten with quantifiers (different writings may use different conventions for separators):
  	\begin{gather*}
  		\forall x \in \mathbb{Z},\ \exists y \in \mathbb{Z}.\ x + y = 0
  	\end{gather*}

 	\section{Number Theory}

 	\begin{itemize}
 		\item $a \mid b$: $a$ is a factor of $b$, i.e. $b$ has remainder $0$ when divided by $a$.
 		\item $a \nmid b$: $a$ is not a factor of $b$, i.e. $a$ does not divide $b$.
 		\item $a \perp b$: $a$ and $b$ are coprimes, i.e. the greatest common factor of $a$ and $b$ is $1$
 		\item $a \bmod b$: The remainder of $a$ divided by $b$.
 		\item $a \equiv b \bmod n$: $a$ is congruent to $b$ modulo $n$, i.e. $n \mid (b - a)$.
 		\item $\mathbb{Z}/n\mathbb{Z}$: The cyclic group (see \nameref{sec:terms}) formed by numbers $\bras{0, 1, \dots, n - 1}$ under addition modulo $n$.
 		\item $\lfloor x \rfloor$: The floor of $x$, i.e. the largest integer less than or equal to $x$.
 		\item $\lceil x \rceil$: The ceiling of $x$, i.e. the smallest integer greater than or equal to $x$.
 		\item $\lfloor x \rceil$ (or $\lbrack x \rbrack$): The nearest integer to $x$.
 	\end{itemize}

 	\section{Calculus}

 	The symbols in calculus are standard to what we've covered in class. However, there are some alternative ways of writing derivatives.

 	Consider function $f(x, y)$. Its partial derivative with respect to $x$ can be written as:
 	\begin{itemize}
 		\item $\frac{\partial f}{\partial x}(x, y)$
 		\item $f_x(x, y)$
 		\item $D_xf(x, y)$
 	\end{itemize}

 	Similarly, the second order derivative of $f(x, y)$ with respect to $x$ and then $y$ can be written as:
 	\begin{itemize}
 		\item $\frac{\partial^2 f}{\partial x \partial y}(x, y)$
 		\item $f_{xy}(x, y)$
 		\item $D_{xy}f(x, y)$
 	\end{itemize}

 	Note that according to Clairaut's theorem, $f_{xy} = f_{yx}$, so the order of the "with respect to" variables does not matter.

 	\subsection{Vector Calculus}

 	The gradient of a real-valued function $f(x_1, \dots, x_n)$, denoted $\nabla f$, is defined as the vector:
 	\begin{gather*}
 	\nabla f =
	\begin{bmatrix}
 		\frac{\partial f}{\partial x_1} \\
 		\vdots \\
 		\frac{\partial f}{\partial x_n}
 	\end{bmatrix}
 	\end{gather*}

 	The gradient marks the steepest direction of ascent for function $f$.

 	Note that the operator $\nabla$ is often considered in its vector form, $\nabla = \braks{\frac{\partial}{\partial x_1}, \dots, \frac{\partial}{\partial x_n}}$. This makes it flexible, and can also be used on a vector-valued function:
 	\begin{itemize}
 		\item $\nabla \cdot f$: The divergence of $f$.
 		\item $\nabla \times f$: The curl of $f$.
 	\end{itemize}

 	\begin{defn}
 		The \emph{Jacobian matrix} of a vector-valued function $f : \bras{x_1, \dots, x_n} \to \bras{f_1, \dots, f_m}$ is an $m \times n$ matrix containing each component of the result taken derivative to each components in the parameter:
 		\begin{gather*}
 		J_f =
 		\begin{bmatrix}
 			\frac{\partial f_1}{\partial x_1} & \dots & \frac{\partial f_1}{\partial x_n} \\
 			\vdots & \ddots & \vdots \\
 			\frac{\partial f_m}{\partial x_1} & \dots & \frac{\partial f_m}{\partial x_n}
 		\end{bmatrix}
 		\end{gather*}
 	\end{defn}

 	Note that there is no established convention for the orientation of the Jacobian matrix; some writings may feature a transposed version of the above definition.

 	\section{Terminologies}
 	\label{sec:terms}

 	TODO.

 	\section{Appendix: \LaTeX\ Notations}

 	This section aims to cover the typesetting of common symbols in \LaTeX.	 This includes all symbols shown in this document, as well as ones we've covered in class.

 	\subsection{Sets Symbols}

 	\begin{multicols}{2}
 	\begin{itemize}
	 	\item $\varnothing$: \ltc{varnothing}
 		\item $\NN$: \ltc{mathbb\{N\}}
 		\item $\mathbb{Z}$: \ltc{mathbb\{Z\}}
 		\item $\QQ$: \ltc{mathbb\{Q\}}
 		\item $\RR$: \ltc{mathbb\{R\}}
 		\item $\RR_+$: \ltc{mathbb\{R\}\_+}
 		\item $\RR_{++}$: \ltc{mathbb\{R\}\_\{++\}}
 		\item $\CC$: \ltc{mathbb\{C\}}
 		\item $\RR^n$: \ltc{mathbb\{R\}\^{}n}
 		\item $\RR^{m \times n}$: \ltc{mathbb\{R\}\^{}\{m \textbackslash times n\}}
 		\item $\bras{x}$: \ltc{lbrack x \textbackslash rbrack}
 		\item $|A|$: \ltcc{\textbackslash lvert A \textbackslash rvert}
 		\item $\overline{A}$: \ltc{overline\{A\}}
 		\item $a \in A$: \ltcc{a \textbackslash in A}
 		\item $A \subset B$: \ltcc{A \textbackslash subset B}
 		\item $A \not\subset B$: \ltcc{A \textbackslash not\textbackslash subset B}
 		\item $A \subseteq B$: \ltcc{A \textbackslash subseteq B}
 		\item $A \not\subseteq B$: \ltcc{A \textbackslash not\textbackslash subseteq B}
 		\item $A \setminus B$: \ltcc{A \textbackslash setminus B}
 		\item $A \cap B$: \ltcc{A \textbackslash cap B}
 		\item $A \cup B$: \ltcc{A \textbackslash cup B}
 		\item $A \sqcup B$: \ltcc{A \textbackslash sqcup B}
 		\item $A \times B$: \ltcc{A \textbackslash times B}
 		\item $\bigcap^n_{i=1} a_i$: \ltc{bigcap\^{}n\_\{i=1\} a\_i}
 		\item $\bigcup^n_{i=1} a_i$: \ltc{bigcup\^{}n\_\{i=1\} a\_i}
 		\item $a \neq b$: \ltcc{a \textbackslash neq b}
 		\item $a \cup \dots \cup b$: \ltcc{a \textbackslash cup \textbackslash dots \textbackslash cup b}
 	\end{itemize}
 	\end{multicols}

	\subsection{Functions}
	\begin{multicols}{2}
 	\begin{itemize}
	 	\item $A \to A$: \ltcc{A \textbackslash to A}
	 	\item $x \mapsto x^2$: \ltcc{x \textbackslash mapsto x\^{}2}
	 	\item $\text{id}_A$: \ltc{text\{id\}\_A}
	 	\item $g \circ f$: \ltcc{g \textbackslash circ f}
 	\end{itemize}
 	\end{multicols}

 	%\subsection{Relations}
 	\begin{multicols}{2}
 	\begin{itemize}
	 	\item $a \preceq b$: \ltcc{a \textbackslash preceq b}
	 	\item $a \cong b$: \ltcc{a \textbackslash cong b}
 	\end{itemize}
 	\end{multicols}

 	\subsection{Formal Logic}
 	\begin{multicols}{2}
 	\begin{itemize}
	 	\item $\lnot A$: \ltc{lnot A}
	 	\item $A \land B$: \ltcc{A \textbackslash land B}
	 	\item $A \lor B$: \ltcc{A \textbackslash or B}
	 	\item $A \implies B$: \ltcc{A \textbackslash implies B}
	 	\item $A \iff B$: \ltcc{A \textbackslash iff B}
	 	\item $\forall a \in A$: \ltc{forall a \textbackslash in A}
	 	\item $\exists a \in A$: \ltc{exists a \textbackslash in A}
 	\end{itemize}
 	\end{multicols}

 	\subsection{Number Theory}
 	\begin{multicols}{2}
 	\begin{itemize}
	 	\item $a \mid b$: \ltcc{a \textbackslash mid b}
	 	\item $a \nmid b$: \ltcc{a \textbackslash nmid b}
	 	\item $a \perp b$: \ltcc{a \textbackslash perp b}
	 	\item $a \bmod b$: \ltcc{a \textbackslash bmod b}
	 	\item $a \equiv b$: \ltcc{a \textbackslash equiv b}
	 	\item $\lfloor x \rfloor$: \ltcc{\textbackslash lfloor x \textbackslash rfloor}
	 	\item $\lceil x \rceil$: \ltcc{\textbackslash lceil x \textbackslash rceil}
	 	\item $\lfloor x \rceil$: \ltcc{\textbackslash lfloor x \textbackslash rceil}
 	\end{itemize}
 	\end{multicols}

 	\subsection{Calculus}
 	\begin{multicols}{2}
 	\begin{itemize}
	 	\item $\partial$: \ltc{partial}
	 	\item $\nabla$: \ltc{nabla}
	 	\item $D_xf(x, y)$: \ltcc{D\_x f(x, y)}
	 	\item $\lim_{x \to 0}$: \ltc{lim\_\{x \textbackslash to 0\}}
 	\end{itemize}
 	\end{multicols}

\end{document}


\section{The detail of the calculations}
The following sub-sections describe the details on some of the calculations.
Consider two consecutive points $(x_1,y_1)$ and $(x_2,y_2)$ where $x_1 <= x <= x_2$ and $x_1 < x_2$, then interpolation is defined as

\begin{description}
    \item[Lin-lin interpolation]
    \begin{equation}
        y = { y_2 ( x - x_1 ) + y_1 ( x_2 - x ) \over ( x_2 - x_1 ) }
    \end{equation}
    \item[Lin-log interpolation]
    \begin{equation}
        y = y_1 \, \left( y_2 \over y_1 \right)^{x - x_1 \over x_2 - x_1}
    \end{equation}
    \item[Log-lin interpolation]
    \begin{equation}
        y = { y_1 \log(x_2/x) + y_2 \log(x/x_1) \over \log(x_2/x_1) }
    \end{equation}
    \item[Log-log interpolation]
    \begin{equation}
        y = y_1 \, \left( x \over x_1 \right)^{\log(y_2/y_1) \over \log(x_2/x_1)}
    \end{equation}
\end{description}

In some calculation we will need the x location for the maximum of the relative error, $( y' - y ) / y$, between the
approximate value, $y'$, and the ``exact'' value,
$y$. This x location occurs where the derivative of the relative error is zero:
\begin{equation}    \label{MaxXFormula}
    { d(( y' - y ) / y ) \over dx} = { d(y'/y - 1) \over dx} = {d(y'/y) \over dx} = { 1 \over y^2 } \left( y {dy' \over dx } - y' { dy \over dx } \right) = 0
\end{equation}

\subsection{Converting log-log to lin-lin}
This section describes how fudge2dmath converts a \highlight{fudge2dmathXY} object with interpolation of
\highlight{f2dmC\_interpolationLogLog} (hence called log-log) to one with interpolation of \highlight{f2dmC\_inter-pol-ation-LinLin}
(hence called lin--lin).

From Eq.~\ref{MaxXFormula} the maximum of the relative error occurs where,
\begin{equation}
    { 1 \over y } \left( {dy' \over dx } - {y' \over y} { dy \over dx } \right) = \left({ 1 \over y }\right) \left\{
        { y_2 - y_1 \over x_2 - x_1 } - \left({y' \over x}\right) {\log(y_2/y_1) \over \log(x_2/x_1)} \right\} = 0
\end{equation}
The solution is
\begin{equation}
    { x \over x_1 } = { a ( x_2 / x_1 - y_2 / y_1 ) \over ( 1 - a ) ( y_2 / y_1 - 1 ) }
\end{equation}
where $a = \log(y_2/y_1) / \log(x_2/x_1)$.


We now glance over some of the very basic terminology from probability
theory needed in our analysis.  Such basic notions are found in all
standard introductory texts on probability, and for details that are
missing in the exposition below, we refer to the book by Grimmett and
Welsh~\cite{2014GrimmettG_WelshD-aa}.

\section{Probability space and random variables}
A triple $(\Omega, \mathcal{F}, P)$ is called a \emph{probability
  space}; $\Omega$ is, in general, a set called \emph{sample space}
which contains all conceptually possible outcomes of an experiment;
$\mathcal{F}$ is a \emph{$\sigma$-algebra} of events (sets) on
$\Omega$; $P$ is a countably-additive measure on $\Omega$ called
\emph{probability}, i.e., a function
$P: \mathcal{F} \mapsto \mathbb{R}$ such that $P(a) \geq 0$ for all
$a \in \mathcal{F}$, $P(\Omega) = 1$, and
$P(\sum_{n=1}^{\infty}a_n) = \sum_{n=1}^{\infty} P(a_n)$ whenever
$a_1, a_2, \cdots$ are (pairwise) disjoint sets in $\mathcal{F}$.  For
example, the \emph{uniform distribution} is defined by choosing
$P(a) = |a|/|\Omega|$ where $|a|$ is the measure (cardinality) of $a$.

Based on the probability space, given by the triple $(\Omega,
\mathcal{F}, P)$, a \emph{discrete random variable} is a function $X:
\Omega \mapsto \mathbb{R}$ such that $X^{-1}(b) := \{ \omega:
X(\omega) \in b \} \in \mathcal{F}$ for every set $b$ in the image of
$X$.
% for every $B \in \mathcal{B}(\mathbb{R})$ where
% $\mathcal{B}(\mathbb{R})$ is the Borel $\sigma$-algebra of
% $\mathbb{R}$.
Notice that, the sets $\{ X \in b \} = \{ \omega: X(\omega) \in b \} $
are ``events'' (members of $\mathcal{F}$) and the corresponding
probabilities are defined as $ P(X \in b) := P(X^{-1}(b))$.
The \emph{expectation} $E(\cdot)$ is the average of a random variable. If
$X \geq 0$ is a random variable on $(\Omega, \mathcal{F}, P)$ then we
define its expectation to be $E(X) = \int X \mathrm{d} P$, which for
a discrete random variable $X$ is:
\begin{equation}
E(X) = \sum_{x \in R(X)} x P(X=x) = \sum_{x} x P(X=x),
\end{equation}

\section{Conditional probability and conditional expectation}
From now on, we focus on discrete random variables, as this is the setup we need. We say that two random variables $X$ and $Y$, are \emph{independent} if
\begin{equation}
P(X=x, Y=y) = P(X=x) P(Y=y),
\end{equation}
for all $x$ and $y$.  The \emph{conditional probability} of
$X=x$ under the condition $Y=y$ is given by
\begin{equation}
P(X=x\,|\,Y=y) = \frac{P(X=x, Y=y)}{P(Y=y)}.
\end{equation}
Therefore, if $X$ and $Y$ are independent, we have
\begin{equation}\label{eqn:indep}
P(X=x\,|\,Y=y) = P(X=x).
\end{equation}

In general, we write, for $G \subset \Omega$,
\begin{equation}
E(X| G)
= \sum_{x \in R(X)} x \frac{P(\omega\in G, X(\omega)=x)}{P(G)}=
\sum_{x \in R(X)} x P(X=x|G).
\end{equation}
In particular, we write
\begin{equation}
E(X|Y=y)=E(X|G)
\mbox{ if }
G=\{Y=y\}.
\end{equation}
which is known as the \emph{conditional expectation} of $X$ given $Y =
y$.
Now, we introduce the conditional random variable $E(X|Y)$ as follows
\begin{equation}
E(X|Y)(\omega)=E(X|Y=Y(\omega)).
\end{equation}
Obviously
\begin{equation}
\{\omega\in \Omega: E(X|Y)=z\}=\bigcup_{y:  E(X|Y=y)=z}\{\omega\in \Omega:  Y(\omega)=y\}
\end{equation}
and, hence
\begin{equation}
P\{\omega\in \Omega:  E(X|Y) (\omega)=z\}=\sum_{y:   E(X|Y=y) =z}P\{\omega\in \Omega:  Y(\omega)=y\}
\end{equation}
namely
\begin{equation}
P( E(X|Y)=z)=\sum_{y:   E(X|Y=y)=z}P(Y=y).
\end{equation}

%and this allows us to further define a random variable
%$E(X|Y): \Omega \mapsto \mathbb{R}$, called the \emph{conditional
%  expectation} of $X$ with respect to $Y$:
%\begin{equation}\label{e:ez}
  % \left(  E(X|Y) \right) (\omega) = E(X| Y=Y(\omega)).
%E(X|Y) = Z, \quad\mbox{where}\quad Z(\omega) = E(X| Y=Y(\omega)), \quad \omega\in \Omega.
%\end{equation}
% we denote by $E(X\,|\,Y)$ that function of the random variable $Y$
% whose value at $Y = y$ is $E(X\,|\, Y=y)$,
\begin{lemma} \label{lem:EXY}
\begin{equation}\label{e:eee}
E(X) = E(E(X\, | \, Y)).
\end{equation}
%Namely
%\begin{equation}
%E(X) = \sum_xx\sum_y P(X= x, \, Y = y)
%\end{equation}
\end{lemma}
\begin{proof}
Denote $Z=E(X|Y)$, it follows that
\begin{align}
E(E(X \, | \, Y))  = E(Z)
&=\sum_z zP(Z=z)\\
&= \sum_zz\sum_{y: w(y)=z}P( Y=y )  \\
&= \sum_y w(y) P( Y = y )  \\
 &  = \sum_y \left(  \sum_x x P(X = x \, | \, Y=y)  \right) P(Y = y) \\
			           %& = \sum_y \sum_x x P(X= x \, | \, Y = y) P(Y = y) \\
			           & = \sum_y \sum_x x P(X= x, \, Y = y) \\
			           & = \sum_y \sum_x x P(Y= y \, | X = x) P(X = x) \\
			           & = \sum_x x P(X = x) \left( \sum_y P(Y = y\, | X = x) \right) \\
			           & = \sum_x x P(X = x) \\
			           & = E(X).
\end{align}
\end{proof}

\begin{lemma}
\begin{equation}
E(X) = \sum_xx\sum_y P(X= x, \, Y = y)
\end{equation}
\end{lemma}
%This is easily verified by the definition of $E(X)$ and the fact that
%\begin{equation}\label{e:mama-mia}
%E(E(X\, | \, Y)) = \sum_{y} E(X\,|\, Y=y) P(Y = y).
%\end{equation}

\newpage

\section{Central Limit Theorem}
\begin{theorem}[Lindeberg--Levy Central Limit Theorem]
  Suppose ${X_1, X_2, \dots}$ is a sequence of independent identical
  distributed (i.i.d.) random variables with $E[X_i] = \mu$ and
  $Var[X_i] = \sigma_2 < \infty$. Then as n approaches infinity, the
  random variables $\sqrt{n}((\frac{1}{n}\sum_{i=1}^{n}X_i) - \mu)$
  converge in distribution to a normal distribution $N(0,\sigma^2)$:
\begin{align}
\sqrt{n}((\frac{1}{n}\sum_{i=1}^{n}X_i) - \mu)&\stackrel{d}{\longrightarrow} \mathcal{N}(0,\sigma^2).
\end{align}
\end{theorem}

\begin{lemma}[Levy's convergence theorem]
Let $(F_n)$ be a sequence of distribution functions, and let $\psi_n$ denote the characteristic function of $F_n$. Suppose that $g(\theta):=lim_n \psi_n(\theta)$ exists for all $\theta \in \mathbb{R}$, and that $g(\cdot)$ is continuous at 0. Then $g=\psi_F$ for some distribution function F, and $F_n \stackrel{w}{\longrightarrow} F$.
\end{lemma}

\begin{proof}
First let's denote
\begin{align}
Z_n=\frac{X_1+...+X_n-n\mu}{ \sqrt{n \sigma^2}}=\sum_{i=1}^{n} \frac{X_i-\mu}{ \sqrt{n \sigma^2}}=\sum_{i=1}^{n} \frac{Y_i}{ \sqrt{n }},
\end{align}
where  in the last step we defined the new random variables $Y_i=\frac{X_i-\mu}{ \sigma}$, each with zero mean and unit variance.

Then we wanna to prove the characteristic function of $Z_n$ converges to the characteristic function of a random variable with standard normal distribution.
The characteristic function of a random variable X is defined as
\begin{align}
\psi_X(\theta)=\mathbb{E}(e^{i\theta X})=\int_\mathbb{R}e^{i\theta X} f_X(x)dx,
\end{align}
where $f_X$ is the probability density function of X.

Property 1. If a random variable X has moments up to k-th order, then the characteristic function $\psi_X$ is k times continuously differentiable on the entire real line. Moreover,
\begin{align}
\mathbb{E}[X^k]=(-i)^k \psi_X^k(0).
\end{align}

Property 2. If $X_1,...,X_n$ are independent random variables, and $a_1,...,a_n$ are some constants, then the characteristic function of the linear combination of the $X_i's$ is
\begin{align}
\psi_{a_1 X_1+...+a_n X_n}(\theta)=\psi_{X_1}(a_1\theta)...\psi_{X_n}(a_n\theta).
\end{align}

The characteristic function of $Z_n$ is given by
\begin{align}
\psi_{Z_n}(\theta)=\psi_{\sum_{i=1}^{n} \frac{Y_i}{ \sqrt{n }}}(\theta)=\psi_{Y_1}(\frac{\theta}{ \sqrt{n }})\cdot \psi_{Y_2}(\frac{\theta}{ \sqrt{n }})...\psi_{Y_n}(\frac{\theta}{ \sqrt{n }})=[\psi_{Y_1}(\frac{\theta}{ \sqrt{n }})]^n,
\end{align}

where in the last step we used the fact that all of the $Y_i$ are identically distributed. The characteristic function of $Y_1$ is, by Taylor's theorem,
\begin{align}
\psi_{Y_1}(\frac{\theta}{ \sqrt{n }})=1-\frac{\theta^2}{2n}+c\frac{\theta^3}{6n^{\frac{3}{2}}}+o(\frac{\theta^3}{n^{\frac{3}{2}}}), \theta  \rightarrow 0.
\end{align}
Therefore
\begin{align}
\psi_{Z_n}(\theta)=\bigg ( 1-\frac{\theta^2}{2n}+c\frac{\theta^3}{6n^{\frac{3}{2}}}+o(\frac{\theta^3}{n^{\frac{3}{2}}}) \bigg )^n \rightarrow e^{-\frac{1}{2}\theta^2}, n  \rightarrow \infty,
\end{align}
which converges to the characteristic function of a random variable with standard normal distribution.
Hence as a corollary of the lemma,
\begin{align}
F_{Z_n}&\stackrel{w}{\longrightarrow} \mathcal{N}(0, 1).
\end{align}
Therefore
\begin{align}
\sqrt{n}((\frac{1}{n}\sum_{i=1}^{n}X_i) - \mu)&\stackrel{d}{\longrightarrow} \mathcal{N}(0,\sigma^2).
\end{align}

\end{proof}

Here the normal distribution $N(0,\sigma^2)$ means $f(x)=\frac{1}{\sqrt{2\pi}}e^{-\frac{x^2}{2\pi}}$ in terms of the probability distribution function.

A sequence $X_1, X_2$, ... of real-valued random variables is said to converge in distribution to a random variable X if
$ \lim _{n\to \infty }F_{n}(x)=F(x)$,
for every number $x \in \mathbb{R}$ at which F is continuous. Here $F_n$ and F are the cumulative distribution functions of random variables $X_n$ and X, respectively.

\section{Random Walk and Brownian Motion}

Define the random variable $X_i$ as follows. $X_i=1$ if the $i_{th}$ coin toss results in heads, and $X_i=-1$ if the $i_{th}$ coin toss results in tails. Thus,
\begin{equation}
  P(X_i=1)=P(X_i=-1)=\frac{1}{2}.
\end{equation}
We also have $E(X_i)=0$ and $\mbox{var}(X_i)=1$.

Fix $ t \geq 0$, choose $\delta$ small enough as the time step s.t. $N=t/\delta\in \mathbb{N}$. Then the current location after $n_{th}$ step is defined as
\begin{equation}
  S_N(n \delta)= \sum_{i=1}^{n} X_i.
\end{equation}
Hence $S_N(t)=S_N(N \delta)= \sum_{i=1}^{N} X_i$ and $S_N(0)= 0$.

Then we rescale $S_N(t)$ by
\begin{equation}
  \bar{S}_N(t)=\sqrt{\delta} S_N(t)=\sqrt{N\delta}\sqrt{N}\frac{S_N(t)}{N}
  =\sqrt{t}\sqrt{N}\frac{\sum_{i=1}^{N} X_i}{N}
\end{equation}
It's known that $\{X_1,...,X_N\}$ is a sequence of independent and
identically distributed random variables drawn from the same
distribution, with $E(X_i)=0$ and $\mbox{var}(X_i)=1$.

Then according to the central limit theorem, as $n$ approaches infinity,
\begin{align}
  \frac{\bar{S}_N(t)}{\sqrt{t}}&=\sqrt{N}\frac{\sum_{i=1}^{N} X_i}{N}\\\notag
  &=\sqrt{N}(\frac{\sum_{i=1}^{N} X_i}{N}-0)\stackrel{d}{\longrightarrow} \mathcal{N}(0,1)\\
  \bar{S}_N(t)&\stackrel{d}{\longrightarrow} \mathcal{N}(0,t).
\end{align}
If we denote
\begin{equation}
  W_t=\lim_{N\rightarrow \infty} \bar{S}_N(t)=\lim_{\delta\rightarrow 0} \sqrt{\delta} \sum_{i=1}^{N} X_i,
\end{equation}
then we want to prove  $W_t$ that we obtained is indeed a Brownian motion.
\begin{definition}
A continuous-time stochastic process $W_t$ is called Bronwian motion if
\begin{enumerate}
\item $W_0=0$
\item $W_t$ is continuous a.s.
\item $W_t \sim \mathcal{N}(0,t)$
\item $W_t$ has independent increments
\end{enumerate}
\end{definition}

\section{Brownian motion as the limit of random walk: proof}

\subsection{Starting Points}
\begin{align}
  W_0&=\lim_{N\rightarrow \infty} \bar{S}_N(0)\\\notag
     &=\lim_{N\rightarrow \infty} \sqrt{\delta} S_N(0)\\\notag
     &=\lim_{N\rightarrow \infty} \sqrt{\delta} 0\\\notag
     &=0\\\notag
\end{align}

\subsection{Continuity}
\begin{align}
   W_t&=\lim_{N\rightarrow \infty} \bar{S}_N(t)=\lim_{N\rightarrow \infty} \sqrt{\delta} S_N(t)\\
   W_{t-\delta}&=\lim_{N\rightarrow \infty} \bar{S}_N(t-\delta)=\lim_{N\rightarrow \infty} \sqrt{\delta} S_N(t-\delta)\\
   |W_{t-\delta}-W_t|&=|\lim_{N\rightarrow \infty} \sqrt{\delta} S_N(t-\delta)-\sqrt{\delta}S_N(t)|\\\notag
   &=|\lim_{N\rightarrow \infty} \sqrt{\delta} S_N((N-1)\delta)-\sqrt{\delta}S_N(N \delta)|\\\notag
   &=|\lim_{N\rightarrow \infty} \sqrt{\delta} X_{N}|\\\notag
   &=\sqrt{\delta},
\end{align}
which tends to 0 as $\delta$ approaches 0. Therefore $W_t$ is continuous in time t.

\subsection{Distribution Function}
We have already proved that
\begin{equation}
   W_t \sim \mathcal{N}(0,t).
\end{equation}

\subsection{Independent Increments}
Fix $ t,s \geq 0$, choose $\delta$ small enough as the time step s.t. $N_1=t/\delta$, $N_2=s/\delta\in \mathbb{N}$.
\begin{align}
   W_t&=\lim_{N_1\rightarrow \infty} \bar{S}_{N_1}(t)=\lim_{N_1\rightarrow \infty} \sqrt{\delta} S_{N_1}(t)\\
   W_{t+s}&=\lim_{N_1+N_2\rightarrow \infty} \bar{S}_{N_1+N_2}(t+s)=\lim_{N_1+N_2\rightarrow \infty} \sqrt{\delta} S_{N_1+N_2}(t+s)\\
   W_{t+s}-W_t&=\lim_{N_1,N_2\rightarrow \infty} \sqrt{\delta} S_{N_1+N_2}(t+s)-\sqrt{\delta} S_{N_1}(t)\\\notag
   &=\lim_{N_1,N_2\rightarrow \infty} \sqrt{\delta} S_{N_1+N_2}((N_1+N_2)\delta)-\sqrt{\delta} S_{N_1}(N_1\delta)\\\notag
   &=\lim_{N_1,N_2\rightarrow \infty} \sqrt{\delta} \sum_{i=1}^{N_1+N_2} X_i-\sqrt{\delta} \sum_{i=1}^{N_1} X_i\\\notag
   &=\lim_{N_1,N_2\rightarrow \infty} \sqrt{\delta} \sum_{i=N_1+1}^{N_1+N_2} X_i\\\notag
   &=\lim_{N_2\rightarrow \infty} \sqrt{\delta} \sum_{i=1}^{N_2} X_i\\\notag
   &=\lim_{N_2\rightarrow \infty} \sqrt{\delta} S_{N_2}(N_2\delta)\\\notag\
   &=\lim_{N_2\rightarrow \infty} \sqrt{\delta} S_{N_2}(s)\\\notag
   &=\lim_{N_2\rightarrow \infty} \bar{S}_{N_2}(s)\\\notag
   &=W_s.
\end{align}
Hence $W_{t+s}-W_t$  has the same distribution as $W_s$ for $\forall t,s \geq 0$.

Then we want to prove for any positive integer n and any $0 = t_0 < t_1 <...< t_n$, the random variables $W_{t_{i+1}} - W_{t_{i}}$, i = 1, ..., n, are mutually independent. Moreover, it suffices to prove $W_{t_{i}}-W_{t_{i-1}}$ and $W_{t_{i+1}}-W_{t_{i}}$ are independent.

If we define $t_{i}=N_{i}\delta$, by definition, $W_{t_{i-1}}=\lim_{\delta\rightarrow0}\sqrt{\delta}\sum\limits_{j=1}^{N_{i-1}}X_j$, $W_{t_{i}}=\lim_{\delta\rightarrow0}\sqrt{\delta}\sum\limits_{j=1}^{N_i}X_j$, $W_{t_{i+1}}=\lim_{\delta\rightarrow0}\sqrt{\delta}\sum\limits_{j=1}^{N_{i+1}}X_j$
~\\So we have $$W_{t_{i}}-W_{t_{i-1}}=\lim_{\delta\rightarrow0}\sqrt{\delta}\sum\limits_{j=N_{i-1}+1}^{N_{i}}X_j=:Y_i$$
$$W_{t_{i+1}}-W_{t_{i}}=\lim_{\delta\rightarrow0}\sqrt{\delta}\sum\limits_{j=N_{i}+1}^{N_{i+1}}X_j=:Y_{i+1}$$
Then we want to prove $Y_{i}$ and $Y_{i+1}$ are independent, or equivalently, $P(Y_i<a, Y_{i+1}<b)=P(Y_i<a)P(Y_{i+1}<b)$
~\\Proof: By definition, $$P(Y_i<a, Y_{i+1}<b)$$
$$=P(\lim_{\delta\rightarrow0}\sqrt{\delta}\sum\limits_{j=n_{i-1}+1}^{n_{i}}X_j<a, \lim_{\delta\rightarrow0}\sqrt{\delta}\sum\limits_{j=n_{i}+1}^{n_{i+1}}X_j<b)$$
$$=\lim_{\delta\rightarrow0}P(\sqrt{\delta}\sum\limits_{j=n_{i-1}+1}^{n_{i}}X_j<a, \sqrt{\delta}\sum\limits_{j=n_{i}+1}^{n_{i+1}}X_j<b)$$
$$=\lim_{\delta\rightarrow0}P(\sqrt{\delta}\sum\limits_{j=n_{i-1}+1}^{n_{i}}X_j<a)P(\sqrt{\delta}\sum\limits_{j=n_{i}+1}^{n_{i+1}}X_j<x_{i+1})$$
$$=P(Y_i<x_i)P(Y_{i+1}<b)$$

Remark 1: Here we used the property that almost sure convergence implies convergence in probability and convergence in probability implies convergence in distribution.

Remark 2:
In this case, we construct a Bronwian motion with i.i.d. random variables $\{X_i\}$ where the domain for $X_i$ is  $\{-1,1\}$. Therefore the domain $\Omega$ for the random variables is $\{-1,1\}\times...\times\{-1,1\}\times...=\{-1,1\}^{\infty}$ in the notation of product space.

Actually according to the Wiener's theroem, we construct a Bronwian motion with any i.i.d. random variables $\{X_i\}$. Here we assume nothing for the random variable as long as it's not determined. Suppose the probability space  for $X_i$ is  $(S, \sigma(S),P')$ . Hence the domain $\Omega$ for the random variables is $S\times...\times S\times...=S^{\infty}$.

For any given time $t$, $W_t$ is a random variable. The value of $W_t$ is completely determined by the i.i.d. random variables $\{X_i\}$. Therefore the domain  for the $W_t$ is $\Omega=S^{\infty}$, where $S$ is the domain for $X_i$. The $\sigma$ -algebra   $\mathcal{F}=\sigma((A_1\times A_2\times ...\times A_n \times...):A_i\in \sigma(S))$. The measure  is $P(A_1\times A_2\times ...\times A_n \times...)=\Pi_{i=1}^{\infty}P'(A_i)$, where $P'$ is the measure on $S$. Therefore $(\Omega, \mathcal{F},P)$ is the probability space for a random variable $W_t$.


\subsection{Smoothing kernel estimation}

\subsubsection{Smoothed kernels}

We have \(K(x-x_i)\)

We can smooth this to:

\(K_h(x-x_i)=\dfrac{1}{h}K(\dfrac{x-x_i}{h})\)

Where \(h>0\) is the smoothing bandwidth.

\(f(x)=\dfrac{1}{n}\sum_{i=1}^nK_h(x-x_i)\)


\subsection{The projection and annihilation matrices}

\subsubsection{The projection matrix}

We have \(X\).

The projection matrix is \(X(X^TX)^{-1}X^T\)

The projection matrix maps from actual y to predicted y

\(\hat y = Py\)

Each entry refers to the covariance between actual and fitted

\(p_{ij}=\dfrac{Cov (\hat y_i, y_j}{Var (y_j)}\)

\subsubsection{The annihilation matrix}

We can get residuals too:

\(u=y-\hat y=y-py=(1-P)y\)

\(1-P\) is called the annihilator matrix

We can now use the propagation of uncertainty

\(\Sigma^f = A\Sigma^x A^T\)

To get:

\(\Sigma^u = (I-P)\Sigma^y (I-P)\)

Annihilator matrix is:

\(M_X=I-X(X^TX)^{-1}X^T\)

Called this because:

\(M_XX=X-X(X^TX)^{-1}X^TX\)

\(M_XX=0\)

Is called residual maker