\chapter{Integral Transforms }
\section{Integral Transforms}
Frequently in mathematical physics we encounter pairs of functions related by an expression of the form
\begin{equation}
g(x)=\int_{a}^{b} f(t) K(x, t) d t\label{FT-01}
\end{equation}
where it is understood that $a, b$, and $K(x, t)$ (called the kernel) will be the same for all function pairs $f$ and $g$. We can write the relationship expressed in Eq. (\ref{FT-01}) in the more symbolic form
\begin{equation}
g(x)=\mathcal{L}{ f(t)}
\end{equation}
thereby emphasizing the fact that Eq. $(\ref{FT-01})$ can be interpreted as an operator equation. The function $g(x)$ is called the integral transform of $f(t)$ by the operator $\mathcal{L}$, with the specific transform determined by the choice of $a, b$, and $K(x, t)$. The operator defined by Eq. (\ref{FT-01}) will be linear:
\begin{align}
\int_{a}^{b}\left[f_{1}(t)+f_{2}(t)\right] K(x, t) d t&=\int_{a}^{b} f_{1}(t) K(x, t) d t+\int_{a}^{b} f_{2}(t) K(t) d t \\
\int_{a}^{b} c f(t) K(\alpha, t) d t&=c \int_{a}^{b} f(t) K(\alpha, t) d t
\end{align}
In order for transforms to be useful, we will shortly see that we need to be able to "undo" their effect. From a practical viewpoint, this means that not only must there exist an operator $\mathcal{L}^{-1}$, but also that we have a reasonably convenient and powerful method of evaluating
\begin{equation}
\mathcal{L}^{-1} g(x)=f(t)
\end{equation}
for an acceptably broad range of $g(x)$. The procedure for inverting a transform takes a wide variety of forms that depend on the specific properties of $K(x, t)$, so we cannot write a formula that is as general as that for $\mathcal{L}$ in Eq. (\ref{FT-01}).\\
Not all superficially reasonable choices for the kernel $K(x, t)$ will lead to operators $\mathcal{L}$ that have inverses, and even for strategically chosen kernels it may be the case that $\mathcal{L}$ and $\mathcal{L}^{-1}$ will only exist for substantially restricted classes of functions. Thus, the entire development of the present chapter is restricted (for any given integral transform) to functions for which the indicated operations can be carried out.\\
One of the frequent uses of integral transforms is to use one, together with its inverse, to form an \textbf{integral representation} of a function that we originally had in an explicit form. This move (which appears to be in the direction of generating greater complexity) has value that arises from the relatively simple behavior of the transforms of differentiation and integration operators. Procedures involving integral representations are also presented in later sections of this chapter.
\begin{figure}[H]
	\centering
	\includegraphics[height=5cm,width=9cm]{FT-01}
\end{figure}
\newpage
\begin{abox}
	Fourier Transform
	\end{abox}

The integral transform that has seen the widest use is the Fourier transform, defined as
\begin{equation}
g(\omega)=\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} f(t) e^{i \omega t} d t
\end{equation}
The notation for this transform is not entirely universal; some writers omit the prefactor $1 / \sqrt{2 \pi}$; we keep it because it causes the transform and its inverse to have formulas that are more symmetrical. In applications involving periodic systems, one occasionally encounters a definition with kernel $\exp \left(2 \pi i \omega t / a_{0}\right)$, where $a_{0}$ is a lattice constant. These differences in notation do not change the mathematics, but cause formulas to differ by powers of $2 \pi$ or $a_{0}$. Caution is therefore advised when combining material from different sources.\\
\section{Infinite Fourier Transform}
\begin{align*}
\text{The fourier transform of }f(x)[-\infty<x<\infty] &:\\
f(s)=F\{f(x)\}&=\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} f(x) e^{i s x} d x\\
\text{Inverse fourier transform of }f(s)[-\infty<x<\infty] &:\\
f(x)=F^{-1}\{f(s)\}&=\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} f(s) e^{-i s x} d s .
\end{align*}
\section{Fourier Sine Transform}
\begin{align*}
f_{s}(\mathrm{~s})&=F_{s}\{f(x)\}=\sqrt{\frac{2}{\pi}} \int_{0}^{\infty} f(x) \sin s x d x \\
f(x)&=F_{s}^{-1}\left\{f_{s}(s)\right\}=\sqrt{\frac{2}{\pi}} \int_{0}^{\infty} f_{s}(s) \sin s x d s
\end{align*}
\section{Fourier Cosine Transform}
\begin{align*}
f_{c}(s)&=F_{c}\{f(x)\}=\sqrt{\frac{2}{\pi}} \int_{0}^{\infty} f(x) \cos s x d x \\
f(x)&=F_{c}^{-1}\left\{f_{c}(s)\right\}=\sqrt{\frac{2}{\pi}} \int_{0}^{\infty} f_{c}(s) \cos s x d s
\end{align*}
\subsubsection{Important Formulas:}
\begin{enumerate}
	\item $\int_{0}^{\infty} \frac{\sin a x}{x} d x=\frac{\pi}{2}$
	\item $\int_{0}^{\infty} e^{-x^{2}} d x=\sqrt{\frac{\pi}{2}}$
	\item $\int_{0}^{\infty} e^{-a x} \sin b x d x=\frac{b}{a^{2}+b^{2}}$
	\item $\int_{0}^{\infty} e^{-a x} \cos b x d x=\frac{a}{a^{2}+b^{2}}$
	\item $\int_{0}^{\infty} e^{-a x^{2}} d x=\frac{1}{2} \sqrt{\frac{\pi}{a}}$
	\item $\int_{0}^{\infty} e^{-a x^{2}} \cos b x d x=\frac{1}{2} \sqrt{\frac{\pi}{a}} e^{-b^{2} / 4 a}$
	\item $\int_{-\infty}^{\infty} e^{-a x^{2}+b x} d x=\sqrt{\frac{\pi}{a}} e^{b^{2} / 4 a}$
\end{enumerate}

\begin{exercise}
	Find the fourier transform of Gaussian distribution function $f(x)=N e^{-a x^{2}}$ where $N$ and 'a' are constant.
\end{exercise}
\begin{answer}
	\begin{align*}
	F\{f(x)\}=\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} f(x) e^{i k x} d x=\frac{1}{\sqrt{2 \pi}} \int_{-\infty} N e^{-a x^{2}} e^{i k x} d x=\frac{N}{\sqrt{2 \pi}} e^{-k^{2} / 4 a}
	\end{align*}
\end{answer}
\begin{exercise}
	Find $F\{\delta(x)\}$
\end{exercise}
\begin{answer}
	\begin{align*}
	F\{f(x)\}=\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} f(x) e^{i k x} d x=\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} \delta(x) e^{i k x} d x=\frac{1}{\sqrt{2 \pi}}\left[\mathrm{e}^{i k x}\right]_{x=0}=\frac{1}{\sqrt{2 \pi}}
	\end{align*}
\end{answer}
\begin{exercise}
	Find the fourier transform of $f(x)=e^{-a|x|} \quad(-\infty<x<\infty) \quad(a>0)$
\end{exercise}
\begin{answer}
	\begin{align*}
	\begin{aligned}
	F\left[e^{-a|x|}\right] &=\frac{1}{\sqrt{2 \pi}}\left[\int_{-\infty}^{0} e^{a x} e^{i s x} d x+\int_{0}^{\infty} e^{-a x} e^{i s x} d x\right]=\frac{1}{\sqrt{2 \pi}}\left[\left(\frac{e^{(a+i s) x}}{(a+i s)}\right)_{-\infty}^{0}+\left(\frac{e^{-(a-i s) x}}{(i s-a)}\right)_{0}^{\infty}\right] \\
	&=\frac{1}{\sqrt{2 \pi}}\left[\frac{1}{a+i s}+\frac{1}{a-i s}\right]=\frac{2 a}{\sqrt{2 \pi}\left(s^{2}+a^{2}\right)} \because
	\end{aligned}
	\end{align*}
\end{answer}

\section{Properties of Fourier Transform}
(i) \textbf{Linearity theorem:}
\begin{align*}
\text{If }F(x)&=a_{1} f_{1}(x)+a_{2} f_{2}(x)+\ldots\text{ then, }\\f(s)&=F[f(x)]=a_{1} f_{1}(s)+a_{2} f_{2}(s)+\ldots
\end{align*}
(ii) \textbf{Change of scale property: }
\begin{align*}
\text{If }F[f(x)]&=f(s),\text{ then }F\{f(a x)\}\\&=\frac{1}{a} f(\mathrm{~s} / a)
\end{align*}
(iii) $\text{If }F\{f(x)\}=f(s),\text{ then }F\left\{f^{*}(x)\right\}=f^{*}(-s)$\\\\
(iv) \textbf{Shifting property:}
 \begin{align*}
\text{ If $f(s)$ is the Fourier transform of }\mathrm{f}(\mathrm{x}), \text{ then }F\{f(x \pm a)\}=e^{\mp i s a} f(s)
 \end{align*}
(v) \textbf{Modulation theorem:} \\\\
If $F\{f(x)\}=f(s)$, then $F\{f(x) \cos a x\}=\frac{1}{2} f(s-a)+\frac{1}{2} f(s+a)$
\section{Parseval Identity for Fourier Transform}
If the Fourier transform of $f(x)$ and $g(x)$ be $f(s) \& g(s)$ respectively, then\\\\
(i) $\int_{-\infty}^{\infty} f(s) g^{*}(s) d s=\int_{-\infty}^{\infty} f(x) g^{*}(x) d x$\\\\
where $g^{*}(s)$ is complex conjugate of $g(s)$ and $g^{*}(x)$ is the complex conjugate of $g(x)$.\\\\
(ii) $\left.\int_{-\infty}^{\infty}|f(s)|\right.^{2} d s=\int_{-\infty}^{\infty}|f(x)|^{2} d x$
\begin{exercise}Using parseval's identity show that $\int_{b}^{\infty}\left(\frac{\sin x}{x}\right)^{2} d x=\pi / 2$
\end{exercise}
\begin{answer}
	\begin{align*}
	F\{f(x)\}&=\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} f(x) e^{i x x} d x=\sqrt{\frac{2}{\pi}} \frac{\sin a s}{s}\\
	\text{Using parseval identity, }\int_{-\infty}^{\infty}|f(x)|^{2} d x&=\int_{-\infty}^{\infty}|f(s)|^{2} d s\\
	\int_{-a}^{\infty} l^{2} d x&=\int_{-\infty}^{\infty} \frac{2}{\pi} \frac{\sin ^{2} a s}{s^{2}} d s \\ 2 a&=\frac{2}{\pi} \int_{-\infty}^{\infty}\left(\frac{\sin a s}{s}\right)^{2} d s\\
	\text{Putting }a s&=x \Rightarrow a d s=d x\\
	\Rightarrow \pi&=\int_{-\infty}^{\infty}\left(\frac{\sin x}{x}\right)^{2} d x \\ \int_{0}^{\infty}\left(\frac{\sin x}{x}\right)^{2} d x&=\frac{\pi}{2}
	\end{align*}
\end{answer}
\begin{exercise}
	If the fourier transform of $f(x)$ is $f(s)$, then find the fourier transform of $\frac{d f}{d x}$
\end{exercise}
\begin{answer}
	\begin{align*}
	F\left[\frac{d f}{d x}\right]&=\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} \frac{d f}{d x} e^{i s x} d x\\&=\frac{1}{\sqrt{2 \pi}}\left[e^{i s x} f(x)\right]_{-\infty}^{\infty}-\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} i s e^{i s x} f(x) d x\\
	\intertext{Since $f(x)$ should be a well behave function for the fourier tranform to exist, therefore, the first term of the above equation should be equal to zero.}
	\text{Then }F\left[\frac{d f}{d x}\right]&=-i s \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} e^{i s x} f(x) d x=-is f( {s})
	\end{align*}
\end{answer}
\begin{exercise}
	Find Fourier transform of $f(x)= \begin{cases}x^{2}, & |x|<a \\ 0, & |x|>a\end{cases}$
\end{exercise}
\begin{answer}
	\begin{align*}
	F\{f(x)\} &=\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} e^{i s x} f(x) d x=\frac{1}{\sqrt{2 \pi}} \int_{-a}^{a} e^{i s x} x^{2} d x \\
	&=\frac{1}{\sqrt{2 \pi}}\left[\left(\frac{e^{i s x}}{i s} \cdot x^{2}\right)_{x=-a}^{a}-\frac{2}{i s} \int_{-a}^{a} x e^{i s x} d x\right] \\
	&=\frac{1}{\sqrt{2 \pi}}\left[\frac{a^{2}}{i s}\left(e^{i s a}-e^{-i s a}\right)-\frac{2}{i s}\left[\left(\frac{x e^{i s x}}{i s}\right)_{x=-a}^{a}-\frac{1}{i s} \int_{-a}^{a} e^{i s x} d x\right]\right]\\
	&=\frac{1}{\sqrt{2 \pi}}\left[\frac{2 a^{2}}{s} \sin (s a)+\frac{2 a}{s^{2}} \cos (s a)-\frac{4}{s^{3}} \sin (s a)\right]
	\end{align*}
\end{answer}
\begin{exercise}
	Find the complex fourier transform $e^{-|x|}$
\end{exercise}
\begin{answer}
	\begin{align*}
	F\left\{e^{-|x|}\right\}&=\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} e^{-|x|} e^{i s x} d x\\&=\frac{1}{\sqrt{2 \pi}}\left[\int_{-\infty}^{0} e^{-|x|} e^{i s x} d x+\int_{0}^{\infty} e^{-|x|} \cdot e^{i s x} d x\right]\\
	&=\frac{1}{\sqrt{2 \pi}}\left[\int_{-\infty}^{0} e^{x} \cdot e^{i s x} d x+\int_{0}^{\infty} e^{-x(1-i s)} d x\right]\\&=\frac{1}{\sqrt{2 \pi}}\left[\left[\frac{e^{x(1+i s)}}{1+i s}\right]_{x=-\infty}^{0}+\left[\frac{e^{-x(1-i s)}}{-(1-i s)}\right]_{x=0}^{\infty}\right] \\
	&=\frac{1}{\sqrt{2 \pi}}\left[\frac{1}{1-i s}+\frac{1}{1+i s}\right]\\&=\frac{1}{\sqrt{2 \pi}}\left(\frac{2}{1+s^{2}}\right)
	\end{align*}
\end{answer}

\begin{exercise}
	The graph of a real periodic function $f(x)$ for the range $[-\infty, \infty]$ is shown below
	\begin{figure}[H]
		\centering
		\includegraphics[height=3cm,width=5cm]{FT01}
	\end{figure}
	Which of the following graphs represents the real part of its Fourier transform?
	 \begin{tasks}(2)
		\task[\textbf{a.}]
		\begin{figure}[H]
			\centering
			\includegraphics[height=2cm,width=2.5cm]{FT-06}
		\end{figure}
		\task[\textbf{b.}]	\begin{figure}[H]
			\centering
			\includegraphics[height=2cm,width=2.5cm]{FT-05}
		\end{figure}
		\task[\textbf{c.}]	\begin{figure}[H]
			\centering
			\includegraphics[height=2cm,width=2.5cm]{FT-03}
		\end{figure}
		\task[\textbf{d.}] 	\begin{figure}[H]
			\centering
			\includegraphics[height=2cm,width=2.5cm]{FT02}
		\end{figure}
	\end{tasks}
\end{exercise}
\begin{answer}
	\begin{align*}
	\text { The given graph in the question is of } f(x)&=\cos \omega x\\
	F(\cos \omega x)=\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{+\infty} \cos \omega x e^{i k x} d x&=\frac{1}{\sqrt{2 \pi}}\left(\int_{-\infty}^{+\infty} \frac{e^{i(k+\omega) x}}{2} d x+\int_{-\infty}^{+\infty} \frac{e^{i(k-\omega) x}}{2} d x\right)\\
	&\Rightarrow F(\cos \omega x) \frac{1}{\sqrt{2 \pi}}\left(\frac{1}{2} \delta(k+\omega)+\frac{1}{2} \delta(k-\omega)\right)
	\intertext{Therefore, the fourier transform of $f(x)=\cos x \omega x$ consist of two delta functions of same height at $k=\omega$ and $k=-\omega$.}
	\end{align*}
		So the correct answer is \textbf{option (b)}
\end{answer}
\newpage
\begin{abox}
	Laplace Transforms
\end{abox}
The Laplace transform $f(s)$ of a function $F(t)$ is defined by
\begin{equation}
f(s)=\mathcal{L}\{F(t)\}=\int_{0}^{\infty} e^{-s t} F(t) d t .
\end{equation}
A few comments on the existence of the integral are in order. The infinite integral of $F(t)$,
\begin{equation*}
\int_{0}^{\infty} F(t) d t,
\end{equation*}
need not exist. For instance, $F(t)$ may diverge exponentially for large $t$. However, if there are some constants $s_{0}, M$, and $t_{0} \geq 0$ such that for all $t>t_{0}$
\begin{equation}
\left|e^{-s_{0} t} F(t)\right| \leq M,\label{LT-01}
\end{equation}
the Laplace transform will exist for $s>s_{0} $. $F(t)$ is then said to be of exponential order. As a counter example, $F(t)=e^{t^{2}}$ does not satisfy the condition given by Eq. (\ref{LT-01}) and is not of exponential order. Thus, $\mathcal{L}\left\{e^{t^{2}}\right\}$ does not exist.
The Laplace transform may also fail to exist because of a sufficiently strong singularity in the function $F(t)$ as $t \rightarrow 0$. For example,
\begin{equation*}
\int_{0}^{\infty} e^{-s t} t^{n} d t
\end{equation*}
diverges at the origin for $n \leq-1$. The Laplace transform $\mathcal{L}\left\{t^{n}\right\}$ does not exist for $n \leq-1$.

\section{Important Laplace Transforms:}
\begin{enumerate}
	\item $\mathcal{L}(1)=\frac{1}{s}$
	\begin{exercise}
		$$
		\mathcal{L}(1)=\frac{1}{s}
		$$
	\end{exercise}
	\begin{answer}
		\begin{align*}
		\mathcal{L}(1)&=\int_{0}^{\infty} 1 \cdot e^{-s t} d t=\left[\frac{e^{-s t}}{-s}\right]_{0}^{\infty}\\&=-\frac{1}{s}\left[\frac{1}{e^{s t}}\right]_{0}^{\infty}=-\frac{1}{s}[0-1]\\&=\frac{1}{s}\\
		\text{ Hence }\mathcal{L}(1)&=\frac{1}{s}
		\end{align*}
	\end{answer}
	\item $\mathcal{L}\left(t^{n}\right)=\frac{n !}{s^{n+1}}(n=0,1,2, \ldots \ldots . .)$
	\begin{exercise}
		$\mathcal{L}\left(t^{n}\right)=\frac{n !}{s^{n+1}}$ where $n$ and $s$ are positive.
	\end{exercise}
	\begin{answer}
		\begin{align*}
		\mathcal{L}\left(t^{n}\right)&=\int_{0}^{\infty} e^{-s t} t^{n} d t\\
		\text{Putting }s t=&x \quad\text{ or }\quad t=\frac{x}{s} \quad\text{ or } \quad d t=\frac{d x}{s}\\
		\text{Thus we have}\mathcal{L}\left(t^{n}\right)&=\int_{0}^{\infty} e^{-x}\left(\frac{x}{s}\right)^{n} \frac{d x}{s_{r}} \quad \Rightarrow \quad \mathcal{L}\left(t^{n}\right)=\frac{1}{s^{n+1}} \int_{0}^{\infty} e^{-x} \cdot x^{n} d x\\
		\Rightarrow \mathcal{L}\left(t^{n}\right)&=\frac{\sqrt{n+1}}{s^{n+1}} \Rightarrow \mathcal{L}\left(t^{n}\right)=\frac{n !}{s^{n+1}}\hspace{2cm}\left[\begin{array}{c}
		{\left[n+1=\int_{0}^{\infty} e^{-x} \cdot x^{n} d x\right] } \\
		\text { and } \quad \mid \overline{n+1}=n !
		\end{array}\right]
		\end{align*}
	\end{answer}
	\item $\mathcal{L}\left(e^{a t}\right)=\frac{1}{s-a}(s>a)$
	\begin{exercise}
		$\mathcal{L}\left(e^{a t}\right)=\frac{1}{s-a}, \quad$ where $s>a$
	\end{exercise}
	\begin{answer}
		\begin{align*}
		\mathcal{L}\left(e^{a t}\right) &=\int_{0}^{\infty} e^{-s t} \cdot e^{a t} d t=\int_{0}^{\infty} e^{-s t+a t} \cdot d t \\
		&=\int_{0}^{\infty} e^{(-s+a) t} \cdot d t=\int_{0}^{\infty} e^{-(s-a) t} \cdot d t=\left[\frac{e^{-(s-a) t}}{-(s-a)}\right]_{0}^{\infty}=-\frac{1}{s-a}\left[\frac{1}{e^{(s-a) t}}\right]_{0}^{\infty}\\
		&=\frac{-1}{(s-a)}(0-1)=\frac{1}{s-a}
		\end{align*}
	\end{answer}
	\item $\mathcal{L}(\cos a t)=\frac{s}{s^{2}+a^{2}}$
	$(s>0)$
	\begin{exercise}
		$L(\cos a t)=\frac{s}{s^{2}+a^{2}}$
	\end{exercise}
	\begin{answer}
		\begin{align*}
		\mathcal{L}(\cos a t)&=\mathcal{L}\left(\frac{e^{t a t}+e^{-i a t}}{2}\right) \quad\left[\because \cos a t=\frac{e^{i a t}+e^{-i a t}}{2}\right]\\
		&=\frac{1}{2}\left[\mathcal{L}\left(e^{i a t}+e^{-i a t}\right)\right]=\frac{1}{2}\left[\mathcal{L}\left(e^{i a t}\right)+\mathcal{L}\left(e^{i a t}\right)\right]=\frac{1}{2}\left[\frac{1}{s-i a}+\frac{1}{s+i a}\right]=\frac{1}{2} \frac{s+i a+s-i a}{s^{2}+a^{2}}\\
		&=\frac{s}{s^{2}+a^{2}}
		\end{align*}
	\end{answer}
	\item $\mathcal{L}(\sin a t)=\frac{a}{s^{2}+a^{2}} \quad(s>0)$
		\begin{exercise}
		$\mathcal{L}(\sin a t)=\frac{a}{s^{2}+a^{2}}$
	\end{exercise}
	\begin{answer}
		\begin{align*}
		\mathcal{L}(\sin a t)&=\mathcal{L}\left[\frac{e^{i a t}-e^{-i a t}}{2 i}\right]\hspace{3cm} \left[\because \sin a t=\frac{e^{i a t}-e^{-i a t}}{2 i}\right]\\
		&=\frac{1}{2 i}\left[L\left(e^{i a t}-e^{-i a t}\right)\right]=\frac{1}{2 i}\left[L\left(e^{i a t}\right)-\mathcal{L}\left(e^{-i a t}\right)\right] \\
		&=\frac{1}{2 i}\left[\frac{1}{s-i a}-\frac{1}{s+i a}\right]=\frac{1}{2 i} \frac{s+i a-s+i a}{s^{2}+a^{2}}=\frac{1}{2 i} \frac{2 i a}{s^{2}+a^{2}}=\frac{a}{s^{2}+2^{2}}
		\end{align*}
	\end{answer}

	\item $\mathcal{L}(\cosh a t)=\frac{s}{s^{2}-a^{2}}\left(s^{2}>a^{2}\right)$
		\begin{exercise}
		$\mathcal{L}(\cosh a t)=\frac{s}{s^{2}-a^{2}}$
	\end{exercise}
	\begin{answer}
		\begin{align*}
		\mathcal{L}(\cosh a t) &=L\left[\frac{e^{a t}+e^{-a t}}{2}\right] \hspace{4cm}\left(\therefore \cosh a t=\frac{e^{a t}+e^{-a t}}{2}\right)\\
		&=\frac{1}{2} \mathcal{L}\left(e^{a t}\right)+\frac{1}{2} L\left(e^{-a t}\right)=\frac{1}{2}\left[\frac{1}{s-a}+\frac{1}{s+a}\right] \hspace{2cm}\left[\mathcal{L}\left(e^{a t}\right)=\frac{1}{s-a}\right]\\
		&=\frac{1}{2}\left[\frac{s+a+s-a}{s^{2}-a^{2}}\right]=\frac{s}{s^{2}-a^{2}}
		\end{align*}
	\end{answer}
	\item $\mathcal{L}(\sinh a t)=\frac{a}{s^{2}-a^{2}}\left(s^{2}>a^{2}\right)$
	\begin{exercise}
		$L(\sinh a t)=\frac{a}{s^{2}-a^{2}}$
	\end{exercise}
	\begin{answer}
		\begin{align*}
		\mathcal{L}(\sinh a t) &=\mathcal{L}\left[\frac{1}{2}\left(e^{a t}-e^{-a t}\right)\right] \\
		&=\frac{1}{2}\left[\mathcal{L}\left(e^{a t}\right)-L\left(e^{-a t}\right)\right]=\frac{1}{2}\left[\frac{1}{s-a}-\frac{1}{s+a}\right]=\frac{1}{2}\left[\frac{s+a-s+a}{s^{2}-a^{2}}\right] \\
		&=\frac{a}{s^{2}-a^{2}}
		\end{align*}
	\end{answer}
\end{enumerate}

\subsubsection{Important properties:}
\begin{enumerate}
	\item Linear Property: $\mathcal{L}\left[a_{1} f_{1}(t)+a_{2} f_{2}(t)\right]=a_{1} \mathcal{L}\left[f_{1}(t)\right]+a_{2} \mathcal{L}\left[f_{2}(t)\right]$
	\item Shifting Property: $\mathcal{L}\left[e^{a t} f(t)\right]=f(s-a)$
	\item Scaling Property: $\mathcal{L}[f(a t)]=\frac{1}{a} f\left(\frac{s}{a}\right)$
	\item $\mathcal{L}\left[t^{n} f(t)\right]=(-1)^{n} \frac{d^{n}}{d s^{n}}(f(s))$
	\item $\mathcal{L}\left[\frac{f(t)}{t}\right]=\int_{-\infty}^{\infty} f(s) d s$
	\item $\mathcal{L}\left[\int_{0}^{t} f(t) d t\right]=\frac{1}{s} f(s)$
	\item $\mathcal{L}\left[f^{\prime}(t)\right]=s L[f(t)]-f(0)$
	\item  $\mathcal{L}\left[f^{\prime \prime}(t)\right]=s^{2} L[f(t)]-f^{\prime}(0)-s f(0)$
	\item Laplace transform of a periodic function $f(t)$ having period ' $T$ ' is $$\mathcal{L}[f({t})]=\frac{1}{1-e^{-s T}} \int_{0}^{T} e^{-s t} f({t}) d {t}$$
	\item Laplace transform of the unit step function $u(t-a)=u_{a}(t)=\left\{\begin{array}{lll}0 & \text { for } & t<a \\ 1 & \text { for } & t \geq a\end{array}\right.$ is $\frac{e^{-a s}}{s}$.
	\item Laplace transform of the dirac delta function $\delta(t-a)$ is $e^{-a s}$.
\end{enumerate}
\begin{exercise}
	Find the laplace transform of $f(t)=(1+\cos 2 t)$
\end{exercise}
\begin{answer}
	\begin{align*}
	\begin{aligned}
	\mathcal{L}[f(t)] &=\int_{0}^{\infty} e^{-s t}\left(1+\cos 2 t \cdot d t=\left(\frac{e^{-s t}}{-s}\right)_{0}^{\infty}+\int_{0}^{\infty} e^{-s t} \cos 2 t d t\right.\\
	&=\frac{1}{s}+\frac{s}{4+s}=\frac{2 s^{2}+4}{s\left(s^{2}+4\right)}
	\end{aligned}
	\end{align*}
\end{answer}
\begin{exercise}
	Find the laplace transform of $f(t)=2 \sin 2 t \cos 4 t$
\end{exercise}
\begin{answer}
	\begin{align*}
	\mathcal{L}[f(t)]=\mathcal{L}[2 \sin 2 t \cdot \cos 4 t]=\mathcal{L}[\sin 6 t-\sin 2 t]=\left(\frac{6}{s^{2}+36}-\frac{2}{s^{2}+4}\right)
	\end{align*}
\end{answer}
\begin{exercise}
	Find the laplace transform of $f(t)=e^{-t}(3 \sinh 2 t-5 \cosh 2 t)$
\end{exercise}
\begin{answer}
	\begin{align*}
	\begin{aligned}
	\mathcal{L}[f(t)] &=\mathcal{L}\left(3 e^{-t} \sinh 2 t\right)-\mathcal{L}\left(5 e^{-t} \cosh 2 t\right) \\
	&=3 \frac{2}{(s+1)^{2}-4}-5 \cdot \frac{(s+1)}{(s+1)^{2}-4}=\frac{(1-5 \mathrm{~s})}{\left(\mathrm{s}^{2}+2 \mathrm{~s}-3\right)} \quad \text { (Using 2nd property) }
	\end{aligned}
	\end{align*}
\end{answer}
\begin{exercise}
	Find the laplace transform of the fuction\\
	$f(t)=1$ for $2 n \leq t \leq 2 n+1$\\
	$=0 \quad$ for $2 n+1 \leq t \leq 2 n+2 \quad(n=0,1,2, \ldots . .)$
\end{exercise}
\begin{answer}
	\begin{align*}
	\mathcal{L}[f(t)] &=\frac{1}{1-e^{-s T}} \int_{0}^{T} e^{-s t} f(t) d t=\frac{1}{1-e^{-2 s}} \int_{0}^{2} e^{-s t} d t \\
	&=\frac{1}{1-e^{-2 s}}\left[\int_{0}^{1} e^{-s t} d t\right]=\frac{1}{1-e^{-2 s}}\left(\frac{1-e^{-s}}{s}\right)\\
	&=\frac{\left(1-e^{-s}\right)}{s\left(1+e^{-s}\right)\left(1-e^{-s}\right)}=\frac{1}{s\left(1+e^{-s}\right)}
	\end{align*}
\end{answer}

\begin{exercise}
	Find the laplace transform of $f(t)=\left(1+t e^{-t}\right)^{3}$
\end{exercise}
\begin{answer}
	\begin{align*}
	f(t)&=\left(1+t e^{-t}\right)^{3}=1+t^{3} e^{-3 t}+3 t e^{-t}+3 t^{2} e^{-2 t} \\
	\mathcal{L}[f(t)]&=\mathcal{L}(1)+\mathcal{L}\left(t^{3} e^{-3 t}\right)+\mathcal{L}\left(3 t e^{-t}\right)+\mathcal{L}\left(3 t^{2} e^{-2 t}\right)\\
	&=\frac{1}{s}+(-1)^{3} \frac{d^{3}}{d s^{3}}\left(\frac{1}{s+3}\right)+3(-1)^{1} \frac{d}{d s}\left(\frac{1}{s+1}\right)+3(-1)^{2} \frac{d^{2}}{d s^{2}}\left(\frac{1}{s+2}\right)\text{ (Using 4th property)}\\
	&=\frac{1}{s}+\frac{6}{(s+3)^{4}}+\frac{3}{(s+1)^{2}}+\frac{6}{(s+2)^{3}} \text {. }
	\end{align*}
\end{answer}

\newpage
\begin{abox}
	Practise Set-1
\end{abox}
\begin{enumerate}

\item The Fourier transform of the derivative of the Dirac $\delta-$ function, namely $\delta^{\prime}(x)$, is proportional to
{\exyear{NET/JRF(DEC-2013)}}

\begin{tasks}(4)
\task[\textbf{A.}] 0
\task[\textbf{B.}] 1
\task[\textbf{C.}] $\sin k$
\task[\textbf{D.}] $i k$
\end{tasks}
\item
	The graph of a real periodic function $f(x)$ for the range $[-\infty, \infty]$ is shown below {\exyear{NET/JRF(DEC-2014)}}
	\begin{figure}[H]
		\centering
		\includegraphics[height=3cm,width=5cm]{FT01}
	\end{figure}
	Which of the following graphs represents the real part of its Fourier transform?
	\begin{tasks}(2)
		\task[\textbf{A.}]
		\begin{figure}[H]
			\centering
			\includegraphics[height=2cm,width=2.5cm]{FT-06}
		\end{figure}
		\task[\textbf{B.}]	\begin{figure}[H]
			\centering
			\includegraphics[height=2cm,width=2.5cm]{FT-05}
		\end{figure}
		\task[\textbf{C.}]	\begin{figure}[H]
			\centering
			\includegraphics[height=2cm,width=2.5cm]{FT-03}
		\end{figure}
		\task[\textbf{D.}] 	\begin{figure}[H]
			\centering
			\includegraphics[height=2cm,width=2.5cm]{FT02}
		\end{figure}
	\end{tasks}
\item The Fourier transform of $f(x)$ is $\tilde{f}(k)=\int_{-\infty}^{+\infty} d x e^{i k x} f(x)$.
If $f(x)=\alpha \delta(x)+\beta \delta^{\prime}(x)+\gamma \delta^{\prime \prime}(x)$, where $\delta(x)$ is the Dirac delta-function (and prime denotes derivative), what is $\tilde{f}(k) ?$
{\exyear{NET/JRF(DEC-2015)}}

\begin{tasks}(2)
\task[\textbf{A.}] $\alpha+i \beta k+i \gamma k^{2}$
\task[\textbf{B.}] $\alpha+\beta k-\gamma k^{2}$
\task[\textbf{C.}]  $\alpha-i \beta k-\gamma k^{2}$
\task[\textbf{D.}] $i \alpha+\beta k-i \gamma k^{2}$
\end{tasks}
\item What is the Fourier transform $\int d x e^{i l x} f(x)$ of
$$
f(x)=\delta(x)+\sum_{n=1}^{\infty} \frac{d^{n}}{d x^{n}} \delta(x)
$$
where $\delta(x)$ is the Dirac delta-function?
{\exyear{NET/JRF(JUNE-2016)}}

\begin{tasks}(4)
\task[\textbf{A.}]  $\frac{1}{1-i k}$
\task[\textbf{B.}] $\frac{1}{1+i k}$
\task[\textbf{C.}] $\frac{1}{k+i}$
\task[\textbf{D.}] $\frac{1}{k-i}$
\end{tasks}
\item The Fourier transform $\int_{-\infty}^{\infty} d x f(x) e^{i k x}$ of the function $f(x)=\frac{1}{x^{2}+2}$ is
{\exyear{NET/JRF(DEC-2016)}}

\begin{tasks}(4)
\task[\textbf{A.}] $\sqrt{2} \pi e^{-\sqrt{2}|| \mid}$
\task[\textbf{B.}] $\sqrt{2} \pi e^{-\sqrt{2 k}}$
\task[\textbf{C.}] $\frac{\pi}{\sqrt{2}} e^{-\sqrt{2 k}}$
\task[\textbf{D.}] $\frac{\pi}{\sqrt{2}} e^{-\sqrt{2}|k|}$
\end{tasks}

\item The Fourier transform $\int_{-\infty}^{\infty} d x f(x) e^{i k x}$ of the function $f(x)=e^{-|x|}$
{\exyear{NET/JRF(JUNE-2018)}}

\begin{tasks}(4)
\task[\textbf{A.}] $-\frac{2}{1+k^{2}}$
\task[\textbf{B.}] $-\frac{1}{2\left(1+k^{2}\right)}$
\task[\textbf{C.}] $\frac{2}{1+k^{2}}$
\task[\textbf{D.}] $\frac{2}{\left(2+k^{2}\right)}$
\end{tasks}
\item The Heaviside function is defined as $H(t)=\left\{\begin{array}{ll}+1, & \text { for } t>0 \\ -1, & \text { for } t<0\end{array}\right.$ and its Fourier transform is given by $-2 i / \omega$. The Fourier transform of $\frac{1}{2}[H(t+1 / 2)-H(t-1 / 2)]$ is
{\exyear{GATE 2015}}

\begin{tasks}(4)
\task[\textbf{A.}] $\frac{\sin \left(\frac{\omega}{2}\right)}{\frac{\omega}{2}}$
\task[\textbf{B.}]  $\frac{\cos \left(\frac{\omega}{2}\right)}{\frac{\omega}{2}}$
\task[\textbf{C.}] $\sin \left(\frac{\omega}{2}\right)$
\task[\textbf{D.}]  0
\end{tasks}
\item The Fourier transform of the function $\frac{1}{x^{4}+3 x^{2}+2}$ up to proportionality constant is
{\exyear{JEST 2017}}

\begin{tasks}(2)
\task[\textbf{A.}] $\sqrt{2} \exp \left(-k^{2}\right)-\exp \left(-2 k^{2}\right)$
\task[\textbf{B.}]$\sqrt{2} \exp (-|k|)-\exp (-\sqrt{2}|k|)$
\task[\textbf{C.}] $\sqrt{2} \exp (-\sqrt{|k|})-\exp (-\sqrt{2|k|})$
\task[\textbf{D.}]$\sqrt{2} \exp \left(-\sqrt{2} k^{2}\right)-\exp \left(-2 k^{2}\right)$
\end{tasks}

\end{enumerate}
\colorlet{ocre1}{ocre!70!}
\colorlet{ocrel}{ocre!30!}
\setlength\arrayrulewidth{1pt}
\begin{table}[H]
	\centering
	\arrayrulecolor{ocre}
	\begin{tabular}{|p{1.5cm}|p{1.5cm}||p{1.5cm}|p{1.5cm}|}
		\hline
		\multicolumn{4}{|c|}{\textbf{Answer key}}\\\hline\hline
		\rowcolor{ocrel}Q.No.&Answer&Q.No.&Answer\\\hline
		1&\textbf{D} &2&\textbf{B}\\\hline
		3&\textbf{C} &4&\textbf{B} \\\hline
		5&\textbf{D} &6&\textbf{C} \\\hline
		7&\textbf{A}&8&\textbf{B}\\\hline
		9&\textbf{D}&10&\textbf{}\\\hline
		11&\textbf{} &12&\textbf{}\\\hline
		13&\textbf{}&14&\textbf{}\\\hline
		15&\textbf{}& &\\\hline

	\end{tabular}
\end{table}
\newpage
\begin{abox}
	Practise Set-2
\end{abox}
\begin{enumerate}
	\item The graph of the function $f(x)=\left\{\begin{array}{ll}1 & \text { for } 2 n \leq x \leq 2 n+1 \\ 0 & \text { for } 2 n+1 \leq x \leq 2 n+2\end{array}\right.$ where $n=(0,1,2, \ldots \ldots)$ is shown below. Its Laplace transform $\tilde{f}(s)$ is
{	\exyear{NET/JRF(DEC-2011)}}

	\begin{figure}[H]
		\centering
		\includegraphics[height=2.7cm,width=7.5cm]{diagram-20211005(9)-crop}
	\end{figure}
	\begin{tasks}(4)
		\task[\textbf{A.}] $\frac{1+e^{-s}}{s}$
		\task[\textbf{B.}] $\frac{1-e^{-s}}{s}$
		\task[\textbf{C.}] $\frac{1}{s\left(1+e^{-s}\right)}$
		\task[\textbf{D.}]  $\frac{1}{s\left(1-e^{-s}\right)}$
	\end{tasks}

	\item The inverse Laplace transforms of $\frac{1}{s^{2}(s+1)}$ is
	{\exyear{NET/JRF(JUNE-2013)}}

	\begin{tasks}(4)
		\task[\textbf{A.}] $\frac{1}{2} t^{2} e^{-t}$
		\task[\textbf{B.}] $\frac{1}{2} t^{2}+1-e^{-t}$
		\task[\textbf{C.}] $t-1+e^{-t}$
		\task[\textbf{D.}] $\frac{1}{2} t^{2}\left(1-e^{-t}\right)$
	\end{tasks}

	\item The Laplace transform of
	$$
	f(t)=\left\{\begin{array}{cc}
	\frac{t}{T}, & 0<t<T \\
	1 & t>T
	\end{array}\right.
	$$
	is
	{\exyear{NET/JRF(DEC-2016)}}

	\begin{tasks}(4)
		\task[\textbf{A.}] $\frac{-\left(1-e^{-s T}\right)}{s^{2} T}$
		\task[\textbf{B.}] $\frac{\left(1-e^{-s T}\right)}{s^{2} T}$
		\task[\textbf{C.}] $\frac{\left(1+e^{-s T}\right)}{s^{2} T}$
		\task[\textbf{D.}] $\frac{\left(1-e^{s T}\right)}{s^{2} T}$
	\end{tasks}

	\item Consider the differential equation $\frac{d y}{d t}+a y=e^{-b t}$ with the initial condition $y(0)=0$. Then the Laplace transform $Y(s)$ of the solution $y(t)$ is
	{\exyear{NET/JRF(DEC-2017)}}

	\begin{tasks}(4)
		\task[\textbf{A.}] $\frac{1}{(s+a)(s+b)}$
		\task[\textbf{B.}] $\frac{1}{b(s+a)}$
		\task[\textbf{C.}] $\frac{1}{a(s+b)}$
		\task[\textbf{D.}] $\frac{e^{-a}-e^{-b}}{b-a}$
	\end{tasks}

	\item If $f(x)=\left\{\begin{array}{ll}0 & \text { for } x<3, \\ x-3 & \text { for } x \geq 3\end{array}\right.$ then the Laplace transform of $f(x)$ is
	{\exyear{GATE 2010}}

	\begin{tasks}(4)
		\task[\textbf{A.}] $s^{-2} e^{3 s}$
		\task[\textbf{B.}] $s^{2} e^{3 s}$
		\task[\textbf{C.}] $s^{-2}$
		\task[\textbf{D.}] $s^{-2} e^{-3 s}$
	\end{tasks}

	\item The Laplace transform of $\frac{(\sin (a t)-a t \cos (a t))}{\left(2 a^{3}\right)}$ is
	{\exyear{JEST 2018}}

	\begin{tasks}(4)
		\task[\textbf{A.}] $\frac{2 a s}{\left(s^{2}+a^{2}\right)^{2}}$
		\task[\textbf{B.}]$\frac{s^{2}-a^{2}}{\left(s^{2}+a^{2}\right)^{2}}$
		\task[\textbf{C.}]$\frac{1}{(s+a)^{2}}$
		\task[\textbf{D.}]$\frac{1}{\left(s^{2}+a^{2}\right)^{2}}$
	\end{tasks}
\end{enumerate}
\colorlet{ocre1}{ocre!70!}
\colorlet{ocrel}{ocre!30!}
\setlength\arrayrulewidth{1pt}
\begin{table}[H]
	\centering
	\arrayrulecolor{ocre}
	\begin{tabular}{|p{1.5cm}|p{1.5cm}||p{1.5cm}|p{1.5cm}|}
		\hline
		\multicolumn{4}{|c|}{\textbf{Answer key}}\\\hline\hline
		\rowcolor{ocrel}Q.No.&Answer&Q.No.&Answer\\\hline
		1&\textbf{C} &2&\textbf{C}\\\hline
		3&\textbf{B} &4&\textbf{A} \\\hline
		5&\textbf{D} &6 &\textbf{D} \\\hline

	\end{tabular}
\end{table}
\newpage
\begin{abox}
	Practise Set-3
\end{abox}
\begin{enumerate}

	\item The Fourier transform of $f(x)=\left\{\begin{array}{cl}-1 & -1<x<0 \\ 1 & 0<x<1 \\ 0 & \text { otherwise }\end{array}\right.$ is
	\begin{tasks}(2)
		\task[\textbf{a.}] $\frac{i}{\omega} \sqrt{\frac{2}{\pi}}(\cos \omega+1)$
		\task[\textbf{b.}] $\frac{i}{\omega} \sqrt{\frac{2}{\pi}}(\cos \omega-1)$
		\task[\textbf{c.}]$-\frac{i}{\omega} \sqrt{\frac{2}{\pi}}(1+\cos \omega)$
		\task[\textbf{d.}] $\frac{i}{\omega} \sqrt{\frac{2}{\pi}}(1-\cos \omega)$
	\end{tasks}
	\begin{answer}
	\begin{align*}
	H(f)&=\int_{-\infty}^{+\infty} h(t) e^{-j 2 \pi f t} d t=\int_{0}^{+\infty} \beta e^{-\alpha t} e^{-j 2 \pi f t} d t=\beta \int_{0}^{+\infty} e^{-(\alpha+j 2 \pi f) t} d t\\
	\Rightarrow H(f)&=\left.\frac{-\beta}{\alpha+j 2 \pi f} e^{-(\alpha+j 2 \pi f) t}\right|_{0} ^{\infty}=\frac{\beta}{\alpha+j 2 \pi f}=\frac{\beta \alpha}{\alpha^{2}+(2 \pi f)^{2}}-j \frac{2 \pi f \beta}{\alpha^{2}+(2 \pi f)^{2}}\\
	\Rightarrow H(f)&=\frac{\beta}{\sqrt{\alpha^{2}+(2 \pi f)^{2}}} e^{j \tan ^{-1}[-2 \pi f / \alpha]}
	\end{align*}
	So the correct answer is \textbf{Option (d)}
\end{answer}
	\item The Fourier transform of $f(x)= \begin{cases}x, & 0<x<a \\ 0, & \text { otherwise }\end{cases}$ is
	\begin{tasks}(2)
		\task[\textbf{a.}]$\frac{1}{\sqrt{2 \pi} \cdot \omega^{2}}\left[e^{-i \omega n a}(1+i a \omega)-1\right]$
		\task[\textbf{b.}]$\frac{1}{\sqrt{2 \pi} \cdot \omega^{2}}\left[e^{-i \omega a}(1-i a \omega)-1\right]$
		\task[\textbf{c.}] $\frac{1}{\sqrt{2 \pi} \cdot \omega^{2}}\left[e^{-1 \text { tox }}(1+i a \omega)+1\right]$
		\task[\textbf{d.}] $\frac{1}{\sqrt{2 \pi} \cdot \omega^{2}}\left[e^{-i \omega a}(1-i a \omega)+1\right]$
	\end{tasks}
	\begin{answer}
	\begin{align*}
	f(x)&= \begin{cases}x, & 0<x<a \\ 0, & \text { otherwise }\end{cases}\\
	F[f(x)]&=\frac{1}{\sqrt{2 \pi}} \int_{0}^{a} x e^{-i \omega x} d x=\frac{1}{\sqrt{2 \pi}}\left[\left(\frac{x e^{-i e x}}{-i \omega}\right)_{0}^{a}-\left(\frac{e^{-i \omega x}}{(-i \omega)^{2}}\right)_{0}^{a}\right]\\&=\frac{1}{\sqrt{2 \pi}}\left[\frac{a e^{-i \omega a}}{-i \omega}-\frac{e^{-i \omega a}-1}{(-i \omega)^{2}}\right]=\frac{1}{\sqrt{2 \pi}}\left[\frac{i a \omega e^{-i e a}}{\omega^{2}}+\frac{e^{-i e a}-1}{\omega^{2}}\right]\\
	&=\frac{1}{\sqrt{2 \pi} \cdot \omega^{2}}\left[e^{-i \omega x}(1+i a \omega)-1\right]
	\end{align*}
	So the correct answer is \textbf{Option (a)}
\end{answer}

\item	If $F\left\{e^{-i 2 \text { avat }}\right\}=f(\omega+a)$, then $F(\cos 2 \pi v a t)=$ ?

\begin{answer}
	\begin{align*}
	\mathrm{F}[\cos 2 \pi v \mathrm{vat}]=\mathrm{F} \frac{\left[\mathrm{e}^{\mathrm{i} 2 \pi \mathrm{vat}}+\mathrm{e}^{-\mathrm{i} 2 \pi \mathrm{vat}}\right]}{2}=\frac{1}{2}\left\{F\left[e^{i 2 \pi v a t}\right]+F\left[e^{-i 2 \pi v a t}\right]\right\}=\frac{1}{2}[f(\omega-a)+f(\omega+a)]
	\end{align*}
\end{answer}

\item	Given that $F[\delta(x-a)]=\exp [-i 2 \pi v a]$, then $F^{-1}[\cos 2 \pi v a]=$ ?

\begin{answer}
	\begin{align*}
	\text{	As }\quad F\{\delta(x-a)\}&=\exp [-i 2 \pi v a] \quad \Rightarrow \delta(x-a)=F^{-1}[\exp (-i 2 \pi v a)]\\
	\text{Now }F^{-1}(\cos 2 \pi v a)&=F^{-1}\left[\frac{\exp (-i 2 \pi v a)+\exp (i 2 \pi v a)}{2}\right]=\frac{1}{2}[\delta(x+a)+\delta(x-a)]
	\end{align*}
\end{answer}

\item	Find the fourier cosine transform of the function $f(x)=e^{-a^{2} x^{2}}$

\begin{answer}
	\begin{align*}
	\begin{aligned}
	f_{c}(s) &=F_{c}[f(x)]=\sqrt{\frac{2}{\pi}} \int_{b}^{\infty} e^{-a^{2} x^{2}} \cos s x d x=\frac{1}{2} \sqrt{\frac{2}{\pi}} \int_{-\infty}^{\infty} e^{-a^{2} x^{2}} \cos s x d x \\
	&=\frac{1}{2} \sqrt{\frac{2}{\pi}} \text { Real part of } \int_{-\infty}^{\infty} e^{-a^{2} x^{2}} e^{i s x} d x=\frac{1}{2} \sqrt{\frac{2}{\pi}} \text { Real part of }\left(\sqrt{\frac{\pi}{a^{2}}} e^{\frac{i^{2} s^{2}}{4 a^{2}}}\right)=\frac{1}{\sqrt{2 a^{2}}} e^{-s^{2} / 4 a^{2}}
	\end{aligned}
	\end{align*}
\end{answer}

\item	Find the sine transform of $\frac{x}{1+x^{2}}$

\begin{answer}
	\begin{align}
	\text { Firstly we shall determine}&\text{ cosine transform of } \frac{1}{1+x^{2}}\notag\\
	F_{c}\left\{\frac{1}{1+x^{2}}\right\}&=\int_{0}^{\infty} \frac{\cos s x}{1+x^{2}} d x=I(\text { say })\notag\\
	\text{then }I&=\int_{0}^{\infty} \frac{\cos s x}{1+x^{2}} d x\label{FT-03}\\
	\text{Dufferentiating w.r.t.s,}\notag\\
	\frac{d I}{d s} &=\int_{0}^{\infty} \frac{-x \sin s x}{1+x^{2}} d x=F_{s}\left\{\frac{-x}{1+x^{2}}\right\}\label{FT-04} \\
	&=-\int_{0}^{\infty} \frac{x^{2} \sin s x d x}{x\left(1+x^{2}\right)}=-\int_{0}^{\infty} \frac{\left(1+x^{2}-1\right) \sin s x \cdot d x}{x\left(1+x^{2}\right)}\notag\\
	&=-\int_{0}^{\infty} \frac{\sin s x}{x} d x+\int_{0}^{\infty} \frac{\sin s x d x}{x\left(1+x^{2}\right)}=-\frac{\pi}{2}+\int_{0}^{\infty} \frac{\sin s x d x}{x\left(1+x^{2}\right)}\notag\\
	&=-\int_{0}^{\infty} \frac{\sin s x}{x} d x+\int_{0}^{\infty} \frac{\sin s x d x}{x\left(1+x^{2}\right)}=-\frac{\pi}{2}+\int_{0}^{\infty} \frac{\sin s x d x}{x\left(1+x^{2}\right)}\label{FT-05}\\
	\text{Differentiating again w.r.t. s,}\notag\\
	\frac{d^{2} I}{d s^{2}}&=\int_{0}^{\infty} \frac{x \cos (s x) d x}{x\left(1+x^{2}\right)}=I\notag\\
	\text { Or, } &\quad \frac{d^{2} I}{d s^{2}}-I=0\notag\\
	\text{The solution of it is }I&=A e^{-s}+B e^{s}\label{FT-06}\\
	\text{ Putting }\mathrm{s}&=0 \text{in (\ref{FT-03}),}\notag\\
	I &=\int_{0}^{\infty} \frac{1}{1+x^{2}} d x=\left(\tan ^{-1} x\right)_{0}^{\infty}=\frac{\pi}{2} \notag\\
	\therefore \quad I &=\frac{\pi}{2} \text { when } s=0\notag\\
	\text{Subjecting (\ref{FT-06}) to this}&\text{ condition}\notag\\
	\frac{\pi}{2}&=A+B\label{FT-07}
	\end{align}
\end{answer}
\item
	Find cosine transform of $f(x)$ if $f(x)= \begin{cases}\cos x, & 0<x<a \\ 0, & x>a\end{cases}$

\begin{answer}
	\begin{align*}
	F_{C}\{f(x)\} &=\sqrt{\frac{2}{\pi}} \int_{0}^{\infty} f(x) \cos (p x) d x \\
	&=\sqrt{\frac{2}{\pi}} \int_{0}^{a} \cos x \cos (p x) d x+\int_{a}^{\infty} 0 \cdot \cos p x d x \\
	&=\sqrt{\frac{2}{\pi}} \frac{1}{2} \int_{0}^{a}[\cos (p+1) x \cos (p-1) x] d x\\
	&=\sqrt{\frac{2}{\pi}} \frac{1}{2}\left[\frac{\sin (p+1) x}{p+1}+\frac{\sin (p-1) x}{p-1}\right]_{x=0}^{a} \\
	&=\sqrt{\frac{2}{\pi}} \frac{1}{2}\left[\frac{\sin (p+1) a}{p+1}+\frac{\sin (p-1) a}{p-1}\right]
	\end{align*}
\end{answer}

\item	Fourier transform of the derivative of the Dirac $\delta$-function, namely, $\delta^{\prime}(x)$, is proportional to
	\begin{tasks}(4)
		\task[\textbf{a.}]0
		\task[\textbf{b.}]1
		\task[\textbf{c.}] $\sin k$
		\task[\textbf{d.}] $\mathrm{i} k$
	\end{tasks}

\begin{answer}
	\begin{align*}
	\text { Fourier transform of } \delta^{\prime}(x)&=\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} \delta^{\prime}(x) e^{i k x} d x\\
	\intertext { Using the property of Dirac delta function, } \int_{-\infty}^{\infty} f(x) \delta^{\prime}(x-a) d x&=-f^{\prime}(a)\\
	\text { So, } \quad F\left[\delta^{\prime}(x)\right]&=-\frac{1}{\sqrt{2 \pi}}(i k)  \\ F\left[\delta^{\prime}(x)\right] &\propto i k
	\end{align*}
	So the correct answer is \textbf{option (d)}
\end{answer}
\item Find the fourier transform of
$f(x)=\left\{\begin{array}{lll}1 & \text { for } & |x|<a \\ 0 & \text { for } & |x|>a\end{array}\right.$
\begin{answer}
	\begin{align}
	\intertext{The fourier transform of a function $f(x)$ is given by}
	F[f(x)]&=\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^{\infty} f(x) \cdot e^{i s x} d x\label{FT15}
	\intertext{Substituting the value of $f(x)$ in (\ref{FT15}), we get}
	F[f(x)] &=\frac{1}{\sqrt{2 \pi}} \int_{-a}^{a} 1 \cdot e^{i s x} d x=\left[\frac{e^{i s x}}{i s}\right]_{-a}^{a}=\frac{1}{\sqrt{2 \pi}} \frac{1}{(i s)}\left[e^{i a s}-e^{-i a s}\right] \notag\\
	&=\frac{1}{\sqrt{2 \pi}} \frac{2}{s} \cdot \frac{e^{i a s}-e^{-i a s}}{2 i}=\frac{1}{\sqrt{2 \pi}} \frac{2 \sin s a}{s}=\sqrt{\frac{2}{\pi}} \frac{\sin s a}{s}\notag
	\end{align}
\end{answer}

\item Find Fourier Sine transform of $\frac{1}{x}$
\begin{answer}
	\begin{align}
	\text{Here, }f(x)&=\frac{1}{x}\notag\\
	F_{s}[f(x)]&=\sqrt{\frac{2}{\pi}} \int_{0}^{\infty} f(x) \sin s x d x\label{FT18}\\
	\text { Putting the value of } &f(x) \text { in }(\ref{FT18}) \text {, we get }\notag\\
	F_{s}\left(\frac{1}{x}\right) &=\sqrt{\frac{2}{\pi}} \int_{0}^{\infty} \frac{\sin s x}{x} d x=\sqrt{\frac{2}{\pi}} \int_{0}^{\infty} \frac{\sin \theta}{\frac{\theta}{s}} \frac{d \theta}{s} \quad \text { Putting } s x=\theta \text { so that } s d x=d \theta \notag\\
	&=\sqrt{\frac{2}{\pi}} \int_{0}^{\infty} \frac{\sin \theta}{\theta} d \theta=\sqrt{\frac{2}{\pi}}\left(\frac{\pi}{2}\right) \quad \text { [Some useful result I on page 1208] Ans. } \notag\\
	&=\sqrt{\frac{\pi}{2}}\notag
	\end{align}
\end{answer}
\item Find the Fourier Sine Transform of $e^{-a x}$
\begin{answer}
	\begin{align}
	\text{Here, }f(x)&=e^{-a x}\notag
\intertext{	The Fourier sine transform of $f(x)$ :}
	F_{s}[f(x)]&=\sqrt{\frac{2}{\pi}} \int_{0}^{\infty} f(x) \sin s x d x\label{FT19}
\intertext{	On putting the value of $f(x)$ in (\ref{FT19}), we get}
	F_{s}\left[e^{-a x}\right]&=\sqrt{\frac{2}{\pi}} \int_{0}^{\infty} e^{-a x} \sin s x d x\notag
	\intertext{On integrating by parts, we get}
	F_{s}\left[e^{-a x}\right] &=\sqrt{\frac{2}{\pi}}\left[\frac{e^{-a x}}{a^{2}+s^{2}}[-a \sin s x-s \cos s x]\right]_{0}^{\infty}\left[\int e^{a x} \sin b x d x=\frac{e^{a x}}{a^{2}+b^{2}}(a \sin b x-b \cos b x)\right] \notag\\
	&=\sqrt{\frac{2}{\pi}}\left[0-\frac{1}{a^{2}+s^{2}}(-s)\right]=\sqrt{\frac{2}{\pi}}\left(\frac{s}{a^{2}+s^{2}}\right)\notag
\end{align}
\end{answer}
\item Find the laplace transform of $f(t)=\frac{1-e^{t}}{t}$

\begin{answer}
	\begin{align*}
	\mathcal{L}\left(\frac{1-e^{t}}{t}\right)&=\int_{s}^{\infty}\left(\frac{1}{s}-\frac{1}{s-1}\right) d s=[\ln s-\ln (s-1)]_{s}^{\infty}=\left[\ln \frac{s}{s-1}\right]_{s}^{\infty}:\text{ (Using 5th property)}\\
	&=[\ln s-\ln (s-1)]_{s}^{\infty}=\left[\ln \frac{s}{s-1}\right]_{s}^{\infty}=\ln \left(\frac{1}{\left.1-\frac{1}{s}\right)_{s}}\right)^{\infty} \\
	&=0-\ln \frac{s}{s-1}=\ln \frac{(s-1)}{s}
	\end{align*}
\end{answer}

\item	Find the inverse laplace transform of
	$$\mathrm{f}(\mathrm{s}) =\frac{\mathrm{s}-2}{(\mathrm{~s}-2)^{2}+25}+\frac{\mathrm{s}+4}{(\mathrm{~s}+4)^{2}+81}+\frac{1}{(\mathrm{~s}+2)^{2}+9} $$

\begin{answer}
	\begin{align*}
	\mathcal{L}^{-1}[\mathrm{f}(\mathrm{s})] &=\mathcal{L}^{-1}\left[\frac{\mathrm{s}-2}{(\mathrm{~s}-2)^{2}+25}\right]+\mathrm{L}^{-1}\left[\frac{\mathrm{s}+4}{(\mathrm{~s}+4)^{2}+81}\right]+\mathrm{L}^{-1}\left[\frac{1}{(\mathrm{~s}+2)^{2}+9}\right] \\
	&=\mathrm{e}^{2 \mathrm{t}} \cos 5 \mathrm{t}+\mathrm{e}^{-4 \mathrm{t}} \cos 9 \mathrm{t}+\frac{\mathrm{e}^{-2 \mathrm{t}}}{3} \sin 3 \mathrm{t}
	\end{align*}
\end{answer}

\item	The graph of the function
	$$f(t)= \begin{cases}1 & \text { for } 2 n \leq t \leq 2 n+1 \\ 0 & \text { for } 2 n+1 \leq t \leq 2 n+2\end{cases}$$
	(Where $n=0,1,2,........$)is shown below.
	\begin{figure}[H]
		\centering
		\includegraphics[height=2.5cm,width=6cm]{LT-01}
	\end{figure}
	Its Laplace transform $\tilde{f}(\mathrm{s})$ is
	\begin{tasks}(2)
		\task[\textbf{a.}]$\frac{1+e^{-s}}{s}$
		\task[\textbf{b.}]$\frac{1-e^{-s}}{s}$
		\task[\textbf{c.}]$\frac{1}{s\left(1+e^{-s}\right)}$
		\task[\textbf{d.}]  $\frac{1}{s\left(1-e^{-s}\right)}$
	\end{tasks}

\begin{answer}
	Given function is a periodic function of period $T=2$
	\begin{align*}
	\mathcal{L}[f(t)] &=\frac{1}{1-e^{-s T}} \int_{0}^{T} e^{-s t} f(t) d t=\frac{1}{1-e^{-2 s}} \int_{0}^{2} e^{-s t} d t \\
	&=\frac{1}{1-e^{-2 s}}\left[\int_{0}^{1} e^{-s t} d t\right]=\frac{1}{1-e^{-2 s}}\left(\frac{1-e^{-s}}{s}\right) \\
	&=\frac{\left(1-e^{-s}\right)}{s\left(1+e^{-s}\right)\left(1-e^{-s}\right)}=\frac{1}{s\left(1+e^{-s}\right)}
	\end{align*}
	So the correct answer is \textbf{option (c)}
\end{answer}

\item	The inverse transform of $\frac{1}{s^2(s+1)}$ is
	\begin{tasks}(2)
		\task[\textbf{a.}]$\frac{1}{2} t^{2} e^{-t}$
		\task[\textbf{b.}]$\frac{1}{2} t^{2}+1-e^{-t}$
		\task[\textbf{c.}]$t-1+e^{-t}$
		\task[\textbf{d.}] $\frac{1}{2} t^{2}\left(1-e^{-t}\right)$
	\end{tasks}

\begin{answer}
	\begin{align*}
	&f(s)=\frac{1}{s^{2}(s+1)}=\frac{(s+1)-s(s+1)+s^{2}}{s^{2}(s+1)}=\frac{1}{s^{2}}-\frac{1}{s}+\frac{1}{s+1} \\
	&\Rightarrow \mathcal{L}^{-1}[f(s)]=t-1+e^{-t}
	\end{align*}
	So the correct answer is \textbf{option (c)}
\end{answer}
\end{enumerate}

CSIR- Mathematical Physics/chapter/Fourier Transform.tex


\section{Matrices}
\subsection{Matrices}
The set of real m × n matrices is denoted $R^{m \times n}$
\subsection{Zero and identity matrices}
\textbullet \textit{Zero}: All elements equals 0.\\
\textbullet \textit{Identity}: All elements equals 0 and diagonal element equals 1.\\
\textbullet \textit{Sparse}: If many entries are 0\\
\textbullet \textit{Diagonal}: off-diagonal entries are zero\\
\textbullet \textit{Triangular}: \textit{upper triangular} if $A_{ij} = 0 \text{ for } i>j$, and it is \textit{lower triangular} if $A_{ij} =0 \text{ for } i<j$\\
\textbullet \textit{Adjacency Matrix}:
For, $R={(1,2), (1,3), (2,1), (2,4), (3,4), (4,1)}$\\
$
A_{ij} = \left\{
  \begin{array}{@{}ll@{}}
    1, & (i,j) \in R\\
    0, & (i, j) \not\in R
  \end{array}\right.
$\\
A relation R on ${1,...,n}$ is represented by the n×n matrix A with $A_{ij} = 1$, if there exists a edge else , $A_{ij} = 0$
\subsection{Transpose, addition and norm}
\textbf{Block matrix Transpose}\\
$
\begin{bmatrix}
    A & B \\
    C & D

\end{bmatrix}^{T}=
\begin{bmatrix}
    A^T & C^T \\
    B^T & D^T
\end{bmatrix}
$\\
\textbf{Symmetric matrix}: $A = A^T$\\
\textbf{Properties of matrix addition}\\
\textbullet Commutativity: $A+B=B+A$\\
\textbullet Associativity: $(A + B) + C = A + (B + C)$\\
\textbullet Addition with zero matrix: $A+0=0+A=A$\\
\textbullet Transpose of sum:  $(A+B)^T = A^T +B^T$\\
If A is a matrix and $\beta$ , $\gamma$ are scalars
$(\beta + \gamma)A = \beta A + \gamma A,
(\beta \gamma)A = \beta(\gamma A)
$\\
\textbf{Matrix norm}
$\Vert A \Vert = \sqrt{\sum_{i=1}^n\sum_{j=1}^m} A^2_{ij}$
matrix norm satisfies the properties of any norm

\subsection{Matrix-vector multiplication}
A is an $m\times n$ matrix and x is an n-vector, then the matrix-vector product $y = Ax$\\

$y_i = \sum_{k=1}^n A_{ik} x_k =A_{i1}x_1 +...+A_{in}x_n$ for $i=1...m$\\
\textbullet \textbf{Row and column interpretations.}\\
$y = Ax$ can be expressed as $y_i=b^T_ix, i=1,..,m$ where $b^T_1,...,b^T_m$ are rows of A\\
\textbullet $y = Ax$ could also be expressd in terms of column
$y = x_1a_1 + x_2a_2 +...+ x_na_n$
\subsection{Complexity}
\textit{addition}: $mn$\\
\textit{sparse matrix addition}: If A or B or both are sparse $min\{nnz(A), nnz(B)\}$\\
\textit{vector multiplication $A_{mxn}$ with n-vector}: $m(2n-1) \approx 2mn$\\
\textit{Matrix Transpose}: 0 flops


\lab{Applications}{Linear Algebra in Statistics}{Statistics}
\label{Stats1}

\objective{This section will teach about using Python to solve problems in statistics. This will include finding means, correlation matrices, and solving least squares problems.}

\section*{Shifting Data by the Mean}

Consider the table below representing students scores in a class.\\

\begin{figure}[h!]
\begin{center}
\begin{tabular}{|c|r|r|r|r|}
	\hline
Student & Homework & Exam 1  & Exam 2 & Final \\
\hline
S1  & 89 & 91 & 77 & 75 \\
S2  & 67 & 72 & 76 & 66 \\
S3  & 72 & 77 & 69 & 70 \\
S4  & 56 & 60 & 55 & 61 \\
S5  & 92 & 98 & 89 & 86 \\
S6  & 83 & 88 & 90 & 84 \\
S7  & 45 & 60 & 55 & 48 \\
\hline
Average  & 72 & 78 & 73 & 70\\
\hline
\end{tabular}\\
\end{center}
\end{figure}

We can shift our data set by subtracting each column by its average value.  This makes it so that the average of each column in the matrix below is zero.  If $W$ represents the matrix of scores, the following Python command will subtract out the average.  Why does this work?
\begin{lstlisting}[style=python]
:X = W - sp.dot(sp.ones((7,1),dtype=sp.float_),sp.mean(W,axis=0).reshape(1,4));X
array([[ 17.,  13.,   4.,   5.],
       [ -5.,  -6.,   3.,  -4.],
       [  0.,  -1.,  -4.,   0.],
       [-16., -18., -18.,  -9.],
       [ 20.,  20.,  16.,  16.],
       [ 11.,  10.,  17.,  14.],
       [-27., -18., -18., -22.]])
\end{lstlisting}

\section*{Angles Between Vectors}

Inner products give information about lengths of vectors and angles between vectors.  Recall that the angle $\theta$ between two vectors is given by
\[
\cos{\theta} = \frac{\ipt{x}{y}}{\norm{x}\norm{y}}
\]
where the norm (or length) of a vector is $\norm{x} = \sqrt{\ipt{x}{x}}$.  Note that the usual inner product in $\mathbb{R}^n$ is given by
\[
\ipt{x}{y} = x^T y
\]
Alternatively, if we can first divide our vectors by their length (these are called unit vectors) and then take the inner product
\[
\cos{\theta} = \left\langle\frac{x}{\norm{x}},\frac{y}{\norm{y}}\right\rangle
\]
Hence, we can find the cosine of the angles between our data columns in $X$ by dividing each column by its length.  There are a couple of ways of doing this.  The following is an interesting way of doing it.  Be sure to explore why it works:
\begin{lstlisting}[style=python]
: Y = X / sp.dot(sp.ones((7,1),dtype=sp.float_),sp.sqrt(sp.diag(sp.dot(X.T,X)).T).reshape(1,4));Y
array([[ 0.39848615,  0.35329218,  0.11386819,  0.15371887],
       [-0.11720181, -0.16305793,  0.08540114, -0.12297509],
       [ 0.        , -0.02717632, -0.11386819,  0.        ],
       [-0.37504578, -0.48917378, -0.51240685, -0.27669396],
       [ 0.46880723,  0.54352643,  0.45547275,  0.49190037],
       [ 0.25784398,  0.27176321,  0.4839398 ,  0.43041282],
       [-0.63288976, -0.48917378, -0.51240685, -0.67636301]])
\end{lstlisting}
Hence we have:
\[
Y=
\begin{bmatrix}
0.3985 & 0.3533 & 0.1139 & 0.1537\\
-0.1172 & -0.1631 & 0.0854 & -0.1230\\
0 &-0.0272 &-0.1139 & 0\\
-0.3750 & -0.4892 & -0.5124 & -0.2767\\
0.4688 & 0.5435 & 0.4555 & 0.4919\\
0.2578 & 0.2718 & 0.4839 & 0.4304\\
-0.6329 & -0.4892 & -0.5124 & -0.6764
\end{bmatrix}
\]
Finally, we get the cosines of the angles between columns by computing $Y^T Y$.  In Python, that's just
\begin{lstlisting}[style=python]
: sp.dot(Y.T,Y)
array([[ 1.        ,  0.97782999,  0.89014869,  0.94908967],
       [ 0.97782999,  1.        ,  0.90978843,  0.92490144],
       [ 0.89014869,  0.90978843,  1.        ,  0.9276955 ],
       [ 0.94908967,  0.92490144,  0.9276955 ,  1.        ]])
\end{lstlisting}
This yields
\[
Y^T Y =
\begin{bmatrix}
1.0000 & 0.9778 & 0.8901 & 0.9491\\
0.9778 & 1.0000 & 0.9098 & 0.9249\\
0.8901 & 0.9098 & 1.0000 & 0.9277\\
0.9491 & 0.9249 & 0.9277 & 1.0000
\end{bmatrix}.
\]
Note that the $(j,k)$ entry of $Y^T Y$ corresponds to the cosine of the angle between the $j^{th}$ and $k^{th}$ columns.  We remark that the diagonals are always equal to one because the angle between a vector and itself is zero and the cosine of zero is one.

\section*{Correlation}

We remark that the cosine of the angle between two vectors is sometimes called the correlation coefficient.  Two columns are said to be
\begin{itemize}
\item Perfectly correlated if the cosine of the angle between them is one.
\item Positively correlated if the cosine of the angle between them is between zero and one.
\item Uncorrelated if the cosine of the angle between them is zero.
\item Negatively correlated if the cosine of the angle between them is between negative one and zero.
\item Perfectly anticorrelated if the cosine of the angle between them is negative one.
\end{itemize}

The notion of correlation is important in establishing the relationships between measurements.  For example, there is a high correlation between those who smoke and those who get lung cancer.  It's important to understand that the high correlation alone does not necessarily imply that smoking causes lung cancer, it only links them statistically.  This is the famous issue of causation versus correlation.  For example, there is a high correlation between crime rates and sales of ice cream.  This doesn't mean that ice cream causes crime or that increases in crime makes people want to eat more ice cream, but both rates do go up in the summer.

\begin{problem}
Download the file lab8.txt from the following link:
\url{http://www.math.byu.edu/~jeffh/teaching/m343h/lab8.txt}
You can load this datafile by typing
\li{lab8 = sp.genfromtxt("lab8.txt")}
The data is now available in the array \li{lab8} (also, make sure Python is working in the directory of lab8.txt).  \li{lab8} consists of two columns.  Find the correlation coefficient of the two columns.  Then plot the original data and see if the value that you got is reasonable.
\end{problem}

\section*{Least Squares}

It is well known that the displacement of a spring is proportional to the force acting upon it, that is, $F = k x$.  The proportionality constant $k$ is called Hooke's spring constant.  Consider a laboratory experiment where different loads are placed on a spring and the displacement is measured and recorded in the table below:
\vspace{5mm}\\
\begin{center}
\begin{tabular}{|c|c|}
	\hline
x & F \\
(cm) & (dyne)\\
\hline
1.04  & 3.11 \\
2.03  &  6.01\\
2.95  &  9.07\\
3.92  &  11.99\\
5.06  &  15.02\\
6.00  &  17.91\\
7.07  &  21.12\\
\hline
\end{tabular}
\end{center}
\vspace{5mm}
To find the spring constant $k$, we simply need to solve the following linear system
\[
\begin{pmatrix}
1.04\\
2.03\\
2.95\\
3.92\\
5.06\\
6.00\\
7.07\\
\end{pmatrix}
\begin{pmatrix}k\end{pmatrix} =
\begin{pmatrix}
3.11 \\
6.01\\
9.07\\
11.99\\
15.02\\
17.91\\
21.12\\
\end{pmatrix}.
\]
However, there is no solution to this system because it is overdetermined.  Instead, we seek the ``best'' $k$ that fits the data.  Least squares (which we mentioned in a previous exercise) allows us to find that ``best'' solution. We can find the least squares solution by computing the following in Python:
\begin{lstlisting}[style=python]
: A = sp.vstack([1.04,2.03,2.95,3.92,5.06,6.00,7.07])
: b = sp.vstack([3.11,6.01,9.07,11.99,15.02,17.91,21.12])
: k = sp.dot(sp.dot(la.inv(sp.dot(A.T,A)),A.T),b);k
: # Using the built in least squares function this is just: k=la.lstsq(A,b);k
array([[ 2.99568294]])
\end{lstlisting}
Hence, we find the spring constant to be $k = 2.9957$.  We plot the data against the best fit as follows:
\begin{figure}[h!]
\label{fig1}
\begin{center}
\includegraphics[scale = .6]{lab9_line_py}
\caption{The graph of the spring data together with its linear fit}
\label{Fig:SpringFit}
\end{center}
\end{figure}

\begin{lstlisting}[style=python]
: x0 = sp.linspace(0,8,100)
: y0 = k[0]*x0
: from matplotlib import pyplot as plt
: plt.plot(A,b,'*',x0,y0)
: plt.show()
\end{lstlisting}
See Figure \ref{Fig:SpringFit} to see how well the line fits the data.

\section*{General Line Fitting}

Suppose that we wish to fit a general line, that is $y=m x+b$, to the data set $\{(x_k,y_k)\}^n_{k=1}$.  Assume that the line does not cross through the origin, as in the previous example.  Then we seek both a slope and a $y$-intercept.  In this case, we set up the following linear system $A x = b$, or more precisely
\[
\begin{pmatrix}
x_1 & 1\\
x_2 & 1\\
x_3 & 1\\
\vdots & \vdots\\
x_n & 1
\end{pmatrix}
\begin{pmatrix}
m\\
b
\end{pmatrix}=
\begin{pmatrix}
y_1\\
y_2\\
y_3\\
\vdots\\
y_n
\end{pmatrix}.
\]
Note that $A$ has rank $2$ as long as not all of the $x_k$ values are the same.  Hence, the least squares solution will fit the best line for this data.

\begin{problem}
Download the file lab9.txt from the following link:
\url{http://www.math.byu.edu/~jeffh/teaching/m343h/lab9.txt}
You can load this datafile by typing
\li{lab9 = sp.genfromtxt("lab9.txt")}
Now the data is available in the array \li{lab9}.
This consists of two columns corresponding to the $x$ and $y$ values of a given data set.  Use least squares to find the slope and $y$-intercept that best fits the data.  Then plot the data points and the line on the same graph.  Finish off the problem with a discussion of what you've learned.
\end{problem}


In Bayesian statistics, we are interested in performing inference on the posterior distribution of a parameter, $\theta$.  This is calculated using Bayes rule
$$
\pi(\theta | x) = \frac{f(x | \theta) p(\theta)}{f(x)}
$$
where $f(x|\theta)$ is the likelihood function of the data, and $p(\theta)$ is the prior distribution on the parameter.  The term on the denominator, $f(x)=\int f(x)|\theta) p(\theta) d\theta$, is a normalising constant, such that $\pi$ is a probability distribution.  In general, and particularly in high dimensions, this normalising constant is difficult to calculate analytically, and so the posterior is only known up to proportionality.
$$
\pi(\theta | x) \propto f(x | \theta) p(\theta)
$$
This means we cannot easily make inferential statements about the parameter $\theta$.  To solve this problem, we use Markov chain Monte Carlo methods to draw samples from the posterior distribution, and use these samples to make inferences on the parameter.

\subsection{Markov Chain Monte Carlo}
Monte Carlo methods are a class of algorithms that replace difficult or impossible analytical probability calculations with numerical approximations.  Given a random variable $X$ on a probability space $(\Omega,\mathcal{F}, \pi)$ with distribution given by $\pi$ consider the problem of calculating

$$
\E_\pi[g(X)] = \int_\Omega g(x) d \pi
$$
If the distribution $\pi$ does not take a well-known form, then analytically solving this expectation is very difficult.  Instead, Monte Carlo integration uses the strong law of large numbers to approximate the integral.  If $X_n$ are a sequence of i.i.d. random variables distributed according to $\pi$, then
$$
\frac{1}{N} \sum_{n=1}^N g(X_n) \to \E_\pi[g(X)] \quad \text{a.s. as $N \to \infty$.}
$$
Hence we can approximate integrals by taking independent samples from the distribution $\pi$.  However, for an arbitrary distribution, it is not necessarily possible to find samples independently in an efficient way.  Markov chain Monte Carlo allows us to dispense with this assumption.  Rather than sampling independently, we can instead construct a Markov chain with $\pi$ as its invariant distribution.  This chain can then be used to generate dependent samples, which can be used for Monte Carlo integration provided the chain is ergodic with respect to $\pi$.

\begin{defn}[Ergodicity]
Let $T:\Omega \to \Omega$ be a probability-preserving transformation on a probability space $(\Omega,\mathcal{F}, \P)$.  Then we say $T$ is \emph{ergodic} if for every $F \in \mathcal{F}$ with $T^{-1}(F)=F$, either $\P(F)=0$ or $\P(F)=1$.
\end{defn}

Intuitively, this condition means that the process explores the whole space, without becoming stuck in a subregion.  The ergodic theorem then provides the theoretical justification that permits the use of dependent samples for Monte Carlo integration.

\begin{theorem}[Ergodic Theorem]\label{thm:ergodic}
 Let $f$ be measurable, $\E_\pi(|f|) < \infty$, and $T$ be an ergodic probability-preserving transformation. Then with probability 1:
 $$
\frac{1}{N} \sum_{n=1}^N f(T^n(x)) \to \E_\pi[f] \quad \text{a.s. as $N \to \infty$.}
$$
\end{theorem}
\noindent In other words, if $T$ is ergodic, time averages converge to space averages.  Hence, if we can construct a Markov chain which is ergodic with respect to a target measure $\pi$, it can be used to generate Monte Carlo samples.  One method of constructing such a Markov chain is by using Langevin dynamics.


\subsection{\(\sigma \)-algebra}

\subsubsection{Review of algebra on a set}

An algebra, \(\Sigma \), on set \(s\) is a set of subsets of \(s\) such that:

\begin{itemize}
\item Closed under intersection: If \(a\) and \(b\) are in \(\Sigma \) then \(a\land b\) must also be in \(\Sigma \)
\item \(\forall ab [(a \in \Sigma \land b \in \Sigma )\rightarrow (a\land b \in \Sigma)]\)
\item Closed under union: If \(a\) and \(b\) are in \(\Sigma \) then \(a\lor b\) must also be in \(\Sigma \).
\item \(\forall ab [(a \in \Sigma \land b \in \Sigma )\rightarrow (a\lor b \in \Sigma)]\)
\end{itemize}

If both of these are true, then the following is also true:

\begin{itemize}
\item Closed under complement: If \(a\) is in \(\Sigma \) then \(s \backslash a\) must also be in \(\Sigma \)
\end{itemize}

We also require that the null set (and therefore the original set, null's complement) is part of the algebra.

\subsubsection{\(\sigma \)-algebra}

A \(\sigma \)-algebra is an algebra with an additional condition:

All countable unions of sets in \(A\) are also in \(A\).

This adds a constraint. Consider the real numbers with an algebra of all finite sets.

This contains all finite subsets, and their complements. It does not contain \(\mathbb{N}\).

However a \(\sigma \)-algebra requires all countable unions to be including, and the natural numbers are a countable union.

The power set is a \(\sigma \)-algebra. All other \(\sigma \)-algebras are subsets of the power set.


\textbf{Solve the boundary value problem $u_{xx}+4u_{x}+e^xu=\sin{(8x)}$ numerically on $[-1,1]$ with boundary conditions $u(\pm1)=0$. To 10 digits of accuracy, what is $u(0)$?}
\newline

We can express the BVP as
\begin{align*}
D^2u+4Du+\text{diag}(e^x)u &= f,\\
\left[D^2+4D+\text{diag}(e^x)\right]u &= f,\\
Mu &= f.
\end{align*}
where $D$ is the Chebyshev differentiation matrix and $f$ is the right hand side of the equation. Hence, we can calculate the solution to our BVP as
\begin{align*}
u &= M^{-1}f.
\end{align*}

The solution, using $N+1=51$ points (odd number of points to have a node exactly at $x=0$), is plotted in the figure 1. In the figure 2 we plot the value of $u(x=0)$ for different values of $N$. We see how with just $10+$ nodes we achieve high accuracy. The value of $u(x=0)$ obtained is
\begin{align*}
u(0) =  0.0095978572.
\end{align*}
The number of nodes needed to achieve 10 digits of accuracy was $N+1 = 23$.
\begin{figure}[H]
\centering
\includegraphics[scale=0.9]{P1_b.png}\caption{Solution to the BVP using Chebyshev matrices and 51 nodes.}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[scale=0.9]{P1_a.png}\caption{Value of $u(0)$ for different number of nodes.}
\end{figure}

\subsection*{Matlab code for this problem}

\begin{verbatim}
%% Homework 4, Problem 1 - Francisco Castillo
clear all; close all; clc;
labelfontsize = 14;

NN = 4:2:50;
for j=1:length(NN)
    N=NN(j);
    [D,x] = cheb(N); % D:(N+1)x(N+1), x:(N+1)x1
    D2 = D^2;
    D = D(2:N,2:N);
    D2 = D2(2:N,2:N);       % Zero Dirichlet BCs
    f = sin(8*x(2:N));
    E = diag(exp(x(2:N)));
    u = (D2+4*D+E)\f;
    u = [0;u;0];
    u0(j) = u(N/2+1);
end

figure
plot(NN,u0,'r*')
grid on
xlabel('$N$','fontsize',labelfontsize,'interpreter','latex')
ylabel('$u(0)$','fontsize',labelfontsize,'interpreter','latex')
set(get(gca,'ylabel'),'rotation',0)
txt = 'Latex/FIGURES/P1_a';
saveas(gcf,txt,'png')

xx = [x(end):.01:x(1)]';
uu = polyval(polyfit(x,u,N),xx);

figure
plot(x,u,'b*')
hold on
plot(xx,uu,'b')
grid on
xlabel('$x$','fontsize',labelfontsize,'interpreter','latex')
ylabel('$u(x)$','fontsize',labelfontsize,'interpreter','latex')
set(get(gca,'ylabel'),'rotation',0)
txt = 'Latex/FIGURES/P1_b';
saveas(gcf,txt,'png')
\end{verbatim}


\textbf{Derive Newtons iteration for this function. Show that the iteration is well-defined so long as $x_k\neq 1$ and that the convergence rate is expected to be similar to that of the bisection method (and certainly not quadratic).}

\vspace{0.3in}
Newton's method for finding successively better approximations to the roots of a real-valued function is the following,
\begin{align*}
x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)}.
\end{align*}
Given the function
\begin{align*}
f(x)=(x-1)^2e^x,
\end{align*}
and its derivative
\begin{align*}
f'(x)=(x^2-1)e^x,
\end{align*}
we can obtain the iteration for this problem
\begin{align*}
x_{n+1}=x_n-\frac{(x_n-1)^2e^{x_n}}{(x_n^2-1)e^{x_n}}=x_n-\frac{(x_n-1)}{(x_n+1)}.
\end{align*}
Note that this is not defined for $x_n=-1$. Clearly the root of the function is $x^*=1$. To find the rate of convergence let $\epsilon_n=x_n-x^*=x_n-1$, and $g(x_n)=\frac{f(x_n)}{f'(x_n)}$. Then, rewriting the first equation in this terms we get
\begin{align*}
\epsilon_{n+1}=\epsilon_n-g(x_n).
\end{align*}
Further,
\begin{align*}
g(x_n)=g(\epsilon_n+1)=g(1)+g'(1)\epsilon_n+\frac{\epsilon_n^2}{2}g''(\xi),
\end{align*}
by Taylor expansion. The values of $g$ and $g'$ are known,
\begin{align*}
g(1)=\frac{f(1)}{f'(1)}=0,~~~~~~g'(x)=\frac{2}{(x+1)^2}\Rightarrow g'(1)=\frac{1}{2}.
\end{align*}
Hence,
\begin{align*}
g(x_n)=\frac{1}{2}\epsilon_n+\frac{\epsilon_n^2}{2}g''(\xi),
\end{align*}
and
\begin{align*}
\lim_{\epsilon\to 0}\frac{\epsilon_{n+1}}{\epsilon_{n}}=\frac{1}{2}.
\end{align*}
Note that the convergence rate is linear and equal to the one of the bisection method. It differes from the quadratic convergence rate talked in class because this function and its derivative have the same root $x^*=1$. For the convergence rate to be quadratic, the root of the function should not be a root of the derivative.

\vspace{0.3in}

\textbf{Implement Newtons method and observe its performance starting from $x_0 = 2$.}

\vspace{0.3in}

After implementing the method in \textsl{Matlab} the root $x^*=1$ was found in 42 iterations, with a time of the order of $0.009~s$. In the following figure we can see the function given and the roots calculated by the Newton's method until convergence.

\begin{figure}[H]
\centering     %%% not \center
{\includegraphics[scale=0.75]{prob1roots.eps}}
\caption{Newton's method performance.}
\end{figure}

\vspace{0.3in}

\textbf{How easy would it be to apply the bisection method? Explain.}

\vspace{0.3in}

For this case the bisection method would not have worked since it relies on a change of sign at both sides of the root, and this is not the case with this function which is nonnegative for all values of $x$.

\subsection*{Matlab code for this problem}
\begin{verbatim}
%% Problem 1
xx = linspace(-3,3,1e4);
f = @(x) (x - 1).^2.*exp(x);
df = @(x) (x.^2 - 1).*exp(x);

tic
[x,niter,conv]=newton(f,df,2,1e-12,100)
time=toc
% To obtain the roots, modified newton function
[x,niter,conv]=newton_vector(f,df,2,1e-12,100);

figure
plot(xx,f(xx),'linewidth',1.5)
hold on
plot(x,0,'r*')
grid on
axis([-2 2.5 -1 7.5])
set(gca,'fontsize',axisfontsize)
xlabel('$x$','interpreter','latex','fontsize',labelfontsize)
ylabel('$y$','interpreter','latex','fontsize',labelfontsize)
saveas(gcf,'Latex/FIGURES/prob2roots','epsc')
\end{verbatim}


\vspace{-4mm}

\subsection{Projections}
\label{vectors:projections}

\vspace{-2mm}

Dot products are used for computing projections.
Assume you're given two vectors $\vec{u}$ and $\vec{n}$ and you want to find the component
of $\vec{u}$ that points in the $\vec{n}$ direction.
The following formula based on the dot product will give you the answer:
\[
 \Pi_{\vec{n}}( \vec{u} ) \equiv \frac{  \vec{u} \cdot \vec{n}  }{ \| \vec{n} \|^2 } \vec{n}.
\]

\vspace{-2mm}

\noindent
This is how to implement this formula in \texttt{SymPy}:
\small
\begin{verbatimtab}
>>> u = Matrix([4,5,6])
>>> n = Matrix([1,1,1])
>>> (u.dot(n) / n.norm()**2)*n
[5, 5, 5]                      # projection of v in the n dir
\end{verbatimtab}
\normalsize

\noindent
In the case where the direction vector $\hat{n}$ is of unit length $\|\hat{n}\| = 1$,
the projection formula simplifies to $\Pi_{\hat{n}}( \vec{u} ) \equiv (\vec{u}\cdot\hat{n})\hat{n}$.

Consider now the plane $P$ defined by $(1,1,1)\cdot[(x,y,z)-(0,0,0)]=0$.
A plane is a two dimensional subspace of $\mathbb{R}^3$.
We can decompose any vector $\vec{u} \in \mathbb{R}^3$ into two parts $\vec{u}=\vec{v} + \vec{w}$
such that $\vec{v}$ lies inside the plane and $\vec{w}$ is perpendicular to the plane (parallel to $\vec{n}=(1,1,1)$).

To obtain the perpendicular-to-$P$ component of $\vec{u}$,
compute the projection of $\vec{u}$ in the direction $\vec{n}$:
\small
\begin{verbatimtab}
>>> w = (u.dot(n) / n.norm()**2)*n
[5, 5, 5]
\end{verbatimtab}
\normalsize

\noindent
To obtain the in-the-plane-$P$ component of $\vec{u}$,
start with $\vec{u}$ and subtract the perpendicular-to-$P$ part:
\small
\begin{verbatimtab}
>>> v = u - (u.dot(n)/n.norm()**2)*n      # same as u - w
[ -1, 0, 1]
\end{verbatimtab}
\normalsize

\noindent
You should check on your own that $\vec{v}+\vec{w}=\vec{u}$ as claimed.

\vspace{-5mm}


\subsection{Cardinality}

\subsubsection{Cardinality of cartesian product}

What about the cardinality of Cartesian products? So if we have sets:

\(\{1,2,3\}\)

\(\{a,b\}\)

We can have the Cartesian product set:

\(\{(1,a),(2,a),(3,a),(1,b),(2,b),(3,b)\}\)

We can see that:

\(|A.B|=|A|.|B|\)
\subsubsection{Cardinality of union and intersection}

\(|A\lor B| = |A|+|B|-|A\land B|\)
\subsubsection{Cardinality of powerset}

\(|P(s)|=2^{|s|}\)

\subsubsection{Cardinality of complement}

\(|a \setminus b|=|a|-|a\land b|\)

\subsubsection{Cardinality of even/odd natural numbers}

What about the cardinality of even numbers? Well, we can define a bijective function between each:

\(f(n)=2n\)

Similarly for odd numbers:

\(f(n)=2n+1\)

So these both have cardinality \(\aleph_0\).


\subsection{Hyperbolic trigonometric functions}
\label{basics:hyperbolic_trigonometric_functions}

The hyperbolic sine and cosine in \texttt{SymPy} are denoted \texttt{sinh} and \texttt{cosh} respectively
and \texttt{SymPy} is smart enough to recognize them when simplifying expressions:

\small
\begin{verbatimtab}
>>> simplify( (exp(x)+exp(-x))/2 )
cosh(x)
>>> simplify( (exp(x)-exp(-x))/2 )
sinh(x)
\end{verbatimtab}
\normalsize
Recall that $x=\cosh(\mu)$ and $y=\sinh(\mu)$ are defined as $x$ and $y$ coordinates of a point on the
the hyperbola with equation $x^2 - y^2 = 1$ and therefore satisfy the identity $\cosh^2 x - \sinh^2 x =1$:

\small
\begin{verbatimtab}
>>> simplify( cosh(x)**2 - sinh(x)**2 )
1
\end{verbatimtab}
\normalsize


\section{Matrix examples}
\subsection{Geometric transformations}
\textbullet \textit{Scaling}: $y = Ax$ with $A = aI$ stretches a vector by the factor $|a|$ (or shrinks it when $|a| < 1$), and it flips the vector (reverses its direction) if $a < 0$\\
\textbullet \textit{Dilation}: $y = Dx$, where D is a diagonal matrix, $D = diag(d1,d2)$. Stretches the vector x by different factors along the two different axes. (Or shrinks, if $|d_i| < 1$, and flips, if $di < 0$.)\\
\textbullet \textit{Rotation Matrix (counter clockwise)}:
$
y =
\begin{bmatrix}
    cos \theta & -sin \theta \\
    sin \theta & cos \theta
\end{bmatrix}x
$\\
\textbullet \textit{Reflection}
Suppose that y is the vector obtained by reflecting x through the line that passes through the origin, inclined $\theta$ radians with respect to horizontal.\\
$
y =
\begin{bmatrix}
    cos (2\theta) & sin(2\theta) \\
    sin (2\theta) & -cos (2\theta)
\end{bmatrix}x
$\\
\textbullet \textit{Projection into a line}
Projection of point x onto a set is the point in the set that is closest to x.\\
$
y =
\begin{bmatrix}
    (1/2)(1 + cos (2\theta)) & (1/2)sin(2\theta) \\
    (1/2)sin (2\theta) & (1/2)(1 - cos (2\theta))
\end{bmatrix}x
$
\subsection{Selectors}
An $m \times n$ selector matrix A is one in which each row is a unit vector (transposed):
\[
\begin{bmatrix}
    e^T_{k_1}\\
    .\\
    .\\
    e^T_{k_m}\\
\end{bmatrix}
\]
When it multiplies a vector, it simply copies the $k_i$th entry of x into the $i$th entry of $y = Ax$:\\
$y = (x_{k_1},x_{k_2},...,x_{k_m})$\\
\textbf{r:s matrix slicing}\\
$A=[0_{m\times(r-1)} I_{m\times m} 0_{m \times (n-s)}]$\\
where $m=s-r+1$
\subsection{Incidence matrix}
\textbf{Directed graph}: A \textit{directed graph} consists of a set of \textit{vertices} (or nodes), labeled 1,...,n, and a set of \textit{directed edges} (or branches), labeled 1,...,m. \\
$
A_{ij} = \left\{
  \begin{array}{@{}ll@{}}
    1, & \text{edge j points to node i} \\
    -1, & \text{edge j points from node i}\\
    0, & \text{otherwise}
  \end{array}\right.
$\\
\subsection{Convolution}
The convolution of an n-vector a and an m-vector b is the (n + m - 1)-vector denoted c = a * b\\
$c_k = \sum_{i+j=k+1} a_ib_j, k=1,..,n+m-1$\\
\textbf{Properties of convolution}\\
\textbullet symmetric: $a * b = b * a$\\
\textbullet associative: $(a*b)*c = a*(b*c)$\\
\textbullet $a*b = 0$ implies that either $a = 0$ or $b = 0$\\
\textbullet A basic property is that for fixed a, the convolution a * b is a linear function of b; and for fixed b, it is a linear function of a, $a * b = T (b)a = T (a)b$ where where T(b) is the (n + m - 1) × n matrix with entries\\
$
T(b)_{ij} = \left\{
  \begin{array}{@{}ll@{}}
    b_{i-j+1}, & 1\leq i-j+1 \leq m\\
    0, & \text{otherwise}
  \end{array}\right.
$\\
\textbf{Complexity of convolution}\\
\textbullet $c= a * b$: 2mn flops\\
\textbullet $T(a)b or T(b)a$: 2mn flops\\
\textbullet Convolution could be calculated faster using \textit{fast Fourier transform (FFT)} : $5(m + n) log_2(m + n) flops$\\


\section{Method of Moments}

Let $X_1, \ldots , X_ n \stackrel{iid}{\sim } \mathbf{P}_{\theta ^*}$ associated with model $(\mathbb {E}, \{ \mathbf{P}_{\theta }\} _{\theta \in \Theta })$, with $\mathbb {E} \subseteq \mathbb {R}$ and $\Theta \subseteq \mathbb {R^d}$, for some d $\geq 1$

Population moments:\\

$m_k(\theta) = \mathbb{E}_{\theta}[X^k_1],  1 \leq k \leq d$\\

Empirical moments:\\

$\widehat{m_k}(\theta) = \bar{X_n^k} = \frac{1}{n} \sum _{i=1}^{n}X_i^k$

Convergence of empirical moments:\\

$\widehat{m_k} \xrightarrow[n \rightarrow \infty]{P, a.s.} m_k$\\

$(\widehat{m_1}, \ldots, \widehat{m_d}) \xrightarrow[n \rightarrow \infty]{P, a.s.} (m_1, \ldots, m_d)$

MOM Estimator $M$ is a map from the parameters of a model to the moments of its distribution. This map is invertible, (ie. it results into a system of equations that can be solved for the true parameter vector $\theta^{*}$). Find the moments (as many as parameters), set up system of equations, solve for parameters, use empirical moments to estimate.

$\displaystyle  \psi : \Theta \displaystyle \to \mathbb {R}^ d$\\

$\displaystyle \theta \displaystyle \mapsto (m_1(\theta ), m_2(\theta ), \ldots , m_ d(\theta ))$\\

$M ^{-1}(m_1(\theta ^*), m_2(\theta ^*), \ldots , m_ d(\theta ^*))$

The MOM estimator uses the empirical moments:\\

$M ^{-1}\left( \frac{1}{n} \sum _{i = 1}^ n X_ i, \frac{1}{n} \sum _{i = 1}^ n X_ i^2, \ldots , \frac{1}{n} \sum _{i = 1}^ n X_ i^ d \right)$\\

Assuming $M^{-1}$ is continuously differentiable at $M(0)$, the asymptotical variance of the MOM estimator is:\\\\

$\sqrt(n)(\widehat{\theta_n^{MM}} - \theta) \xrightarrow[n \rightarrow \infty]{(d)} N(0,\Gamma)$\\

where,

$\Gamma (\theta )=\left[\frac{\partial M^{-1}}{\partial \theta } (M(\theta ))\right]^{T} \Sigma (\theta )\left[\frac{\partial M^{-1}}{\partial \theta } (M(\theta ))\right]$

$\Gamma(\theta) = \nabla_{\theta}(M^{-1})^{T} \Sigma \nabla_{\theta}(M^{-1}) $

$\Sigma_{\theta}$ is the covariance matrix of the random vector of the moments $(X_1^1,X_1^2 \ldots, X_1^d)$.


\chapter{The Cartesian Product}
The \emph{Cartesian product} $\times$ is a binary operation on sets which produces a new set containing all ordered pairs where the first element comes from the left-hand set and the second element comes from the right-hand set. If $A = \{a,b,c,\dots\}$ and $\Gamma = \{\alpha,\beta,\gamma,\dots\}$, then $A \times \Gamma = \{(a,\alpha), (a,\beta), \dots, (b,\alpha), \dots\}$.
Specifically, the Cartesian product is defined as
\[ A \times B = \{(a,b) : a \in A, b \in B\}. \]
While we call it a binary operation, we can have an arbitrary number of sets upon which it can act; consider the generalization to $n$ sets
\[ A \times B \times C \times \cdots = \{(a,b,c,\dots) : a \in A, b \in B, c \in C, \dots\}, \]
which produces an $n$-tuple.
We use the Cartesian product to describe the domain and range of new binary functions that we create. If we imagine a new binary operation $\odot$ which acts upon sets $A$ and $B$ and maps to $C$, we could define it as
\[ \odot : A \times B \to C. \]
For the same of brevity, we often use an exponent to represent the number of times a Cartesian product has been taken over the same vector space, so
\[ A \times A \times A = A^3 \]
and
\[A  \times A \times B \times B \times B = A^2 \times B^3. \]


\section{Decision Tree Learning (20 points)}

In this problem, we want to construct a decision tree to find out if a person will enjoy beer.

\paragraph{Definitions.} Let there be $k$ binary-valued attributes in the data.

We pick an attribute that maximizes the gain at each node:
\begin{equation}\label{G}
%	G = \max [I(D) - (I(D_L) + I(D_R))];
	G = I(D) - (I(D_L) + I(D_R));
\end{equation}
where $D$ is the given dataset, and $D_L$ and $D_R$ are the sets on left and right hand-side branches after division. Ties may be broken arbitrarily.

There are three commonly used impurity measures used in binary decision trees: Entropy, Gini index, and Classification Error. In this problem, we use Gini index and define $I(D)$ as follows\footnote{As an example, if $D$ has 10 items, with 4 positive items (\emph{i.e.} 4 people who enjoy beer), and 6 negative items (\emph{i.e.} 6 who do not), we have $I(D) = 10 \times (1 - (0.16 + 0.36))$.}:
\[
	I(D) = |D| \times \left(1-\sum_i p_i^2\right),
\]
where:
\begin{itemize}
	\item $|D|$ is the number of items in $D$;
	\item $1 - \sum_i p_i^2$ is the gini index;
	\item $p_i$ is the probability distribution of the items in $D$, or in other words, $p_i$ is the fraction of items that take value $i\in\{+,-\}$. Put differently, $p_+$ is the fraction of positive items and $p_-$ is the fraction of negative items in $D$.
\end{itemize}

Note that this intuitively has the feel that the more evenly-distributed the numbers are, the lower the $\sum_i p_i^2$, and the larger the impurity.

\subquestion{(a) [10 Points]}
Let $k = 3$. We have three binary attributes that we could use: ``likes wine'', ``likes running'' and ``likes pizza''.  Suppose the following:
\begin{itemize}
	\item There are 100 people in sample set, 40 of whom like beer and 60 who don't.
	\item Out of the 100 people, 50 like wine; out of those 50 people who like wine, 20 like beer.
	\item Out of the 100 people, 30 like running; out of those 30 people who like running, 20 like beer.
	\item Out of the 100 people, 80 like pizza; out of those 80 people who like pizza, 30 like beer.
	\end{itemize}
\task{What are the values of $G$ (defined in Equation~\ref{G}) for wine, running and pizza attributes? Which attribute would you use to split the data at the root if you were to maximize the gain $G$ using the gini index metric defined above?}

\subquestion{(b) [10 Points]}
Let's consider the following example:
\begin{itemize}
\item There are $100$ attributes with binary values $a_1, a_2, a_3, \ldots, a_{100}$.
\item Let there be one example corresponding to each possible assignment of 0's and 1's to the values $a_1, a_2, a_3 \ldots, a_{100}$. (Note that this gives us $2^{100}$ training examples.)
\item Let the values taken by the target variable $y$ depend on the values of $a_1$ for $99 \%$ of the datapoints. More specifically, of all the datapoints where $a_1=1$, let $99 \%$ of them are labeled $+$. Similarly, of all the datapoints where $a_1 = 0$, let $99 \%$ of them are labeled with $-$.  (Assume that the values taken by $y$ depend on $a_2, a_3, \dots, a_{100}$ for fewer than $99\%$ of the datapoints.)
\item Assume that we build a complete binary decision tree (\emph{i.e.}, we use values of all attributes).
\end{itemize}

\task{Explain what the decision tree will look like. (A one line explanation will suffice.) Also, in 2-3 sentences, identify what the desired decision tree for this situation should look like to avoid overfitting, and why.(The desired decision tree isn't necessarily a complete binary decision tree)}

\subsection*{What to submit}
\begin{enumerate}[(i)]
	\item Values of $G$ for wine, running and pizza attributes. [part (a)]
	\item The attribute you would use for splitting the data at the root. [part (a)]
	\item Explain what the decision tree looks like in the described setting. Explain how a decision tree should look like to avoid overfitting. (1-2 lines each) [part (b)]
\end{enumerate}