\chapter{Frequency Selective Filters in CT}

Recall the response of stable CT LTI systems to periodic inputs. Given a stable LTI system with frequency response $H(j\omega)$
\[
x(t) = \sum\limits_{k = -\infty}^{\infty} a_k e^{jk\omega_0 t} \longrightarrow y(t) = \sum\limits_{k = -\infty}^{\infty} a_k H(jk\omega_0) e^{jk\omega_0 t}
\]

Note the output is equivalent to a signal with Fourier series coefficients $b_k = a_k H(jk\omega_0)$. That is the Fourier coefficients are scaled by the frequency response at the harmonic frequency $k\omega_0$.

Similarly for aperiodic signals, given a stable LTI system with frequency response $H(j\omega)$
\[
x(t) = \frac{1}{2\pi} \int\limits_{-\infty}^{\infty} X(j\omega) e^{j\omega t}\; d\omega \longrightarrow y(t) = \frac{1}{2\pi} \int\limits_{-\infty}^{\infty} X(j\omega) H(j\omega) e^{j\omega t}\; d\omega
\]

Note the output is equivalent to a signal with Fourier Transform $Y(j\omega) = X(j\omega) H(jk\omega_0)$. That is the Fourier transform at each continuous frequency $\omega$ is scaled by the frequency response at that frequency.

We can use this behavior to our advantage. In many applications we want to modify the values of $a_k$ or $X(j\omega)$ selectively, passing them unmodified, increasing (amplifying) them, or decreasing (attenuating) them. This is accomplished by designing a frequency response. Such systems are called frequency selective \emph{filters} and come in 4 basic types:
\begin{itemize}
\item Low-pass Filters attenuate high frequencies while passing through lower frequencies. They are often used to reduce the effects of high-frequency noise in a signal and to prepare it for sampling (so-called anti-aliasing filters). They are the most common filter.
\item High-pass Filters attenuate lower frequencies while passing through higher frequencies. While less common, they are often used to remove the DC component ($\omega$ = 0) of a signal and to compute the derivative of a signal.
\item Bandpass Filters attenuate frequencies outside a band of frequencies. They can be viewed as a combination of a high-pass and low-pass filter. They are commonly used to select a range of frequencies for further processing and are central to many communication technologies.
\item Notch or Bandstop Filters attenuate frequencies inside an often narrow band of frequencies. Common applications are the removal of one or more corrupting signals mixed into another signal.
\end{itemize}

While the design of such filters is outside the scope of this course, you are now equipped to understand and apply them based on your knowledge of the Fourier methods covered over the past several weeks.

\section{Ideal Filters}

The above filter types each have an ideal (although unrealizable) form.

Low-pass filters remove frequency content above a threshold, $\omega_c$, called the \emph{cutoff frequency}. They have an ideal frequency response
\[
H(j\omega) = \left\{ \begin{array}{lc}
  1 & -\omega_c < \omega < \omega_c\\
  0 & \text{else}
\end{array}
\right.
\]
with magnitude and phase plot
\begin{center}
  \includegraphics[scale=1]{graphics/lowpass-ideal.pdf}
\end{center}
The range of frequencies below $|\omega_c|$ are called the pass-band. The range of frequencies above $|\omega_c|$ are called the stop-band.

High-pass filters remove frequency content below the cutoff frequency $\omega_c$. They have an ideal frequency response
\[
H(j\omega) = \left\{ \begin{array}{lc}
  0 & -\omega_c < \omega < \omega_c\\
  1 & \text{else}
\end{array}
\right.
\]
with magnitude and phase plot
\begin{center}
  \includegraphics[scale=1]{graphics/highpass-ideal.pdf}
\end{center}
The range of frequencies above $|\omega_c|$ are called the pass-band. The range of frequencies below $|\omega_c|$ are called the stop-band.

Bandpass filters remove frequency content outside a band of frequencies called the pass-band. They have an ideal frequency response
\[
H(j\omega) = \left\{ \begin{array}{lc}
  1 & -\omega_0 - \frac{B}{2} < \omega < -\omega_0+\frac{B}{2}\\[1em]
  1 & \omega_0 -\frac{B}{2} < \omega < \omega_0+\frac{B}{2}\\[1em]
  0 & \text{else}
\end{array}
\right.
\]
where $\omega_0$ is the \emph{center frequency} and $B$ is the \emph{bandwidth}. The frequencies outside this range are in the stop-band. The magnitude and phase plot looks like
\begin{center}
  \includegraphics[scale=1]{graphics/bandpass-ideal.pdf}
\end{center}

Finally, notch or bandstop filters remove frequency content inside a band of frequencies (the stop band) defined by the center frequency $\omega_0$ and bandwidth $B$. The ideal frequency response is
\[
H(j\omega) = \left\{ \begin{array}{lc}
  0 & -\omega_0 - \frac{B}{2} < \omega < -\omega_0+\frac{B}{2}\\[1em]
  0 & \omega_0 -\frac{B}{2} < \omega < \omega_0+\frac{B}{2}\\[1em]
  1 & \text{else}
\end{array}
\right.
\]
with magnitude and phase plot
\begin{center}
  \includegraphics[scale=1]{graphics/bandstop-ideal.pdf}
\end{center}
Often the bandstop filter has a very narrow bandwidth, thus it "notches" out a frequency component of the input signal.

\section{Practical Filters}

Ideal CT filters cannot be implemented in practice because they are non-causal and thus physically impossible. To see why consider the impulse response of the ideal low-pass filter:

\[
h(t) = \mathcal{F}^{-1} \left\{ H(j\omega) \right\} = \frac{1}{2\pi} \int\limits_{\-\omega_c}^{\omega_c} e^{j\omega t} \; d\omega = \frac{1}{\pi t}\sin(\omega_c t)
\]
which has nonzero values for $t < 0$, and thus corresponds to a non-casual system. Ideal filters also have zero phase which cannot be achieved in practice.

Practical filters are described by a frequency response that is a ratio of two polynomials in $j\omega$, i.e.
\[
H(j\omega) = \frac{K \cdot(j\omega - b_1)\cdot(j\omega - b_2)\cdots (j\omega - b_M)}{(j\omega - a_1)\cdot(j\omega - a_2)\cdots (j\omega - a_N)}
\]
where $K$ is a constant that controls the gain at DC, and the zero or more complex coefficients $b_k$ and the one or more complex coefficients $a_k$ are called the \emph{zeros} and \emph{poles} of the filter respectively. Such systems correspond to differential equations as we have covered before and are physically realizable as circuits if all poles and zeros are real or come in conjugate pairs. The processes of designing filters consists of choosing the poles and zeros, or equivalently choosing the coefficients of the numerator and dominator polynomials. This is covered in ECE 3704, ECE 4624, and other upper-level courses.

Practical filters differ from ideal filters in that they cannot be zero over any finite range of frequencies and cannot transition discontinuously between stop and pass bands. Instead they must vary over the bands and transition smoothly, with a degree of variation and sharpness that is a function of the order of the filter and the exact form of the frequency response polynomials. Thus practical filters are described by additional parameters that define the stop and pass-bands.

The overall gain of the filter is the magnitude of the frequency response at a frequency that depends on the filter type, zero for a low-pass filter and the center frequency for a band-pass filter. The pass-band is defined by the frequency at which the magnitude of the frequency response drops below the overall gain, often -3dB = $\frac{\sqrt{2}}{2}$. The stop-band is defined similarly, as the frequency at which the magnitude of the frequency response drops further below the overall gain, often -20dB = 0.1 or -40dB = 0.01. The \emph{transition bandwidth} is defined as the difference in the stop-band and pass-band frequencies. The \emph{pass-band ripple} is defined as the maximum deviation from the overall gain, over the pass-band.

\section{First-order and second-order systems as filters}

Given the equivalence of LTI systems and linear, constant-coefficient differential equations, block diagrams, impulse responses, and frequency responses, filters can be represented in any of these ways.

We have covered extensively first-order and second-order CT systems and seen how they can be represented variously as circuits, differential equations, block diagrams, and as frequency responses. We now see how they can describe simple filters and serve as building blocks for higher-order filters.

\begin{example} Consider a low-pass filter with the desired characteristics of having a pass-band of -3dB at 1kHz, and a stop-band of -20dB at 10kHz. Suppose this is implemented as a first-order "Butterworth" filter, which can be realized by an RC circuit.
\begin{center}
  \begin{circuitikz}[american voltages,scale=0.8, every node/.style={transform shape}]
    \draw
    (5,2.5) node[op amp, yscale=-1] (opamp1) {}
    (0,3) to[R,l=$R$,o-] (opamp1.+)
    (3,0) to[C, l=$C$] (3,3)
    (0,0) to[short,o-o] (7,0)
    (0,3) to[open, v=$x(t)$] (0,0)
    (7.5,2.5) to[open, v=$y(t)$] (7.5,0)
    (opamp1.-) |- (5,1) -| (opamp1.out)
    (opamp1.out) to[short,-o] (7,2.5);
  \end{circuitikz}
\end{center}

  where $R=99.2k\Omega$ and $C=1.6$nF. This is equivalent to the differential equation
  \[
  \frac{dy}{dt}(t) + a y(t) = a x(t)
  \]
  where $a=\frac{1}{RC}$, or the block diagram
  \begin{center}
  \tikzstyle{block} = [draw, fill=gray!20, rectangle,
    minimum height=2em, minimum width=2em]
  \tikzstyle{sum} = [draw, fill=gray!20, circle, node distance=1cm]
  \tikzstyle{input} = [coordinate]
  \tikzstyle{output} = [coordinate]
  \tikzstyle{pinstyle} = [pin edge={to-,thin,black}]

  \begin{tikzpicture}[auto]
    \node [input, name=input] at (0,0) {};
    \node[block] at (4,-1) (block2) {$\int$};

    \node [shape=coordinate, name=conn] at (4,0) {};
    \node [shape=coordinate, name=conn2] at (2,-2) {};
    \node [shape=coordinate, name=conn3] at (4,-2) {};
    \node [sum, right of=input,node distance=2cm] (sum) {$\Sigma$};
    \node [output, right of=conn3] (output) {};

    \draw (sum) -- node {$a$} (conn);
    \draw (conn) -- (block2);
    \draw (block2) -- (conn3);
    \draw (conn3) -- (conn2);
    \draw [->] (conn2) -| node[pos=0.95] {$-$} (sum);
    \draw [draw,->] (input) -- node {$x(t)$} (sum);
    \draw [->] (conn3) -- node[pos=1] {$y(t)$} (output);
  \end{tikzpicture}
  \end{center}

  This system has the impulse response
  \[
  h(t) = ae^{-at}u(t)
  \]
  and the frequency response
  \[
  H(j\omega) = \frac{a}{j\omega + a}
  \]
  If we plot the frequency response as a Bode plot, we see the DC gain is 0dB, and the response passes through -3dB and -20dB at the expected frequencies $2\pi*1000 \approx 6.3\times 10^3$ rad/s and $2\pi*10000 \approx 6.3\times 10^4$ rad/s. Thus the transition bandwidth is 9kHz.
  \begin{center}
\includegraphics[scale=0.7]{graphics/lecture24_1.png}
  \end{center}

\end{example}

\begin{example} Suppose we wish to sharpen the transition band for the previous example so that has a pass-band of -3dB at 1kHz, and a narrower stop-band of -20dB at 5kHz. This requires a second-order filter, and can be realized by a circuit called the Sallen-Key.

  \begin{center}
    \begin{circuitikz}[american voltages,scale=0.8, every node/.style={transform shape}]
      \draw
      (7,3.5) node[op amp] (opamp1) {}
      (0,4) to[R,l=$R_1$,o-] (2,4)
      (2,4) to[short] (2,5)
      (2,5) to[C,l=$C_1$] (8.2,5)
      (8.2,5) to[short] (opamp1.out)
      (2,4) to[R, l=$R_2$] (4,4)
      (4,4) to[C, l=$C_2$] (4,0)
      (0,0) to[short,o-o] (12,0)
      (4,4) to[short] (opamp1.-)
      (opamp1.+) to[short] (5.8,1.75)
      (5.8,1.75) to[short] (8.2,1.75)
      (opamp1.out) to[short] (8.2,1.75)
      (opamp1.out) to[short, -o] (12,3.5)
      (0,4) to[open, v=$x(t)$] (0,0)
      (12,3.5) to[open, v=$y(t)$] (12,0);
    \end{circuitikz}
  \end{center}
  where $R_1=74.2k\Omega$, $R_2=91.3M\Omega$, $C_1=1.6$nF and $C_2=160$pF. This is equivalent to the differential equation
  \[
  \frac{d^2y}{dt^2}(t) + 2\alpha \frac{dy}{dt}(t) + \omega_0^2 y(t) = \omega_0^2 x(t)
  \]
  where
  \[
  \alpha = \frac{R_1+R_2}{2R_1 R_2 C_1} \;\text{ and }\; \omega_0^2 = \frac{1}{R_1 R_2 C_1 C_2}
  \]
  or the block diagram
  \begin{center}
    \tikzstyle{block} = [draw, fill=gray!20, rectangle,
      minimum height=2em, minimum width=2em]
    \tikzstyle{sum} = [draw, fill=gray!20, circle, node distance=1cm]
    \tikzstyle{input} = [coordinate]
    \tikzstyle{output} = [coordinate]
    \tikzstyle{pinstyle} = [pin edge={to-,thin,black}]

    \begin{tikzpicture}[auto]
      \node [input, name=input] at (0,0) {};
      \node [sum, right of=input,node distance=2cm] (sum) {$\Sigma$};
      \node [sum, below of=sum,node distance=2cm] (sum2) {$\Sigma$};
      \node[block] at (4,-1) (block3) {$\int$};
      \node[block] at (4,-3) (block4) {$\int$};

      \node [shape=coordinate, name=conn1] at (4,0) {};
      \node [shape=coordinate, name=conn2] at (4,-2) {};
      \node [shape=coordinate, name=conn3] at (4,-4) {};
      \node [shape=coordinate, name=conn4] at (2,-4) {};
      \node [output, right of=conn3] (output) {};

      \draw [->] (input) -- node {$\omega_0^2$} (sum);
      \draw (sum) -- (conn1);
      \draw [->] (conn1) -- (block3);
      \draw (block3) -- (conn2);
      \draw [->] (conn2) -- (block4);
      \draw [->] (conn2) -- node {$2\alpha$} (sum2);
      \draw (block4) -- (conn3);
      \draw (conn3) -- node {$\omega_0^2$} (conn4);
      \draw [->] (conn3) -| (sum2);
      \draw [->] (sum2) -- node[pos=0.95] {$-$} (sum);
      \draw [->] (conn3) -- node[pos=1] {$y(t)$} (output);
      \node at (-0.5,0) {$x(t)$};
    \end{tikzpicture}
  \end{center}

  This system has the frequency response
    \[
  H(j\omega) = \frac{\omega_0^2}{\omega_0^2-\omega^2 + j2\alpha\omega}
  \]

  If we plot the frequency response as a Bode plot using the resistor and capacitor values above, we see the DC gain is 0dB, and the response passes through -3dB at the expected frequency $2\pi*1000 \approx 6.3\times 10^3$ rad/s. At the frequency $2\pi*5000 \approx 3.14\times 10^4$ rad/s the response passes through about -28dB. Thus this circuit has a transition bandwidth even narrower than that designed (it is slightly better).
  \begin{center}
    \includegraphics[scale=0.7]{graphics/lecture24_2.png}
  \end{center}
  Note the price we pay for this decreased transition bandwidth is a larger phase shift (and a two more components).
\end{example}

\section{Higher-Order Filters}

We can continue to increase the steepness of the passband to stop-band transitions by increasing the order of the filter. This is typically accomplished using a serial connection of systems, called \emph{stages} in filter parlance, where each stage is a first-order or second-order system.

Recall in a series connection of systems the overall impulse response is the convolution of the individual responses. If  we assume each stage is stable then, by the convolution property, the overall frequency response is given by the product of their individual frequency responses.
\begin{center}
\begin{tikzpicture}[auto, node distance=2cm,>=latex',scale=1, every node/.style={transform shape}]
    % We start by placing the blocks
    \node [input, name=input] {};
    \node [block, right of=input] (system1) {$H_1(j\omega)$};
    \node [block, right of=system1,node distance=4cm] (system2) {$H_2(j\omega)$};
    \node [output, right of=system2] (output) {};
    \node [right of=output,node distance=1cm] {$Y(j\omega)$};
    \node [left of=input,node distance=1cm] {$X(j\omega)$};
    % Once the nodes are placed, connecting them is easy.
    \draw [->] (input) -- (system1);
    \draw [->] (system1) -- (system2);
    \draw [->] (system2) -- (output);
\end{tikzpicture}
\end{center}

\[
H(j\omega) = \frac{Y(j\omega)}{X(j\omega)} = H_1(j\omega)\cdot H_2(j\omega)
\]

Writing each response in polar form
\[
H_1(j\omega)\cdot H_2(j\omega) = |H_1(j\omega)|\cdot |H_2(j\omega)| e^{\angle H_1(j\omega)+ \angle H_2(j\omega)}
\]
we note that the magnitudes multiply and the phases add. That means we can use additional stages to reinforce the attenuation of previous stages. Note this requires in the circuit that the stages be impedance isolated, thus the use of the opamps at the end of CT filters. Again the price we pay for increasing the order of the filter and decreasing the transition frequency is increased phase shift in the signal.

\newpage
Matlab code for plotting the first-order example Bode plot:
\begin{verbatim}
R = 99.2e3;
C = 1.6e-9;
a = 1/(R*C);

H = tf([a],[1,a]);
[mag,ph,w] = bode(H);

% Create a nice bode plot
hFig = figure();
hold on;

subplot(2,1,1);
hm = semilogx(w,20*log10(squeeze(mag)));
grid on;
hTitle  = title ('Frequency Response - first order');
hYLabel1 = ylabel('Magnitude (dB)');
set(gca, 'FontSize', 14, 'YTick', -60:10:20, ...
    'Box', 'off', 'LineWidth', 2);

subplot(2,1,2);
hp = semilogx(w,squeeze(ph*(pi/180)));
grid on;
hYLabel2 = ylabel('Phase (radians)');
hXLabel = xlabel('Frequency (rad/s)');
set(gca, 'FontSize', 14, 'Box', 'off', 'LineWidth', 2);

set(hm, 'linewidth', 2);
set(hp, 'linewidth', 2);
set([hXLabel, hYLabel1, hYLabel2]  , ...
     'FontSize'   , 14          );
set( hTitle                    , ...
     'FontSize'   , 14          , ...
     'FontWeight' , 'bold'      );
\end{verbatim}

\newpage
Matlab code for plotting the second-order example Bode plot:
\begin{verbatim}
R1 = 74.2e3;
R2 = 1.33e6;
C1 = 1.6e-9;
C2 = 160e-12;

a = (R1+R2)/(R1*R2*C1);
b = 1/(R1*R2*C1*C2);

H = tf([b],[1,a,b]);
[mag,ph,w] = bode(H);

% Create a nice bode plot
hFig = figure();
hold on;

subplot(2,1,1);
hm = semilogx(w,20*log10(squeeze(mag)));
grid on;
hTitle  = title ('Frequency Response - second order');
hYLabel1 = ylabel('Magnitude (dB)');
set(gca, 'FontSize', 14, 'YTick', -90:10:20, ...
    'Box', 'off', 'LineWidth', 2);

subplot(2,1,2);
hp = semilogx(w,squeeze(ph*(pi/180)));
grid on;
hYLabel2 = ylabel('Phase (radians)');
hXLabel = xlabel('Frequency (rad/s)');
set(gca, 'FontSize', 14, 'Box', 'off', 'LineWidth', 2);

set(hm, 'linewidth', 2);
set(hp, 'linewidth', 2);

set([hXLabel, hYLabel1, hYLabel2]  , ...
     'FontSize'   , 14          );
set( hTitle                    , ...
     'FontSize'   , 14          , ...
     'FontWeight' , 'bold'      );
\end{verbatim}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS624: Analysis of Algorithms
% Copyright 2015 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/beacon
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 7}

Suppose $T$ is a rooted tree with root $r$ and $x \neq r$ is a vertex in $T$.
Further, suppose $a$ and $b$ are both ancestors of $x$.
And to make things simple, suppose that $a \neq b$.
Prove that either $a$ is an ancestor of $b$ or $b$ is an ancestor of $a$.

\subsection*{Solution}

Proof is given by contradiction.
Assume distinct nodes $a$ and $b$ have no ancestorhood relationship.
Since they are ancestors of $x$ however, there exists a path of the form $r \rightarrow \cdots \rightarrow a \rightarrow \cdots \rightarrow x$ that does not pass node $b$.
Similarly, there exists a path of the form $r \rightarrow \cdots \rightarrow b \rightarrow \cdots \rightarrow x$ that does not pass from node $a$.
This leads to a simple loop of the form $r \rightarrow \cdots \rightarrow b \rightarrow \cdots \rightarrow x \rightarrow \cdots \rightarrow a \rightarrow \cdots \rightarrow r$ which contradicts definition of the tree.
Therefore initial assumption does not hold true and any path of the form $r \rightarrow \cdots \rightarrow a \rightarrow \cdots \rightarrow x$ would pass node $b$ at some point.
Hence either $a$ is an ancestor of $b$ or $b$ is an ancestor of $a$ and the proof is complete.


\lab{Statistical Distributions}{Statistical Distributions}
\label{lab:distributions}
\objective{This section will review the most common distributions and thus assumes prior exposure to basic terms and concepts.}

\section*{Discrete Distributions}
\emph{Discrete distributions} are distributions whose support is finite or countable at most.
The \emph{support} of a distribution refers to the points in $\mathbb{R}$ that have non-zero probability.
In other words, discrete distributions are a collection of point masses.

\begin{table*}[h]
\centering
\begin{tabular}{|c|c|c|c|}
	\hline
 & Uniform &  Binomial & Poisson \\
\hline
\hline
Support&$x \in \{a,\dots,b\}$&$x \in \{0,\dots,n\}$&$x \in \{0,1,2,\dots\}$\\ \hline
PMF&$\frac{1}{b-a+1}$&$\dbinom{n}{x}p^x(1-p)^{n-x}$&$\frac{e^\lambda \lambda^x}{x!}$\\ \hline
Parameters&$b>a$, $a,b \in \mathbb{Z}$&$n \in \mathbb{N}$, $p \in [0,1]$& $\lambda>0$\\ \hline
Mean $E(X)$&$\frac{a+b}{2}$&$np$& $\lambda$\\ \hline
Variance&&$np(1-p)$& $\lambda$\\ \hline
Description&
\parbox{1in}{models a single occurrence of one of several events of equal probability} & \parbox{1in}{models $n$ events with binary outcomes, where $p$ is the probability of a success ($x=1$). Note, when $n=1$, this is referred to as the Bernoulli distribution, or a Bernoulli trial.}&\parbox{1in}{models the number of events that occur in a given time period, given rate $\lambda$}\\
\hline
\end{tabular}
\end{table*}

\section*{Continuous Distributions}
\emph{Continuous distributions} are distributions whose support is uncountable, i.e. some subset of $\mathbb{R}$.
\begin{table*}[h]
\centering
\begin{tabular}{|c|c|c|c|}
	\hline
 & Normal& Gamma&Beta\\ \hline \hline
Support &$x \in \mathbb{R}$ & $x>0$ &$x \in [0,1]$\\  \hline
PDF & $\frac{1}{\sqrt{2\pi \sigma^2}} e^{-\frac{1}{2\sigma^2}(x-\mu)^2}$ & $\frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha -1} e^{-\beta x}$ &$\frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)} x^{\alpha-1} (1-x)^{\beta-1}$\\  \hline
Parameters & $\mu \in \mathbb{R}$, $\sigma^2 > 0$ & $\alpha,\beta>0$ &$\alpha,\beta>0$\\  \hline
Mean &  $\mu$& $\frac{\alpha}{\beta}$ &$\frac{\alpha}{\alpha + \beta}$\\  \hline
Variance &$\sigma^2$& $\frac{\alpha}{\beta^2}$&$\frac{\alpha \beta}{(\alpha + \beta)^2(\alpha + \beta +1)}$\\ \hline
Description&The bell curve&\parbox{1in}{When $\alpha=1$, this is called the exponential distribution. $\alpha$ is called the shape, $\beta$ is called the rate.}&\parbox{1in}{models probabilities or proportions. When $\alpha = \beta =1$, this is called the continuous uniform distribution}\\
\hline
\end{tabular}
\end{table*}

\section*{Normal Distribution}
The SAT is a standardized test common for college admissions in the United States.
It is comprised of three sections, critical reading, writing and mathematics.
Each section is graded out of 800 possible points, with a minimum score of 200.
Each section is intended to have a mean score of 500 with a standard deviation of 100.
The distribution of SAT section scores is well approximated by the normal distribution
(though these scores are discrete and to the nearest 10, i.e. $\mbox{score} \in \{200,210,220,\dots,790,800\}$, we are only approximating it).
\begin{figure}[h]
\includegraphics[width=\textwidth]{normaldist.pdf}
\end{figure}

The scores of the three sections are added together to produce the test score.
Thus the test in aggregate has 2400 possible points and a minimum score of 600.
As each of the three sections has a mean score of 500 and a standard deviation of 100, it seems logical that the test score would also has a normal distribution,
but with mean of 1500 points and a standard deviation of 300 points.
But is that true? One method of evaluating this is to simulate the test scores and then find the mean and standard deviation of our simulated data.
NumPy provides a tool to generate random data from the normal distribution, numpy.random.normal.

\begin{problem}
Generate 10,000 random test scores by adding together three vectors of 10,000 random draws from the $N(\mu=500,\sigma=100)$ distribution.
Find the mean and standard deviation of the 10,000 test scores.
Are they close to 1500 and 300, respectively?
Also, make a histogram of the test scores to visually inspect for whether or not the test scores are also normally distributed.
\label{prob:randomTests}
\end{problem}

\begin{figure}[h]
\includegraphics[width=\textwidth]{scorehist.pdf}
\caption{Histogram of the simulated test scores.}
\end{figure}
In the plot above, we see that the the 10,000 randomly generated test scores appear to be fairly bell-shaped, i.e. normally distributed.
The mean of these test scores should, in fact, be about 1500.
However, the standard deviation is not $3(100) = 300$, but around 173.2, or $\sqrt{3*100^2}=\sqrt{3}(100) \approx 173.205081$.
This is because, for independent normal random variables $X_1 \sim N(\mu_1,\sigma^2_1), X_2 \sim N(\mu_2,\sigma^2_2),X_3 \sim N(\mu_3,\sigma^2_3),$
\[ (X_1 +X_2 +X_3) \sim N(\mu_1+\mu_2+\mu_3, \sigma^2_1+\sigma^2_2+\sigma^2_3). \]
Said succinctly, the mean of the sum is the sum of the means, and the variance of the sum is the sum of the variances.
But as the standard deviation is the square root of the variance, this same idea doesn't work.
Yet we do see that the standard deviation of the sum is the square root of the sum of the variances (the new variance):
\[ \sqrt{100^2+100^2+100^2} = \sqrt{30000} \approx 173.205081 \]

\section*{Multivariate Normal Distribution}
Note that in problem \ref{prob:randomTests}, we assumed that the scores of all three tests were independent.
This should seem like an unreasonable assumption, because we expect there to be at least a little correlation in the test scores,
i.e. a student with a high reading score is more likely to score high on the other sections compared to a student with a low reading score.
To employ correlations in these draws, we turn to the multivariate normal distribution.
\begin{table*}[h]
\centering
\begin{tabular}{|c|c|}
\hline
&Multivariate Normal distribution  \\ \hline \hline
Support &$\mathbf{x} \in \mathbb{R}^k$ \\ \hline
PDF & $(2\pi)^{-k/2}|\Sigma|^{-1/2}e^{-\frac{1}{2}(\mathbf{x} - \mathbf\mathbf{\mu})^T \Sigma^{-1} (\mathbf{x}-\mathbf{\mu})}$ \\ \hline
 Parameters&$\mathbf{\mu}\in \mathbb{R}^k, \Sigma \in \mathbb{R}^{k\times k}$ \\ \hline
\end{tabular}
\end{table*}
Note that there is no mean and variance included above.
Similar to the normal distribution, they are simply $\mu$ and $\Sigma$ respectively. Also note that for $k=1$, this is the normal distribution.
Furthermore, this distribution can be viewed as having $k$ components, each of which is normally distributed.
Thus, $\mathbf{\mu}$ is a vector of the means of each of these normal distributions.
Similarly, the diagonal of the matrix $\Sigma$ are the variances of the $k$ components.
The off-diagonal elements of $\Sigma$ are covariances, or in other words, $\Sigma_{i,j} = Cov(x_i,x_j) = \sigma_i \sigma_j \rho_{ij}$, where $\rho_{ij}$ is the correlation between $x_i$ and $x_j$.
Hence, for $i=j$, $\Sigma_{i,j} = \sigma^2_i$ as the correlation between an object and itself is 1.

Returning to our SAT example, we can use the multivariate normal distribution with $k=3$.
Because each distribution of section scores has a mean of 500 and a variance of 10,000, $\mathbf{\mu} = (500,500,500)$ and $\Sigma_{1,1} = \Sigma_{2,2} = \Sigma_{3,3} = 10,000$.
Now let's assume the correlation between the reading and writing sections is $.7$, and the others are $.5$.
This means for $\mathbf{x} = (x_{\text{math}},x_{\text{read}},x_{\text{write}})$
\[ \Sigma =  \begin{bmatrix*}
10000 & 5000 & 5000 \\
5000 & 10000 & 7000 \\
5000 & 7000 & 10000 \end{bmatrix*} \]

\begin{problem}
Using the above $\mathbf{\mu}$ and $\Sigma$, generate 10,000 random test scores by adding together the vectors created by 10,000 random draws from the
\li{numpy.random.multivariate_normal} function.
Find the mean and standard deviation of the 10,000 test scores and make a histogram of the test scores to visually
inspect for whether or not the test scores are also normally distributed.
Which of the models discussed appears to be more appropriate?
\end{problem}

Note that the mean of the generated scores should be around 1500 and the standard deviation $\sqrt{64000} \approx 253$.
Note that 64,000 is the sum of all elements in $\Sigma$, however the explanation is beyond the scope of this lab.
\begin{figure}[h]
\includegraphics[width=\textwidth]{scorehist-mvn.pdf}
\caption{Histogram of the simulated test scores, generated via the multivariate normal distribution.}
\end{figure}

\section*{Central Limit Theorem}
The Central Limit Theorem, simply stated, says that for independently and identically distributed random variables, $X_i$ , $i = 1, \dots, n$, as $n \rightarrow \infty$,
\[
\frac{1}{n} \sum^n_{i=1} X_i = \bar{x}_n \sim N\left(E(X), \frac{Var(X)}{n}\right)
\]
In other words, for a large enough sample size, the sample mean is approximately normally distributed.
\begin{figure}[h]
\includegraphics[width=\textwidth]{multiplot.pdf}
\caption{A histogram of the means of 10,000 samples of size 10, 20, and 30 respectively, from the Exponential(1) distribution (same as the Gamma(1,1) distribution). Notice that as $n$ increases, it leaves the shape of the original distribution (exponential) and becomes more and more bell-curve-shaped.}
\label{fig:CLT}
\end{figure}

\begin{problem}
Create plots similar to \ref{fig:CLT} for the Poisson(3), Beta(.1,.1), Uniform(0,10), Normal($\mu=0$, $\sigma^2=100$), Gamma($\alpha = 2$, $\beta = 3$), and the Binomial($n=20$, $p=.8$). Notice how the discrete distributions appear ``more continuous'' as $n$ grows, due to more values being possible.
\end{problem}


\section*{Problem 3 Solution}

We need to find the minimum required volume of UO$_2$, $V$, so let's begin by expressing that volume of UO$_2$ in terms of the number of UO$_2$ molecules it contains.

First, we use $M_{\text{UO}_2}$ as the total mass of UO$_2$ and $\rho_{\text{UO}_2}$ as the density of UO$_2$ (about 10.41 g/cm$^3$ for our purposes)
$$ V = \frac{M_{\text{UO}_2}}{\rho_{\text{UO}_2}} $$
Then, we can write the mass of UO$_2$ as the number of UO$_2$ molecules, $N_{\text{UO}_2}$, times the mass of a UO$_2$ molecule, $m_{\text{UO}_2}$. Since we know that the UO$_2$ is enriched such that 3\% of the molecules contain $^{235}$U, we can say $m_{\text{UO}_2} = 0.03m_{\text{U5}} + 0.97m_{\text{U8}} + 2m_{\text{O}}$. Then
$$ M_{\text{UO}_2} = N_{\text{UO}_2}(0.03m_{\text{U5}} + 0.97m_{\text{U8}} + 2m_{\text{O}}) $$
We can use this equality in our total volume equation to find
\begin{equation}
\label{vol}
V = \frac{N_{\text{UO}_2}(0.03m_{\text{U5}} + 0.97m_{\text{U8}} + 2 m_{\text{O}})}{\rho_{\text{UO}_2}}
\end{equation}
Now we need to relate this volume, in terms of number of UO$_2$ molecules, to the electric power produced by the reactor. Say $P_E$ is the electric power, $P_T$ is thermal power, and $\varepsilon$ is the efficiency.
$$ P_E = \varepsilon P_T $$
The total energy produced by the reactor in time $t$ can be found by simply multiplying the power by the time of production.
$$ E = P_T t $$
or equivalently
$$ P_T = \frac{E}{t} .$$
The energy from one fission is $E_f$ and the energy harnessed from one fission is $0.95E_f$. The total energy harnessed can then be expressed in terms of the number of fissions, $N_f$ and the energy captured per event,
$$ E = 0.95E_f N_f $$
If we consider the extreme and highly unrealistic case that \textit{all} $^{235}$U atoms fission, then the number of $^{235}$U atoms required is $N_{\text{U5}} = N_f$. Since $^{235}$U is 3\% of the total uranium by atom, then $N_{\text{U5}} = 0.03N_{\text{U}}$. Taking this one step further, there is a one-to-one ratio of uranium atoms to UO$_2$ molecules, so $N_{\text{U}} = N_{\text{UO}_2}$.

We combine these facts together to relate the electric power produced to the number of uranium dioxide molecules required.
\begin{align*}
P_E	&= \varepsilon P_T \\
	&= \varepsilon \frac{E}{t} \\
	&= \frac{0.95\varepsilon E_f N_f}{t} \\
	&= \frac{(0.95)(0.03)\varepsilon E_f N_{\text{UO}_2}}{t} \\
\end{align*}
We solve for $N_{\text{UO}_2}$ to get
\begin{equation}
\label{num}
N_{\text{UO}_2} = \frac{P_E t}{(0.95)(0.03)\varepsilon E_f}
\end{equation}

Now, we use equation (\ref{num}) for the number of UO$_2$ molecules required for the given power in equation (\ref{vol}) for the volume of UO$_2$ required. We find
$$ V = \frac{P_E t(0.03m_{\text{U5}} + 0.97m_{\text{U8}} + 2 m_{\text{O}})}{(0.95)(0.03)\varepsilon E_f\rho_{\text{UO}_2}} $$
Finally, we can include our values. We note the following
\begin{itemize}
\item $P_E = 1000\text{ MWE} = 10^9\text{ J/s}$
\item $t = 1\text{ yr} = 31,557,600\text{ s}$
\item $E_f = 200\text{ MeV} = 3.204 \times 10^{-11}$ J
\item $m_{\text{U5}} = 235.044\text{ amu} = 3.9028 \times 10^{-22}$ g
\item $m_{\text{U8}} = 238.051\text{ amu} = 3.9528 \times 10^{-22}$ g
\item $m_{\text{O}} = 15.995\text{ amu} = 2.656 \times 10^{-23}$ g
\end{itemize}
$$ V = \frac{(10^9\text{ J/s})(31557600\text{ s})(0.03(3.9028 \times 10^{-22}\text{ g}) + 0.97(3.9528 \times 10^{-22}\text{ g}) + 2(2.656 \times 10^{-23}\text{ g}))}{(0.95)(0.03)(0.33)(3.204 \times 10^{-11}\text{ J})(10.41\text{ g/cm}^3)} $$
$$\boxed{ V = 4.51\text{ m}^3 }$$


%!TEX root = /home/renaud/Documents/EPL/tfe/latex/tfe.tex
\section{Discrete-time stability as an autocovariance}
The stability criterion is based on the two-way relationship between graphs and Markov chains: On one hand, any graph has an associated Markov chain where the states are the nodes of the graph and the transitions probabilities between states are given by the weights of the edges. On the other hand, any Markov chain can be represented by a graph whose edges are weighted according to the transition probabilities. Concretely, consider a graph of $n$ nodes whose $n \times n$ weighted adjacency matrix is denoted $\b A$. Let $\b q = \b A \b 1$; $q_i$ is thus the total weight of the outgoing edges from node $i$. Let $\b Q = \mathrm{diag}(\b q)$. Then, by normalizing the rows of $\b A$ we get the matrix $\b M = \b Q^{-1}\b A$, the transition probability matrix. $\b M$ is row-stochastic (or right-stochastic) and $[\b M]_{ij}$ is the probability to go from node $i$ to node $j$.
Consider a particle moving in the network according to the transition probabilities in $\b M$. Now let $\b p_t$ be the $1 \times n$ probability vector at Markov time $t$, namely that $p_{t,i}$ is the probability that the particle is located in node $i$ at time $t$. The dynamics of the discrete-time Markov process are given by :
\begin{equation} \label{eq:discreteMP}
	\b p_{t+1} = \b p_t \b K^{-1}\b A = \b p_t \b M.
\end{equation}
Now, suppose that the Markov chain is ergodic, i.e. that it is possible to go from every state to every state and that the Markov process is aperiodic. The ergodicity assumption implies that any initial state will asymptotically reach the same stationary solution. Let $\bs \pi$ be that stationary distribution, obtained by solving $\bs \pi = \bs \pi \b M$, and $\b \Pi = \rm{diag}(\bs \pi)$. Now, let $\b x_t$ be the $n$-dimensional random indicator vector describing the position of a particle undergoing the above dynamics: $x_{t,i} = 1$ if the particle is located in node $i$ at time $t$, and $0$ otherwise. At stationarity, the \textit{autocovariance matrix} of $\b x$ is
\begin{subequations}
	\begin{align}
 	\b C(\b x_{t_0},\b x_{t_0+t}) &\triangleq \E\left[(\b x_{t_0} - \E[\b x_{t_0}])^{\t}(\b x_{t_0+t} - \E[\b x_{t_0+t}])\right] \\
 		&= \E\left[(\b x_{t_0} - \bs \pi)^{\t}(\b x_{t_0+t} - \bs \pi)\right] \\
 		&= \E\left[\b x_{t_0}^{\t} \b x_{t_0+t}\right] - \E[\b x_{t_0}^{\t}] \bs \pi - \bs \pi^{\t} \E\left[\b x_{t_0+t}\right] + \bs \pi^{\t} \bs \pi \\
 		&= \bs \Pi \b M^t - \bs \pi^{\t} \bs \pi,
 	\end{align}
\end{subequations}
where the fact that $\b C(\b x_{t_0},\b x_{t_0+t})$ only depends on the time difference $t$ at stationarity is readily verified. Here, $^\t$ is the transposed sign and $\b M^t$ is $\b M$ at the power $t$. $[\b C(\b x_{t_0},\b x_{t_0+t})]_{ij}$ is interpreted as the correlation between $\b x_{t_0,\,i}$ and $\b x_{t_0+t,\,j}$. The independence on the initial time $t_0$ implies that it can indifferently be chosen equal to $0$.

Suppose now a partition $\P$; we note $\b H_{\P}$ the indicator matrix of $\P$. If $c$ is the number of communities in $\P$, $\b H_{\P}$ is a binary $n \times c$ matrix such that
\begin{equation}
	[\b H_{\P}]_{ik} =
	\begin{cases}
		1  & \quad \mbox{if node $i$ is in community $k$},\\
	    0  & \quad \text{otherwise}.\\
	\end{cases}
\end{equation}
Let us define $\mathcal{H}_{\P} : \R[n \times n] \rightarrow \R[c \times c] : \b B \mapsto \mathcal{H}_\P(\b B) = \b H_{\P}^{\t} \b B \b H_{\P}$. Let $\b X$ be any $n \times n$ matrix, then $\b Y = \mathcal{H}_{\P}(\b X)$ is a $c \times c$ matrix such that
\begin{equation}
	[\b Y]_{kl} = \sum_{i \in \C_k} \sum_{j \in \C_l} [\b X]_{ij},
\end{equation}
where $\C_k$ and $\C_l$ denote communities $k$ and $l$ of partition $\P$. One could thus say that operator $\mathcal{H}_{\P}$ returns the \textit{clustered version} of any $n \times n$ matrix, namely the matrix where the contributions of every nodes belonging to the same community are gathered by summing them. Finally, let $\b y_t = \b H_{\P}^{\t} \b x_t$ denote the $c$-dimensional community indicator vector: $\b y_{t,\,k}$ is equal to $1$ if the particle is in community $k$ at time $t$ and zero otherwise.
Using those notations and the interpretation of $\mathcal{H}_{\P} $, the \textit{clustered autocovariance matrix} for partition $\P$ at time $t$ is defined as
\begin{subequations}
	\begin{align}
		\b R_t(\P) &= \mathcal{H}_{\P}\left(\b C(\b x_{t_0},\b x_{t_0+t})\right)\\
			&= \b C(\b y_{t_0},\b y_{t_0+t})\\
			&= \b H_{\P}^{\t}(\b \Pi\b M^t - \bs \pi^{\t}\bs \pi)\b H_{\P}.
	\end{align}
\end{subequations}
Notice that $\b R_t$ depends only on the topology of the graph and on the partition. If the graph has well defined communities given by $\P$ \textit{over a given time scale}, we expect that the particle is more likely to remain within the starting community over that time scale. This implies that the values of $\b y_{0,\,i}$ and $\b y_{t,\,i}$ are positively correlated for $t$ in that time scale, which in turn implies large diagonal elements in $\b R_t(\P)$ and hence a large trace of $\b R_t(\P)$. The elements of $\b R_t(\P)$ are interpreted as follows in terms of the random walk of a particle: $[\b R_t(\P)]_{kl}$ is the probability that a particle is in community $\C_l$ after $t$ discrete time-steps if it has started in $\C_k$ minus the probability that two independent random walkers are in $\C_k$ and $\C_l$, evaluated at stationarity. A good partition is such that there is a high likelihood of remaining in the starting community over a given time scale. The definition of the stability of a \textit{clustering} $\P$ follows naturally:
\begin{equation}
	r_t(\P) = \min_{0 \le s \le t} \sum_{i = 1}^{c} [\b R_s]_{ii} = \min_{0 \le s \le t} \trace(\b R_s).
\end{equation}
Note that taking the minimum for all times up to $t$ implies that the stability of the clustering at time $t$ is large only if it is large for all times preceding $t$. This allows to assign a low stability to partitions where there is a high probability of leaving the community and coming back to it later. According to \cite{delvenne2013stability}, this minimization is unnecessary in most cases and we have $r_t(\P) \approx \trace(\b R_t)$. Nevertheless, taking the minimization ensures maximum generality and allows for example to deal with almost bipartite graphs where $\trace(\b R_s)$ can be oscillatory.

All the definitions introduced until now are for a given partition $\P$. But what we ultimately want to compute is the optimal partition in the sense of stability, hence the one that maximizes the stability measure. Clearly, the optimal partition might be different for each Markov time $t$. Computing the optimal clustering for each Markov time gives the \textit{stability curve of the graph} :
\begin{equation}
	r_t = \max_{\P} r_t(\P).
\end{equation}
Now we understand how Markov time acts as an intrinsic resolution parameter: as Markov time grows, the number of communities is expected to decrease, since there are more possibilities for a random walker to escape a community when the time window increases. Hence, communities get bigger (or coarser) as Markov time increases. Interestingly, one can prove that in the case of \textit{undirected} networks, stability at time 1 is equivalent to the well-known \textit{configuration modularity} measure. But this equivalence does not hold for \textit{directed} networks and therefore does not concern the present work.

At this stage, an important remark has to be made about the assumption of ergodicity. The verification of this assumption is often far from being obvious, especially in the case of big undirected networks. The trick in that case is to introduce "à la Google" random teleportations.\footnote{In the original PageRank proposed by S. Brin and L. Page in 1998 (ref. \cite{grin1998anatomy}), this consists essentially in applying a perturbation to the transition probability matrix between web pages in order to ensure that at least one row of the matrix is positive, which implies the convergence of the Power Method. If we note the teleportation probability $\tau$, the perturbation can be interpreted as follows: a web surfer follows a link in his current page with probability $1-\tau$ and jumps to an arbitrary web page with probability $\tau$.} Let $\tau$ be the \textit{teleportation probability}. Then, if a random walker is located on a node with at least one outlink (which is always the case for the networks that we will consider), it follows one of the outlinks with probability $1-\tau$. Otherwise, the node is called a \textit{dangling node} and the random walker is teleported with a uniform probability to another random node. The corresponding perturbation of the transition probability matrix is, in the most general case:
\begin{equation} \label{eq:M_teleport}
	\widetilde{\b M} = (1-\tau)\b M + \frac{1}{n}[(1-\tau)\b d + \tau \b 1]\b 1^{\t},
\end{equation}
where $n$ is the number of nodes, $\b d$ is a binary $n \times 1$ vector whose entries are equal to $1$ if the corresponding node is a dangling node and $0$ otherwise, and $\b 1$ is the $n \times 1$ unity vector. In the case that we will consider in the next section, $\b d$ is the zero vector. This perturbation is known to make the dynamics ergodic, ensuring the existence and uniqueness of the stationary solution $\bs \pi$.


%        File: AnalyticalDuctModes.tex
%     Created: Fri Feb 11 10:00 PM 2022 E
% Last Change: Fri Feb 11 10:00 PM 2022 E
%
\documentclass[a4paper]{article}
\usepackage{mathtools}
\begin{document}
This paper outlines the mathematical model that describes sound propagation
within a duct. This is commonly used as the starting basis for predicting
turbomachinery duct noise. The goal of this document is check equation (4.1)
in Kousen's work \cite{Kousen1999} which was obtained from \cite{Kerrebrock1992}.
First, we will derive the analytical solution to the wave equation obtained
from the linearized Euler equations. This will provide a useful background by
showing why the Bessel's Function's zero crossing is needed to determine the
solution. Spolier, it is because the wave equation can be simplified to be
of the same form as Bessel's (Differential) Equation.

\section{}

The equation presented in Kousen's work

\begin{equation}
    \gamma_{acoustic} =
    - \frac{k M_x}{1 - M_x^2} \pm
    \frac{1}{1-M_x^2}
    \sqrt{k^2 + (M_x^2-1) \kappa_{m \mu}^2}
    \label{eqn:4_1}
\end{equation}

where $\kappa _{m \mu}$ is a ``separation constant for the $m^{th}$
circumferential and $\mu^{th}$ radial mode in the solution of the convected
wave equation.

It is not clear where to find the values of separation constants needed to
calculate the analytical solution for a uniform flow. In order to check
this work, this should be further documented. Although values of $\kappa$ are
not listed in Kousen's paper \cite{Kousen1999}, the reference [44] in \cite{Kousen1999}
(\cite{Kerrebrock1992}), defines the separation constant as $\mu $

The pressure fluctuations are governed by the wave equation derived from the
radial linearized euler equation,

\begin{align*}
    \frac{1}{A^2}\left(
        \frac{\partial^2 \tilde{p}}{\partial t^2}
    +
        \vec{V}\cdot \left(
            \frac{\partial\tilde{p}}{\partial t} +
            \frac{1}{\tilde{r}}\frac{\partial \tilde{p} }{\partial \tilde{r}} +
            \frac{\partial \tilde{p}}{\partial \theta} +
            \frac{\partial \tilde{p}}{\partial x}
        \right)  \right)-
        \left(
            \frac{\partial^2 \tilde{p}}{\partial t^2} +
            \frac{1}{\tilde{r}}\frac{\partial \tilde{p}}{\partial r} +
            \frac{1}{\tilde{r}^2} \frac{\partial^2 \tilde{p}}{\partial \theta^2} +
            \frac{\partial^2 \tilde{p}}{\partial x^2}
        \right) &= 0
\end{align*}

If we assume that the pressure fluctuation is

\begin{equation}
    \tilde{p}(x,r,\theta,t) = X(x) R(r) \Theta(\theta) T(t)
\end{equation}

where,

\begin{align*}
    X(x) &=
    A_1 e^{ik_x x} +
    B_1 e^{-ik_x x }\\
    \Theta(\theta) &=
    A_2 e^{i k_{\theta} \theta } +
    B_2 e^{-ik_{\theta} \theta }\\
    T(t) &=
    A_3 e^{i \omega t } +
    B_3 e^{-i\omega t  }
\end{align*}

Then the radial dependence is governed by,

\begin{align*}
    \frac{ d^2 \tilde{p}}{d r} +
    \frac{1}{r}
    \frac{d \tilde{p}}{d r} + \left[
    k^2 - 2 k k_x M_x + k_x^2 M_x^2- \frac{m^2}{r^2}  - k_x^2\right]\tilde{p}
\end{align*}

Applying algebraic jujitsu, this can be simplified, let's define a new variable,
$N$

$$- N^2 = k_x^2 M_x^2 - 2 k k_x M_x - k_x^2 $$
$$-N^2 = -(1 -  M_x^2)k_x^2 - 2 k k_x M_x  $$

\begin{align*}
    \frac{ d^2 p}{d r} +
    \frac{1}{r}
    \frac{d p}{d r} + \left[
    k^2 - N^2 - \frac{m^2}{r^2}  \right]p
\end{align*}

Let $k_r^2 = k^2 - N^2$

\begin{align*}
    \frac{ d^2 p}{d r} +
    \frac{1}{r}
    \frac{d p}{d r} + \left[
    k_r^2  - \frac{m^2}{r^2}  \right]p = 0
\end{align*}

Looking at the radial wavenumber,

\begin{align*}
    k_r^2 &= k^2 - N^2 \\
          &= k^2-(1 - M^2)^2 k_x^2 - 2 k k_x M_x \\
    0 &=  -(1 - M^2) ^2 k_x ^2 -  \left( 2M_x k \right)k_x +(k^2 - k_r^2)
\end{align*}

Where the roots to this equation are the axial wavenumber, which matches
the definition of the constant $\mu$,

\begin{equation}
    (\mu )^2 = -(1 - M^2) k^2 - 2 M \omega k / A_T + \left( \omega/A_T \right)^2
    \label{<+label+>}
\end{equation}

Applying the quadratic formula and taking

\begin{align*}
    A &= - \beta^2 \\
    B &= - 2M_x k\\
    C &= k^2 - k_2^2
\end{align*}

Note B is negative when $M_x$ is positive,

(I feel like N should change based on $M_x's$ sign)

\begin{align*}
    k_x &= \frac{2M_x k \pm \sqrt{4 M_x^2 k^2 + 4 \beta^2 \left( k^2 - k_r^2 \right)}}{-2\beta^2}\\
        &= \frac{-M_x k \pm \sqrt{k^2 - k_r^2}}{\beta^2}
\end{align*}
\section{Unsteady Flow}
The linearized Euler equations (LEE) for mean axial shear and swirling flow
in cylindrical coordinates are

We can rearrange the equations to reflect Equations 2.33-2.36. Note that the momentum equation in the $\theta$ and $x$ directions remain unchanged. The term $ \frac{\partial(rv_r')}{\partial r}  = \frac{\partial (r)}{\partial r}v_r' + \frac{\partial v_r'}{\partial r} r$ in the Energy equation
\begin{align*}
\frac{1}{\bar{\rho} A^2}\left(
\frac{\partial p'}{\partial t} +
\frac{V_{\theta}}{r}
\frac{\partial p'}{\partial \theta} +
V_x
\frac{\partial p'}{\partial x}
\right) +
\frac{V_{\theta}^2}{A^2 r}v_r'+
\frac{\partial v_r'}{\partial r} + \frac{v_r'}{r} +
\frac{1}{r}
\frac{\partial v_{\theta}'}{\partial \theta}
 +
\frac{\partial v_x'}{\partial x}
&= 0\\
\frac{\partial  v_r' }{\partial t} +
\frac{V_{\theta}}{r} \frac{\partial v'_r}{\partial \theta} -
\frac{2V_{\theta}v'_{\theta}}{r} +
V_x \frac{\partial v_r'}{\partial x} &= \frac{1}{\bar{\rho}} \frac{\partial p'}{\partial r}+\frac{V_{\theta}}{\bar{\rho} r A^2}   p'
\\
\frac{\partial  v_{\theta}' }{\partial t} +
v_r' \frac{\partial  V_{\theta}  }{\partial r} +
\frac{V_{\theta}}{r} \frac{\partial v'_{\theta}}{\partial \theta} +
\frac{v'_rV_{\theta}}{r} +
V_x \frac{\partial v_{\theta}'}{\partial x}
= -\frac{1}{\bar{\rho} r}	\frac{\partial p'}{\partial \theta}\\
\frac{\partial  v_x' }{\partial t} +
v_r' \frac{\partial  V_x  }{\partial r} +
\frac{V_{\theta}}{r} \frac{\partial v'_x}{\partial \theta} +
V_x \frac{\partial v_x'}{\partial x}
&= -\frac{1    }{\bar{\rho}}  \frac{\partial p'}{\partial x}
\end{align*}

Starting with equation 2.28 (Wave Equation) in Kousen's paper \cite{Kousen1999},

\begin{equation}
    \frac{1}{A^2}\frac{D^2\tilde{p}}{Dt^2} -
    \nabla^2 \tilde{p} =
    2 \bar{\rho} \frac{d V_x}{d r} \frac{\partial  \tilde{v}_r}{ \partial x}
    \label{eqn:KousensWaveEquation}
\end{equation}

If there is uniform axial flow (or no flow), $dV_x/dr = 0$,

\begin{align*}
    \frac{1}{A^2}\left(
        \frac{\partial^2 \tilde{p}}{\partial t^2} +
        \vec{V}\cdot \vec {\nabla} (\tilde{p})
    \right) -
    \nabla^2
    \tilde{p} &=
    0 \\
\end{align*}

Substituting the definitions for $\nabla$ and $\nabla^2$ in cylindrical
coordinates gives,

\begin{align*}
    \frac{1}{A^2}\left(
        \frac{\partial^2 \tilde{p}}{\partial t^2}
    +
        \vec{V}\cdot \left(
            \frac{\partial\tilde{p}}{\partial t} +
            \frac{1}{\tilde{r}}\frac{\partial \tilde{p} }{\partial \tilde{r}} +
            \frac{\partial \tilde{p}}{\partial \theta} +
            \frac{\partial \tilde{p}}{\partial x}
        \right)  \right)-
        \left(
            \frac{\partial^2 \tilde{p}}{\partial t^2} +
            \frac{1}{\tilde{r}}\frac{\partial \tilde{p}}{\partial r} +
            \frac{1}{\tilde{r}^2} \frac{\partial^2 \tilde{p}}{\partial \theta^2} +
            \frac{\partial^2 \tilde{p}}{\partial x^2}
        \right) &= 0
\end{align*}

For no flow, $\vec{V} = 0$, therefore,

\begin{align*}
    \frac{1}{A^2}\left(
        \frac{\partial^2 \tilde{p}}{\partial t^2}
    \right) -
        \left(
            \frac{\partial^2 \tilde{p}}{\partial t^2} +
            \frac{1}{\tilde{r}}\frac{\partial \tilde{p}}{\partial  r}  +
            \frac{1}{\tilde{r}^2} \frac{\partial^2 \tilde{p}}{\partial \theta^2} +
            \frac{\partial^2 \tilde{p}}{\partial x^2}
        \right) &= 0
\end{align*}

Using the isentropic relation, $\tilde{p} = p/\bar{\rho} A^2$.
To dimensionalize the equation, this relation is substituted and both sides
are multiplied by $\bar{\rho}A^2$,

\begin{align*}
    \frac{1}{A^2}\left(
        \frac{\partial^2 {p}}{\partial t^2}
    \right) -
        \left(
            \frac{\partial^2 {p}}{\partial t^2} +
            \frac{1}{\tilde{r}}\frac{\partial p}{\partial r} +
            \frac{1}{\tilde{r}^2} \frac{\partial^2 p}{\partial \theta^2} +
            \frac{\partial^2 p}{\partial x^2}
        \right) &= 0
\end{align*}
\section{Analytical Solution}
The process of separation of variables(seperation indeterminatarum) was first written and formalized by John Bernoulli in a letter to Leibniz. The method
of separation of variables requires an assumed solution as well as initial and boundary
conditions. For a partial differential equation, the assumed solution can be a
linear combination of solutions to a system of ordinary differential equations that
comprises the partial differential equation. Since $p$ is a function of four
variables, the solution is assumed to be a linear combination of four solutions.
Each solution is assumed to be Euler's identity, a common ansant for linear partial
differential equations and boundary conditions.  The pressure field can be assumed to be,

\begin{equation}
    p(x,r,\theta,t) = X(x) R(r) \Theta(\theta) T(t)
\end{equation}

where,

\begin{align*}
    X(x) &=
    A_1 e^{ik_x x} +
    B_1 e^{-ik_x x }\\
    \Theta(\theta) &=
    A_2 e^{i k_{\theta} \theta } +
    B_2 e^{-ik_{\theta} \theta }\\
    T(t) &=
    A_3 e^{i \omega t } +
    B_3 e^{-i\omega t  }
\end{align*}

The next step is to rewrite the wave equation in terms of $X$, $R$, $\Theta$,
and $T$. To further simplify the result, each term is divided by $p$.
Before the substitution, the derivatives of the assumed solutions need to be
evaluated.

\subsection{Evaluating the derivatives}
By looking at each derivative individually, the process will be clearer to
follow,
\subsubsection{Temporal Derivatives}

\begin{align*}
    \frac{\partial p}{\partial t}
    &=
    \frac{\partial }{\partial t}  \left( XR\Theta T \right) \\
    &=
    XR\Theta\frac{\partial T}{\partial t}
\end{align*}

\begin{align*}
    \frac{1}{p}\frac{\partial p}{\partial t}
    &=
    \frac{ 1}{X R \Theta T}  \left( XR\Theta\frac{\partial T}{\partial t} \right) \\
    &=\frac{ 1}{ T}\frac{\partial T}{\partial t}
\end{align*}

\begin{align*}
    \frac{\partial^2 p}{\partial t^2}
    &=
    \frac{\partial^2 }{\partial t^2}  \left( XR\Theta T \right) \\
    &=
    XR\Theta\frac{\partial^2 T}{\partial t^2}
\end{align*}

\begin{align*}
    \frac{1}{p}\frac{\partial^2 p}{\partial t^2}
    &=
    \frac{ 1}{X R \Theta T}  \left( XR\Theta\frac{\partial^2 T}{\partial t^2} \right) \\
    &=\frac{ 1}{ T}\frac{\partial^2 T}{\partial t^2}
\end{align*}

\begin{align*}
    \frac{\partial T}{\partial t} &=
    \frac{\partial}{\partial t}
        \left(
        A_3 e^{i \omega t} + B_3 e^{-i \omega t}
    \right)  \\
    &=
    \frac{\partial}{\partial t} \left(A_3 e^{i \omega t}  \right) +
    \frac{\partial}{\partial t} \left(B_3 e^{-i \omega t}  \right)\\
    &= i \omega A_3 e^{i \omega t} - i \omega B_3 e^{i \omega t}
\end{align*}

\begin{align*}
    \frac{\partial^2 T}{\partial t^2} &=
    \frac{\partial^2}{\partial t^2}
        \left(
        i \omega A_3 e^{i \omega t} + i \omega B_3 e^{-i \omega t}
    \right)  \\
    &=
    \frac{\partial^2}{\partial t^2} \left(i \omega A_3 e^{i \omega t}  \right) +
    \frac{\partial^2}{\partial t^2} \left(- i \omega B_3 e^{-i \omega t}  \right)\\
    &= (i \omega)^2 A_3 e^{i \omega t} - (i \omega)^2 B_3 e^{i \omega t}
\end{align*}

\begin{align*}
    \frac{1}{T}\frac{\partial^2 T}{\partial t^2}
    &=
    (i\omega)^2 \\
    &= -\omega^2
\end{align*}

\subsubsection{Radial Derivatives}
\begin{align*}
    \frac{\partial p}{\partial r}
    &=
    \frac{\partial }{\partial r}  \left( XR\Theta T \right) \\
    &=
    X\Theta T\frac{\partial R}{\partial r}
\end{align*}

\begin{align*}
    \frac{1}{p}\frac{\partial p}{\partial r}
    &=
    \frac{ 1}{X R \Theta T}  \left( X\Theta T\frac{\partial R}{\partial r} \right) \\
    &=\frac{ 1}{ R}\frac{\partial R}{\partial r}
\end{align*}

\begin{align*}
    \frac{\partial^2 p}{\partial r^2}
    &=
    \frac{\partial^2 }{\partial r^2}  \left( XR\Theta T \right) \\
    &=
    X\Theta T\frac{\partial^2 R}{\partial r^2}
\end{align*}

\begin{align*}
    \frac{1}{p}\frac{\partial^2 p}{\partial r^2}
    &=
    \frac{ 1}{X R \Theta T}  \left( X\Theta T \frac{\partial^2 R}{\partial r^2} \right) \\
    &=\frac{ 1}{ R}\frac{\partial^2 R}{\partial r^2}
\end{align*}
The radial derivatives will be revisited once the remaining derivatives are evaluated,

\subsubsection{Tangential Derivatives}

\begin{align*}
    \frac{\partial p}{\partial \theta }
    &=
    \frac{\partial }{\partial t}  \left( XR\Theta T \right) \\
    &=
    XRT\frac{\partial \Theta}{\partial \theta}
\end{align*}

\begin{align*}
    \frac{1}{p}\frac{\partial p}{\partial \theta}
    &=
    \frac{ 1}{X R \Theta T}  \left( XR\Theta\frac{\partial T}{\partial \theta} \right) \\
    &=\frac{ 1}{ \Theta}\frac{\partial \Theta}{\partial \theta}
\end{align*}

\begin{align*}
    \frac{\partial^2 p}{\partial \theta^2}
    &=
    \frac{\partial^2 }{\partial \theta^2}  \left( XR\Theta T \right) \\
    &=
    XRT\frac{\partial^2 \Theta }{\partial \theta^2}
\end{align*}

\begin{align*}
    \frac{1}{p}\frac{\partial^2 p}{\partial \theta^2}
    &=
    \frac{ 1}{X R \Theta T}  \left( XRT\frac{\partial^2 \Theta}{\partial \theta^2} \right) \\
    &=\frac{ 1}{ \Theta}\frac{\partial^2 \Theta}{\partial \theta^2}
\end{align*}

\begin{align*}
    \frac{\partial \Theta}{\partial \theta} &=
    \frac{\partial}{\partial \theta}
        \left(
            A_2 e^{i k_{\theta} \theta} + B_2 e^{-i k_{\theta} \theta}
        \right)  \\
    &=
    \frac{\partial}{\partial \theta} \left(A_2 e^{i k_{\theta} \theta}  \right) +
    \frac{\partial}{\partial \theta} \left(B_2 e^{-i k_{\theta} \theta}  \right)\\
    &= i k_{\theta} A_2 e^{i k_{\theta} \theta} - i k_{\theta} B_2 e^{i k_{\theta} \theta}
\end{align*}

\begin{align*}
    \frac{\partial^2 \Theta }{\partial \theta^2} &=
    \frac{\partial^2}{\partial \theta^2}
        \left(
        i k_{\theta} A_2 e^{i k_{\theta} \theta} - i k_{\theta} B_2 e^{i k_{\theta} \theta}
    \right)  \\
    &=
    \frac{\partial^2}{\partial \theta^2} \left(i k_{\theta} A_2 e^{i k_{\theta} \theta}  \right) +
    \frac{\partial^2}{\partial \theta^2} \left(- i k_{\theta} B_2 e^{-i k_{\theta} \theta}  \right)\\
    &= (i k_{\theta})^2 A_2 e^{i k_{\theta} \theta } - (i k_{\theta})^2 B_2 e^{i k_{\theta} \theta}
\end{align*}

\begin{align*}
    \frac{1}{\Theta}\frac{\partial^2 \Theta}{\partial \theta^2}
    &=
    (ik_{\theta})^2 \\
    &= -k_{\theta}^2
\end{align*}

\subsubsection{Axial Derivatives}

\begin{align*}
    \frac{\partial p}{\partial x}
    &=
    \frac{\partial }{\partial x}  \left( XR\Theta T \right) \\
    &=
    R\Theta T \frac{\partial X}{\partial x}
\end{align*}

\begin{align*}
    \frac{1}{p}\frac{\partial p}{\partial x}
    &=
    \frac{ 1}{X R \Theta T}  \left( R\Theta\frac{\partial X}{\partial x} \right) \\
    &=\frac{ 1}{ X}\frac{\partial X}{\partial x}
\end{align*}

\begin{align*}
    \frac{\partial^2 p}{\partial x^2}
    &=
    \frac{\partial^2 }{\partial x^2}  \left( XR\Theta T \right) \\
    &=
    R\Theta T \frac{\partial^2 X}{\partial x^2}
\end{align*}

\begin{align*}
    \frac{1}{p}\frac{\partial^2 p}{\partial x^2}
    &=
    \frac{ 1}{X R \Theta T}  \left( R\Theta T \frac{\partial^2 X}{\partial x^2} \right) \\
    &=\frac{ 1}{ X}\frac{\partial^2 X}{\partial x^2}
\end{align*}

\begin{align*}
    \frac{\partial X}{\partial x} &=
    \frac{\partial}{\partial t}
        \left(
        A_3 e^{i k_x t} + B_3 e^{-i \omega t}
    \right)  \\
    &=
    \frac{\partial}{\partial t} \left(A_1 e^{i k_x x}  \right) +
    \frac{\partial}{\partial t} \left(B_1 e^{-i k_x x }  \right)\\
    &= i k_x A_1 e^{i k_x x } - i k_x B_1 e^{i k_x x}
\end{align*}

\begin{align*}
    \frac{\partial^2 X}{\partial x^2} &=
    \frac{\partial^2}{\partial x^2}
        \left(
        i k_x A_1 e^{i k_x x} + i k_x B_1 e^{-i k_x x}
    \right)  \\
    &=
    \frac{\partial^2}{\partial x^2} \left(i k_x A_1 e^{i k_x x}  \right) +
    \frac{\partial^2}{\partial x^2} \left(- i k_x B_1 e^{-i k_x x}  \right)\\
    &= (i k_x)^2 A_1 e^{i k_x x} - (i k_x)^2 B_1 e^{i k_x x}
\end{align*}

\begin{align*}
    \frac{1}{X}\frac{\partial^2 X}{\partial x^2}
    &=
    (i k_x)^2 \\
    &= -k_x^2
\end{align*}

Substituting this back into the wave equation yields ,

\begin{align*}
    \frac{1}{A^2}\left(
        \frac{\partial^2 {p}}{\partial t^2}
    \right) &=
        \left(
            \frac{\partial^2 {p}}{\partial t^2} +
            \frac{1}{\tilde{r}}\frac{\partial p}{\partial r} +
            \frac{1}{\tilde{r}^2} \frac{\partial^2 p}{\partial \theta^2} +
            \frac{\partial^2 p}{\partial x^2}
        \right)
\end{align*}

\begin{equation}
    \frac{1}{A^2} \frac{1}{T}\frac{\partial^2 T}{\partial t^2} =
    \frac{1}{R}\frac{\partial^2 R}{\partial r^2 } +
    \frac{1}{r}\frac{1}{R}\frac{\partial R}{\partial r}  +
    \frac{1}{r^2}\frac{1}{\Theta}\frac{\partial \Theta}{\partial \theta} +
    \frac{1}{X}\frac{\partial^2 X}{\partial x^2}
    \label{eqn:waveode}
\end{equation}

Notice that each term is only a function of its associated independent variable.
So, if we vary the time, only the term on the left-hand side can vary. However,
since none of the terms on the right-hand side depend on time, that means the
right-hand side cannot vary, which means that the ratio of time with its second
derivative is independent of time. The practical upshot is that each of these
terms is constant, which has been shown. The wave numbers are the \textit{separation constants}
that allow the PDE to be split into four separate ODE's. Substituting the separation constants
into Equation (\ref{eqn:waveode}) gives,

\begin{equation}
    -\frac{\omega^2}{A^2}  =
    \frac{1}{R}
    \left(
    \frac{\partial^2 R}{\partial r^2 } +
    \frac{1}{r}\frac{\partial R}{\partial r}
\right) -
    \frac{k_{\theta}^2}{r^2}-
    k_x^2
    \label{eqn:waveode2}
\end{equation}
Note that the dispersion relation states $\omega = k A$

\begin{equation}
    \frac{1}{R}
    \left(
    \frac{\partial^2 R}{\partial r^2 } +
    \frac{1}{r}\frac{\partial R}{\partial r}
\right) -
    \frac{k_{\theta}^2}{r^2}-
    k_x^2 + k^2 = 0
    \label{eqn:waveode3}
\end{equation}
The remaining terms are manipulated to follow the same form as \textit{Bessel's Differntial
Equation} ,

\begin{equation}
    x^2 \frac{d^2 y}{dx^2} + x \frac{dy }{dx } + (x^2 - n^2) y = 0
    \label{eqn:besselODE}
\end{equation}

The general solution to Bessel's differential equation is a linear combination of
the Bessel functions of the first kind, $J_n(x)$ and of the second kind, $Y_n(x)$
. The subscript $n$ refers to the order of Bessel's equation.

\begin{equation}
    y(x) = AJ_n(x) + BY_n(x)
    \label{eqn:besselsolution}
\end{equation}

By rearranging Equation (\ref{eqn:waveode3}), a comparison can be made to Equation
(\ref{eqn:besselODE}) to show that the two equations are of the same form.

The first step is to revisit the radial derivatives that have not been addressed.
As was done for the other derivative terms, the radial derivatives will also
be set equal to a separation constant, $-k_r^2$.

\begin{align}
    \underbrace{\frac{1}{R}
    \left(
    \frac{\partial^2 R}{\partial r^2 } +
    \frac{1}{r}\frac{\partial R}{\partial r}
\right) -
    \frac{k_{\theta}^2}{r^2}}_{-k_r^2}-
    k_x^2 + k^2 = 0
    \label{eqn:wavenumber_without_kr}
\end{align}

The reader may be curious as to why the tangential separation constant $k_{\theta}$ is
included within the definition of the radial separation constant.

Recall the ODE for the tangential direction,

\begin{align*}
    \frac{\partial \Theta}{\partial \theta} \frac{1}{\Theta} = - k_{\theta}^2\\
    \frac{\partial \Theta}{\partial \theta} \frac{1}{\Theta} + \Theta k_{\theta}^2 = 0
\end{align*}

where the solution is more or less,

\begin{align*}
    \Theta(\theta) = e^{i k_{\theta} \theta}
\end{align*}

In order to have non trivial, sensible solutions, the value of $\Theta(0)$ and
$\Theta(2\pi)$ need to be the same, and this needs to be true for any multiple
of $2\pi$ for a fixed r. Taking $\Theta$ to be one, a unit circle, it can be shown that the domain
is only going to be an integer multiple. Therefore, there is an implied periodic
azimuthal boundary condition, i.e. $0<\theta\leq 2 \pi$ and $k_{\theta}=m$.

Continuing with the radial derivatives\ldots

\begin{align*}
    -k_r^2 =\frac{1}{R}
    \left(
    \frac{\partial^2 R}{\partial r^2 } +
    \frac{1}{r}\frac{\partial R}{\partial r}
\right) -
    \frac{m^2}{r^2}
\end{align*}
To further simplify, the chain rule is used to do a change of variables, $x = k_r r$
\begin{align*}
    \frac{\partial R}{\partial r} &= \frac{dR}{dx}\frac{dx}{dr}\\
    &=
    \frac{dR}{dx}\frac{d}{dr}\left( k_r r \right) \\
    &=
    \frac{dR}{dx} k_r
\end{align*}

\begin{align*}
    \frac{\partial^2 R}{\partial r^2} &= \frac{d^2R}{dx^2}\left(\frac{dx}{dr}\right)^2 +
    \frac{dR}{dr}\frac{d^2x}{dr^2}\\
    &=
    \frac{d^2R}{dx^2}\frac{d}{dr} k_r^2 + k_r \frac{d^2r}{dr^2}\\
    &=
    \frac{d^2R}{dx^2}\frac{d}{dr} k_r^2
\end{align*}

Substituting this into Equation (\ref{eqn:waveode3}),
\begin{equation}
    \left(\frac{d^2R}{dx^2}k_r^2 +
    \frac{1}{r}\frac{d^2R}{dx^2}k_r\right) +
    \left(k_r^2 - \frac{m^2}{r^2}\right)R
    \label{eqn:waveode4}
\end{equation}
Dividing Equation \ref{eqn:waveode4} by $k_r^2$,

\begin{equation}
    \left(\frac{d^2R}{dx^2} +
    \frac{1}{k_r r}\frac{d^2R}{dx^2}\right) +
    \left(1  - \frac{m^2}{k_r^2 r^2}\right)R
    \label{eqn:waveode5}
\end{equation}

\begin{equation}
    \left(\frac{d^2R}{dx^2} +
    \frac{1}{x^2}\frac{d^2R}{dx^2}\right) +
    \left(1  - \frac{m^2}{x^2}\right)R
    \label{eqn:waveode6}
\end{equation}

Multiplying Equation (\ref{eqn:waveode6}) by $x^2$ gives,

\begin{equation}
    \frac{d^2R}{dr^2}x^2 +
    \frac{dR}{dr}x +
    \left( x^2 - m^2 \right)R
    \label{eqn:finalradialode}
\end{equation}
which matches the form of Bessel's equation

Therefore, the solution goes from this,
\begin{equation}
    y(x) = AJ_n(x) + BY_n(x)
    \label{eqn:besselsolution}
\end{equation}
to this,

\begin{equation}
    R(r) = (AJ_n(k_r r) + BY_n(k_r r))
    \label{eqn:besselsolution}
\end{equation}
where the coefficients $A$ and $B$ are found after applying radial
boundary conditions. %and there is an exponential dependence.

\subsubsection{Hard Wall boundary condition}
\begin{align*}
    \frac{\partial p}{\partial r}|_{r = r_{min}}  =\frac{\partial p}{\partial r}|_{r = r_{max}} = 0 \rightarrow
    \frac{\partial}{\partial r} \left( X\Theta T R \right) &= 0 \\
    X \Theta T\frac{\partial R}{\partial r}  &= 0 \\
    \frac{\partial R}{\partial r}  &= 0
\end{align*}

where,

\begin{align*}
    \frac{ \partial R}{\partial r}|_{r_{min}} &= AJ_n'(k_r r_{min}) + B Y_n' (k_r r_{min}) = 0
    \rightarrow B = -A \frac{J_n'(k_r r_{min})}{Y_n'(k_r r_{min})}
\end{align*}

\begin{align*}
    \frac{ \partial R}{\partial r} &= AJ_n'(k_r r_{max}) + B Y_n' (k_r r_{max}) = 0 \\
                                   &= AJ_n'(k_r r_{max}) - A\frac{J_n' (k_r r_{min})}{Y_n'(k_r r_{min})} Y_n' (k_r r_{max}) = 0 \\
                                   &= \frac{J_n'(k_r r_{min})}{J_n' (k_r r_{max})} - \frac{Y_n'(k_r r_{min})}{Y_n' (k_r r_{max})} = 0
\end{align*}
where $k_r r$ are the zero crossings for the derivatives of the Bessel functions of the first and second kind.

In summary, the wave equation for no flow in a hollow duct with hard walls is obtained
from Equation (\ref{eqn:wavenumber_without_kr}).
\begin{equation}
    k^2 = k_r^2 + k_x^2
    \label{eqn:wavenumber_equation}
\end{equation}

Solving for the axial wavenumber gives,

\begin{equation}
    k_x = \sqrt{k^2 - k_r^2}
    \label{eqn:axial_wavenumber_equation}
\end{equation}

\section{Uniform Flow}

To get the same equation but for uniform flow, the same procedure can be followed.

Starting with Equation 2.27 redimensionalized,

\begin{align*}
    \frac{ d^2 \tilde{p}}{d \tilde{r}^2} +
    \frac{1}{\tilde{r}}
    \frac{d \tilde{p}}{d \tilde{r}} +
    \frac{2 \bar{\gamma} \left( \frac{d M_x}{d \tilde{r}} \right)}
    {\left( k - \bar{\gamma} M_x \right)}\frac{d \tilde{p}}{d \tilde{r}}+
    \left[ \left( k - \bar{\gamma} M_x \right)^2 - \frac{m^2}{\tilde{r}^2}-
    \bar{\gamma}^2 \right] \tilde{p}
\end{align*}

Let's separate the new terms from the old ones,

\begin{align*}
    \frac{ d^2 \tilde{p}}{d \tilde{r}^2} +
    \frac{1}{\tilde{r}}
    \frac{d \tilde{p}}{d \tilde{r}} +
    \frac{2 \bar{\gamma} \left( \frac{d M_x}{d \tilde{r}} \right)}
    {\left( k - \bar{\gamma} M_x \right)}\frac{d \tilde{p}}{d \tilde{r}}+
    \left[ \left( k - \bar{\gamma} M_x \right)^2 - \frac{m^2}{\tilde{r}^2}-
    \bar{\gamma} \right] \tilde{p}
\end{align*}

Recalling the non-dimensional definitions,
\begin{align*}
    \tilde{p} &= \frac{p}{\bar{\rho} A^2} \\
    \tilde{r} &= \frac{r}{r_T} \\
    \frac{\partial \tilde{p}}{\partial \tilde{r}} &=
    \frac{ \partial \tilde{p}}{\partial r} \frac{\partial r}{ \partial \tilde{r}}  \\
    &= \frac{ \partial \tilde{p}}{\partial r} \frac{\partial }{ \partial \tilde{r}} \left( \tilde{r} r_T \right) \\
    &=
    \frac{ \partial \tilde{p}}{\partial r}  r_T \\
    \frac{\partial^2 \tilde{p}}{\partial \tilde{r}^2} &=
    \frac{ \partial^2 \tilde{p}}{\partial r^2}  (r_T)^2+
    \frac{ \partial \tilde{p}}{\partial r} \frac{\partial^2 r}{ \partial \tilde{r}^2} \\
    &= \frac{ \partial^2 \tilde{p}}{\partial r^2}  (r_T)^2
\end{align*}

\begin{align*}
    \frac{\partial}{\partial r} \left( \frac{p}{\bar{\rho} A^2} \right)
    &=
    \frac{\left(\frac{\partial}{\partial r} \left(  p\right) \bar{\rho} A^2 -
    \underbrace{\frac{\partial \bar{\rho}A^2}{\partial r}}_0 p \right)}{\left( \bar{\rho} A^2 \right)^2}\\
    &= \frac{1}{\bar{\rho}A^2} \frac{\partial p}{\partial r}
\end{align*}

\begin{align*}
    \frac{ d^2 \tilde{p}}{d \tilde{r}^2} +
    \frac{1}{\tilde{r}}
    \frac{d \tilde{p}}{d \tilde{r}}-
    \frac{m^2}{\tilde{r}^2}\tilde{p}-
    \bar{\gamma}^2  \tilde{p}
 +
    \frac{2 \bar{\gamma} \left( \frac{d M_x}{d \tilde{r}} \right)}
    {\left( k - \bar{\gamma} M_x \right)}\frac{d \tilde{p}}{d \tilde{r}}+
    \left( k - \bar{\gamma} M_x \right)^2\tilde{p}
\end{align*}

If there is only uniform flow, then $dM_x/dr = 0$,

\begin{align*}
    \frac{ d^2 \tilde{p}}{d \tilde{r}^2} +
    \frac{1}{\tilde{r}}
    \frac{d \tilde{p}}{d \tilde{r}}-
    \frac{m^2}{\tilde{r}^2}\tilde{p}-
    \bar{\gamma}^2  \tilde{p}
 +
    \left( k - \bar{\gamma} M_x \right)^2\tilde{p}
\end{align*}

Re-dimensionalizing,

\begin{align*}
    \frac{1}{\bar{\rho} A^2}\left[
    \frac{ d^2 p}{d r} r_T^2+
    \frac{r_T}{r}
    \frac{d p}{d r} r_T -
    \frac{m^2}{r^2}r_T^2 p - k_x^2r_T^2  p\right]
    + \left( \frac{\omega }{A}r_T - k_x r_T M_x \right)^2p
\end{align*}

Expanding the last term and substituting $\omega/A = k$

\begin{align*}
    \frac{1}{\bar{\rho} A^2}\left[
    \frac{ d^2 p}{d r} r_T^2+
    \frac{r_T}{r}
    \frac{d p}{d r} r_T -
    \frac{m^2}{r^2}r_T^2 p - k_x^2r_T^2  p\right]
    +\left( r_T^2\left(
        k^2 - 2 k k_x M_x + k_x^2 M_x^2 \right)
    \right)p
\end{align*}
Canceling out $r_T/\bar{\rho}A$ in every term

\begin{align*}
    \frac{ d^2 p}{d r} +
    \frac{1}{r}
    \frac{d p}{d r} + \left[
    k^2 - 2 k k_x M_x + k_x^2 M_x^2- \frac{m^2}{r^2}  - k_x^2\right]p
\end{align*}

Continue here,

Defining

$$- N^2 = k_x^2 M_x^2 - 2 k k_x M_x - k_x^2 $$
$$-N^2 = -(1 -  M_x^2)k_x^2 - 2 k k_x M_x  $$
$$-N^2 =  -\beta^2 k_x^2 - 2 k k_x M_x  $$

\begin{align*}
    \frac{ d^2 p}{d r} +
    \frac{1}{r}
    \frac{d p}{d r} + \left[
    k^2 - N^2 - \frac{m^2}{r^2}  \right]p
\end{align*}

Let $k_r^2 = k^2 - N^2$

\begin{align*}
    \frac{ d^2 p}{d r} +
    \frac{1}{r}
    \frac{d p}{d r} + \left[
    k_r^2  - \frac{m^2}{r^2}  \right]p
\end{align*}

Looking at the radial wavenumber,

\begin{align*}
    k_r^2 &= k^2 - N^2 \\
          &= k^2-\beta^2 k_x^2 - 2 k k_x M_x \\
    0 &=  -\beta ^2 k_x ^2 -  \left( 2M_x k \right)k_x +(k^2 - k_r^2)
\end{align*}

Where the roots to this equation are the axial wavenumber,

Applying the quadratic formula and taking

\begin{align*}
    A &= - \beta^2 \\
    B &= - 2M_x k\\
    C &= k^2 - k_2^2
\end{align*}

Note B is negative when $M_x$ is positive,

(I feel like N should change based on $M_x's$ sign)

\begin{align*}
    k_x &= \frac{2M_x k \pm \sqrt{4 M_x^2 k^2 + 4 \beta^2 \left( k^2 - k_r^2 \right)}}{-2\beta^2}\\
        &= \frac{-M_x k \pm \sqrt{k^2 - k_r^2}}{\beta^2}
\end{align*}

\section{Annular Duct Axial Wavenumber solution}

In \cite{Amr2001}, the axial wavenumber for annular ducts is reported, if the
annulus is narrow ($(r_{max} - r_{min})/r_{mean} << 1$ , then the radius, mean
velocity, density, and speed of sound can be expressed in terms of a perturbation
expansion of the form,

\begin{align}
    r &= 1 + \epsilon \tilde{r} + \cdots,\\
    M_t &= M_x(1) e_x + M_{\theta}(1) e_{\theta} + O(\epsilon)
    \rho_o &= 1 + O(\epsilon)
    c_o^2 &=   1 + O(\epsilon)
\end{align}

where $\epsilon << 1$

\begin{equation*}
    \frac{-(\tilde{\omega} - m M_{\theta})M_x \pm \sqrt{\left( \tilde{\omega} - m M_{\theta}^2 \right) - \beta\left( m^2 + \Gamma_{m,n} ^2 \right)}}{\beta^2}
\end{equation*}

where
$\tilde{\omega} = \omega - m M_{\theta}$
$\beta = 1- M_x^2$
$\Gamma_{m,n}= \frac{n^2 \pi^2}{\left( r_{max} - r_{min} \right)^2}$
\bibliographystyle{ieeetr}
\bibliography{references.bib}
\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS622: Theory of Formal Languages
% Copyright 2014 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/beacon
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 3}

Let $A = \{a,b\}$.
Prove that there are no words $x, y \in A^*$ such that $xay = ybx$.
Prove that there is no word $x \in \{a,b\}^*$ such that $ax = xb$.

\subsection*{Solution}

It is prerequisite of equality for any two words $u, v \in \{a,b\}^*$ to share an equal number of each symbols.
It is shown that the two words $u=xay$ and $v=ybx$ fail to satisfy this prerequisite.

Suppose there are $m$ total $a$ symbols and $n$ total $b$ symbols in x and y.
Therefore, total number of $a$ symbols in $u$ is $m+1$ whereas total number of $a$ symbols in $v$ is $m$.
Thus prerequisite is not met and $u \neq v$.

Similarly, if $u = ax$ and $v = xb$, number of $a$ symbols in $u$ is always one more than number of $a$ symbols in $v$ thus $u \neq v$.


\section*{Exercises}

\begin{ex}
  Let
  \begin{equation*}
    A = \begin{mymatrix}{cc}
      a & b \\
      c & d \\
    \end{mymatrix}.
  \end{equation*}
  An operation is done to get from $A$ to a matrix $B$. In each case,
  identify which operation was done and explain how it will affect the
  value of the determinant.
  \begin{enumerate}
  \item
    \begin{equation*}
      B = \begin{mymatrix}{cc}
        a & c \\
        b & d \\
      \end{mymatrix}
    \end{equation*}
  \item
    \begin{equation*}
      B = \begin{mymatrix}{cc}
        c & d \\
        a & b \\
      \end{mymatrix}
    \end{equation*}
  \item
    \begin{equation*}
      B = \begin{mymatrix}{cc}
        a   & b   \\
        a+c & b+d \\
      \end{mymatrix}
    \end{equation*}
  \item
    \begin{equation*}
      B = \begin{mymatrix}{cc}
        a  & b  \\
        2c & 2d \\
      \end{mymatrix}
    \end{equation*}
  \item
    \begin{equation*}
      B = \begin{mymatrix}{cc}
        b & a \\
        d & c \\
      \end{mymatrix}
    \end{equation*}
  \end{enumerate}
  \begin{sol}
    \begin{enumerate}
    \item The transpose was taken and $\det(B) = \det(A)$.
    \item Two rows were switched and $\det(B) = -\det(A)$.
    \item The first row was added to the second row and $\det(B) = \det(A)$.
    \item The second row was multiplied by 2 and $\det(B) = 2\det(A)$.
    \item Two columns were switched and $\det(B) = -\det(A)$.
    \end{enumerate}
  \end{sol}
\end{ex}

\begin{ex}
  Let $A$ be an $n\times n$-matrix and suppose there are $n-1$ rows
  such that the remaining row is a linear combinations of these $n-1$
  rows. Show $\det(A) = 0$.
  \begin{sol}
    By assumption, we can obtain a row of zeros by doing row
    operations. Row operations do not change whether the determinant
    is zero, so the determinant must have been zero all along.
  \end{sol}
\end{ex}

\begin{ex}
  \label{ex:determinant3}
  Let $A$ be an $n\times n$-matrix. Show that if $\det(A) \neq 0$ and
  $A\vect{x}=\vect{0}$, then $\vect{x}=\vect{0}$.
  \begin{sol}
    If $\det(A) \neq 0$, then $A^{-1}$ exists. Therefore
    $\vect{x} = A^{-1}A\vect{x} = A^{-1}\vect{0} = \vect{0}$.
  \end{sol}
\end{ex}

\begin{ex}
  Using only Theorems~\ref{thm:determinant-of-triangular-matrix} and
  {\ref{thm:determinant-of-product}}, show that
  $\det(kA) = k^n\det(A)$ for an $n\times n$-matrix $A$ and scalar
  $k$.
  \begin{sol}
    The matrix $kI$ has $k$ down the main diagonal and has determinant
    equal to $k^n$ by
    Theorem~\ref{thm:determinant-of-triangular-matrix}. Using
    Theorem~\ref{thm:determinant-of-product}, it follows that
    $\det(kA) = \det(kIA) = \det(kI) \det(A) = k^n\det(A)$.
  \end{sol}
\end{ex}

\begin{ex}
  Construct two random $2\times 2$-matrices $A$ and $B$ and verify
  that $\det(A)\det(B) = \det(AB)$.
  \begin{sol}
    \begin{equation*}
      \det
      \paren{\begin{mymatrix}{cc}
          1 & 2 \\
          3 & 4
        \end{mymatrix} \begin{mymatrix}{rr}
          -1 & 2 \\
          -5 & 6
        \end{mymatrix}} = -8,
    \end{equation*}
    \begin{equation*}
      \det\paren{\begin{mymatrix}{cc}
        1 & 2 \\
        3 & 4
      \end{mymatrix}}\det\paren{\begin{mymatrix}{rr}
        -1 & 2 \\
        -5 & 6
      \end{mymatrix}} = -2\times 4 = -8.
    \end{equation*}
  \end{sol}
\end{ex}

\begin{ex}
  Is it true that $\det(A+B) = \det(A) + \det(B)$? If this is so,
  explain why. If it is not so, give a counterexample.
  \begin{sol}
    This is not true at all. Consider $A = \begin{mymatrix}{cc}
      1 & 0 \\
      0 & 1
    \end{mymatrix}$ and $B = \begin{mymatrix}{rr}
      -1 & 0 \\
      0 & -1
    \end{mymatrix}$. Then $\det(A)=1$, $\det(B)=1$, and $\det(A+B)=0$.
  \end{sol}
\end{ex}

\begin{ex}
  An $n\times n$-matrix is called \textbf{nilpotent}%
  \index{nilpotent matrix}%
  \index{matrix!nilpotent} if there exists some positive integer $k$
  such that $A^k = 0$. If $A$ is a nilpotent matrix, what are the
  possible values of $\det(A)$?
  \begin{sol}
    Since $A^k=0$, we have $\det(A)^k=\det(A^k)=\det(0)=0$. Therefore,
    it must be the case that $\det(A)=0$.
  \end{sol}
\end{ex}

\begin{ex}
  A square matrix is said to be \textbf{orthogonal}%
  \index{matrix!orthogonal}%
  \index{orthogonal matrix} if $A^TA = I$. Thus the inverse of an
  orthogonal matrix is its transpose. What are the possible values of
  $\det(A)$ if $A$ is an orthogonal matrix?
  \begin{sol}
    If $A$ is orthogonal, we have $\det(A)^2 = \det(A^T)\det(A) =
    \det(A^TA) = \det(I) = 1$. Therefore the only possible values for
    $\det(A)$ are $\pm 1$.
  \end{sol}
\end{ex}

\begin{ex}
  Let $A$ and $B$ be two $n\times n$-matrices. We say that $A$ is
  \textbf{similar}%
  \index{matrix!similar}%
  \index{similar matrices} to $B$, in symbols $A\similar B$, if there
  exists an invertible matrix $P$ such that $A = P^{-1}BP$. Show that
  if $A\similar B$, then $\det(A) = \det(B)$.
  \begin{sol}
    $\det(A) = \det(P^{-1}BP) = \det(P^{-1}) \det(B) \det(P) =
    \frac{1}{\det(P)} \det(B) \det(P) = \det(B)$.
  \end{sol}
\end{ex}

\begin{ex}
  Find the determinant of
  \begin{equation*}
    A = \begin{mymatrix}{ccc}
      1 & 1 & 1 \\
      1 & a & a^2 \\
      1 & b & b^2 \\
    \end{mymatrix}.
  \end{equation*}
  For which values of $a$ and $b$ is this matrix invertible? Hint:
  after you compute the determinant, you can factor out $(a-1)$ and
  $(b-1)$ from it.
  \begin{sol}
    The determinant is
    \begin{equation*}
      \begin{absmatrix}{ccc}
        1 & 1 & 1 \\
        1 & a & a^2 \\
        1 & b & b^2 \\
      \end{absmatrix}
      = \begin{absmatrix}{cc}
        a & a^2 \\
        b & b^2 \\
      \end{absmatrix}
      - \begin{absmatrix}{cc}
        1 & 1 \\
        b & b^2 \\
      \end{absmatrix}
      + \begin{absmatrix}{cc}
        1 & 1 \\
        a & a^2 \\
      \end{absmatrix}
      = ab^2 - ba^2 - b^2 + b + a^2 - a
      = (a-1)(b-1)(b-a).
    \end{equation*}
    Therefore the determinant is $0$ if $a=1$, $b=1$, or $a=b$. In all
    other cases, the determinant is non-zero and the matrix is invertible.
  \end{sol}
\end{ex}

\begin{ex}
  Assume $A$, $B$, and $C$ are $n\times n$-matrices and $ABC$ is
  invertible. Use determinants to show that each of $A,B$, and $C$ is
  invertible.
  \begin{sol}
    This follows because $\det(ABC) = \det(A)\det(B)\det(C)$ and if
    this product is non-zero, then each determinant in the product is
    non-zero. Therefore, each of these matrices is invertible.
  \end{sol}
\end{ex}

\begin{ex}
  Suppose $A$ is an upper triangular matrix. Show that $A^{-1}$ exists
  if and only if all elements of the main diagonal are non-zero. Is it
  true that $A^{-1}$ will also be upper triangular? Explain. Could the
  same be concluded for lower triangular matrices?
  \begin{sol}
    The given condition is what it takes for the determinant to be
    non-zero. Recall that the determinant of an upper triangular
    matrix is just the product of the entries on the main diagonal.
    The inverse will also be upper triangular; this can be seen by
    noting that every invertible upper triangular can be written as a
    product of upper triangular elementary matrices; the inverse of
    each such elementary matrix is upper triangular, and therefore so
    is their product. The analogous statement about lower triangular
    matrices is also true.
  \end{sol}
\end{ex}

\begin{samepage}  % Inhibit page break within the exercise. A student
                  % misread a "false" statement at the top of the page
                  % as a theorem.
\begin{ex}
  Specify whether each statement is true or false. If true, provide a
  proof. If false, provide a counterexample.
  \begin{enumerate}
  \item If $A$ is a $3\times 3$-matrix with determinant zero, then one
    column must be a multiple of some other column.

  \item If any two columns of a square matrix are equal, then the
    determinant of the matrix equals zero.

  \item For two $n\times n$-matrices $A$ and $B$,
    $\det(A+B) = \det(A) + \det(B)$.

  \item For an $n\times n$-matrix $A$, $\det(3A) = 3\det(A)$.

  \item If $A^{-1}$ exists, then $\det(A^{-1}) = \det(A)^{-1}$.

  \item If $B$ is obtained by multiplying a single row of $A$ by $4$,
    then $\det(B) = 4\det(A)$.

  \item For an $n\times n$-matrix $A$, we have
    $\det(-A) = (-1)^n\det(A)$.

  \item If $A$ is a real $n\times n$-matrix, then
    $\det(A^TA) \geq 0$.

  \item If $A^k = 0$ for some positive integer $k$, then
    $\det(A) = 0$.

  \item If $A\vect{x} = 0$ for some $\vect{x} \neq 0$, then
    $\det(A) = 0$.
  \end{enumerate}
  \begin{sol}
    \begin{enumerate}
    \item False. Consider $\begin{mymatrix}{rrr}
        1 & 1 & 0 \\
        1 & 2 & 1 \\
        0 & 1 & 1 \\
      \end{mymatrix}$.
    \item True.
    \item False.
    \item False.
    \item True.
    \item True.
    \item True.
    \item True.
    \item True.
    \item True.
    \end{enumerate}
  \end{sol}
\end{ex}
\end{samepage}


\clearpage
\newpage

\section{1D FMM, $1/(x-x')$}
\label{sec:1dfmm}

Here we give the pseudo-code and algorithm of \cite{dutt1996fast} that can be used to accelerate the computation of the 1-dimensional kernel $1/(x-x')$ that appears in the fast scalar spherical transform in Section \ref{sec:fastscasphfilt}. Specifically, this deals with the computation of
\begin{equation}
f(x_j) = \sum_k^N \dfrac{a_k}{x_j - x_k} \label{eqdir}
\end{equation}

This kernel is similar to the kernel for electrostatic potentials (e.g., $1/\vert x - x'\vert$), except with the important difference that the denominator retains its sign.

Let $M$ be the number of observation points $x_j$, $N$ be the number of source points, $x_k$, that have amplitudes $a_k$ which can be real or complex. In matrix form, this requires $O(NM)$ operations to perform the matrix vector multiplication.  The algorithm in \cite{dutt1996fast} reduces this to $O(Np + Mp)$ where $p$ is the number of expansion terms needed to achieve a specific accuracy.

This algorithm is based on the observation that the far-field expansions of local groups of sources under this kernel can be aggregated up a binary tree-structure, expanded about another center at fixed cost, then dissaggregated. It exploits the properties of Chebyshev polynomials on the domain $x = [-1,1]$ in order to accomplish this procedure with pre-defined precision.  Two versions of the algorithm are given in \cite{dutt1996fast}; we repeat the first algorithm here; the second uses SVD to accelerate the computation.

For $x_j$ and $x_k$ in the range $x = [a, b]$, the inputs can be rescaled with the affine transformation
\begin{eqnarray}
x' &=& \dfrac{2}{b-a}(x + a) -1 \label{af1} \\
x &=& \dfrac{b-a}{2}(x' + 1) - a \label{af2}
\end{eqnarray}

Using \eqref{af2} in \eqref{eqdir} yields $x_j'$ and $x_k'$ on $x = [-1, 1]$.
\begin{equation}
f(x_j') = \dfrac{2}{b-a}\sum_k^N \dfrac{a_k}{x_j' - x_k'}
\end{equation}

Therefore only the source and observation points need to be transformed using \eqref{af1}, and the scale factor $2/(b-a)$ applied to the amplitudes, the algorithm is otherwise equivalent.

%\subsection{Basic Algorithm}

\paragraph{Setup}
\begin{itemize}
  \setlength{\itemsep}{1pt}
  \setlength{\parskip}{0pt}
  \setlength{\parsep}{0pt}
\item  $p$ is an integer that is the size of the Chebyshev expansions.  $p$ is the same for all expansions on $x = [-1,1]$ and given by $p = \lceil -\log_5(\epsilon) \rceil$ where $0 < \epsilon < 1$ is the desired precision.

\item $t_1, ..., t_p$ are the Chebyshev nodes of order $p$ on $x = [-1,1]$, given by
\begin{equation}
t_i = \cos\left(\dfrac{2i-1}{p}\dfrac{\pi}{2}\right) \label{fmm1prepeqfirst}
\end{equation}

\item The expansion functions are given by the polynomials
\begin{equation}
u_j(t) = \prod_{\substack{k=1 \\ k \neq j}}^p \dfrac{t-t_k}{t_j-t_k}
\end{equation}

\item The far-field due to sources in $x = [x_o-r,x_o+r]$ is given by
\begin{eqnarray}
f_{\textrm{far}}(x) &=& \sum_j^p \Phi_{j}  u_j\left(\dfrac{3r}{x - x_o}\right) \\
\Phi_{j} &=& \sum_k^p a_k \dfrac{t_j}{3r - t_j(x_k - x_o)} \label{phi1}
\end{eqnarray}

\noindent where $\Phi_{k}$ are the far-field expansion coefficients.

\item The local field in $x = [y_o-r,y_o+r]$ is given by
\begin{eqnarray}
f_{\textrm{loc}}(x) &=& \sum_j^p \Psi_{j}  u_j\left(\dfrac{x - y_o}{r}\right) \label{psi1}\\
\Psi_{j} &=& \sum_k^p  \dfrac{a_k}{rt_j - (x_k - y_o)}
\end{eqnarray}

\noindent where $\Psi_{k}$ are the local field expansion coefficients.

\item $s$ is an integer and is the number of points in each subinterval at the finest level. It is recommended to set $s \approx 2p$.

\item $nlevs = \lceil \log_2(N/s) \rceil$ is the level of finest subinterval and the total number of levels.

\item $\Phi_{l,i}$ are the $p$-term far-field coefficients at level $l$, subinterval $i$.

\item $\Psi_{l,i}$ are the $p$-term local-field coefficients at level $l$, subinterval $i$.

\item $M_L$ and $M_R$ are $p \times p$ matrices that aggregate the far-field expansions of left and right subinterval (children) to a far-field expansion at the next higher level (parent).  These are given by (\cite[Eq. 78]{dutt1996fast}  has a typo)
\begin{eqnarray}
M_L(i,j) &=& u_j\left(\dfrac{3 t_i}{6  + t_i}\right) \\
M_R(i,j) &=& u_j\left(\dfrac{3 t_i}{6  - t_i}\right)
\end{eqnarray}

\item $S_L$ and $S_R$ are $p \times p$ matrices that disaggregate the local expansion of a parent to its left and right children
\begin{eqnarray}
S_L(i,j) &=& u_j\left(\dfrac{t_i -1 }{2}\right) \\
S_R(i,j) &=& u_j\left(\dfrac{t_i + 1}{2}\right)
\end{eqnarray}

\item $T_1$, $T_2$, $T_3$, $T_4$ are $p \times p$ matrices that translate the far-field expansions of the well-separated subdivisions to the local expansion of a subinterval at the same level.  The far-field subdivisions are separated from the local subinterval by $-3$, $-2$, $2$ and $3$ positions, respectively.

\begin{eqnarray}
T_1(i,j) &=& u_j\left(\dfrac{3 }{t_i - 6}\right) \\
T_2(i,j) &=& u_j\left(\dfrac{3 }{t_i - 4}\right) \\
T_3(i,j) &=& u_j\left(\dfrac{3 }{t_i + 4}\right) \\
T_4(i,j) &=& u_j\left(\dfrac{3 }{t_i + 6}\right)  \label{fmm1prepeqlast}
\end{eqnarray}

\end{itemize}

\paragraph{Algorithm psuedo-code}
\begin{enumerate}
\item Set the expansion size $p$, choose $s$, and compute $nlevs$.  Precompute Chebyshev coefficients and translation matrices.

\item Determine the far-field expansions at the finest level.

\textbf{do} $i = 1, ..., 2^{nlevs}$
\begin{addmargin}[1em]{2em}
Compute $p$-term far-field expansions $\Phi_{nlevs,i}$ using \eqref{phi1} due to sources at $x_k$ which lie in subinterval $i$ at level $nlevs$.
\end{addmargin}
\textbf{end}

\item Determine the $p$-term far-field expansion at each subinterval at every level by shifting and adding the far-field expansions of the subintervals children

\textbf{do} $l = nlevs-1, ..., 1$
\begin{addmargin}[1em]{2em}
\textbf{do} $i = 1, ..., 2^{l}$
\begin{addmargin}[1em]{2em}
$\Phi_{l,i} = M_L \cdot \Phi_{l+1,2i-1}  + M_R \cdot \Phi_{l+1,2i} $
\end{addmargin}
\textbf{end}
\end{addmargin}
\textbf{end}

\item Determine $p$-term local expansion at each subinterval at each level by 1) disaggregating the parent's local expansion, 2) adding the far-field translation to local translation of well-separated subintervals at the same level, but that have not been accounted for at the parent's level. (The equations for this step given in \cite{dutt1996fast} do not work as they appear, but the following does)

\textbf{do} $l = 1,...,nlevs-1$
\begin{addmargin}[1em]{1em}
\textbf{do} $i = 1, ..., 2^{l}$
\begin{addmargin}[1em]{1em}
$\Psi_{l+1,2i-1} = S_L \cdot  \Psi_{l,i} + T_2 \cdot  \Phi_{l+1,2i-3} + T_3  \cdot \Phi_{l+1,2i+1} + T_4 \cdot  \Phi_{l+1,2i+2}$ \\
$\Psi_{l+1,2i} \quad = S_R  \cdot \Psi_{l,i} + T_1 \cdot  \Phi_{l+1,2i-3} + T_2  \cdot \Phi_{l+1,2i-2} + T_3  \cdot \Phi_{l+1,2i+2}$
\end{addmargin}
\textbf{end}
\end{addmargin}
\textbf{end}

\item Evaluate the local expansion at the finest level

\textbf{do} $i = 1, ..., 2^{nlevs}$
\begin{addmargin}[1em]{2em}
Evaluate $p$-term local expansions $\Psi_{nlevs,i}$ using \eqref{psi1} at points $x_j$ which lie in subinterval $i$ at level $nlevs$.
\end{addmargin}
\textbf{end}

\item Add the near-neighbor contributions directly

\textbf{do} $i = 1, ..., 2^{nlevs}$
\begin{addmargin}[1em]{2em}
For each point $x_j$ in subinterval $i$ at level $nlevs$, compute the contribution of all $x_k$ in subintervals $i-1$, $i$, $i+1$ using \eqref{eqdir}, and add the result to the local expansion already evaluated before.
\end{addmargin}
\textbf{end}

\end{enumerate}

\paragraph{Routines}
\mbox{}\\
\mbox{}\\
The 1D FMM is implemented with two routines \texttt{fmm1prep} and  \texttt{fmm1}.  \texttt{fmm1prep} is a preparatory function that precomputes the setup parameters of the algorithm and translation matrices using \eqref{fmm1prepeqfirst} through \eqref{fmm1prepeqlast}.  It takes as inputs observation points $x_j$ and source points $x_k$ on $x = [-1, 1]$, source amplitudes $a_k$ (real or complex), and precision $\epsilon$.  $s$ is optional, the default is $s = 2p$. The outputs are $M_L$, $M_R$, $S_L$, $S_R$, $T_1$, $T_2$, $T_3$, $T_4$, etc.  \texttt{fmm1} takes the outputs from \texttt{fmm1prep} and executes the algorithm given by the pseudo-code. The pair of routines is set up so that \texttt{fmm1} can be called with new source amplitudes for the same source and observation points and the same outputs of \texttt{fmm1prep}.

Both routines rely on linear indexing to bookkeep the values of variables. The total number of expansions needed, and thus total number of subdivisions on the binary tree, is

\begin{equation}
B = \sum_{i=1}^{nlevs} 2^i = 2^{nlevs + 1} - 2
\end{equation}

The expansion coefficients vectors $\Phi_{l,i}$ and $\Psi_{l,i}$ are stored on $p \times B$ arrays and accessed with column index

\begin{equation}
I(l,i) = 2^l - 2 + i
\end{equation}

\noindent for $ l \ge 1, i = 1,...,2^l $.  This is provided by the helper function \texttt{box2ind}, which has been replaced by inline computation in the code.  The basis functions, $u_j(t)$, are provided in the routine $\texttt{fmm1u}$.  Note, the expansions at the top-most level, $l=1$, are never used except to initialize one of the loops with zeros.

Because Matlab's matrix vector multiplication is highly optimized, it is actually faster for small problems to precompute the elements of the matrix and let Matlab do the computation directly.  This works to a point.  These routines require comparably no storage and are the only path forward for very large problems.

{\footnotesize
\VerbatimInput{\code/FastMultipoleMethod/fmm1/fmm1.m}
}

{\footnotesize
\VerbatimInput{\code/FastMultipoleMethod/fmm1/fmm1prep.m}
}

{\footnotesize
\VerbatimInput{\code/FastMultipoleMethod/fmm1/fmm1u.m}
}

{\footnotesize
\VerbatimInput{\code/FastMultipoleMethod/fmm1/box2ind.m}
}


\title{Laplace Approximation}

\subsection{Laplace Approximation}

(This tutorial follows the
\href{/tutorials/map}{Maximum a posteriori estimation} tutorial.)

Maximum a posteriori (MAP) estimation approximates the posterior $p(\mathbf{z} \mid \mathbf{x})$
with a point mass (delta function) by simply capturing its mode. MAP is
attractive because it is fast and efficient. How can we use MAP to construct a
better approximation to the posterior?

The Laplace approximation
\citep{laplace1986memoir}
is one way of improving a MAP estimate. The idea
is to approximate the posterior with a normal distribution centered at the MAP
estimate,
\begin{align*}
  p(\mathbf{z} \mid \mathbf{x})
  &\approx
  \text{Normal}(\mathbf{z}\;;\; \mathbf{z}_\text{MAP}, \Lambda^{-1}).
\end{align*}
This requires computing a precision matrix $\Lambda$. Derived from a
Taylor expansion, the Laplace approximation uses the Hessian of the
negative log joint density at the MAP estimate.
It is defined component-wise as
\begin{align*}
  \Lambda_{ij}
  &=
  \frac{\partial^2}{\partial z_i \partial z_j} -\log p(\mathbf{x}, \mathbf{z}).
\end{align*}
For flat priors (which reduces MAP to maximum likelihood), the
precision matrix is known as the observed Fisher information
\citep{fisher1925theory}.
Edward uses TensorFlow's automatic differentiation, making this
second-order gradient computation both simple and efficient to
distribute.

For more details, see the \href{/api/}{API} as well as its
implementation in Edward's code base.

\subsubsection{References}\label{references}


\subsection{Free Undamped Vibrations ($b = 0$)}
\noindent
In this case, our equation simplifies to
\begin{equation*}
	my'' + ky = 0
\end{equation*}
The two roots of ou auxiliary equation are
\begin{equation*}
	r = \pm i \sqrt{\frac{k}{m}} = \pm i\omega
\end{equation*}
So, our solution becomes
\begin{equation*}
	y = C_1\cos{(\omega t)} + C_2\sin{(\omega t)}
\end{equation*}
This is the same $\omega$ from physics that means angular frequency, so the same physics formulas apply, like $T = \frac{2\pi}{\omega}$.\\

\noindent
We can simplify this a bit further. If we think of the $\cos$ and $\sin$ components as being sides of a right triangle like so,
\begin{center}
	\includegraphics[width=0.5\textwidth]{./higherOrder/freeVibrs/triangle.png}
\end{center}
then we rewrite our equation as
\begin{equation*}
	y = A\left( \frac{C_1}{\sqrt{C_1^2 + C_2^2}}\cos{(\omega t)} + \frac{C_2}{\sqrt{C_1^2 + C_2^2}}\sin{(\omega t)} \right)
\end{equation*}
Note that since $\left(\frac{C_1}{A}\right)^2 + \left(\frac{C_2}{A}\right)^2 = 1$, we can rewrite these coefficients as $\cos{\phi}$ and $\sin{\phi}$ respectively where
\begin{equation*}
	\phi = \begin{cases}
		\arctan{(\frac{C_2}{C_1})} & C_1 > 0 \\
		\arctan{(\frac{C_2}{C_1})} + \pi & C_1 \leq 0
	\end{cases}
\end{equation*}
So, our equation becomes
\begin{equation*}
	y = A\left(\cos{(\omega t)}\cos{\phi} + \sin{(\omega t)}\sin{\phi}\right)
\end{equation*}
Using the $\cos$ angle addition formula
\begin{equation*}
	y = A\cos{(\omega t - \phi)}
\end{equation*}
\begin{center}
	\includegraphics[width=0.5\textwidth]{./higherOrder/freeVibrs/undampedfree.png}
\end{center}
As we can see, an undamped free vibration will simply oscillate back and forth without decay.

\ifodd\includeHigherOrderExamples\input{./higherOrder/freeVibrs/undampedFreeVibrs_example.tex}\fi


\subsection{Markov property}

For a process with the Markov property, only the current state matters for all probability distributions.

\(P(x_{t+n}|x_t)=P(x_{t+n}|x_t, x_{t-1}...)\)


\section{Spectral Analysis}

Sometimes it is useful to describe the properties of the time series
in a frequency domain. The spectrum is defined as
\[
f_{ab}(\lambda) = \frac{\sigma^2}{2\pi} \sum_{h = -\infty}^{\infty} \gamma_{ab}(h) \exp
( -i \lambda h)
\]

There is a one-to-one correspondence between the spectrum and the
autocovariance function
\[
\sigma^2 \gamma_{ab}(h) = \int_{-\pi}^{\pi} f(\lambda) \exp ( i\lambda h) d\lambda
\]

We call $|f_{aa}|^2$ the power spectrum. A natural way of estimating a
power
spectrum is using the periodogram which
is the modulus of the Fourier transform of the data
\[
I(\lambda) = \frac{1}{2\pi T} | \sum_{t=1}^T Y_t \exp (-i \lambda t) |^2
\]
We usually compute the periodogram at the Fourier frequencies
$\lambda_j = (2 \pi j)/T, j=1,\dots,T/2$. These have desirable
statistical properties

The periodogram is also useful for detecting periodicities
(deterministic ones) in the signal. It is a mathematical fact that if
the data $Y_1,...Y_T$ has a period $p$, the the periodogram will have
peaks at frequencies $\lambda = 2 \pi T/p$ and its multiples.

\centerline{\epsfig{figure=Plots/plot-10-04.ps,angle=270,width=\textwidth}}

\subsection{An Application}
Figured 4a and 4c shows recorded ECoG signal for two channels for
a subject that has received a sensory stimulus at some point
during the recording. A straightforward way of estimating the
spectrum of a stationary process is the periodogram
\[
I(\lambda) = \frac{1}{2\pi T} \left| \sum_t Y(t) \exp(i\lambda t)
\right|^2.\]
In Figures 4b and 4c the periodogram of this data is shown. Brain
researchers have speculated that the so-called $\alpha$ $(8-13
Hz.)$, $\beta$ $(15-25 Hz.)$, and $\gamma$ $(>30 Hz.)$ bands of
human brain signals can indicate functional activation of
sensorimotor cortex. Notice that the
periodogram exhibits a peak around frequencies 10 Hz., 20 Hz. and
60 Hz.. If we were to approximate the ECoG signal as a stationary
processes, we would describe it as having periodic components
around these frequencies. However, we are interested in learning
how the signal changes when the subjects are given a stimuli. Thus
it seems more appropriate to model the signal as a non-stationary
processes and study the time-varying spectral density.

A straightforward estimate of a time-varying spectral density
would be the dynamic periodogram. Basically, for each time $t_0$
we consider a window around that point of size $h(t_0)$ and
estimate a weighted periodogram
\[
I(t_0;\lambda) = \frac{1}{2\pi h(t_0)} \left| \sum_t
w\left(\frac{t-t_0}{h(t_0)}\right) Y(t) \exp(i\lambda t) \right|^2.
\]
Figures 4c and 4e show the estimated time-varying spectral
densities for the signals of channels 19 and 20 (lighter colors
represent higher values)
The figure seems to suggest that the $\alpha$ band changes power
and frequency after the stimulus (time 0).

\centerline{\epsfig{figure=Plots/plot-10-05.ps,width=\textwidth}}
\centerline{\epsfig{figure=Plots/plot-10-06.ps,width=\textwidth}}
\centerline{\epsfig{figure=Plots/plot-10-07.ps,width=\textwidth}}