\section{Basically a recap of what we did last time}
PSet 1 due today. Last time, we showed:
\begin{theorem}
There exists a map $S_p(X)\times S_q(Y)\to S_{p+q}(X\times Y)$ that is:
	\begin{itemize}
	\item Natural, in the sense that if $f:X\to X^\prime$ and $g:Y\to Y^\prime$, and $a\in S_p(X)$ and $b\in S_p(Y)$ so that $a\times b\in S_{p+q}(X\times Y)$, then $f_\ast(a)\times g_\ast(b)=(f\times g)_\ast(a\times b)$.
	\item Bilinear, in the sense that $(a+a^\prime)\times b=(a\times b)+(a^\prime\times b)$, and $a\times (b+b^\prime)=a\times b+a\times b^\prime$.
	\item The Leibniz rule is satisfied, i.e., $\partial(a\times b)=(\partial a)\times b + (-1)^{|a|}a\times \partial b$.
	\item Normalized, in the following sense. Let $x\in X$ and $y\in Y$. Write $i_x:Y\to X\times Y$ sending $y\mapsto (x,y)$, and write $i_y:X\to X\times Y$ sending $x\mapsto (x,y)$. If $b\in S_q(Y)$, then $c^0_x\times b=(i_x)_\ast b\in S_q(X\times Y)$, and if $a\in S_p(X)$, then $a\times c^0_y=(i_y)_\ast a\in S_p(X\times Y)$.
	\end{itemize}
\end{theorem}
We were a little hasty in the end, so we're going to recall some things.
\begin{proof}[Proof sketch]
There were two steps.
\begin{enumerate}
\item It's enough to define $\iota_p\times \iota_q\in S_{p+q}(\Delta^p\times \Delta^q)$ where $\iota_n:\Delta^n\to\Delta^n$ is the identity, because every other simplex is $\iota_n$ pushed forward, and the cross product is supposed to be natural.
\item Induction on $p+q$. The first thing we use is the Leibniz rule, namely $\partial(\iota_p\times\iota_q) = (\partial\iota_p)\times\iota_q + (-1)^p\iota_p\times\partial\iota_q$. Is this a boundary? A necessary thing for anything to be a boundary is that it's a cycle. But because $\Delta^p\times\Delta^q$ is homeomorphic to a star-shaped region, it's contractible - therefore $ H_{p+q-1}(\Delta^p\times\Delta^q)=0$, a \emph{sufficient} condition for something to be a boundary is that it's a cycle! We showed that $\partial(\iota_p\times\iota_q)$ is a cycle, and therefore a boundary. We just need to choose \emph{some} class $[a]$ in $S_{p+q}(\Delta^p\times\Delta^q)$ such that $\partial([a])=(\partial\iota_p)\times\iota_q + (-1)^p\iota_p\times\partial\iota_q$, and this works.
\begin{enumerate}
\item Naturality is left to the reader.
\item Let's check the Leibniz rule. Let $\sigma:\Delta^p\to X$ and $\tau:\Delta^q\to Y$. What's $\partial(\sigma\times\tau)$? We start by asking how we define $\sigma\times \tau$. Well, this is just $\sigma_\ast\iota_p\times\tau_\ast\iota_q$. Because of naturality, this is just $(\sigma\times\tau)_\ast(\iota_p\times\iota_q)$. This means that $\partial(\sigma\times\tau)=\partial((\sigma\times\tau)_\ast(\iota_p\times\iota_q))$. Now, we can use the naturality of the boundary map to see that this is just $(\sigma\times\tau)_\ast\partial(\iota_p\times\iota_q)$, which we can expand as:
\begin{align*}
(\sigma\times\tau)_\ast\partial(\iota_p\times\iota_q)& =(\sigma\times\tau)_\ast((\partial\iota_p)\times\iota_q + (-1)^p\iota_p\times\partial\iota_q) \\
& = \sigma_\ast(\partial\iota_p)\times\tau_\ast\iota_q + (-1)^{p}(\sigma_\ast\iota_p\times\tau_\ast(\partial\iota_q))\\
& = \partial(\sigma_\ast\iota_p)\times\tau_\ast\iota_q + (-1)^{p}(\sigma_\ast\iota_p\times\partial(\tau_\ast\iota_q))\\
& = \partial\sigma\times\tau + (-1)^{p}\sigma\times\partial\tau
\end{align*}
\end{enumerate}
\end{enumerate}
\end{proof}
A key fact in this whole thing is that $ H_{p+q-1}(\Delta^p\times\Delta^q)=0$. This method of proof, namely of reducing to things that have zero homology (aka acyclic spaces) is called the \emph{method of acyclic models}.

What happens on the level of homology? Let's abstract a little bit. Suppose we have three chain complexes $A_\bullet$, $B_\bullet$, and $C_\bullet$, to be thought of as $S_\ast(X)$, $S_\ast(Y)$, and $S_\ast(X\times Y)$. Suppose we have maps $\times: A_p\times B_q\to C_{p+q}$ that satisfies bilinearity and the Leibniz formula. What does this induce in homology?
\begin{lemma}
This determines a bilinear map $ H_p(A)\times H_q(B)\xrightarrow{\times} H_{p+q}(C)$.
\end{lemma}
\begin{proof}
Let $[a]\in H_p(A)$ where $a\in Z_p(A)$ such that $\partial a=0$ (i.e., $a$ is a cycle). Let $[b]\in H_q(B)$ where $b\in Z_q(A)$ such that $\partial b=0$. We want to define $[a]\times [b]\in H_{p+q}(C)$. We hope that $[a]\times [b]=[a\times b]$. We need to check that $a\times b$ is a cycle; let's check. By Leibniz, $\partial(a\times b)=\partial a\times b+(-1)^pa\times\partial b$. Because $a,b$ are boundaries, this is zero. We still need to check that this thing is well-defined. Let's pick another $[a^\prime]=[a]$ and $[b^\prime]=[b]$. We want $[a\times b]=[a^\prime\times b^\prime]$. In other words, we need that $a\times b$ differs from $a^\prime\times b^\prime$ by a boundary. We can write $a^\prime=a+\partial\overline{a}$ and $b^\prime=b+\partial\overline{b}$. What's $a^\prime\times b^\prime$? It's:
	\begin{equation*}
	a^\prime\times b^\prime=(a+\partial\overline{a})+(b+\partial\overline{b})
	= a\times b+\left(a\times\partial\overline{b} + (\partial\overline{a})\times b+(\partial\overline{a})\times(\partial\overline{b})\right)
	\end{equation*}
But, well, $\partial(a\times\overline{b})=\partial a\times\overline{b}+(-1)^pa\times\partial\overline{b}=(-1)^pa\times\partial\overline{b}$, and $\partial(\overline{a}\times b)=\partial\overline{a}\times b$, and $\partial(\overline{a}\times\partial\overline{b})=\partial\overline{a}\times\partial\overline{b}$. This means that $a^\prime\times b^\prime=a\times b+\partial((-1)^{-p}(a\times \overline{b}) + \overline{a}\times b + \overline{a}\times\partial\overline{b})$. They differ by a boundary, so it's well-defined.

The last step is to check bilinearity, which is left to the reader.
\end{proof}
This gives the following result.
\begin{theorem}
There is a map $ H_p(X)\times H_q(Y)\to H_{p+q}(X\times Y)$ that's natural, bilinear, and normalized. This map is also \emph{unique} (unlike the map $S_p(X)\times S_q(Y)\to S_{p+q}(X\times Y)$, which isn't unique because we there are uncountably many choices of $\iota_p\times\iota_q$, all differing by a boundary), because the map $\times:S_p(X)\times S_q(Y)\to S_{p+q}(X\times Y)$ is unique up to chain homotopy!
\end{theorem}
Let's go back to homotopy invariance. Recall that if $f_0\sim f_q:X\to Y$, then $f_{0,\ast}=f_{1,\ast}:S_\ast(X)\to S_\ast(Y)$. We proved this by showing that this reduces to showing that the two inclusions $i_0,i_1:X\to X\times I$ induce the same map on $S_\ast(X)\to S_\ast(X\times I)$. The chain homotopy $h_X:i_{0,\ast}\sim i_{1,\ast}$ is defined as follows. Let $c\in S_p(X)$. We need to give an element of $S_{p+1}(X\times I)$, so if $\iota:\Delta^1\to I$ is the obvious map, we just define $h_X(c)=(-1)^p c\times\iota$. Let's check that.

Let's compute $\partial h_Xc$. This is $\partial((-1)^p c\times \iota)=(-1)^p\partial(c\times\iota)$, which, expanded out, is:
	\begin{equation*}
	\partial((-1)^p c\times \iota)=(-1)^p\partial(c\times\iota)=(-1)^p(\partial c)\times\iota+(-1)^{2p}c\times\partial\iota
	\end{equation*}
Well, $\partial\iota=c_1^0-c_0^0\in S_0(I)$, so this is equal to $(-1)^p(\partial c)\times\iota+c\times c^0_1 - c\times c_0^0$. But $c\times c^0_1=(i_1)_\ast c-(i_0)_\ast c$. Therefore, this is $(-1)^p(\partial c)\times\iota + (i_1)_\ast c-(i_0)_\ast c$. On the other hand, what's $h_X\partial c=(-1)^{p-1}(\partial c)\times\iota$. Let's add them together:
	\begin{equation*}
	\partial h_Xc+h_X\partial c=(-1)^p(\partial c)\times\iota + (i_1)_\ast c-(i_0)_\ast c+(-1)^{p-1}(\partial c)\times\iota=(i_1)_\ast c - (i_0)_\ast c
	\end{equation*}
So this is, by definition, a chain homotopy!

I just want to mention that there's an explicit choice of $\iota_p\times\iota_q$. This is called the Eilenberg-Zilber chain. You're highly encouraged to think about this yourself. We're going to consider $\Delta^{p+q}\to\Delta^p\times\Delta^q$. They're all affine maps, sending vertices to pairs of vertices. We're going to think of an ordered map $\omega:[p+q]\to[p]\times[q]$. We can complete the diagram to get:
	\begin{equation*}
	\xymatrix{ & [p]\\
	[p+q]\ar[ur]^{pr_2\omega}\ar[dr]_{pr_1\omega}\ar[r]^\omega & [p]\times[q]\ar[u]^{pr_2}\ar[d]^{pr_1}\\
	 & [q]}
	\end{equation*}
We also want $\omega$ to be injective (which requires that it takes $(0,0)$ to $(p,q)$). We can draw out a ``staircase'' in the $p\times q$ grid, and the area under the staircase defined by $\omega$ is denote $A(\omega)$. Define $\iota_p\times\iota_q=\sum(-1)^{A(\omega)}\overline{\omega}$ where $\overline{\omega}$ is the corresponding affine map $\Delta{p+q}\to\Delta^p\times\Delta^q$. It's combinatorially annoying to check that this satisfies the conditions of the theorem, but it's a good exercise to check it out. It's in a paper by Eilenberg-Moore.


\section{Appendix}

\subsection{Definitions}
\begin{definition}{(Scaled inverse $\chi^2$ distribution).}\label{def:scaledInverseChi}
  Let $\nu > 0$ and $\tau^2 > 0$ be parameters representing degrees of freedom and scale, respectively. The family of \emph{scaled inverse $\chi^2$ distributions} is characterized by its probability density function, namely
  \begin{align*}
    p(x) \propto x^{-(1 + \nu / 2)} \EXP{\frac{-\nu \tau^2}{2 x}} \quad \text{for} \, x \in (0, \infty) \,,
  \end{align*}
  where the constant of integration is ignored for clarity.
  We write $X \sim \scaledInvChi{\nu, \tau^2}$ to denote that the random variable $X$ follows a scaled inverse $\chi^2$ distribution with parameters $\nu$ and $\tau^2$.
\end{definition}

\begin{definition}{(Conjugate prior).}\label{def:conjugate_prior}
Let the likelihood $p(y \mid \theta)$ be given and assume that the prior distribution $p(\theta)$ is a member of some family $\mathcal{F}$ of probability distributions.
We say that $p(\theta)$ is a \emph{conjugate prior} if the posterior $p(\theta \mid y)$ is also a member of $\mathcal{F}$.
\end{definition}

\subsection{Figures}

\begin{figure}[H]
\begin{center}
\begin{tikzpicture}%
  [vertex/.style={circle,draw=black,fill=white, minimum size=1cm},
  node distance=2.5cm,
  >=latex,
  on grid]
  \node[vertex] (phi) {$\phi$};
  \node[rectangle, draw=black, minimum size=0.9cm,left=2cm of phi] (zeta) {$\zeta$};
  \node[vertex,below left=1.5cm and 2cm of phi] (theta1) {$\theta_1$};
  \node[vertex,below right=1.5cm and 2cm of phi] (thetaJ) {$\theta_J$};
  \node[below=1.5cm of phi] (dots1) {$\dots$};
  \node[rectangle, draw=black, minimum size=0.9cm, left=2cm of theta1] (u1) {$u_1$};
  \node[rectangle, draw=black, minimum size=0.9cm, right=2cm of thetaJ] (uJ) {$u_J$};
  \node[vertex,below=2cm of theta1] (y1) {$y(1)$};
  \node[vertex,below=2cm of thetaJ] (yJ) {$y(J)$};
  \node[rectangle, draw=black, minimum size=0.9cm, left=2cm of y1] (x1) {$x(1)$};
  \node[rectangle, draw=black, minimum size=0.9cm, right=2cm of yJ] (xJ) {$x(J)$};
  \node[below=3.5cm of phi] (dots1) {$\dots$};
  \draw[->]
    (zeta) edge (phi)
    (phi) edge (theta1)
    (phi) edge (thetaJ)
    (u1) edge (theta1)
    (uJ) edge (thetaJ)
    (theta1) edge (y1)
    (thetaJ) edge (yJ)
    (x1) edge (y1)
    (xJ) edge (yJ);
\end{tikzpicture}
\end{center}
\caption{A generic two-level Bayesian hierarchical model depicted as a directed acyclical graph modeling generic observations $y(j)$ in groups $j = 1,\mydots, J$. Circled parameters denote random quantities while parameters contained in squares denote fixed quantities.}
\label{fig:group_sem}
\end{figure}
\subsection{Proofs}

\begin{proof}[Derivation of Results in Table \ref{tab:comp_uniform_bay_ml}.]
  For the normal data problem at hand the derivation of the maximum likelihood estimators, $\bar{y}$ for $\mu$ and $(n-1) s^2 / n$ for $\sigma^2$, is well known.
  We note that the "ML Variance" denotes the theoretical variance of the estimators.
  Further, the moments of the scaled inverse $\chi^2$ distributions are available in closed form.
  In particular, if $X \sim \scaledInvChi{\nu, \tau^2}$, then $\Exp{X} = \nu \tau^2 / (\nu - 2)$ and $\var{X} = 2 \nu^2 \tau^4 / ((\nu - 2)^2 (\nu - 4))$.

  Let us first consider the parameter $\mu$.
  As the t-distribution is parameterized over its mean and variance we can simply read off these values.
  Further, as the t-distribution is symmetric the MAP is equal to its mean.

  Let us now consider the parameter $\sigma^2$.
  Plugging in the respective parameters, we get
  \begin{align*}
    \Exp{\sigma^2 \mid y} = \frac{n - 1}{n - 3} \sigma^2
  \end{align*}
  directly, and
  \begin{align*}
    \var{\sigma^2 \mid y} = \frac{2 (n-1)^2}{(n-3)^2(n-5)} \sigma^4 \,.
  \end{align*}
  To get the MAP for $\sigma^2$ we need to maximize the posterior of $\sigma^2$.
  Note that we can drop any integration constants, i.e. we need to solve
  \begin{align*}
    \maximize_{x > 0} \left\{x^{-(1 + \frac{n-1}{2})} \EXP{\frac{-(n-1) s^2}{2 x}}\right\} \,,
  \end{align*}
  where the term in curly brackets is just the posterior density of $\sigma^2$ evaluated at $x$.
  Using that the constant of integration is positive we can readily see that the posterior is concave.
  Hence, differentiating with respect to $x$ and setting this to zero yields the desired result.
\end{proof}

\begin{proof}[Derivation of Results in Table \ref{tab:comp_conjugate_bay_ml}.]
  The proof from above applies here by mutatis mutandis.
\end{proof}

\begin{remark}
The subsequent proofs presented here follow \citet{gelmanbda04}; however, we contribute detailed remarks.
\end{remark}

\begin{proof}[Proof of Proposition \ref{prop:posterior_uniform}.]
  Consider first the object $\mu \mid \sigma^2, y$.
  We get
  \begin{align*}
    p(\mu \mid \sigma^2, y) \propto p(y \mid \mu, \sigma^2) p(\mu \mid \sigma^2) \propto p(y \mid \mu, \sigma^2) \,,
  \end{align*}
  where the last step follows as the priors are assumed to be independent.
  Note then
  \begin{align*}
    p(\mu \mid \sigma^2, y) &\propto \EXP{-\frac{1}{\sigma^2}\sum_i (y_i - \mu)^2} = \EXP{-\frac{n}{\sigma^2} \frac{1}{n}\sum_i (y_i^2 - 2y_i \mu + \mu^2)}\\
    &=\EXP{-\frac{n}{\sigma^2} (\bar{y^2} - 2 \bar{y} \mu + \mu^2)} \propto
    \EXP{-\frac{1}{\sigma^2 / n} (\mu - \bar{y})^2} \,,
  \end{align*}
  where $\bar{y^2} = \frac{1}{n}\sum_i y_i^2$ and the last step is only proportional as we switch $\bar{y^2}$ for $\bar{y}^2$.
  Note that proportionality here is with respect to $\mu$.
  We thus get $\mu \mid \sigma^2, y \sim \normal{\bar{y}, \sigma^2/n}$ as our first intermediate result.

  Consider now $\sigma \mid y$.
  As we already derived the joint posterior we can compute the marginal posterior of $\sigma^2$ by integrating out $\mu$.
  Note that $\sum_i (y_i - \mu)^2 = [(n-1)s^2 + n(\bar{y} - \mu)^2]$, where $s^2$ denotes the (unbiased) sample variance. Hence
  \begin{align*}
    p(\sigma^2 \mid y) &\propto \int p(\mu, \sigma^2 \mid y) \mathrm{d} \mu\\
    &\propto \int \sigma^{-(n+2)} \EXP{-\frac{1}{2\sigma^2}\sum_i (y_i - \mu)^2} \mathrm{d} \mu\\
    &= \sigma^{-(n+2)} \int \EXP{-\frac{1}{2\sigma^2}\sum_i (y_i - \mu)^2} \mathrm{d} \mu\\
    &= \sigma^{-(n+2)} \int \EXP{-\frac{1}{2\sigma^2}[(n-1) s^2 + n(\bar{y} - \mu)^2]} \mathrm{d} \mu\\
    &= \sigma^{-(n+2)} \EXP{-\frac{1}{2\sigma^2}[(n-1) s^2]} \int \EXP{\frac{1}{2\sigma^2/n}(\mu - \bar{y})^2} \mathrm{d} \mu\\
    &= \sigma^{-(n+2)} \EXP{-\frac{1}{2\sigma^2}[(n-1) s^2]} \sqrt{2 \pi \sigma^2 / n} \\
    &\propto (\sigma^2)^{-(n+1)/2} \EXP{-\frac{1}{2\sigma^2}[(n-1) s^2]} \,,
  \end{align*}
  where the second to last step follows simply by considering the constant of integration of the normal distribution of $\mu \mid \sigma^2, y$.
  Note that here we consider proportionality with respect to $\sigma^2$.
  By inspection we see that $\sigma^2 \mid y \sim \text{scaled-Inv-} \chi^2(n-1, s^2)$, which proves our first claim.

  To finish the proof we integrate the joint posterior over $\sigma^2$ to get the marginal posterior of $\mu$.
  We evaluate the integral by substitution using
  $z = \sfrac{a}{2 \sigma^2}$ with $a = (n-1)s^2 + n(\mu - \bar{y})^2$.

  Then,
  \begin{align*}
    p(\mu \mid y) &= \int_{(0, \infty)} p(\mu, \sigma^2 \mid y) \mathrm{d}\sigma^2\\
    &\propto  \int_{(0, \infty)} (\sigma^2)^{-(n+2)/2} \EXP{-\frac{1}{2\sigma^2}[(n-1) s^2 + n(\mu - \bar{y})^2]} \mathrm{d} \sigma^2\\
    &\propto \int_{(0, \infty)} (\sigma^2)^{-(n+2)/2}\EXP{-z} [(\sigma^2)^2 / a] \mathrm{d}z\\
    &= \int_{(0, \infty)} (\sigma^2)^{-(n-2)/2} / a \EXP{-z} \mathrm{d}z\\
    &= a^{-n/2}\int_{(0, \infty)} z^{(n-2)/2}\EXP{-z} \mathrm{d}z\\
    &= a^{-n/2} \, \Gamma(n/2)\\
    &\propto a^{-n/2}\\
    &= [(n-1)s^2 + n(\mu - \bar{y})^2]^{-n/2}\\
    &\propto \left[1 + \frac{1}{n-1} \frac{(\mu - \bar{y})^2}{s^2 / n}\right]^{-n/2} \,
  \end{align*}
  where $\Gamma$ denotes the gamma function (which is finite on the positive real numbers).
  This concludes the proof by implying that $\mu \mid y \sim t_{n-1}(\bar{y}, s^2/n)$,
\end{proof}

\begin{proof}[Proof of Proposition \ref{prop:posterior_conjugate}.]
Let us first state equation \ref{eq:conjugate_posterior} and the premise again.
We have to show that
\begin{align*}
  p(\mu, \sigma^2 \mid y) \propto& (\sigma^2)^{-\frac{3 + \nu_0 + n}{2}} \times\\
  & \times \EXP{-\frac{1}{2 \sigma^2} \left[\nu_0\sigma_0^2 + \kappa_0(\mu - \mu_0)^2 + (n-1)s^2 + n(\bar{y} - \mu)^2 \right]}
\end{align*}
is $\NormalscaledInvChi{\mu_n, \sigma_n^2/\kappa_n; \nu_n, \sigma_n^2}$ with $\nu_n = \nu_0 + n$, $\kappa_n = \kappa_0 + n$, $\mu_n =\frac{\kappa_0}{\kappa_0 + n}\mu_0 + \frac{n}{\kappa_0 + n}\bar{y}$, $\sigma_n^2 = \left[\nu_0 \sigma_0^2 + (n-1)s^2 + \frac{\kappa_0 n}{\kappa_0 + n} (\bar{y} - \mu_0)^2\right] /\nu_n$.
By definition of the normal-scaled-inverse-$\chi^2$ distribution $\nu_n = \nu_0 + n$ follows trivially.
Let us therefore consider the term in square brackets in the exponential.
We have to show that
\begin{align*}
  \left[\nu_0\sigma_0^2 + \kappa_0(\mu - \mu_0)^2 + (n-1)s^2 + n(\bar{y} - \mu)^2 \right] = \nu_n \sigma_n^2 + \kappa_n (\mu - \mu_n)^2 \,.
\end{align*}
Plugging in for $\sigma_n^2$ we get for the right-hand side
\begin{align*}
  \nu_n \sigma_n^2 + \kappa_n (\mu - \mu_n)^2 = \nu_0 \sigma_0^2 + (n-1)s^2 + \frac{\kappa_0 n}{\kappa_0 + n} (\bar{y} - \mu_0)^2 + \kappa_n (\mu- \mu_n)^2 \,.
\end{align*}
Therefore we only need to check
\begin{align*}
  \kappa_0(\mu - \mu_0)^2 + n(\bar{y} - \mu)^2 = \frac{\kappa_0 n}{\kappa_0 + n} (\bar{y} - \mu_0)^2 + \kappa_n (\mu- \mu_n)^2 \,.
\end{align*}
Expanding the right-hand side we get
\begin{align*}
\frac{\kappa_0 n}{\kappa_0 + n} &(\bar{y} - \mu_0)^2 + \kappa_n (\mu- \mu_n)^2\\
&=\frac{\kappa_0 n}{\kappa_n}\left[\bar{y}^2 - 2\bar{y}\mu_0 + \mu_0^2 \right] + \kappa_n \left[\mu^2 - 2\mu\mu_n + \mu_n^2 \right]\\
&=\frac{\kappa_0 n}{\kappa_n}\left[\bar{y}^2 - 2\bar{y}\mu_0 + \mu_0^2 \right] + \kappa_n \left[\mu^2 - 2\mu\frac{\kappa_0}{\kappa_n}\mu_0 - 2\mu\frac{n}{\kappa_n}\bar{y} + \frac{\kappa_0^2}{\kappa_n^2}\mu_0^2 + \frac{n^2}{\kappa_n^2} \bar{y}^2 + 2\frac{\kappa_0}{\kappa_n} \frac{n}{\kappa_n}\mu_0\bar{y} \right]\\
&=\frac{\kappa_0 n}{\kappa_n}\left[\bar{y}^2 - 2\bar{y}\mu_0 + \mu_0^2 \right] + \kappa_n \mu^2 - 2\mu \kappa_0 \mu_0 - 2\mu n\bar{y} + \frac{\kappa_0^2}{\kappa_n}\mu_0^2 + \frac{n^2}{\kappa_n} \bar{y}^2 + 2 \kappa_0 n \mu_0 \bar{y} / \kappa_n\\
&= \left(\kappa_0 \mu^2 -2\mu \kappa_0 \mu_0 \right) + \left(n \mu^2 - 2 n \mu \bar{y} \right) + \frac{\kappa_0 n}{\kappa_n}\bar{y}^2 + \frac{\kappa_0 n}{\kappa_n}\mu_0^2 + \frac{\kappa_0^2}{\kappa_n} \mu_0^2 + \frac{n^2}{\kappa_n} \bar{y}^2\\
&= \left(\kappa_0 \mu^2 -2\mu \kappa_0 \mu_0 \right) + \left(n \mu^2 - 2 n \mu \bar{y} \right) + \bar{y}^2 \left(\frac{\kappa_0 n}{\kappa_n} + \frac{n^2}{\kappa_n}\right) + \mu_0^2\left(\frac{\kappa_0 n}{\kappa_n} + \frac{\kappa_0^2}{\kappa_n}\right)\\
&= \left(\kappa_0 \mu^2 -2\mu \kappa_0 \mu_0 + \kappa_0 \mu_0^2 \right) + \left(n \mu^2 - 2 n \mu \bar{y} + n \bar{y}^2\right)\\
&=\kappa_0(\mu - \mu_0)^2 + n(\bar{y} - \mu)^2 \,,
\end{align*}
which was what we wanted.
\end{proof}

\begin{proof}[Proof of Proposition \ref{prop:marginal_posterior}.]
We continue to use the notation of the previous proof.
As in the proof of proposition \ref{prop:posterior_uniform} we first compute the distribution of $\mu \mid \sigma^2, y$ and then derive the posterior of $\sigma^2$ by integrating $\mu$ out.
Note that we actually defined $\mu \mid \sigma^2 \sim \normal{\mu_0, \sigma^2/\kappa_0}$.
Hence,
\begin{align*}
  p(\mu \mid \sigma^2, y) &\propto p(y \mid \mu, \sigma^2) p(\mu \mid \sigma^2)\\
  &\propto \EXP{-\frac{1}{2\sigma^2/n} (\mu - \bar{y})^2} \EXP{-\frac{1}{2\sigma^2/\kappa_0}(\mu - \mu_0)^2}\\
  &= \EXP{-\frac{1}{2\sigma^2}\left[n(\mu - \bar{y})^2 + \kappa_0(\mu - \mu_0)^2 \right]}\\
  &= \EXP{-\frac{1}{2\sigma^2}\left[\mu^2(\kappa_0 + n) - 2\mu(\kappa_0\mu_0 + n\bar{y}) + (\mydots) \right]}\\
  &= \EXP{-\frac{1}{2\sigma^2 / \kappa_n}\left[\mu^2 - 2\mu(\kappa_0\mu_0 + n\bar{y})/\kappa_n + (\mydots)/\kappa_n \right]}\\
  &= \EXP{-\frac{1}{2\sigma^2 / \kappa_n}\left(\mu - \mu_n^2\right) + (\mydots)}\\
  &\propto \EXP{-\frac{1}{2\sigma^2 / \kappa_n}\left(\mu - \mu_n^2\right)} \,,
\end{align*}
which implies that $\mu \mid \sigma^2, y \sim \normal{\mu_n, \sigma^2 / \kappa_n}$, where we used $(\mydots)$ to denote constants independent of $\mu$.

Now we can use this result as
\begin{align*}
  p(\sigma^2 \mid y) &= \int p(y, \sigma^2 \mid y) \mathrm{d}\mu\\
  &\propto \int (\sigma^2)^{-\frac{3 + \nu_n}{2}} \EXP{\frac{1}{2\sigma^2} \left[\nu_n \sigma_n^2 + \kappa_n(\mu_n - \mu)^2 \right]}\mathrm{d}\mu\\
  &\propto (\sigma^2)^{-\frac{3 + \nu_n}{2}} \int \EXP{\frac{1}{2\sigma^2} \nu_n \sigma_n^2}\EXP{\frac{1}{2\sigma^2 / \kappa_n} (\mu_n - \mu)^2 }\mathrm{d}\mu\\
  &\propto (\sigma^2)^{-\frac{3 + \nu_n}{2}}\EXP{\frac{1}{2\sigma^2} \nu_n \sigma_n^2} \int \EXP{\frac{1}{2\sigma^2 / \kappa_n} (\mu_n - \mu)^2 }\mathrm{d}\mu\\
  &\propto (\sigma^2)^{-\frac{3 + \nu_n}{2}}\EXP{\frac{1}{2\sigma^2} \nu_n \sigma_n^2} \sqrt{2 \pi \sigma^2 / \kappa_n}\\
  &\propto (\sigma^2)^{-(1 + \frac{\nu_n}{2})} \EXP{-\frac{1}{2\sigma^2}\nu_n\sigma_n^2}\,,
\end{align*}
from which we can conclude that $\sigma^2 \mid y \sim \scaledInvChi{\nu_n, \sigma_n^2}$.

We end the proof by deriving the marginal posterior of $\mu$ using an analogous approach as in the proof of Proposition \ref{prop:posterior_uniform}.
Define $a := \left[\nu_n\sigma_n^2 + \kappa_n(\mu_n - \mu)^2 \right]$. We solve for the posterior by integrating $\sigma^2$ out using the substitution $z = \frac{a}{2\sigma^2}$. Then
\begin{align*}
  p(\mu \mid y) &= \int_{(0, \infty)} p(\mu, \sigma^2 \mid y) \mathrm{d}\sigma^2\\
  &\propto \int_{(0, \infty)}(\sigma^2)^{-\frac{3 + \nu_n}{2}} \EXP{\frac{1}{2\sigma^2} \left[\nu_n \sigma_n^2 + \kappa_n(\mu_n - \mu)^2 \right]}\mathrm{d}\sigma^2\\
  &\propto \int_{(0, \infty)}(\sigma^2)^{-\frac{3 + \nu_n}{2}} \EXP{\frac{a}{2\sigma^2}}\mathrm{d}\sigma^2\\
  &\propto \int_{(0, \infty)}(a / 2z)^{-\frac{3 + \nu_n}{2}} \EXP{-z} \frac{a}{2 z^2} \mathrm{d}z\\
  &\propto \int_{(0, \infty)}a^{-\frac{3 + \nu_n}{2}}a z^{\frac{3 + \nu_n}{2}}z^{-2} \EXP{-z} \mathrm{d}z\\
  &= a^{-\frac{1 + \nu_n}{2}} \int_{(0, \infty)} z^{\frac{\nu_n - 1}{2}} \EXP{-z} \mathrm{d}z\\
  &= a^{-\frac{1 + \nu_n}{2}} \Gamma\left(\frac{\nu_n + 1}{2}\right)\\
  &\propto a^{-\frac{1 + \nu_n}{2}}\\
  &= \left[\nu_n \sigma_n^2 + \kappa_n(\mu_n - \mu)^2 \right]^{-\frac{1 + \nu_n}{2}}\\
  &= \left[\nu_n \sigma_n^2\left(1 + \frac{1}{\nu_n}\frac{(\mu_n - \mu)^2}{\sigma_n^2 / \kappa_n}\right) \right]^{-\frac{1 + \nu_n}{2}}\\
  &\propto \left[1 + \frac{1}{\nu_n}\frac{(\mu_n - \mu)^2}{\sigma_n^2 / \kappa_n} \right]^{-\frac{1 + \nu_n}{2}} \,,
\end{align*}
which concludes the proof by implying that $\mu \mid y \sim t_{\nu_n}(\mu_n, \sigma_n^2 / \kappa_n)$.
\end{proof}


% !TEX root = main.tex

%-------------------------------------------------
\section{Statistical models}\label{sec:statistical_models}

%%-----------------------------
%%\subsection{Statistics}
%
%Let $\mathbf{X}=(X_1,X_2,\ldots,X_n)$ be a random vector.
%
%\begin{definition}
%Any transformation $T:\R^n\to\R$ is called a \emph{statistic}.
%\end{definition}
%
%There are two interpretations:
%\ben
%\it
%$T(\boldX)$ is a random variable on $(\Omega,\prob)$,
%\[\begin{array}{rccl}
%T(\boldX):	& \Omega 	& \to 		& \R \\
%			& \omega	& \mapsto	& T\big[\boldX(\omega)\big]
%\end{array}\]
%\it
%$T$ is a random variable on $(\R^n,\prob_{\mathbf{X}})$,
%\[\begin{array}{rccl}
%T:	& \R^n 		& \to 		& \R \\
%	& \boldx	& \mapsto	& T(\boldx)
%\end{array}\]
%where $\prob_X(B) = \prob(\mathbf{X}\in B)$ is the probability measure induced on subsets of $\R^n$ by $\mathbf{X}$.
%\een
%
%For the latter interpretation to be useful, the distribution of $T$ (over subsets of $\R$) must be determined from the distribution of the random vector (over subsets of $\R^n$).

%-----------------------------
%\subsection*{Statistical models}

To estimate the underlying distribution $F_X$ from a random sample $X_1,X_2,\ldots,X_n$ of observations from the distribution of $X$, let us assume that $F_X$ belongs to some \emph{parametric family} of distributions.
%\[
%\mathcal{M} = \big\{F(x\,;\mathbf{\theta}) : \mathbf{\theta}\in\Theta\big\}
%\]
%where $\mathbf{\theta}$ is an unknown vector of parameters and $\Theta$ is the set of all such vectors.
%
\begin{definition}
A \emph{statistical model} is a parametric family of CDFs,
\[
\mathcal{M} = \big\{F(x\,;\mathbf{\theta}) : \mathbf{\theta}\in\Theta\big\}
\]
where $\mathbf{\theta}$ is a vector of parameters, and $\Theta$ is called the \emph{parameter space}.
%the set of all such vectors called the
%\it $\mathcal{M}$ is called a \emph{statistical model};
%\it $\Theta$ is called the \emph{parameter space}.
%\een
\end{definition}

\begin{example}
\bit
\it
The family of exponential distributions: \par
\qquad $\mathcal{M} = \big\{F(x;\lambda): \lambda>0\big\}$ where $F(x;\lambda)=1-e^{-\lambda x}$ for $x>0$.
\it
The family of uniform distributions: \par
\qquad$\mathcal{M} = \big\{F(x;a,b): a<b\big\}$ where $F(x;a,b)=\displaystyle\frac{x-a}{b-a}$ for $a\leq x\leq b$.
\eit
\end{example}

%The members of a statistical model $\mathcal{M}$ are indexed by the parameter $\theta$.
%Estimating the distribution of $X$ from a random sample of observations amounts to estimating $\theta$.

\begin{remark}
Let $\mathcal{M}=\big\{F(x\,;\mathbf{\theta}):\mathbf{\theta}\in\Theta\big\}$ be a statistical model and suppose that $F_X\in\mathcal{M}$. Then estimating $F_X$ amounts to estimating the ``true'' value of the parameter $\theta$.
\end{remark}

\begin{definition}
%Let $F_X\in\big\{F(x\,;\mathbf{\theta}):\mathbf{\theta}\in\Theta\big\}$.
\ben
\it To estimate a particular value for $\theta$ is called \emph{point estimation}.
\it To estimate a range of values for $\theta$ is called \emph{interval estimation}.
\it To assert whether or not $\theta$ lies in some range is called \emph{hypothesis testing}.
\een
\end{definition}

%-----------------------------
\subsection{Estimators}

%Let $\mathbf{X}=(X_1,X_2,\ldots,X_n)$ be a random vector.

\begin{definition}
Let $X$ be a random variable on $\Omega$ and let $\mathbf{X}=(X_1,X_2,\ldots,X_n)$ be a random sample from the distribution of $X$. Any transformation $T:\R^n\to\R$ of $\mathbf{X}$ is called a \emph{sample statistic}.
\end{definition}

There are two interpretations:
\ben
\it
$T(\boldX)$ is a random variable on $(\Omega,\prob)$,
\[\begin{array}{rccl}
T(\boldX):	& \Omega 	& \to 		& \R \\
			& \omega	& \mapsto	& T\big[\boldX(\omega)\big]
\end{array}\]
\it
$T$ is a random variable on $(\R^n,\prob_{\mathbf{X}})$,
\[\begin{array}{rccl}
T:	& \R^n 		& \to 		& \R \\
	& \boldx	& \mapsto	& T(\boldx)
\end{array}\]
where $\prob_{\mathbf{X}}(B) = \prob(\mathbf{X}\in B)$ is the distribution of $\mathbf{X}$.
\een

For the latter interpretation to be useful, the distribution of the sample statistic $T$ over subsets of $\R$ must be deduced from the distribution of the random sample $\mathbf{X}$ over subsets of $\R^n$.

\begin{definition}
\ben
%\it $\theta$ is called the \emph{true value} of the parameter.
\it A statistic $T(\mathbf{X})$ used to estimate an unknown parameter $\theta$ is called an \emph{estimator} of $\theta$.
%These are commonly denoted by $\hat{\theta}(\mathbf{X})$.
\it For any particular sample realisation $\mathbf{x}$ the value $T(\boldx)$ is called an \emph{estimate} of $\theta$.
\een
Note that estimators of $\theta$ are often denoted by $\hat{\theta}$.
\end{definition}

\begin{example}\label{exa:estimate_bernoulli}
A coin has an unknown probability $\theta$ of landing on heads.
%\bigskip
An appropriate statistical model for this experiment is the family of \emph{Bernoulli} distributions:
\[
\mathcal{M} = \big\{F(x\,;\theta):0\leq\theta\leq 1\big\}
\qquad\text{where}\quad
F(x;\theta) = \begin{cases}
0				& x < 0, \\
1-\theta		& 0 \leq x < 1, \\
1				& x \geq 1. \\
\end{cases}
\]
Let $\mathbf{X}=(X_1,X_2,\ldots,X_n)$ be a random sample from the $\text{Bernoulli}(\theta)$ distribution. An estimator for $\theta$ is provided by the following statistic, called the \emph{sample mean}:
\[\begin{array}{rccl}
T:	& \R^n 		& \to 		& \R \\
	& \boldx	& \mapsto	& \displaystyle\frac{1}{n}\sum_{i=1}^n x_i.
\end{array}\]
The distribution of $T$ is determined by the common distribution of the individual observations $X_i$. In this case $X_i\sim\text{Bernoulli}(\theta)$ so %$nT\sim\text{Binomial}(n,\theta)$ and
\[
%\prob(T = k/n) = \binom{n}{k}\theta^{k}(1-\theta)^{n-k}.
\prob\left(T = \frac{k}{n}\right) = \binom{n}{k}\theta^{k}(1-\theta)^{n-k}
\qquad\text{for $k=0,1,2,\ldots,n$}.
\]
\end{example}


\lab{Weightloss and Predator-Prey Models}{Weightloss and Predator-Prey Models}
\label{lab:Weightloss}

\section*{A Weightloss Model}
Here we will look at a model of weight change based on basic thermodynamics and kinematics.
The main idea behind weight change is simple.
If a person's energy intake is more than their energy expended, then they gain weight.
If their intake is less, then they lose weight.
Let the \emph{energy balance} $EB$ be the difference between \emph{energy intake} $EI$ and \emph{energy expenditure} $EE$, so that
\begin{equation}
\label{eqn:EB}
EB = EI - EE.
\end{equation}
When the energy intake is greater than the energy expended, the balance is positive and weight is gained.
Similarly, the balance is negative and weight is lost if the energy intake is less than the energy expended.

Body weight at time $t$ is the sum of the weight of fat and lean tissue; that is,  $BW(t) = F(t) + L(t).$
These quantities can be described by the compartmental model
\begin{subequations}
\label{eqn:compartment}
\begin{align}
\rho_F \dfrac{dF(t)}{dt} &= (1-p(t)) EB(t),\label{eqn:compartment:a}\\
\rho_L \dfrac{dL(t)}{dt} &= p(t) EB(t),\label{eqn:compartment:b}
\end{align}
\end{subequations}
where $p(t)$ and $1-p(t)$ represent the proportion of the energy balance ($EB(t)$) that results in a change in the quantity of lean or fatty tissue, respectively.
Constants $\rho_L$ and $\rho_F$ represent the energy density of lean and fatty tissue (about $1800$ and $9400$ kcal/kg).

Next we need to find expressions for $p(t)$ and $EB(t)$ in terms of $L$ and $F$ (the dependent variables), $PAL$ and $EI$ (possibly varying parameters), and other constant parameters.

 The proportion $p(t)$ will vary with $F$ and $L$; from Forbes' Law \cite{Fo.2} we have that
\begin{equation}
\label{eqn:forbes}
\dfrac{dF}{dL} = \dfrac{F}{10.4}.
\end{equation}
Hence,
\[
\dfrac{F}{10.4} = \dfrac{dF}{dL} = \dfrac{dF/dt}{dL/dt} = \dfrac{\dfrac{(1-p(t)) EB(t)}{\rho_F}}{\dfrac{p(t) EB(t)}{\rho_L}} = \dfrac{\rho_L}{\rho_F} \dfrac{1-p(t)}{p(t)}.
\]
Solving for $p(t)$ gives Forbes' equation
\begin{equation}
\label{eqn:Forbes2}
p(t) = \dfrac{C}{C+F(t)}\quad\mbox{where}\quad C=10.4\dfrac{\rho_L}{\rho_F}.
\end{equation}

We will use two expressions for energy expenditure (EE).
First, we have the formula
\begin{equation}
\label{eqn:EE0}
EE = PAL \times RMR,
\end{equation}
where $PAL$ is your physical activity level and $RMR$ your resting metabolic rate.
Your resting metabolic rate can be determined by using the Mifflin equation.
This equation is an estimate based on a population study and is widely used in the literature.
It takes into account your gender, age (A) in years, and height (H) in meters:
\begin{equation}
\label{eqn:RMR}
RMR = \begin{cases} 9.99 W + 625 H + 5 A + 5 & \mbox{if male}\\ 9.99 W + 625 H + 5 A -161 & \mbox{if female.}\end{cases}
\end{equation}
Your physical activity level can be determined by using the table below.
\begin{table}[h]
\begin{center}
\begin{tabular}{|l|l|}
\hline
1.40--1.69 & People who are sedentary and do not exercise regularly, spend \\
& most of their time sitting, standing, with little body displacement
\\
\hline
1.70--1.99 & People who are active, with frequent body displacement throughout  \\
& the day or who exercise frequently\\
\hline
2.00--2.40 & People who engage regularly in strenuous work or exercise for \\
& several hours each day\\
\hline
\end{tabular}
\caption{This is a rough guide for physical activity level (PAL).
% For more detailed estimates, see \cite{Heym} and Appendix \ref{PAL_appendix}.
}
\end{center}\label{tab:PAL_table}
\end{table}

The second expression for energy expenditure comes from decomposing more precisely the different ways that energy is expended:
\begin{equation}
\label{eqn:EE}
EE = \underbrace{\delta BW}_\text{\parbox{1cm}{physical\\activity}} + \underbrace{\beta_{tef} EI}_\text{\parbox{1cm}{thermic\\effect of\\eating}} + \underbrace{\beta_{at} EI + \gamma_F F + \gamma_L L + \eta_F \dfrac{dF}{dt} + \eta_L \dfrac{dL}{dt}  + K}_\text{resting metabolic rate (RMR)},
\end{equation}
where $\gamma_F = 22$ kcal/kg/d, $\gamma_L = 3.2$ kcal/kg/d, $\eta_F = 180$ kcal/kg, and $\eta_L = 230$ kcal/kg; see \cite{Hall.2, Hall.4}.
Further, we let $\beta_{tef}=0.10$ and $\beta_{at}=0.14$ denote the coefficients for the thermic effect of feeding and adaptive thermogenesis, respectively.
The parameter $\delta$ is the coefficient representing the amount of energy expended from physical activity per kilogram of body mass.
Notice that $\gamma_L$ is significantly larger than $\gamma_F$.
This means that lean tissue metabolizes energy much faster than fatty tissue.
As a result, there are instances where one may want to increase their lean body mass through resistance training so that they are better able to support a higher caloric intake without significant weight gain.
Finally, we remark that the constant $K$ can be tuned to an individual's body type directly through RMR and fat measurement, and is assumed to remain constant over time.

% Assumptions made/Areas to improve:
% include more accurate approximation of PAL (given in appendix), BMI (show to vary with race), account for variation in body type.

Thus, since the input $EI$ is assumed to be known, we can use \eqref{eqn:EE} and \eqref{eqn:Forbes2} to write \eqref{eqn:compartment} in terms of $F$ and $L$, thus allowing us to close the system of ordinary differential equations (ODEs).

Specifically, we have
\begin{align*}
RMR(t) = \frac{EE}{PAL}&= K + \gamma_F F(t) + \gamma_L L(t) + \eta_F \dfrac{dF}{dt} + \eta_L \dfrac{dL}{dt}  + \beta_{at} EI\\
% &= K + \gamma_F F(t) + \gamma_L L(t) + \dfrac{\eta_F}{\rho_F} (1-p(t)) EB(t) + \dfrac{\eta_L}{\rho_L} p(t) EB(t)  + \beta_{at} EI,\\
\dfrac{1}{PAL}\left(EE - EI + EI \right) &= K + \gamma_F F(t) + \gamma_L L(t) \\
&+ \left(\dfrac{\eta_F}{\rho_F} (1-p(t)) + \dfrac{\eta_L}{\rho_L} p(t) \right) EB(t) + \beta_{at} EI.\\
\left(\dfrac{1}{PAL}-\beta_{at}\right) EI &= K + \gamma_F F(t) + \gamma_L L(t) \\
&+ \left(\dfrac{\eta_F}{\rho_F} (1-p(t)) + \dfrac{\eta_L}{\rho_L} p(t) + \dfrac{1}{PAL}\right) EB(t).
\end{align*}
% Thus,
% \[
% \left(\dfrac{1}{PAL}-\beta_{at}\right) EI = K + \gamma_F F(t) + \gamma_L L(t) + \left(\dfrac{\eta_F}{\rho_F} (1-p(t)) + \dfrac{\eta_L}{\rho_L} p(t) + \dfrac{1}{PAL}\right) EB(t).
% \]
Solving for $EB(t)$ in the last equation yields
\begin{equation}
\label{eqn:EB2}
EB(t) = \dfrac{\left( \dfrac{1}{PAL} - \beta_{at} \right) EI - K - \gamma_F F(t) - \gamma_L L(t)}{\dfrac{\eta_F}{\rho_F} (1-p(t)) + \dfrac{\eta_L}{\rho_L} p(t) + \dfrac{1}{PAL}}.
\end{equation}
% To find $K$, we note that
% \begin{equation}
% \label{eqn:K}
% K = \left(\dfrac{1}{PAL}-\beta_{at}\right) EI - \gamma_F F(t) - \gamma_L L(t) - \eta_F \dfrac{dF}{dt} - \eta_L \dfrac{dL}{dt}  - \dfrac{1}{PAL} EB.
% \end{equation}
In equilibrium ($EB = 0$), this gives us
\begin{equation}
\label{eqn:K2}
K = \left(\dfrac{1}{PAL}-\beta_{at}\right) EI - \gamma_F F - \gamma_L L.
\end{equation}
Thus, for a subject who has maintained the same weight for a while, one can determine $K$ by using \eqref{eqn:K2}, if they know their average caloric intake and amount of fat (assume $L=BW-F$).
The function \li{weight_odesystem} in the following code implements \eqref{eqn:compartment}.

\begin{lstlisting}
from math import log
# Fixed Constants:
rho_F = 9400.
rho_L = 1800.
gamma_F = 3.2
gamma_L = 22.
eta_F = 180.
eta_L = 230.
C = 10.4  # Forbes constant
beta_AT = 0.14  # Adaptive Thermogenesis
beta_TEF = 0.1   # Thermic Effect of Feeding
K = 0

def forbes(F):
    C1 = C * rho_L / rho_F
    return C1 / (C1 + F)

def energy_balance(F, L, EI, PAL):
    p = forbes(F)
    a1 = (1. / PAL - beta_AT) * EI - K - gamma_F * F - gamma_L * L
    a2 = (1 - p) * eta_F / rho_F + p * eta_L / rho_L + 1. / PAL
    return a1 / a2

def weight_odesystem(t, y, EI, PAL):
    F, L = y[0], y[1]
    p, EB = forbes(F), energy_balance(F, L, EI, PAL)
    return np.array([(1 - p) * EB / rho_F , p * EB / rho_L])

def fat_mass(BW, age, H, sex):
    BMI = BW / H**2.
    if sex == 'male':
        return BW * (-103.91 + 37.31 * log(BMI) + 0.14 * age) / 100
    else:
        return BW * (-102.01 + 39.96 * log(BMI) + 0.14 * age) / 100

\end{lstlisting}

\begin{problem}
Consider the initial value problem
\begin{subequations}
\label{eqn:weight_prob1}
\begin{align*}
\rho_F \dfrac{dF(t)}{dt} &= (1-p(t)) EB(t),\\
\rho_L \dfrac{dL(t)}{dt} &= p(t) EB(t),\\
F(0) &= F_0, \\
L(0) &= L_0.
\end{align*}
\end{subequations}
The ode is given above by the function \li{weight_odesystem}.
To solve this IVP for a specific individual we need initial conditions $F_0$ and $L_0.$
The function \li{fat_mass} given earlier calculates $F_0$ based on an individual's body weight (kg), age, height (meters), and gender.
$L_0$ is then given by $L_0 = BW - F_0$.

Suppose a 38 year old female, standing 5'8'' and weighing 160 lbs, reduces her intake from 2143 to 2025 calories/day, and increases her physical activity from little to no exercise (PAL=1.4) to exercising to 2-3 days per week (PAL=1.5).
Find and graph the solution curve for this single-stage weightloss intervention over a period of 5 years.

Note the provided code requires quantities in metric units (kilograms, meters, days) while our graph is converted to units of pounds and days.
\end{problem}

\begin{figure}
\centering
\includegraphics[width=\textwidth]{weightloss.pdf}
\caption{The solution of the weight change model for problem 1.}
\label{fig:weightloss}
\end{figure}

\begin{problem}
Modify the preceding problem to handle a two stage weightloss intervention:
Suppose for the first 16 weeks intake is reduced from 2143 to 1600 calories/day and physical activity is increased from little to no exercise (PAL=1.4) to an hour of exercise 5 days per week (PAL=1.7).
The following 16 weeks intake is increased from 1600 to 2025 calories/day, and exercise is limited to only 2-3 days per week (PAL=1.5).

Find and graph the solution curve over a period of 32 weeks.
\end{problem}

\section*{Two Predator-Prey Models}
The Lotka-Volterra predator-prey model is a well-known
system of odes given by
\begin{align*}
	\frac{du}{dt} &= au - buv,\\
	\frac{dv}{dt} &= -cv + duv.
\end{align*}
In these odes $u$ and $v$ represent the prey and predator populations, respectively. Here $a$ represents the rate of growth of the prey, and $bu$ the amount of prey being eaten.
Similarly, $c$ represents the rate of natural predator death, and $du$ the growth of the predator population due to the quantity of prey eaten.

Let us look at the dynamics of this system.
First we note that there are exactly two equilibria (fixed points): either $(u,v) = (0,0)$ corresponding to the extinction of both species, or $(u,v) = (\frac{c}{d},\frac{a}{b})$.
Furthermore, from the odes we can see that if $v=0$ (there is an absence of any predators) then the population of prey will grow exponentially.

To get a better idea of the dynamics of this system we will graph its phase portrait.
We begin by nondimensionalizing the system to reduce the number of parameters:
Let $U = \frac{d}{c}u,$ $V = \frac{b}{a}v$, $\bar{t} = at,$ and $\alpha = \frac{d}{a}$.
Substituting into the original odes we obtain the nondimensional system of equations
\begin{align*}
	\frac{dU}{d\bar{t}} &= U(1-V),\\
	\frac{dV}{d\bar{t}} &= \alpha V (U-1).
\end{align*}
In the following code we plot the phase portrait.
To plot the direction field for the equations we use \li{numpy}'s \li{meshgrid} function and \li{matplotlib}'s \li{quiver} function.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{Lotka_Volterra.pdf}
\caption{The solution of the nondimensionalized Lotka-Volterra predator-prey equations with parameter $\alpha = 1/3$.
This solution has initial conditions $(u,v) = (3/4, 3/4)$.}
\label{fig:pred-prey_Lotka_Voterra}
\end{figure}

\begin{lstlisting}
from scipy.integrate import odeint
a, b = 0., 13.                    # (Nondimensional) Time interval for one 'period'
alpha = 1. / 3                    # Nondimensional parameter
dim = 2                           # dimension of the system
y0 = np.array([1 / 2., 1 / 3.])   # initial conditions

# Note: swapping order of arguments to match the calling convention
# used in the built in IVP solver.
def Lotka_Volterra(y, x):
    return np.array([y[0] * (1. - y[1]), alpha * y[1] * (y[0] - 1.)])

subintervals = 200
# Using the built in ode solver
Y = odeint(Lotka_Volterra, y0, np.linspace(a, b, subintervals))

# Plot the direction field
Y1, Y2 = np.meshgrid(np.arange(0, 4.5, .2), np.arange(0, 4.5, .2), sparse=True, copy=False)
U, V = Lotka_Volterra((Y1, Y2), 0)
Q = plt.quiver(Y1[::3, ::3], Y2[::3, ::3],  U[::3, ::3],  V[::3, ::3], pivot='mid', color='b', units='dots',width=3.)
# Plot the 2 Equilibrium points
plt.plot(1, 1, 'ok', markersize=8)
plt.plot(0, 0, 'ok', markersize=8)
# Plot the solution in phase space
plt.plot(Y[:,0], Y[:,1], '-k', linewidth=2.0)
plt.plot(Y[::10,0], Y[::10,1], '*b')

plt.axis([-.5, 4.5, -.5, 4.5])
plt.title("Phase Portrait of the Lotka-Volterra Predator-Prey Model")
plt.xlabel('Prey',fontsize=15)
plt.ylabel('Predators',fontsize=15)
plt.show()
\end{lstlisting}

\begin{problem}
Compute the solutions $(u,v)$ of
\begin{align*}
	\frac{dU}{d\bar{t}} &= U(1-V),\\
	\frac{dV}{d\bar{t}} &= \alpha V (U-1).
\end{align*}
for initial conditions $(1/2, 3/4)$, $(1/16, 3/4)$, and $(1/40, 3/4)$.
Add these solutions to the phase portrait of the Lotka-Volterra model.
Can you see any limitations of this model?
\end{problem}

\begin{figure}
\centering
\includegraphics[width=\textwidth]{Lotka_Volterra_Phase_Portrait.pdf}
\caption{The phase portrait for the nondimensionalized Lotka-Volterra predator-prey equations with parameters $\alpha = 1/3$.
The portrait includes the direction field, the two equilibrium points, and the graph of the solution with initial conditions $(u,v) = (3/4, 3/4)$. }
\label{fig:pred-prey_Lotka_Voterra_Phase_Portrait}
\end{figure}

We have already noticed that in the absence of predators, the Lotka-Volterra equations predict that the prey population will grow exponentially.
The logistic predator-prey equations change this dynamic by adding a term to give the prey population a carrying capacity $K$:
\begin{align*}
	\frac{du}{dt} &= au\left(1 -\frac{u}{K}\right) - buv,\\
	\frac{dv}{dt} &= -cv + duv.
\end{align*}
Let $U = \frac{u}{K},$ $V = \frac{b}{a}v$, $\bar{t} = at,$  $\alpha = \frac{dK}{a}$, and $\beta = \frac{c}{dK}$.
Then the nondimensional logistic equations are
\begin{align*}
	\frac{dU}{d\bar{t}} &= U(1-U-V),\\
	\frac{dV}{d\bar{t}} &= \alpha V (U-\beta).
\end{align*}

\begin{problem}
Compute the solutions $(u,v)$ of
\begin{align*}
	\frac{dU}{d\bar{t}} &= U(1-U-V),\\
	\frac{dV}{d\bar{t}} &= \alpha V (U-\beta).
\end{align*}
for initial conditions $(1/3, 1/3)$ and $(1/2, 1/5)$.
Do this for parameter values $\alpha, \beta = 1, .3$ and also for values $\alpha, \beta = 1, 1.1$.
Create a phase portrait for the logistic equations using both sets of parameter values.
Remember to plot the direction field, all equilibrium points, and the orbits of the solutions.
\end{problem}


\chapter{NP-Complete}

The complexity class known as \emph{NP-Complete} is a special subset
of $NP$, such that any problem in \emph{NP} can be translated to a
problem in \emph{NP-Complete} in polynomial time.

\section{CIRCUIT-SAT}

Given a boolean circuit, is it possible to provide a set of inputs
that cause the output to be \emph{True}?

In 1971, Cook proved that \emph{CIRCUIT-SAT} is \emph{NP-Complete}.
The proof is beyond the scope of these notes.  Cook showed that all
operations of a polynomial-sized Turing machine can be performed in
polynomial time using an instance of this problem.  In other words,
Cook showed that a Turing machine can be implemented using circuits
(surprise!!).

\section{Reduction}

Given at least one \emph{NP-Complete} problem, we can prove any other
problem $L$ is \emph{NP-Complete} by showing that $L \in NP$ and that
given an instance $x$ of a proven \emph{NP-Complete} problem we can
translate $x$ to an instance of $L$ in polynomial time, such that by
solving our generated instance of $L$, we can solve $x$.

The steps to do this are:

\begin{enumerate}
\item Show $L \in NP$.  We do this by first showing that the
  certificate for $L$ is polynomial with respect to the input, and
  that a certificate for $L$ can be verified in polynomial time.

\item Select a known \emph{NP-Complete} $L'$, ideally one that is
  similar to $L$.

\item Describe a polynomial time algorithm that maps any instance $x
  \in L'$ to an instance $f(x) \in L$.

\item Show that we can solve $x \in L'$ if and only if we can solve
  $f(x) \in L$.
\end{enumerate}

\section{SAT}

Given a boolean formula, are there values of the variables that cause
the formula to be \emph{True}?

First let us show that $SAT \in NP$ by using the truth values as a
certificate, which is clearly polynomial in size because it is
necessarily a subset of the input.  To verify, we evaluate the formula
given the truth values and return the output.

Next, we select \emph{CIRCUIT-SAT} from which to reduce to this
problem.

Then given an instance of \emph{CIRCUIT-SAT}, we transform the circuit
to a boolean formula such that the circuit is satisfiable if and only
if the formula is satisfiable.  We can do this easily by mapping gates
to boolean operators.

Since the instances of each problem are equivalent, it should be easy
to see that one is satisfiable if and only if the other is.

\section{3CNF-SAT}

Is a given boolean formula in \emph{3CNF} form satisfiable?  A boolean
formula is said to be in \emph{3CNF} form if there are no more than
$3$ variables in each clause, and within a clause there are only
\emph{OR} operators and between clauses there are only \emph{AND}
operators.

By the same proof as \emph{SAT} above, $3CNF-SAT \in NP$.

We reduce from \emph{SAT}.  Given an instance of \emph{SAT}, we use
equivalences to reduce all boolean operators to \emph{AND}, \emph{OR},
and \emph{NOT}.  Then we use DeMorgan's law to put the formula into
\emph{3CNF} form.

Since the instances of each problem are equivalent, it should be easy
to see that one is satisfiable if and only if the other is.

\section{CLIQUE}

Given a simple, undirected graph $G$, find a \emph{complete} subgraph
of $G$ of at least size $k$.

Let the certificate be the set of vertices on which there is a
complete subgraph.  Since this is necessarily a subset of the input
vertices, it must be of polynomial size.  We can verify the set of
vertices are complete by checking that every pair of vertices $i,j$
where $i \neq j$ are adjacent, and we can easily check if the set is
of size $k$ by counting them.

We reduce from \emph{3CNF-SAT}.  Given an instance $\Phi$ of \emph{3CNF-SAT}
with $k$ clauses, we would like to construct an instance of
\emph{CLIQUE}.  Our graph has $3k$ vertices, one for each variable in
each clause.  Clause $r$ has vertices $v^r_1,v^r_2,v^r_3$.  We put an
edge between $v^r_i$ and $v^s_j$ if and only if:

\begin{itemize}
\item $r \neq s$
\item their literals are consistent
\end{itemize}

\begin{lemma}
  If $\Phi$ is satisfiable, then $G$ must contain a clique of size $k$.
\end{lemma}

\begin{proof}
  Assume $\Phi$ is satisfiable, which is to say that $\Phi$ has a
  satisfying truth assignment.  We select the vertices corresponding
  to that truth assignment.  Since this truth assignment satisfies
  $\Phi$, there must exist one selected vertex in each box.  Each of
  these is connected to every vertex in a different box, therefore
  there exists a clique in $G$.
\end{proof}

\begin{lemma}
  If $G$ contains a clique of size $k$, then $\Phi$ is satisfiable.
\end{lemma}

\begin{proof}
  Assume $G$ contains a clique of size $k$.  One vertex of the clique
  must be in each box.  Assign all variables corresponding to the
  vertices of the clique to be true.  This must be a satisfying
  assignment because at least one true variable exists in each clause,
  and there are no inconsistent truth values.
\end{proof}

\begin{theorem}
  $\Phi$ is satisfiable if and only if $G$ contains a clique of size $k$.
\end{theorem}

\begin{proof}
  This follows from the above lemmas.
\end{proof}

\section{VERTEX-COVER}

Given a graph $G=(V,E)$, does there exist a set of vertices $V'$ of
size at most $k$ such that every edge in $E$ is incident to one vertex
in $V'$?

Let the certificate be the set of vertices $V'$.  This is a subset of
$V$, so it is polynomial with respect to the input.  We can verify
that $V'$ is of size at most $k$ by counting the vertices in $V'$.  We
can verify that the vertices in $V'$ form a vertex cover of $G$ by
checking that every edge in $E$ is incident to at least one vertex in
$V'$ in polynomial time.

We reduce from \emph{Clique}.  Let $\overline G = (V, \overline E)$ be
the complement of the graph $G$, where $\overline E = \{ (u,v) | (u,v)
\not \in E \}$.

\begin{lemma}
  $\overline G$ has a vertex cover of size $|V| - k$ if $G$ has a
  clique of size $k$.
\end{lemma}

\begin{proof}
  Assume $G$ has a clique $V'$ of size $k$.  $V \ V'$ is a vertex
  cover of $G$.  Let $(u,v) \in \overline E$, $u$ and $v$ are not both
  in $V'$.  Either $u$ or $v$ is in $V \ V'$.
\end{proof}

\begin{lemma}
  $G$ has a clique of size $k$ if $\overline G$ has a vertex cover of
  size $|V| - k$.
\end{lemma}

\begin{proof}
  Assume $\overline G$ has a vertex cover $V'$ of size $|V| - k$.  For
  all edges $(u,v) \in \overline E$, either $u$ is in $V'$ or $v$ is
  in $V'$.  For all vertices $u$,$v$ and $u \neq v$ then if neither
  $u$ nor $v$ is in $V'$ then $(u,v) \in E$.
\end{proof}

\begin{theorem}
  $G$ has a clique of size $k$ if and only if $\overline G$ has a
  vertex cover of size $|V| - k$.
\end{theorem}

\begin{proof}
  This follows from the preceding lemmas.
\end{proof}

\section{HAM-CYCLE}

Given a graph $G=(V,E)$, does there exist a simple cycle of at least
size $k$ that contains every vertex in $V$?

The proof that this is \emph{NP-Complete} is beyond the scope of these
notes, and not terribly interesting.

\section{TSP}

Given $G=(V,E)$ which is the complete graph on $n$ vertices and each
edge $(u,v)$ has a cost $c(u,v)$, does $G$ have a cycle which visits
each vertex (except the start) exactly once with total cost of at most
$k$?  We call such a cycle a \emph{TSP-tour}

Let the certificate be a permutation of the vertices which is
$\BigOh{|V| + 1}$, so polynomial in size.  We can verify a certificate
by summing the edges between each of the subsequent vertices in the
permutation and checking that the sum is at most $k$.

We reduce from \emph{HAM-CYCLE}.  Given a graph $G=(V,E)$ on which we
would like to solve \emph{HAM-CYCLE}, we create $G'=(V,E')$ where $E'
= \{ (u,v) | u,v \in V, u \neq v \}$.  And we create a cost function:

\begin{math}
  c(u,v) = \left\{
    \begin{array}{l l}
      0 & \text{if } (u,v) \in E \\
      1 & \text{if } (u,v) \not \in E \\
    \end{array} \right.
\end{math}

It should be easy to see that $G$ has a \emph{HAM-CYCLE} if and only
if $G'$ has a \emph{TSP-tour} of $k \leq 0$.

\section{SUBSET-SUM}

Given a set $S$ of integers, does there exist a subset whose sum is
exactly $k$?

Let the certificate be the subset of $S$, which is clearly polynomial
with respect to the input.  We can verify the subset by calculating
the sum and comparing it to $k$ in polynomial time.

We reduce from \emph{3CNF-SAT}.  Given a \emph{3CNF} formula over
variables $x_1, ..., x_n$, and clauses $c_1,...,c_k$, we assume
without loss of generality that all variables appear in at least one
clause and no clause contains a variable and its negation.

We create two tables the columns of which are labeled with the
variables in order then the clauses in order.

The first table's rows are labeled by each variable followed by its
negation.  The value in the table at a given row and column is 1
either if the column is labeled with a variable and the row is labeled
with either that variable or its negation; or the column is labeled
with a clause and that clause contains exactly that variable (or
exactly that negated variable).

The second table's rows are labeled with $s_1,s_1'$ for $c_1$, and
similarly for $c_2,...,c_k$.  The value in the row labeled $s_i$ is 1
in the column labeled $c_i$, and the value in the row labeled $s_i'$
is 2 in the column labeled $c_i$, and 0 otherwise.

The rows of these tables give the digits of numbers in some base
(let's say 10), and the target sum is the number with $n$ 1's followed
by $k$ 4's.

It should be easy to see that we can find a subset which is exactly
the target if and only if the \emph{3CNF} formula is satisfiable.

\section{0/1-Integer Programming}

Beyond the scope of these notes, but interesting.

\section{3-COLOR}

Given a graph $G=(V,E)$, can we label the vertices of a graph using 3
colors such that no adjacent vertices have the same color?

Beyond the scope of these notes, but interesting.


\section{Discontinuity Types}
There are four major types of discontinuity.
\begin{enumerate}[label=]
	\item \textbf{Removable: } If $f$ is discontinuous at $c$ but we can remove the discontinuity by setting $f$ equal to its limit at $c$, then $f$ has a removable discontinuity at $c$.
	\item \textbf{Jump: } If $f$ is discontinuous at $c$, and both of the one-sided limits exist but are different, then $f$ has a jump discontinuity at $c$.
	\item \textbf{Infinite: } If $f$ has a vertical asymptote at $c$, meaning one or both sides go to $\pm\infty$, then $f$ has an infinite discontinuity at $c$.
	\item \textbf{Oscillating: } If $f$ oscillates without limit at $c$, then $f$ has an oscillating discontinuity at $c$. An example of such a function would be $\sin{\frac{1}{x}}$ at $x=0$.
\end{enumerate}

It might seem strange that $\sin{\frac{1}{x}}$ has an oscillating discontinuity at $x=0$ because we were able to find the limit as $x$ approaches of 0 of $x\sin{\frac{1}{x}}$, a very similar function.
However, remembering how we applied the Ham Sandwich Theorem to find this limit, we see that the $x$ term bounds the amplitude of the oscillations, allowing the limit to be $0$.

\begin{example}
	For the following function state the following: its domain, any discontinuities and their types, what values should redefine the function to remove any removable discontinuities (give the extended function).
	\begin{equation*}
		f(x) = \frac{x^3-7x-6}{x^2-9}
	\end{equation*}
\end{example}
\begin{answer}
	Polynomials are continuous on their entire domain of all real numbers.
	So, rational functions like $f$ can only be discontinuous when the denominator is equal to $0$.
	This happens in two places: $x=3$ and $x=-3$.
	We'll check the limits from each side at each of these points to determine the type of discontinuity.
	For $x=3$,
	\begin{equation*}
		\lim_{x\to 3^+}{f(x)} = \lim_{x\to 3^-}{f(x)} = \lim_{x\to 3}{f(x)} = \lim_{x\to 3}{\frac{(x+2)(x+1)(x-3)}{(x+3)(x-3)}} = \lim_{x\to 3}{\frac{(x+2)(x+1)}{(x+3)}} = \frac{20}{6} = \frac{10}{3}.
	\end{equation*}

	So, $f$ has a removable discontinuity at $x=3$ because the left and right limits are the same.
	For $x=3$,
	\begin{equation*}
		\lim_{x\to -3^+}{f(x)} = -\infty \text{ and } \lim_{x\to -3^+}{f(x)} = \infty.
	\end{equation*}

	So, $f$ has an infinite discontinuity at $x=-3$ because both of the left and right limits go to $\pm\infty$.
	The value we got from the limits at $x=3$ gives us the value we need to redefine $f$ as to remove the discontinuity.
	The extended function is therefore
	\begin{equation*}
		f_{e}(x) = \begin{cases}
			f(x) & x \neq 3 \\
			\frac{10}{3} & x = 3
		\end{cases}
	\end{equation*}
\end{answer}


\chapter{First Order Logic}

\emph{"All men are mortal. Socrates is a man. Socrates is mortal."}

\section{Syntax of First-Order Logic}

We start with the symbols of a \textbf{first-order language} $\mathbb{L}$

There are two types of symbols

\begin{itemize}
    \item \textbf{Logical symbols}
    \item Non-logical symbols, a.k.a. \textbf{parameters}
\end{itemize}

\subsection{Symbols}

\subsubsection{Logical Symbols}

In a first-order language $\mathbb{L}$, we have the following symbols

\begin{enumerate}
    \item \textbf{Parentheses}. Two symbols `(' and `)'.
    \item \textbf{Logical connective symbols}. $\to$ and $\neg$
    \item \textbf{Variables}. An enumerable list of symbols $v_1,\dots,v_n,\dots$
    \item \textbf{Identity or Equalily Symbol} $=$ or $\doteq$. It may or may not be present in a particular first-order language
\end{enumerate}

Notice that we do not need $\vee$, $\wedge$. $\leftrightarrow$ because $\{\to, \neg\}$ is complete.

\subsubsection{Parameters}

\begin{enumerate}
    \item \textbf{Universal quantifier}. $\forall$
    \item For each $n>0$, there is a set (possibly empty) of objects called n-ary (or n-place) \textbf{predicate symbols}
    \item For each $n>0$, there is a set (possibly empty) of objects called n-ary (or n-place) \textbf{function symbols}
    \item A set of (possibly empty) of objects \textbf{constant symbols}
\end{enumerate}

\subsubsection{Further Requirements}

\begin{itemize}
    \item $\doteq$ is a 2-ary predicate symbols
    \item There is at least one predicate symbol
    \item The symbols are distinct, and no symbol is equal to a finite sequence of other symbols
\end{itemize}

\subsubsection{Example: Set Theory as First-Order Logic}

The Set Theory can be described by the following language

\begin{itemize}
    \item Equality
    \item Predicate symbols: 2-place $\dot{\in}$
    \item Constant symbols: empty set $\dot{\emptyset}$
    \item Function symbols: None
\end{itemize}

Note that the symbols are (currently) just interpreted as symbols and they do not have semantic meanings.

\begin{remark}
    We do not put restrictions or requirements on number of predicate, function or constant symbols.
\end{remark}

\subsection{Expressions}

An \textbf{expression} in a language $\mathbb{L}$ is a finite sequence of symbols.

\subsubsection{Terms}

\begin{definition}[Term Building Operation]
    \label{def:TermBuildingOperation}
    Given any n-ary function symbol $f$, the term-building operation $\mathcal{F}_f$ is defined by
    \[ \mathcal{F}_f (\sigma_1,\dots,\sigma_n) = f \sigma_1\dots\sigma_n \]
    We call $\sigma_i$ the arguments to $f$
\end{definition}

\begin{definition}[Term]
    \label{def:Term}
    A \textbf{term} is an expression built up from constant symbols and variables by applying some finite times (zero or more times) of term-building operations.
\end{definition}

For example, let $f$ and $g$ be 2-ary and 3-ary function symbols, then $gfc_1c_2v_3c_1$ is a term.

\begin{definition}[Term Sequence]
    \label{def:TermSequence}
    A \textbf{term sequence} is a finite sequence $t_1,\dots,t_n$ of expressions s.t. each $t_i$ is
    \begin{itemize}
        \item either a variable, a constant
        \item or is in the form of $f\sigma_1\dots\sigma_k$ where $f$ is a $f$-ary function and each $\sigma_1,\dots,\sigma_k$ occurs earlier in the sequence
    \end{itemize}
\end{definition}

\begin{proposition}
    An expression $t$ is a term iff there is a term sequence $t_1,\dots,t_n$ such that $t=t_n$
\end{proposition}

\subsubsection{Atomic Formulas}

\begin{definition}[Atomic Formula]
    \label{def:AtomicFormula}
    An expression is an \textbf{atomic formula} if it is of the form $P t_1\dots t_n$ where $t_1,\dots,t_n$ are terms and $P$ is a n-ary predicate symbol.
\end{definition}

\subsection{Well-Formed Formulas}

\begin{definition}[Formula-Building Operations]~{}
    \label{def:FormulaBuildingOperation}
    \begin{itemize}
        \item $\xi_\neg(\alpha) = (\neg \alpha)$
        \item $\xi_\to(\alpha, \beta) = (\alpha\to\beta)$
        \item $\mathcal{Q}_i(\gamma) = \forall v_i\gamma$
    \end{itemize}
\end{definition}

\begin{definition}[Well-Formed Formula]
    A \textbf{well-formed formula} (wff) is an expression built up from atomic formulas by applying some finite times of term-building operations.
\end{definition}

\begin{definition}[Well-Formed Sequence]
    A \textbf{well-formed sequence} is a finite sequence $\alpha_1,\dots,\alpha_n$ of expressions such that each $\alpha_i$ is
    \begin{itemize}
        \item either an atomic formula
        \item or is of the form of $(\neg \beta)$ or $(\beta\to\gamma)$ where $\beta$ and $\gamma$ occur earlier in the list
        \item or is of the form $\forall v_i\beta$ where $\beta$ occurs earlier in the list
    \end{itemize}
\end{definition}

\begin{proposition}
    The expression $\alpha$ is a wff if there is a well-formed sequence $\alpha_1,\dots,\alpha_k$ such that $\alpha = \alpha_k$
\end{proposition}

\subsection{Abbreviations}

\begin{itemize}
    \item $(\alpha\vee\beta)$ abbreviates $((\neg\alpha)\to\beta)$
    \item $(\alpha\wedge\beta)$ abbreviates $(\neg(\alpha \to (\neg\beta)))$
    \item $(\alpha\leftrightarrow\beta)$ abbreviates $(\alpha\to\beta)\wedge(\beta\to\alpha)$
    \item $\exists x\alpha$ abbreviates $(\neg\forall x(\neg\alpha))$
    \item $u\doteq t$ abbreviates $\doteq ut$
    \item $u \dot{\neq} t$ abbreviates $\dot{\neq} ut$
    \item Outer-most parentheses can be omitted
    \item $\neg$, $\forall$, $\exists$ apply to as little as possible
    \item $\wedge$, $\vee$ apply to as little as possible, subject to previous operators
    \item Grouping for repeated connectives is to the right
\end{itemize}

\subsection{Free Occurrence of Variables}

\begin{definition}[Free Occurrence]
    The variable $x$ \textbf{occurs free} in an atomic wff $\varphi$ iff it occurs in $\varphi$.

    $x$ \textbf{occurs free} in $\neg\alpha$ iff $x$ occurs free in $\alpha$.

    $x$ \textbf{occurs free} in $\alpha\to\beta$ iff $x$ occurs free in $\alpha$ or in $\beta$.

    $x$ \textbf{occurs free} in $\forall y \alpha$ iff $x$ occurs free in $\alpha$ and $x \neq y$.
\end{definition}

\begin{definition}[Sentence]
    $\varphi$ is a \textbf{sentence} iff no variable occurs free in $\varphi$.
\end{definition}
\begin{remark}
    Sentences are usually represented by $\sigma$ or $\tau$.
\end{remark}

We provide some examples

\begin{itemize}
    \item $\dot{0} \dot{<} \dot{1}$ does not have any free occurrence. It is a sentence,
    \item $\forall x(x\dot{<}y)$. $y$ occurs free but $x$ does not.
    \item $\forall x(\neg x \dot{<} \dot{0})$. No free occurrence.
    \item $\forall x\forall y (x \dot{<} y \to \exists z x\dot{<}z\wedge z\dot{<}y)$. No free occurrence.
\end{itemize}

\section{Semantics of First-Order Logic}

\subsection{Structures}

\begin{definition}
    Given a first order language $\mathbb{L}$, a \textbf{structure} $\mathfrak{A}$ for $\mathbb{L}$ consists of
    \begin{itemize}
        \item A non-empty set called the \textbf{universe} or \textbf{domain} of the structure, written as $|\mathfrak{A}|$
        \item For each n-ary predicate symbol $P$ of $\mathcal{L}$, other than $\doteq$, an n-ary relation $\mathbb{P}^{\mathfrak{A}}$ on $|\mathfrak{A}|$
        \item $\doteq^{\mathfrak{A}}$ is the identity relation on $|\mathfrak{A}|$. $\doteq^{\mathfrak{A}} = \{(a,b)|a,b\in|\mathfrak{A}, a =b|\}$
        \item For each n-ary function symbol $f$ of $\mathbb{L}$, an n-ary ooperation on the universe, i.e. an n-ary function $f^{\mathfrak{A}}: |\mathfrak{A}|\times\cdots\times|\mathfrak{A}|\mapsto|\mathfrak{A}|$
        \item For each constant symbol $c$ of $\mathbb{L}$, $c^{\mathfrak{A}}\in|\mathfrak{A}|$
    \end{itemize}
\end{definition}

\subsection{Assignments}

Let $\mathfrak{A}$ be a structure for language $\mathbb{L}$. Let $V$ be the set of variables, and $T$ be the set of terms.

\begin{definition}[Assignment Functions]
    An \textbf{assignment} for $\mathfrak{A}$ is a function $s:V\mapsto|\mathfrak{A}|$.
\end{definition}

\begin{definition}[Assignment to Terms]
    An assignment $s$ is extended to a function $\bar{s}:T\mapsto|\mathfrak{A}|$.
    \begin{itemize}
        \item $\bar{s}(v) = s(v)$ if $v$ is a variable
        \item $\bar{s}(c) = c^{\mathfrak{A}}$ if $c$ is a constant
        \item $\bar{s}(ft_1\dots t_n) = f^{\mathfrak{A}}(\bar{s}(t_1),\dots,\bar{s}(t_n))$ if $f$ is a n-ary function symbol and $t_1,\dots,t_n$ are terms
    \end{itemize}
\end{definition}

\subsubsection{Changing the Assignment Function}

Let $s$ be an assignment function, $x$ be a variable and $a\in\mathfrak{A}$. Then $s(x|a)$ is the new assignment, where for each variable $y$

\[s(x|a)(y) = \begin{cases}
    s(y) &\quad \text{if $(y\neq x)$}\\
    a &\quad \text{if $(y=x)$}
\end{cases}\]

This operation actually ``overrides'' the assignment of $s$ to $x$ and makes the assignment to $x$ equal to $a$.

\section{Satisfaction}

Given a first-order language $\mathbb{L}$, let $\mathfrak{A}$ be a structure for $\mathbb{L}$, let $s$ be an assignment for $\mathbb{L}$ and let $\varphi$ be a wff in $\mathbb{L}$. We denote $\mathfrak{A}$ to satisfy $\varphi$ with $s$ by $\vDash_{\mathfrak{A}}\varphi[s]$

Informally, it means ``\emph{The translation of $\varphi$ determined by $\mathfrak{A}$, where a variable $x$ is translated as $s(x)$, is true}''

\subsection{Satisfaction for Atomic Formulas}

\begin{definition}
    Given a language $\mathbb{L}$ and a structure $\mathfrak{A}$, let $s$ be an assignment, let $P$ be a n-ary predicate,
    \[ \vDash_{\mathfrak{A}} Pt_1\dots t_n[s] \Leftrightarrow (\bar{s}(t_1),\dots,\bar{s}(t_n)) \in P^{\mathfrak{A}} \]
    \[ \vDash_{\mathfrak{A}} \doteq t_1t_2[s] \Leftrightarrow \bar{s}(t_1) = \bar{s}(t_2) \]
\end{definition}

\subsection{Satisfaction for WFF}

\begin{definition}
    Suppose $\vDash_{\mathfrak{A}}\alpha[s]$ and $\vDash_{\mathfrak{A}}\beta[s]$ have been defined, then
    \begin{itemize}
        \item $\vDash_{\mathfrak{A}} \neg\alpha[s]$ iff not $\vDash_{\mathfrak{A}}\alpha[s]$
        \item $\vDash_{\mathfrak{A}}\alpha\to\beta[s]$ iff $\vDash_{\mathfrak{A}}\alpha[s]\Longrightarrow\vDash_{\mathfrak{A}}\beta[s]$
        \item $\vDash_{\mathfrak{A}}\forall x \alpha[s]$ iff $\forall a\in|\mathfrak{A}|$, $\vDash_{\mathfrak{A}}\alpha[s(x|a)]$
    \end{itemize}
\end{definition}

If $\vDash_{\mathfrak{A}}\varphi[s]$, we say \emph{$\mathfrak{A}$ satisfies $\varphi$ with $s$}, or \emph{$s$ satisfies $\varphi$ in the structure $\mathfrak{A}$}

\subsubsection{Satisfaction Depends Only on Variables that Occur Free}

\begin{lemma}
    \label{lem:LemmaForFreeOccurrenceThm}
    Let $\mathfrak{A}$ be a structure for $\mathbb{L}$, $s_1, s_2$ be two assignment for $\mathfrak{A}$ and $t$ be a term of $\mathbb{L}$.

    If $s_1(x)=s_2(x)$ for every $x$ that occurs in $t$, then
    \[ \bar{s}_1(t) = \bar{s}_2(t) \]
\end{lemma}
\begin{proof}
    Proof by induction on $t$.
    \begin{itemize}
        \item[Base] If $t=c$ is a constant. It is straightforward that $\bar{s}_1(t) = \bar{s}_2(t) = c$. If $t=x$ is a variable, then by assumption we know that $\bar{s}_1(x) = s_1(x) = s_2(x) = \bar{s}_2(x)$. So we are done.
        \item[Induction] Consider a term $t=ft_1\dots t_n$. By inductive hypothesis we know that $\forall i$, $\bar{s}_1(t_i) = \bar{s}_2(t_i)$. $\bar{s}_1(t) = f^{\mathfrak{A}}(\bar{s}_1(t), \dots, \bar{s}_1(t) = f^{\mathfrak{A}}(\bar{s}_2(t), \dots, \bar{s}_2(t)) = \bar{s}_2(t)$.
    \end{itemize}
\end{proof}

\begin{theorem}
    Let $\mathfrak{A}$ be a structure for $\mathbb{L}$, $s_1, s_2$ be two assignment for $\mathfrak{A}$ and $\varphi$ be a wff of $\mathbb{L}$.

    If $s_1(x)=s_2(x)$ for every $x$ that occurs free in $\varphi$, then
    \[ \vDash_{\mathfrak{A}}\varphi[s_1] \iff \vDash_{\mathfrak{A}}\varphi[s_2] \]
\end{theorem}

\begin{proof}
    Prove by induction on $\varphi$.
    \begin{itemize}
        \item[Base] If $\varphi$ is an atomic formula $Pt_1\dots t_n$.
        \[ \vDash_{\mathfrak{A}} \varphi [s_1] \Leftrightarrow P^{\mathfrak{A}}(\bar{s}_1(t_1),\dots,\bar{s}_1(t_n)) \]
        \[ \vDash_{\mathfrak{A}} \varphi [s_2] \Leftrightarrow P^{\mathfrak{A}}(\bar{s}_2(t_1),\dots,\bar{s}_2(t_n)) \]

        We need to prove that the two RHSes are equivalent. By Lemma~\ref{lem:LemmaForFreeOccurrenceThm} we know that all the terms are equal under $s_1$ and $s_2$, and therefore $\vDash_{\mathfrak{A}} \varphi [s_1] = \vDash_{\mathfrak{A}} \varphi [s_2]$.

        \item[Induction] Consider $\varphi=\neg\alpha$.
        \[ \vDash_{\mathfrak{A}}(\neg\alpha)[s_1] \Leftrightarrow \nvDash_{\mathfrak{A}}\alpha[s_1] \Leftrightarrow \nvDash_{\mathfrak{A}}\alpha[s_2] \Leftrightarrow \vDash_{\mathfrak{A}}(\neg\alpha)[s_2] \]

        The case $\varphi = \alpha\to\beta$ is similar.

        Consider the case $\forall x \alpha$. We want to prove
        \[ \sat{A}{\forall x\alpha}{s_1} \Leftrightarrow \sat{A}{\forall x\alpha}{s_2} \]
        which is equivalent to
        \[ \forall a\in|\mathfrak{A}|\sat{A}{\alpha}{s_1(x|a)} \Leftrightarrow \forall a \in |\frakA| \sat{A}{\alpha}{s_2(x|a)} \]

        We only need to prove that
        \[ \forall y \text{occurring free in $\alpha$}, s_1(x|a)(y) = s_2(x|a)(y) \]

        If $y\neq x$, then $y$ is still occurring free in $\alpha$, and by inductive hypothesis they should equal. If $y=x$, then both sides are $a$. So we are done.
    \end{itemize}
\end{proof}
\begin{remark}
    This theorem is somewhat similar to the theorem in sentential logic, which states that we only need to consider sentence symbols. Similarly, in first-order logic, we only need to consider variables that occur free.
\end{remark}

\begin{definition}
    Let $\varphi$ be a wff s.t. all variables occurring free in $\varphi$ are included amoing $v_1,\dots,v_k$. Given $a_1,\dots,a_k\in\frakA$.

    \[ \assignSat{A}{\varphi}{a_1,\dots,a_k} \]

    means that $\sat{A}{\varphi}{s}$ for some $s:V\mapsto|\frakA|$ s.t. $s(v_i) = a_i$
\end{definition}

\begin{corollary}
    If $\sigma$ is a sentence then
    \begin{itemize}
        \item either $\sat{A}{\sigma}{s}$ for every assignment $s$. We say $\sigma$ is true in $\frakA$.
        \item or $\unsat{A}{\sigma}{s}$ for every assignment $s$. We say $\sigma$ is false in $\frakA$
    \end{itemize}
\end{corollary}

Therefore a sentence does not depend on $s$, and we can simply write $\sentSat{A}{\sigma}$ or $\sentunsat{A}{\sigma}$.

\subsection{Elementary Equivalence}

\begin{definition}[Elementary Equivalence]
    \label{def:ElementaryEquivalence}
    Let $\mathfrak{A}$ and $\mathfrak{B}$ be structures for the same language $\mathbb{L}$. $\mathfrak{A}$ and $\mathfrak{B}$ are \textbf{elementarily equivalent} ($\mathfrak{A} \equiv \mathfrak{B}$) if for every \emph{sentence} of $\mathbb{L}$
    \[ \sentSat{A}{\sigma} \iff \sentSat{B}{\sigma} \]
\end{definition}

\begin{remark}
    Elementary equivalence only take into consideration sentences.
\end{remark}

\begin{proposition}
    $\mathfrak{Q}$ and $\mathfrak{R}$ are elementary equivalent. But this is beyond the scope of the course.
\end{proposition}

\section{Models}

\subsection{Models}

\begin{definition}[Model]
    $\mathfrak{A}$ is a \textbf{model} of the sentence $\sigma$ if $\sentSat{A}{\sigma}$, i.e. if $\sigma$ is true in $\mathfrak{A}$. $\mathfrak{A}$ is a \textbf{model} of a set $\Sigma$ of sentences if $\mathfrak{A}$ is a model for every sentence in $\Sigma$. i.e. every sentence in $\Sigma$ is true in $\mathfrak{A}$.
\end{definition}

For example, consider a first-order language $\mathbb{L}$, with 2-ary predicate symbols $\dot{P}$ and $\doteq$. Given a structure $\mathfrak{A}$ of $\mathbb{L}$,

\begin{itemize}
    \item $\mathfrak{A}$ is a model of $\forall x \forall y x \doteq y$
    \begin{itemize}
        \item $\Leftrightarrow \sat{A}{x\doteq y}{s(x|a)(y|b)}$ for every $a,b \in |\mathfrak{A}|$
        \item $\Leftrightarrow$ $a = b$ for every $a,b \in|\mathfrak{A}|$
        \item $\Leftrightarrow$ $|\mathfrak{A}|$ contains only one element.
        \item Note that $|\frakA|$ cannot be empty because the universe of a structure must be non-empty.
    \end{itemize}
    \item $\mathfrak{A}$ is a model of $\forall x \forall y \dot{P}xy$
    \begin{itemize}
        \item iff $P^\mathfrak{A}(a,b)$ for all $a,b\in|\mathfrak{A}|$
        \item iff $\dot{P}^{\frakA} = |\frakA| \times |\frakA|$
    \end{itemize}
    \item $\mathfrak{A}$ is a model of $\forall x \forall y \neg\dot{P}xy$
    \begin{itemize}
        \item iff $P^{\frakA}(a,b)$ does not hold for all $a,b\in|\frakA|$
        \item iff $\dot{P}^{\frakA} = \emptyset$
    \end{itemize}
    \item $\frakA$ is a model of $\forall x \exists y \dot{P}xy$
    \begin{itemize}
        \item iff forall $a\in|\frakA|$, there is a $b\in|\frakA|$ s.t. $\dot{P}^{\frakA}(a,b)$
        \item iff the domain of $\dot{P}^{\frakA}$ is $|\frakA|$
        \item Conversely, if we want the range of $\dot{P}^{\frakA}$ is $|\frakA|$, we can write $\forall y \exists x \dot{P}xy$
    \end{itemize}
\end{itemize}

\subsection{Linearly Ordered Structures}

\begin{definition}[Trichotomy]
    Let $R$ be a binary relation, $R$ satisfies \textbf{trichotomy} if exactly one of the following is true
    \[ (a,b) \in R \quad (b,a) \in R \quad a = b \]
\end{definition}

\begin{definition}[Linear Ordering]
    A binary relation $R$ is a \textbf{linear ordering} on $A$ if $R$ is transitive and satisfies trichotomy on $A$.
\end{definition}

\begin{definition}
    Let $\mathbb{L}$ be the language with a binary relation symbol $\dot{R}$ and $\doteq$ (and no other symbols). Let $\frakA = (A,R)$, i.e. $A=|\frakA|$ and $R = \dot{R}^{\frakA}$.
    \begin{itemize}
        \item $\frakA$ is transitive if $R$ is transitive
        \item $\frakA$ is a linearly ordered structure if $R$ is a linear ordering on $\frakA$
    \end{itemize}
\end{definition}

For a set of structures with some certain properties, the set can be defined by a sentence.

Let $\frakA = (A,R)$,
\begin{itemize}
    \item $\frakA$ is transitive iff $\sentSat{A}{\sigma}$, where $\sigma=\forall x \forall y \forall z \dot{R}xy \to \dot{R}yz \to \dot{R}xz$. Therefore $\sigma$ defines the set of all transitive structures
    \item $\frakA$ is linearly ordered iff $\sentSat{A}{\sigma}$ where
    \begin{itemize}
        \item $\sigma_1 = \forall x \forall y \forall z \dot{R}xy \to \dot{R}yz \to \dot{R}xz$
        \item $\sigma_2 = \forall x \forall y (\dot{R}xy \vee x=y \vee \dot{R}yx)$
        \item $\sigma_3 = \forall x \forall y (\dot{R}xy \to \neg\dot{R}yx)$
        \item $\sigma = \sigma_1 \wedge \sigma_2 \wedge \sigma_3$
    \end{itemize}
    Therefore $\sigma$ defines the set of all linearly ordered structures
    \item $\domain{R} = A$ iff $\sentSat{A}{\sigma}$ where $\sigma = \forall x \exists y \dot{R}xy$
    \item $\range{R} = A$ iff $\sentSat{A}{\sigma}$ where $\sigma = \forall y \exists x \dot{R}xy$
    \item $R$ is a (total) function iff $\sentSat{A}{\sigma}$ where
    \begin{itemize}
        \item $\sigma_4 =  \forall x \exists y \dot{R}xy$
        \item $\sigma_5 = \forall x \forall y \forall z \dot{R}xy \to \dot{R}xz \to x \doteq z$
        \item $\sigma = \sigma_4 \wedge \sigma_5$
    \end{itemize}
\end{itemize}

\subsection{Elementary Class}

\begin{definition}[Elementary Class]
    A set of strucutures $\mathcal{K}$ is an \textbf{elementary class} if there exists a sentence $\sigma$ s.t.
    \[ \mathcal{K} = \{ \frakA | \frakA \text{ is a model of $\sigma$} \} \]
    i.e.
    \[ \mathcal{K} = \{ \frakA | \sentSat{A}{\sigma} \} \]
\end{definition}

For example, the set of all graphs is an elementary class.

Let $\mathbb{L}$ be the language with a binary predicate symbol $\dot{E}$ and $\doteq$, and no other symbols. A structure $\mathfrak{G} = (G, E)$ for $\mathbb{L}$ is a graph if
\begin{itemize}
    \item $E$ is symmetric (undirected graph)
    \item For every $a \in G$, $(a, a) \notin E$ (no self-loops)
\end{itemize}

To show that the set of graphs defined above is an elementary class, we need to show that there is a sentence $\sigma$ s.t.
\[ \text{$G$ is a graph} \Leftrightarrow \sentSat{G}{\sigma} \]

Therefore $\sigma$ should be able to represent the symmetric and non-reflexible properties.

\[\sigma = (\forall x \forall y (\dot{E}xy \to \dot{E}yx)) \wedge (\forall x(\neg \dot{E}xx))\]

\subsubsection{In the Wider Sense}

\begin{definition}[Elementary Class in the Wider Sense]
    A set of structures $\mathcal{K}$ is an \textbf{elementary class in the wider sense} ($EC_\Delta$) if there is a set $\Sigma$ of sentences s.t.
    \[ \mathcal{K} = \{ \frakA | \frakA \text{ is a model of } \Sigma \} \]
    i.e.
    \[ \mathcal{K} = \{ \frakA | \sentSat{A}{\sigma} \text{ for every $\sigma \in \Sigma$} \} \]
\end{definition}

For example, we can use a set $\Sigma$ to define the set of strucutures whose univerise is infinite.

\[ \Sigma = \{ \lambda_2, \lambda_3, \dots, \lambda_n, \dots \} \]

where $\lambda_i$ denotes ``At least $i$ elements exists in $|\frakA|$'', for example $\lambda_2 = \exists x \exists y x \neq y$.

However, it is hard to decide whether there is a single sentence $\sigma$ s.t. $\frakA$ is a model of $\sigma$ iff $|\frakA|$ is infinite.

\section{Logical Implications and Satisfiability}

\subsection{Logical Implications}

\begin{definition}
    Let $\Gamma$ be a set of wffs and $\varphi$ be a wff. $\Gamma$ \textbf{logically implies} $\varphi$, written as
    \[ \Gamma \vDash \varphi \]
    if for every structure $\frakA$ and every assignment $s$, if $\frakA$ satisfies $\Gamma$ with $s$, then $\frakA$ satisfies $\varphi$ with $s$
\end{definition}

\begin{theorem}
    For a set of sentences $\Sigma$, and a sentence $\sigma$, $\Sigma\vDash\sigma$ iff for every model $\frakA$ of $\Sigma$, $\frakA$ is a model of $\sigma$.
\end{theorem}

As before, we denote $\{\alpha\}\vDash\beta$ by $\alpha\vDash\beta$

\subsection{Logical Equivalence}

\begin{definition}[Logical Equivalence]
    $\alpha$ and $\beta$ are logically equivalent, written as $\alpha\vDash\Dashv\beta$ if $\alpha\vDash\beta$ and $\beta\vDash\alpha$
\end{definition}

\subsection{Valid Formulas}

\begin{definition}[Valid WFFs]
    Let $\varphi$ be a wff in the language $\mathbb{L}$. $\varphi$ is \textbf{valid} if $\semanticalImply{\emptyset}{\varphi}$, written as $\tautology{\varphi}$.
\end{definition}

For example,

\begin{itemize}
    \item $x \doteq x$ is valid
    \item $\exists x \doteq x$ is valid
    \item $\forall x \exists y x\dot{\neq}y$ is not valid
    \item $\dot{P}x \vee \neg \dot{P}x$ is valid
    \item $\exists x(\dot{P}x \to \forall x \dot{P}x)$ is valid
\end{itemize}

We detail the proof of the last example

\begin{itemize}
    \item $\sat{A}{\exists x (Px\to\forall x Px)}{s}$
    \item[$\Leftrightarrow$] There is some $a \in |\frakA|$ s.t. $\sat{A}{Px \to \forall x Px}{s(x|a)}$
    \item[$\Leftrightarrow$] $\sat{A}{\dot{P}(x)}{s(x|a)} \Rightarrow \sat{A}{\forall x Px}{s(x|a)}$
    \item[$\Leftrightarrow$] $\sat{A}{\dot{P}(x)}{s(x|a)} \Rightarrow \sat{A}{\forall x Px}{s(x|a)}$
    \item[$\Leftrightarrow$] There is some $a$ s.t. if $a \in \dot{P}^{\frakA}$, then for every $b \in |\frakA|$, $b\in\dot{P}^{\frakA}$
    \item If there is some $a$ s.t. $a \notin \dot{P}^{\frakA}$, then the consequence holds trivially
    \item If there is no $a$ s.t. $a \notin \dot{P}^{\frakA}$, this means that for every $a \in |\frakA|$, $a\in\dot{P}^{\frakA}$. This is exactly the consequence so we are done
\end{itemize}

\subsection{Satisfiability}

\begin{definition}[Satisfiability]
    \begin{itemize}
        \item The wff $\varphi$ is \textbf{satisfiable} if there is some structure $\frakA$ and some assignment $s$ s.t. $\sat{A}{\varphi}{s}$.
        \item The set of wffs $\Gamma$ is \textbf{satisfiable} if there is some structure $\frakA$ and some assignment $s$ s.t. $\sat{A}{\varphi}{s}$ for every $\varphi$ in $\Gamma$.
    \end{itemize}
\end{definition}

\begin{theorem}
    $\varphi$ is not satisfiable iff $\neg\varphi$ is valid
\end{theorem}

\section{Definability}

\begin{definition}[Relations Defined by WFFs]
    Let $\frakA$ be a structure, and $\varphi$ be a wff, and $n$ be such that the variables occurring free in $\varphi$ are included among $v_1,\dots,v_n$

    The n-ary relation \textbf{defined by $\varphi$ in $\frakA$} is
    \[ \{ (a_1,\dots,a_n) | \assignSat{A}{\varphi}{a_1,\dots,a_n}\} \]
\end{definition}

Let $\mathfrak{N} = (\mathbb{N},\le, +, 1)$, the 2-ary relation $\{(a,b)|a < b\}$ is defined by
\[ v_1 \dot{+}\dot{1}\doteq v_2 \]

To show this, we will show that $(a,b) \in R \iff \assignSat{N}{\varphi}{a,b}$.

If $(a,b) \in R$, $\assignSat{N}{\varphi}{a,b} \iff a+1 \le b$

Conversely, if $\assignSat{N}{\varphi}{a,b}$, it's equivalent to $a+1 \le b$, which mathematically implies that $a < b$

\begin{definition}[Definability]
    The relation $R$ is \textbf{definable in the structure $\frakA$} if there is some wff $\varphi$ that defines it in $\frakA$.
\end{definition}

We show some examples for definability of functions. Let $\mathfrak{N} = (\mathbb{N}, <, +, x, 0, 1)$.

\begin{itemize}
    \item $v_1 \dot{+} v_2 \doteq v_3$ defines $\{ (a,b,c) | a + b = c\}$ which is the same as function $f$, where $f(a,b) = a + b$.
\end{itemize}

\subsection{Definable Relations}

\begin{definition}[Relations Defined by WFFs]
    Let
    \begin{itemize}
        \item $\frakA$ be a structure
        \item $\varphi$ be a wff and $n$ be such that the variables occurring free in $\varphi$ are included among $v_1,\dots,v_n$
    \end{itemize}

    The $n$-ary relation defined by $\varphi$ in $\frakA$ is
    \[ \{ (a_1,\dots,a_n) | \assignSat{A}{\varphi}{a_1,\dots,a_n} \} \]
\end{definition}

For example,

\begin{itemize}
    \item Let $\mathfrak{R} = (\mathbb{R},<,+,\times,0,1)$. The $1$-ary relation $\{ a \in \mathbb{R} | 0 \le a \}$ is defined by
    \[ \exists v_2, v_1 \doteq v_1 \times v_2 \]
    \item Let $\mathfrak{R} = (\mathbb{R},<,+,\times,0,1)$。 The $2$-ary relation $\{ (a,b) | a < b \}$ is defined by
    \[ \exists v_3 (v_1\dot{+}(\dot{1}\dot{+}v_3)\doteq v_2) \]
\end{itemize}

\begin{definition}[Definable Relations]
    The relation $R$ is \textbf{definable in structure} $\frakA$ if there is some wff that defines it in $\frakA$

    Let $f$ be a n-ary function $f$ whose domain is a subset $|\frakA| \times \dots \times |\frakA|$ and whose range is a subset of $|\frakA|$, $f$ is definable in $\frakA$ if the $(n+1)$-ary relation
    \[ \{ (a_1,\dots,a_n, b) | f(a_1,\dots,a_n) = b \} \]
    is definable in $\frakA$.
\end{definition}

Consider $\mathfrak{N} = (\mathbb{N},<,+,\times,0,1)$.
\begin{itemize}
    \item $v_1 + v_2 = v_3$ defines $\{ (a,b,c) | a + b = c \}$, which is the same as function $f(a,b) = a + b$
    \item $v_1 + v_3 = v_2$ defines $\{ (a,b,c) | a+c = b \}$, which is the same as function $f(a,b)$
    \[ f(a,b) = \begin{cases}
        b - a &\quad a \le b\\
        Undefined &\quad o.w.
    \end{cases} \]
\end{itemize}

\begin{lemma}
    Given a structure $\frakA$, the set of definable relations is \emph{enumerable}
\end{lemma}
\begin{lemma}
    Not every subset of $\mathbb{N}$ is definable.
\end{lemma}

The proof of the two lemmas are similar. Note that the set of wffs is enumerable, and every wff may define only one relation. And the set of all subsets of $\mathbb{N}$ is uncountable.

We now move from $\mathbb{N}$ to $\mathbb{R}$ and consider a more generall case. Consider whether the following subsets of $\mathbb{R}$ are definable in $\mathfrak{R} = (\mathbb{R}, <)$

\begin{itemize}
    \item $\emptyset$. Yes.
    \item $\mathbb{N}$. Yes.
    \item Anything else?
\end{itemize}

\section{Homomorphisms}

\begin{definition}[Homomorphism]
    Let $\mathfrak{A}$ and $\mathfrak{B}$ be structures for $\mathbb{L}$. A \textbf{homomorphism} from $\frakA$ to $\mathfrak{B}$ is a function $h:|\frakA| \mapsto |\mathfrak{B}|$ s.t.
    \begin{itemize}
        \item For every n-ary predicate symbol $R$, other than $\doteq$, and $a_1,\dots,a_n \in |\frakA|$,
        \[ (a_1,\dots,a_n) \in R^{\frakA} \iff (h(a_1),\dots,h(a_n))\in R^{\mathfrak{B}} \]
        \item For every n-ary function symbol $f$, and $a_1,\dots,a_n \in |\frakA|$,
        \[ h(f^\frakA(a_1,\dots,a_n)) = f^\mathfrak{B}(h(a_1),\dots,h(a_n)) \]
        \item For every constant symbol $c$
        \[ h(c^\frakA) = c^\mathfrak{B} \]
    \end{itemize}
\end{definition}

\begin{definition}[Onto of Homomorphism]
    $h$ is a homomorphism of $\frakA$ \textbf{onto} $\mathfrak{B}$ if $h$ is a homomorphism from $\mathfrak{A}$ to $\mathfrak{B}$ and $h$ maps $\frakA$ onto $\mathfrak{B}$.
\end{definition}

\begin{definition}[Isomorphism]
    A homomorphism $h$ from $\mathfrak{A}$ to $\mathfrak{B}$ is an \textbf{isomorphism} if $h$ is one-to-one
\end{definition}

\begin{definition}[Isomorphic]
    The structures $\frakA$ and $\mathfrak{B}$ are \textbf{isomorphic}, denoted by $\frakA \cong \mathfrak{B}$ if there is some \emph{isomorphism} of $\frakA$ \emph{onto} $\mathfrak{B}$. (One-to-one correspondence)
\end{definition}

\begin{definition}[Automorphism]
    An \textbf{automorphism} of $\frakA$ is an isomorphism of $\frakA$ onto $\frakA$
\end{definition}

\subsubsection{Examples of Homomorphism}

For example, let $\frakA = (\naturalSet, <^\naturalSet, +^\naturalSet)$, $\mathfrak{B} = (\mathbb{E}, <^\mathbb{E}, +^\mathbb{E})$, where $\mathbb{E}$ is the set of even non-negative integers. Then we claim that $h(n)=2n$ is an isomorphism of $\mathfrak{A}$ onto $\mathfrak{B}$

To do this, we show that 1) $h$ is a homomorphism; 2) $h$ is one-to-one; 3) $h$ is onto

\begin{itemize}
    \item[$\dot{<}$] $(a,b) \in <^\mathbb{N} \iff (h(a), h(b)) = (2a,2b) \in <^\mathbb{E}$
    \item[$\dot{+}$] $h(a +^\mathbb{N} b) = (h(a) +^\mathbb{E} h(b)) = (2a +^\mathbb{E} 2b)$
\end{itemize}

However, let $\mathfrak{C} = (\mathbb{O}, <^\mathbb{O}, +^\mathbb{O})$, where $\mathbb{O}$ is the set of all odd non-negative integers, then there is no isomorphism of $\frakA$ onto $\mathfrak{C}$. Infact $\mathfrak{C}$ is not even a valid strucutre because $+^\mathbb{O}$ is not closed.

\subsubsection{Automorphism of $\mathfrak{R}=(\realSet, <)$}

Consider which of the following $h$ are automorphisms of $\mathfrak{R}$. Note that to show this, we need to show that 1) $h$ is a homomorphism; 2) $h$ is one-to-one; 3) $h$ is onto; and 4) $h$ maps $\realSet$ to $\realSet$.

\begin{itemize}
    \item The identity function. Obviously yes.
    \item $h(a) = a + 3$. Yes.
    \item $h(a) = a - 4$. Yes.
    \item $h(a) = 2a$. Yes.
    \item $h(a) = -a$. Yes.
    \item $h(a) = ka + l$. Yes if $k>0$.
    \item $h(a) = a^3$. Yes.
    \item $h(a) = a^2$. No.
\end{itemize}

\subsubsection{Automorphism of $\mathfrak{N} = (\naturalSet, <)$}

Obviously the identity function is an automorphism. We consider other cases.

If we map $0$ to any $n>0$, i.e. $h(0) = n > 0$. Since $h$ is onto, there exists some $m > 0$ s.t. $h(m) = 0$. And here comes a problem
\[ m > 0 \iff h(m) = 0 > h(0) = n > 0 \]
Therefore $0$ can only be mapped to $0$, i.e. $h(0) = 0$

Similarly, $h(1)$ can only be mapped to $1$, and for each $n$, $h(n) = n$. Therefore the identity function $h(n) = n$ is the \emph{only} automorphism of $\naturalStruct$.

\subsection{Substructures}

We now consider a special kind of isomorphism

\begin{definition}[Substructures]
    Let $\frakA = (A,\dots)$ and $\frakB = (B,\dots)$ be structures for $\mathbb{L}$. $\frakA$ is a \textbf{substructure} of $\frakB$, denoted by $\frakA \subseteq \frakB$ if
    \begin{itemize}
        \item $A \subseteq B$
        \item For every $k$-ary predicate symbol,
        \[ P^\frakA = P^\frakB \cap A^k \]
        Note that this is to guarantee that $(a_1,\dots,a_k)\in P^\frakB \Longrightarrow (a_1,\dots,a_k)\in P^\frakA$
        \item For every $k$-ary function $f$ and every $k$-tuple of $A$
        \[ f^\frakA(a_1,\dots,a_k) = f^\frakB(a_1,\dots,a_k) \]
        \item For every constant $c$
        \[ c^\frakA = c^\frakB \]
    \end{itemize}
\end{definition}
\begin{remark}
    Substructures are defined under identity map $h(x)=x$. The identity map is an isomorphism of $\frakA$ into $\frakB$ iff
    \begin{enumerate}
        \item For each predicate $P$, $P^\frakA$ is the restriction of $P^\frakB$ to $A$
        \item For each function $f$, $f^\frakA$ is the restriction of $f^\frakB$ to $A$
        \item $c^\frakA = c^\frakB$.
    \end{enumerate}
    If these conditions are met, then $\frakA$ is called a \textbf{substructure} of $\frakB$.
\end{remark}

\subsection{Homomorphism Theorem}

\begin{lemma}
    \label{lem:HomomorphismLemma}
    Let $\frakA$ and $\frakB$ be structures for the language $\mathbb{L}$. Let $h$ be a homomorphism from $\frakA$ to $\frakB$, and $s:V\to|\frakA|$ be an assignment for $\frakA$. Then for every term $t$ of $\mathbb{L}$
    \[ h(\bar{s}(t)) = \overline{h\circ s}(t) \]
\end{lemma}
\begin{proof}
    Prove by induction.
    \begin{itemize}
        \item[] \textbf{Base Case.} If $t=c$, then
        \[ h(\bar{s}(c)) = h(c^\frakA) = c^\frakB = \overline{h\circ s(c)} \]
        If $t=v$, then
        \[ h(\bar{s}(x)) = h(s(x)) = h \circ s(x) \]
        \item[] \textbf{Inductive Case.} If $t = ft_1,\dots,t_n$,
        \[ h(\bar{s}(t)) = h(f^\frakA(\bar{s}(t_1),\dots,\bar{s}(t_n))) = f^\frakB(\bar{s}(t_1),\dots,\bar{s}(t_n)) \]
        \[ \overline{h\circ s} = f^\frakB(\overline{h\circ s}(t_1),\dots,\overline{h\circ s}(t_n)) \]
        From inductive hypothesis we know $\overline{h\circ s}(t_k) = h(\bar{s}(t_k))$, and we are done.
    \end{itemize}
\end{proof}

\begin{theorem}[Homomorphism Theorem]
    \label{thm:HomomorphismTheorem}
    Let $h$ be a homomorphism form $\frakA$ to $\frakB$ and $s$ be an assignment function for $\frakA$. The statement
    \[ \sat{A}{\varphi}{s} \iff \sat{B}{\varphi}{h \circ s} \]
    \begin{itemize}
        \item is true for every quantifier-free wff $\varphi$ not containing $\doteq$
        \item is true for every quantifier-free wff $\varphi$ if $h$ is one-to-one
        \item is true for every wff $\varphi$ wff $\varphi$ not containing $\doteq$ if $h$ is onto
        \item is true for every wff $\varphi$ if $h$ is an isomorphism of $\frakA$ onto $\frakB$ ($\frakA \cong \frakB$)
    \end{itemize}
\end{theorem}
\begin{proof}
    Prove by induction on $\varphi$.
    \begin{itemize}
        \item[] \textbf{Base Case.} Let $\varphi = Pt_1\dots t_n$.
         Since $h$ is a homomorphism, by definition we have
         \[ (\bar{s}(t_1),\dots,\bar{s}(t_n))\in P^\frakA \iff (h(\bar{s}(t_1)),\dots, h(\bar{s}(t_n))) \in P^\frakB \]
         Further, by Lemma~\ref{lem:HomomorphismLemma},
         \[ (\overline{h\circ s}(t_1),\dots,\overline{h\circ s}(t_2)) = (h(\bar{s}(t_1)),\dots, h(\bar{s}(t_n))) \in P^\frakB \]
         So we are done.
         \item[] \textbf{Inductive Case.}\begin{itemize}
             \item If $\varphi = \neg \alpha$, by induction hypothesis
             \[ \sat{A}{\alpha}{s} \iff \sat{B}{\alpha}{h \circ s} \]
             We can negate both sides
             \[ \unsat{A}{\alpha}{s} \iff \unsat{B}{\alpha}{h \circ s} \]
             Therefore,
             \[ \sat{A}{\neg\alpha}{s} \iff \sat{B}{\neg\alpha}{h\circ s} \]
             \item If $\varphi = \alpha \to \beta$.
             \[ \sat{A}{\alpha\to\beta}{s} \iff \sat{B}{\alpha\to\beta}{h\circ s} \]
             is equivalent to
             \[ \text{If} \sat{A}{\alpha}{s} \text{then} \sat{A}{\beta}{s} \iff \text{If} \sat{B}{\alpha}{h \circ s} \text{then} \sat{B}{\beta}{s} \]
             By induction hypothesis, we are done.
         \end{itemize}
    \end{itemize}
    Until now we have proved (a). We now further consider $\doteq$ and $\forall$

    We start from $\doteq$, which is a base case,
    If $\varphi = t_1\doteq t_2$
        \[ \sat{A}{t_1\doteq t_2}{s} \iff \sat{B}{t_1\doteq t_2}{h\circ s} \]
        \[ \bar{s}(t_1) = \bar{s}(t_2) \iff \overline{h\circ s}(t_1) = \overline{h\circ s}(t_2) \iff h(\bar{s}(t_1)) = h(\bar{s}(t_2)) \]
        $\Leftrightarrow$ always holds. However, $\Leftarrow$ requires an additional condition that $h$ is one-to-one.

    Finally, consider $\varphi=\forall x\alpha$ in the inductive case.
    \[ \sat{A}{\forall x \alpha}{s} \iff \sat{B}{\forall x \alpha}{h\circ s} \]
    \[ \text{For any $a\in|\frakA|$,} \sat{A}{\alpha}{s(x|a)} \iff \text{For any $b\in|\frakB|$,}\sat{B}{\alpha}{(h\circ s)(x|b)} \]
    Assume LHS, we prove RHS. Note that since $a$ and $b$ are arbitrary, we cannot relate them without additional conditions. Therefore to prove this we would require $h$ to be \emph{onto}. Then there is some $a'\in|\frakA|$ s.t. $h(a')=b$.
    \[ \sat{B}{\alpha}{(h\circ s)(x|h(a'))} \iff \sat{B}{\alpha}{h\circ(s(x|a'))} \]
    By LHS, we have $\sat{A}{\alpha}{s(x|a')}$, and by hypothesis, we have $\sat{B}{\alpha}{h\circ (s(x|a'))}$, so we are done.

    Assume RHS, we prove LHS. Let $b=h(a)$,
    \[ \sat{B}{\alpha}{(h\circ s)(x|h(a))} \iff \sat{B}{\alpha}{h\circ(s(x|a))} \]
    By hypothesis, we have $\sat{A}{\alpha}{s(x|a)}$. Done.
\end{proof}

\begin{corollary}
    \label{cor:HomomorphismToElementaryEquiv}
    If $\frakA \cong \frakB$, then $\frakA \equiv \frakB$. Recall that $\equiv$ means Elementary Equivalence (Def~\ref{def:ElementaryEquivalence}).
\end{corollary}

Corollary~\ref{cor:HomomorphismToElementaryEquiv} follows immediately from Homomorphism Theorem~\ref{thm:HomomorphismTheorem} because sentences do not care about assignments.

But the converse is not true. Take $\mathfrak{R} = (\realSet, <)$ and $\mathfrak{Q}=(\mathbb{Q}, <)$ as a counter example. We have claimed that they are elementary equivalent (though the proof is beyond the scope).

\begin{corollary}[Automorphism Theorem]
    \label{cor:AutomorphismTheorem}
    Let $h$ be an automorphism of $\frakA$. Let $R$ be an n-ary relation on $|\frakA|$ that is definable in $\frakA$. For every n-tuple $(a_1,\dots,a_n)$ of elements of $\frakA$,
    \[ (a_1,\dots,a_n) \in R \iff (h(a_1),\dots,h(a_n)) \in R \]
\end{corollary}
\begin{proof}
    Since $R$ is definable in $\frakA$,
    \begin{itemize}
        \item[] $(a_1,\dots,a_n) \in R$
        \item[$\iff$] $\assignSat{A}{\varphi}{a_1,\dots,a_n}$
        \item[$\iff$] $\assignSat{A}{\varphi}{h(a_1),\dots,h(a_n)}$
        \item[$\iff$] $(h(a_1),\dots,h(a_n))\in R$
    \end{itemize}
\end{proof}

Corollary~\ref{cor:AutomorphismTheorem} is often used for proving some relations are \emph{not definable}. For example, consider $\mathfrak{R} = (\realSet, <)$, its subset $\mathbb{N}$ is not definable in $\realStruct$.

Assume $\naturalSet$ is definable in $\realStruct$. Let $h(a)=a^3$. Obviously $h$ is an automorphism of $\realStruct$.

\begin{itemize}
    \item[] $a \in \mathbb{N}$
    \item[$\iff$] $\assignSat{R}{\varphi}{a}$
    \item[$\iff$] $\assignSat{R}{\varphi}{h(a)}$
    \item[$\iff$] $h(a)\in\naturalSet$
    \item[$\iff$] $a^3 \in \naturalSet$.
\end{itemize}

This leads to a contradiction. If $a^3 = 2$, then no $a \in \naturalSet$.


%!TEX root = RBM.tex

\subsubsection{Annealed Importance Sampling\protect\footnote{Available at \protect\url{https://github.com/lzhbrian/MCMC/blob/ master/src/partition/AIS.m} in Matlab}}

\para{Algorithm}
Annealed Importance Sampling(AIS)\cite{neal2001annealed,salakhutdinov2009learning} is probably one of the most preferable estimating methods avaible.

Previous work \cite{mackay2003information} have shown that if $P_{A}$ and $P_{B}$ in the SIS method is not close enough, the estimator would be very poor.

Based on SIS, the main idea of this algorithm is to gradually alter the value from an known $Z_{A}$ to our required $Z_{B}$ (or $Z_{K}$), by the following identity:
\begin{equation}
\frac{Z_{K}}{Z_{0}} = \frac{Z_{1}}{Z_{0}} \frac{Z_{2}}{Z_{1}} ... \frac{Z_{K}}{Z_{K-1}}
\end{equation}
where
\begin{equation}
\frac{Z_{K}}{Z_{k+1}} = \frac{1}{M} \sum_{i=1}^{M} \frac{P_{k+1}^{*}(\mathbf x^{(i)})}{P_{k}^{*}(\mathbf x^{(i)})}
~~where~ x^{(i)} \sim P_{k}
\end{equation}
in which we can get $x_{k+1}$ from:
\begin{equation}
\begin{aligned}
p(h^{A}_{j}=1|\mathbf v) &= sigmoid\Bigg( (1-\beta_{k})\Bigg(\sum_{i}W^{A}_{ij}v_{i}+a^{A}_{j}\Bigg) \Bigg) \\
p(h^{B}_{j}=1|\mathbf v) &= sigmoid\Bigg( \beta_{k}\Bigg(\sum_{i}W^{B}_{ij}v_{i}+a^{B}_{j}\Bigg) \Bigg) \\
p(v'_{i}=1|\mathbf h) &= sigmoid\Bigg( (1-\beta_{k})\Bigg(\sum_{j}W^{A}_{ij}h_{i}^{A}+b^{A}_{i}\Bigg) \\
& + \beta_{k}\Bigg(\sum_{j}W^{B}_{ij}h_{j}^{B}+b^{B}_{i}\Bigg) \Bigg)
\end{aligned}
\end{equation}
this procedure is shown in Figure~\ref{fig:xkxk1}.

\begin{figure}[tb]
% \vspace{-0.5in}
  	\centering
  	\includegraphics[width=0.4\textwidth]{figure/xkxk1.png}
% \vspace{-0.2in}
	\caption{The transition process from $x_{k}$ to $x_{k+1}$ which leaves $P_{k}(\mathbf v)$ invariant.}
	\label{fig:xkxk1}
\end{figure}

Note that model A indicates an initial model which we can easily compute all its configurations. Commonly, we choose an RBM model with $\theta = \{0,0,0\}$

$\mathbf \beta$ in the above equations is defined by users as a set of inverse temperatures $\{0= \beta_{1} < \beta_{2} < ... < \beta_{K} =1\}$, which can define a sequence of
\begin{equation}
P_{k}(\mathbf x) \propto P_{A}^{*}(\mathbf x)^{1-\beta_{k}} P_{B}^{*}(\mathbf x)^{\beta_{k}}
\end{equation}
where
\begin{equation}
P^{*}_{k}(\mathbf v)=\sum_{h^{A}h^{B}}e^{(1-\beta_{k})E(\mathbf v, \mathbf h^{A};\theta_{A})+\beta_{k}E(\mathbf v,\mathbf h^{B};\theta_{B})}
\end{equation}

\para{Initialize $Z_{A}$ with dataset}
In \cite{salakhutdinov2009learning}, Ruslan also notice a method to make $Z_{A}$ near $Z_{B}$. As the length \& time limit, we will not specify the process here.

Originally, we initialize model A with a configuration of $\theta=\{0,0,0\}$. This method can use the training data to initialize the visible bias $\mathbf b$ to a desired value s.t. we can get a better outcome of the estimation.

In our real practice, we find that this method take less than 0.005 second to initialize $\mathbf b$ even for a very big model (i.e.784 visible and 500 hidden units), but have strongly improved the result as we will mention in the next subsection.

	\begin{algorithm}
        \caption{Annealed Importance Sampling}
        \begin{algorithmic}
        	\Require Required $\beta_{k}$ s.t. $0 = \beta_{0} < \beta_{1} < ... < \beta_{K} = 1$
        	\State Initialize $b_{A}$ by dataset
        	\State Sample $\mathbf x_{1}$ from $P_{A} = P_{0}$
            \For{$k = 1 \to K-1$}
                \State Sample $\mathbf x_{k+1}$ given $\mathbf x_{k}$ using $T_{k}(\mathbf x_{k+1} \longleftarrow \mathbf x_{k+1})$
			\EndFor
			\State Set $\omega_{AIS} = \prod_{k=1}^{K}P^{*}_{k}(\mathbf x_{k})/P^{*}_{k-1}(\mathbf x_{k})$
        \end{algorithmic}
    \end{algorithm}