\chapter{Deductive Calculus of First Order Logic}

\section{Deductive Calculus}

Proofs are (purely) syntactic constructs that caputre \emph{derivability of facts}. Deductive calculi provide descriptions of proofs in logic. There are more than one forms for deductive calculi.

We adopt the Hilbert-style deductive calculus, which contains

\begin{itemize}
    \item A set $\Lambda$ of wffs called \textbf{logical axioms}
    \item A \emph{single} \textbf{rule of inference} for forming a new wff from a pair of wffs
\end{itemize}

We then systematically generate a set of wffs from the axioms by using the rules of inference. They are called \textbf{derivable} wffs.

Another deductive calculus is called the natural calculus, which has many rules of inference and one single axiom (\emph{“排中律”}, that a proposition is either True or False).

Yet another deductive calculus is called the sequent calculus, which contains no axiom and a set of symmetric rules.

\subsection{Introduction to Soundness and Completeness}

The goal is to prove that for any language $\mathbb{L}$, the following are equivalent

\begin{itemize}
    \item The set of derivable wffs in $\mathbb{L}$
    \item The set of valid wffs in $\mathbb{L}$
\end{itemize}

This is done by proving the soundness and completeness

\begin{theorem}[Soundness]
    \label{thm:FOSoundness}
    Every derivable of wff is valid.
\end{theorem}

\begin{theorem}[Completeness]
    \label{thm:FOCompleteness}
    Every valid wff is derivable.
\end{theorem}

\section{Logical Axioms}

\begin{definition}[Generalization]
    A \textbf{generalization} of the wff $\alpha$ is any wff obtained by putting zero or more universal quantifiers in front of $\alpha$
\end{definition}

For example, $\forall x\forall y\forall y\alpha$ is a generalization of $\alpha$.

Note that every wff is a generalization of itself.

\begin{definition}[Axioms]
    Let $\mathbb{L}$ be a first-order language. The set $\Lambda$ of logical axioms of $\mathbb{L}$ consists of all generalizations of the wffs in the following groups
    \begin{axiom}
        \label{axiom:InstanceOfTautology}
        Instances of tautologies.
    \end{axiom}
    \begin{axiom}
        \label{axiom:Substitution}
        Wffs of the form $\forall x\alpha \to \alpha_t^x$ such that the term $t$ is \textbf{substitutable} for $x$ in $\alpha$. As a special case, $\forall x\alpha \to \alpha$, where we replace $x$ with $t=x$
    \end{axiom}
    \begin{axiom}
        \label{axiom:PushUniversalIntoImplication}
        Wffs of the form $\forall x(\alpha\to\beta) \to (\forall x \alpha\to \forall x \beta)$.
    \end{axiom}
    \begin{axiom}
        \label{axiom:QuantifyBoundedVar}
        Wffs of the form $\alpha\to\forall x \alpha$ if $x$ \emph{does not occur free} in $\alpha$
    \end{axiom}
    \begin{axiom}
        \label{axiom:Equality}
        Wffs of the form $x \doteq x$
    \end{axiom}
    \begin{axiom}
        \label{axiom:EqualitySubstitution}
        Wffs of the form $x \doteq y \to (\alpha \to \alpha')$ where $\alpha$ is atomic and $\alpha'$ is obtained from $\alpha$ by replacing zero or more free occurrences of $x$ in $\alpha$ by $y$
    \end{axiom}
\end{definition}

\begin{lemma}
    A wff $\varphi$ is valid $\iff$ $\forall{x}\varphi$ is valid.
\end{lemma}

For example,

\begin{itemize}
    \item $\alpha=Py$, $Py \to \forall x Py$ is an instance of Axiom \ref{axiom:QuantifyBoundedVar}
    \item $\alpha=Py$, $\forall y(Py\to \forall xPy)$ is a \emph{generalization} of instance of Axiom \ref{axiom:QuantifyBoundedVar}
    \item $\alpha = Px$, $x \doteq y \to Px\to Py$ is an instance of Axiom \ref{axiom:EqualitySubstitution}
    \item $\alpha = Px$, $x \doteq y \to Px \to Px$ is also an instance of Axiom \ref{axiom:EqualitySubstitution} because zero or more $x$ can be substituted
    \item $\alpha = Qxx$, $x\doteq y \to Qxx \to Qxy$ is also an instance of Axiom~\ref{axiom:EqualitySubstitution}
\end{itemize}

Despite being axioms, we can prove the validity of some of these axioms. We detail the proof of \ref{axiom:InstanceOfTautology} and \ref{axiom:Substitution}. Proof of others are trivial.

\subsection{Instance of Tautologies}

In this section we show the validity of Axiom~\ref{axiom:InstanceOfTautology}.

\begin{definition}[Instance of WFFs of Sentential Logic]
    Let $\alpha_1,\dots,\alpha_n,\dots$ be an infinite sequence of wffs of the first-order language of $\mathbb{L}$, $\varphi$ be a wff of sentential logic with junst the connectives $\to$ and $\neg$, $\varphi^\ast$ be the wff of $\mathbb{L}$ obtained by replacing every occurrence of the sentence symbol $A_n$ in $\varphi$ by $\alpha_n$ for each $m$. We say that $\varphi^\ast$ is an \textbf{instance} of $\varphi$
\end{definition}

For example, let $\varphi=A_1\to A_3$, $\varphi^\ast = \alpha_1 \to \alpha_3$ is an instance of $\varphi$.

\begin{definition}[Instance of Tautologies]
    For any tautologies $\varphi$ in the sentential logic, $\varphi^\ast$ is an instance of the tautology $\varphi$
\end{definition}

\begin{lemma}
    Given a structure $\frakA$ for language $\mathbb{L}$,
    \begin{itemize}
        \item $s$ be an assignment function
        \item $\varphi$ be a wff of sentential logic
        \item $\varphi^\ast$ be the instance of $\varphi$
        \item $v$ be the truth assignment such that
        \[ v(A_i) = T \iff \sat{A}{\alpha_i}{s} \]
    \end{itemize}
    Then
    \[ \bar{v}(\varphi) = T \iff \sat{A}{\varphi^\ast}{s} \]
\end{lemma}
\begin{proof}
    Prove by induction.
    \begin{itemize}
        \item[] \textbf{Base.} $\varphi = A_i$ follows immediately from the assumption that $v(A_i) = T \iff \sat{A}{\alpha_i}{s}$
        \item[] \textbf{Inductive.} \begin{enumerate}
            \item $\varphi = \neg \beta$.
            \begin{itemize}
                \item[] $\bar{v}(\neg\beta) = T \iff \sat{A}{(\neg\beta)^\ast}{s}$
                \item[$\equiv$] $\bar{v}(\beta) = F \iff \unsat{A}{\beta^\ast}{s}$
                \item[$\equiv$] $\bar{\beta} = T \iff \sat{A}{\beta^\ast}{s}$
            \end{itemize}
            \item $\varphi = \gamma \to \beta$
            \begin{itemize}
                \item[] $\bar{v}(\gamma\to\beta) = T \iff \sat{A}{\gamma\to\beta}{s}$
                \item[$\equiv$] If $\bar{v}(\gamma) = T$ then $\bar{v}(\beta) = T$ $\iff$ If $\sat{A}{\gamma}{s}$ then $\sat{A}{\beta}{s}$
            \end{itemize}
        \end{enumerate}
    \end{itemize}
\end{proof}

\begin{corollary}
    Every instance of a tautology is valid.
\end{corollary}

\subsection{Substitutions}

\begin{definition}[Substitution for Terms]
    Let $u$ be a term, $x$ be a variable, and $t$ be a term. $u_t^x$ is the result of replacing every occurrence of $x$ in $u$ by $t$
\end{definition}

\begin{definition}[Substitution for Formulas]
    For a wff $\alpha$, a variable $x$, let $\alpha_t^x$ be the result of replacing every free occurence of $x$ in $\alpha$ by $t$,

    \begin{itemize}
        \item If $\alpha$ is atomic. $\alpha = Pu_1,\dots,u_n$, then $\alpha_t^x = Pu_{1t}^x,\dots,u_{nt}^x$
        \item If $\alpha = \neg \beta$, then $\alpha_t^x = \neg\beta_t^x$
        \item if $\alpha = \beta\to\gamma$, then $\alpha_t^x = \beta_t^x \to \gamma_t^x$
        \item If $\alpha = \forall y \beta$, then $\alpha_t^x = \alpha$ if $y=x$, and $\alpha_t^x = \forall y \beta_t^x$
    \end{itemize}
\end{definition}

Now we return to Axiom~\ref{axiom:Substitution}. Is every $\forall x \alpha \to \alpha_t^x$ valid?

Of course the answer is ``No'', or otherwise we would not have required that $t$ is \emph{substitutable} for $x$ in $\alpha$.

To see this, let $\alpha = \exists y x\neq y$, $t=y$, then

\[ \forall x \exists y x\neq y \to \exists y y\neq y \]

which is obviously not valid.

The problem here is that by this substitution we made $x$ ``local/bounded'' (\emph{capture of free variables}), and the new formula is semantically different from the original one.

\begin{definition}[Substitutability]
    Let $\alpha$ be a wff, $x$ be a variable, and $t$ be a term. $t$ is substitutable for $x$ in $\alpha$ if
    \begin{itemize}
        \item $\alpha$ is atomic
        \item $\alpha = \neg\beta$ and $t$ is substitutable for $x$ in $\beta$.
        \item $\alpha = \beta \to \gamma$ and $t$ is substitutable for $x$ in $\beta$ and $\gamma$
        \item $\alpha = \forall y \beta$ and
        \begin{itemize}
            \item either $x$ does not occur free in $\forall y \beta$
            \item or $x$ occurs free in $\forall y\beta$ (which implies $x\neg y$), and $t$ is substitutable for $x$ in $\beta$, and $y$ does not occur in $t$
        \end{itemize}
    \end{itemize}
\end{definition}

We can check this with our previous example $\alpha = \exists y x\neq y$, $t=y$. $\alpha = \neg\forall y (\neg x \neq y)$. Notice that $y$ ocurrs in $t$ ($t=y$), therefore $y$ is not substitutable for $x$ in $\alpha$.

\subsubsection{Substitution Lemma}

\begin{lemma}
    Given a first-order language $\mathbb{L}$, let $\frakA$ be a structure for $\mathbb{L}$, $s$ be an assignment for $\frakA$, $u$ and $t$ be two terms and $x$ be a variable. Then
    \[ \bar{s}(u_t^x) = \overline{s(x|\bar{s}(t))}(u) \]
\end{lemma}

This looks very intuitive. It just states that the assignment for substitution is equal to first changing the assignment function and then apply the assignment on original term.

\begin{proof}
    Prove by induction.
    \begin{itemize}
        \item[] \textbf{Base.} (1) If $u = c$, trivial. (2) If $u=x$, then LHS would be $\bar{s}(t)$, RHS would also be $\bar{s}(t)$. (3) $u=y\neq x$, then LHS and RHS will both be $\bar{s}(y)$.
        \item[] \textbf{Inductive.} If $u = ft_1\dots t_n$.
        \[ \bar{s}(u_t^x) = \bar{s}(ft_{1t}^x\dots f(t_{nt}^x)) = f(\bar{s}(t_{1t}^x\dots t_{nt}^x)) \]
        which can be shown equal to RHS by applying inductive hypothesis
    \end{itemize}
\end{proof}

\begin{lemma}[Substitution Lemma]
    \label{lem:SubstitutionLemma}
    Let $s$ be an assignment function for $\frakA$. If $t$ is substitutable for $x$ in $\alpha$, then
    \[ \sat{A}{\alpha_t^x}{s} \iff \sat{A}{\alpha}{s(x|\bar{s}(t))} \]
\end{lemma}
\begin{proof}
    Prove by induction on $\alpha$.
    \begin{itemize}
        \item[] \textbf{Base.} $\alpha = Pt_1\dots t_n$
        \begin{itemize}
            \item[] $\sat{A}{(Pt_1\dots t_n)}{s}$
            \item[$\iff$] $\sat{A}{Pt_{1t}^x\dots t_{nt}^x}{s}$
            \item[$\iff$] $(\bar{s}(t_{1t}^x),\dots,\bar{s}(t_{nt}^x)) \in P^\frakA$
            \item[$\iff$] $(\bar{s'}(t_1),\dots,\bar{s'}(t_n)) \in P^\frakA$
            \item[$\iff$] $\sat{A}{\alpha}{s(x|\bar{s}(t))}$
        \end{itemize}
        \item[] \textbf{Inductive.} \begin{enumerate}
            \item  $\alpha = \neg \beta$
            \begin{itemize}
                \item[] $\sat{A}{\neg \beta_t^x}{s}$
                \item[$\iff$] $\unsat{A}{\beta_t^x}{s}$
                \item[$\iff$] $\unsat{A}{\beta}{s'}$
                \item[$\iff$] $\sat{A}{\neg\beta}{s'}$
            \end{itemize}
            \item $\alpha = \beta \to \gamma$. Similar.
            \item $\alpha = \forall y \beta$.
            \begin{itemize}
                \item[] $\sat{A}{\forall y \beta_t^x}{s}$
                \item[$\iff$] For any $d \in |\frakA|$, $\sat{A}{\beta_t^x}{s(y|d)}$
                \item[] $\sat{A}{\forall y\beta}{s'}$
                \item[$\iff$] For any $d \in |\frakA|$, $\sat{A}{\beta}{s(x|\bar{s}(t))(y|d)}$
                \item[] Since $y \neq x$, we can exchange the order of assignment overwriting
                \item[] For any $d \in |\frakA|$, $\sat{A}{\beta}{s(y|d)(x|\bar{s}(t))}$. For brevity denote $s(y|d)$ by $s''$. By induction hypothesis, we have $\sat{A}{\beta_t^x}{s''} \iff \sat{A}{\beta}{s''(x|\bar{s}''(t))}$. And by Substitutability, we have $\bar{s}''(t) = \bar{s}(t)$. Then we have proved the lemma.
            \end{itemize}
        \end{enumerate}
    \end{itemize}
\end{proof}

\subsubsection{Validity of Axiom~\ref{axiom:Substitution}}

\begin{theorem}
    If $t$ is substitutable for $x$ in $\alpha$, then $\forall \alpha \to \alpha_t^x$ is valid.
\end{theorem}
\begin{proof}
    If $\sat{A}{\forall x \alpha}{s}$, since $s(x|\bar{s}(t))$ is an instance of ``for all $x$'', it holds that $\sat{A}{\alpha}{s(x|\bar{s}(t))}$. Then by the Substitution Lemma~\ref{lem:SubstitutionLemma}, we have $\sat{A}{\alpha_t^x}{s}$.
\end{proof}

Now we have proved that every member in the set of axioms $\Lambda$ is valid.

\section{Deductions}

\subsection{Rule of Inference}

\begin{definition}[Modus Ponens]
    Given any wffs $\alpha$ and $\beta$, the rule of \textbf{modus ponens} provides the operation for deriving $\beta$ from $\alpha\to\beta$ and $\alpha$.
\end{definition}

\subsection{Deduction}

\begin{definition}[Deduction]
    Let $\Gamma$ be a set of wffs of $\mathbb{L}$. A \textbf{deduction from $\Gamma$} is a finite sequence
    \[ \alpha_0,\dots,\alpha_n \]
    of wffs such that for every $\alpha_i$, at least one of the following holds
    \begin{itemize}
        \item $\alpha\in\Gamma$
        \item $\alpha\in\Lambda$
        \item $\alpha$ is inferred by modus ponens from two wffs $\alpha_j$ and $\alpha_k$ s.t. $j,k < i$.
    \end{itemize}
\end{definition}

\begin{definition}
    $\Gamma\vdash\alpha$ ($\alpha$ is a theorem of $\Gamma$) if there is a deduction $\alpha_0,\dots,\alpha_n$ from $\Gamma$ s.t. $\alpha=\alpha_n$.

    We write $\vdash\alpha$ for $\emptyset\vdash\alpha$
\end{definition}

Notice that deduction is purely based on symtax, and the logical implications we have learned before requires semantics.

As an example, we show that $\vdash Px \to \exists yPy$. Since $\Gamma = \emptyset$, we can only derive the conclusion from the axioms and modus ponens.

\[ Px \to \exists yPy \iff Px \to \neg\forall y \neg Py \]

$\forall y \neg Py \to \neg x$ is in axiom group~\ref{axiom:Substitution}. Further, $(\forall y \neg Py \to \neg x) \to (Px \to \neg \forall y \neg Py)$ is an instance of tautology $(A_1 \to \neg A_2) \to (A_2 \to \neg A_1)$. We can then finish the proof by applying modus ponens.

The proof can be formatted into a tree. But I cannot draw it so please refer to slides. A key problem is that few human beings (if any) is able to prove things like this, as it gradually ``factorizes'' the target wffs into members in $\Gamma$ or $\Lambda$. Fortunately we will later see some more helper rules to help normal people prove theorems in first-order logic.

\subsection{Properties of Deductions}

\begin{itemize}
    \item If $\Gamma\vdash\varphi$, then there must be a finite subset $\Delta$ of $\Gamma$ s.t. $\Gamma\vdash\varphi$. This is guaranteed by definition of deduction: it is finite.
    \item If $\alpha_1,\dots,\alpha_n$ is a deduction from $\Gamma$ and $\beta_1,\dots,\beta_m$ is a deduction from $\Gamma$, then $\alpha_1,\dots,\alpha_n,\beta_1,\dots,\beta_m$ is also a deduction from $\Gamma$
    \item If $\varphi\in\Gamma$ then $\Gamma\vdash\varphi$
    \item If $\Gamma\vdash\varphi$ and $\Gamma\subseteq\Delta$ then $\Delta\vdash\varphi$
    \item If $\Gamma\vdash\alpha$ and $\Gamma\vdash\alpha\to\beta$, then $\Gamma\vdash\beta$
    \item (Cut Rule) If $\Gamma\vdash\alpha$ and $\alpha\to\beta$, then $\Gamma\vdash\beta$. This property allows proving some theorem $\beta$ by one or more lemmas $\alpha$
    \item If $\Gamma\to\alpha$ then for any $\beta$, $\Gamma\vdash\beta\to\alpha$. Notice that $\alpha\to\left( \beta\to\alpha \right)$ is an instance of tautology.
    \item If $\Gamma\vdash\alpha\wedge\beta$, then $\Gamma\vdash\alpha$ and $\Gamma\vdash\beta$. Notice that $\alpha\wedge\beta = \neg\left( \alpha\to\neg\beta \right)$, and that $\neg\left( \alpha\to\neg\beta \right) \to \alpha$ (or $\beta$) are two instances of tautologies.
    \item If $\Gamma\vdash\alpha$ and $\Gamma\vdash\beta$ then $\Gamma\vdash\alpha\wedge\beta$
    \item If $\Gamma\vdash\alpha$ or $\Gamma\vdash\beta$ then $\Gamma\vdash\alpha\vee\beta$
    \item If $\Gamma\vdash\alpha\to\beta$ then $\Gamma;\alpha\to\beta$
\end{itemize}

\subsection{Relations between Tautologies and Derivations}

\begin{definition}
    In first-order logic, a set of wffs $\Gamma$ \textbf{tautologically implies} $\varphi$ if there is a wff $\alpha$ and a set $\Sigma$ of wffs in the sentential logic such that for some $*$ of $\mathbb{L}$
    \begin{itemize}
        \item For every $\beta\in\Sigma$, $\beta^*\in\Gamma$
        \item $\varphi=\alpha^*$
        \item $\Sigma$ tautologically implies $\alpha$ in the sentential logic ($\Sigma\vDash\alpha$)
    \end{itemize}
\end{definition}

\begin{lemma}
    $\Gamma\vdash\varphi$ iff $\Gamma\cup\Lambda$ tautologically implies $\varphi$
\end{lemma}

\begin{lemma}[Rule T (Enderton)]
    \label{lem:RuleT}
    If $\Gamma\vdash\alpha_1,\dots,\Gamma\vdash\alpha_n$ and $\{\alpha_1,\dots,\alpha_n\}$ tautologically implies $\beta$ then $\Gamma\vdash\beta$
\end{lemma}

\section{Deduction Theorem and Generalization Theorem}

\subsection{The Deduction Theorem}

\begin{theorem}[Deduction Theorem]
    \label{thm:DeductionTheorem}
    If $\Gamma;\alpha\to\beta$, then $\Gamma\vdash\alpha\to\beta$
\end{theorem}
\begin{proof}
    By induction on $\Gamma;\alpha\to\beta$.
    \begin{itemize}
        \item[] \textbf{Base.} If $\beta\in\Lambda$, then
        \[ \vdash\beta \Rightarrow \vdash \alpha\to\beta \Rightarrow \Gamma\vdash\alpha\to\beta \]
        If $\beta\in\Gamma$, then
        \[ \Gamma\vdash\beta \Rightarrow \Gamma\vdash\alpha\to\beta \]
        If $\beta=\alpha$, then it is an instance of tautology so it is in $\Lambda$
        \item[] \textbf{Inductive.} If $\Gamma;\alpha\vdash\gamma\to\beta$ and $\Gamma;\alpha\vdash\gamma$. By IH, $\Gamma\vdash\alpha\to(\gamma\to\beta)$ and $\Gamma\vdash\alpha\to\gamma$. Notice that $\{ \alpha\to(\gamma\to\beta), \alpha\to\gamma \}$ tautologically imply $\alpha\to\beta$. Then we apply ``Rule T''~\ref{lem:RuleT}.
    \end{itemize}
\end{proof}

\subsubsection{Contraposition}

\emph{Contraposition} is a corollary of the deduction theorem

\begin{theorem}[Contraposition]
    $\Gamma;\varphi\vdash\neg\psi$ iff $\Gamma;\psi\vdash\neg\varphi$
\end{theorem}
\begin{proof}
    \begin{itemize}
        \item[$\Rightarrow$] If $\Gamma;\varphi\vdash\neg\psi$, then by Deduction Theorem~\ref{thm:DeductionTheorem}, $\Gamma\vdash\varphi\to\neg\psi$. Notice that $\left( \varphi\to\neg\psi \right) \to \left( \psi\to\neg\varphi \right)$ is an instance of tautological implication in sentential logic. Therefore by Rule T~\ref{lem:RuleT} $\Gamma\vdash\psi\to\neg\varphi$, and $\Gamma;\psi\to\neg\varphi$. The converse is similar
    \end{itemize}
\end{proof}

\subsection{The Generalization Theorem}

\begin{theorem}[Generalization Theorem]
    \label{thm:GeneralizationTheorem}
    If $\Gamma\vdash\varphi$ and $x$ does not occur free in any member of $\Gamma$ then $\Gamma\vdash\forall x \varphi$
\end{theorem}
\begin{proof}
    By induction on $\varphi$
    \begin{itemize}
        \item[] \textbf{Base.} If $\varphi\in\Gamma$, then $x$ does not occur free in $\varphi$. Notice that we have $\varphi\to\forall x \varphi \in \Lambda$ (axiom~\ref{axiom:QuantifyBoundedVar}), and we have $\varphi\in\Gamma$, by modus ponens we have $\forall x \varphi$.

        If $\varphi\in\Lambda$, $\forall x\varphi$ is a generalization of $\varphi$.
        \item[] \textbf{Inductive.} $\Gamma\vdash\gamma\to\varphi$; $\Gamma\vdash\gamma$. By inductive hypothesis, $\Gamma\to\forall x(\gamma\to\varphi)$, and $\Gamma\vdash\forall x \gamma$. Notice that $\forall x(\gamma\to\varphi) \to \forall x \gamma\to\forall x\varphi$ (axiom~\ref{axiom:PushUniversalIntoImplication}).
    \end{itemize}
\end{proof}

The Deduction Theorem and the Generalization Theorem are ``meta-theorem''s that applies to deductive calculi. In Hilbert-style Calculus, we use the six logical axioms to derive these meta-theorems. In other calculi, they may use other axioms, or directly use these meta-theorems.

\subsubsection{Generalization on Constants}

\begin{theorem}
    \label{thm:GeneralizationOnConsts}
    If $\Gamma\vdash\varphi$ and $c$ is a constant symbol that is not in any member of $\Gamma$, then there is some variable $y$ not in $\varphi$ s.t. $\Gamma\vdash\forall y \varphi_y^c$ ($c$ replaced by $y$).

    Furthermore, there is a deduction of $\forall y \varphi_y^c$ from $\Gamma$ in which $c$ does not occur.
\end{theorem}

Intuitively, if $\varphi$ can be derived from $\Gamma$, then the constant $c$ in $y$ is somewhat like a quantified variable $y$.

\begin{proof}
    Since $\Gamma\vdash\varphi$, we have a deduction
    \[ \varphi_0,\varphi_1,\dots,\varphi_n \]
    s.t. $\varphi_n=\varphi$.

    We first show by induction (on deduction $\Gamma\vdash\varphi_i$) that
    \[ \varphi_{0y}^c,\dots,\varphi_{ny}^c \]
    is a deduction from $\Gamma$.
    \begin{itemize}
        \item[] \textbf{Base.} If $\varphi_i \in \Gamma$, since $c$ does not occur in members of $\Gamma$, we have $\varphi_{iy}^c = \varphi_i$. Then $\Gamma\vdash\varphi_{iy}^c$. If $\varphi_i\in\Lambda$, it can be verified by checking all 6 axioms.
        \item[] \textbf{Inductive.} If $\varphi_j = \varphi_k\to\varphi_i$, $\varphi_k$, $k,j < i$. By IH, $\Gamma\vdash\varphi_{ky}^c \to \varphi_{iy}^c$ and $\Gamma\vdash\varphi_{ky}^c$. We are done by Modus Ponens
    \end{itemize}

    We have shown so far that $\Gamma\vdash\varphi_y^c$ and $y$ does not occur in $\varphi$ (and the finite deduction sequence $\Delta$ that derives $\varphi$). Therefore we have $\Gamma\vdash\forall y \varphi_y^c$

    It also follows from this proof that the deduction does not contain $c$
\end{proof}

\begin{corollary}[Corollary 24G]
    \label{coroll:Corollary24G}
    If $\Gamma\vdash\varphi_c^x$, and $c$ is a constant symbol that is not in $\varphi$ or any member of $\Gamma$, then $\Gamma\vdash\varphi$
\end{corollary}

\begin{corollary}[Rule El]
    If $\Gamma;\varphi_c^x\vdash\psi$ and $c$ does not occur in $\varphi,\psi,\Gamma$, then $\Gamma;\exists x \varphi\vdash\psi$.

    Furthermore, there is a deduction of $\psi$ from $\Gamma;\exists x \varphi$ in which $c$ does not occur.
\end{corollary}
\begin{proof}
    To be completed.
\end{proof}

\subsection{The Re-Replacement Lemma}

\begin{lemma}[Re-Replacement Lemma]
    \label{lem:ReReplacementLemma}
    If $y$ does not occur in $\varphi$, then $x$ is substitutable for $y$ in $\varphi_y^x$ and $\varphi_{yx}^{xy} = \varphi$
\end{lemma}

\subsection{Consistency}

\begin{definition}[Consistency]
    $\Gamma$ is \textbf{inconsistent} if there is some wff $\alpha$ s.t. $\Gamma\vdash\alpha$ and $\Gamma\vdash\neg\alpha$

    $\Gamma$ is \textbf{consistent} if it is not inconsistent.
\end{definition}

Some properties of consistency

\begin{itemize}
    \item If $\Gamma$ is inconsistent then for every $\beta$, $\Gamma\vdash\beta$
    \begin{itemize}
        \item[] $\Gamma\vdash\alpha$, $\Gamma\vdash\alpha$
        \item[$\Rightarrow$] $\Gamma;\beta\vdash\alpha$ $\Gamma;\neg\beta\vdash\alpha$ (Add $\beta$ to $\Gamma$ and the deduction should still hold)
        \item[$\Rightarrow$] $\Gamma\vdash\neg\beta\to\alpha$ $\Gamma\vdash\neg\beta\to\neg\alpha$ (Deduction Theorem)
        \item[$\Rightarrow$] Notice that $\neg\beta\to\alpha, \neg\beta\to\neg\alpha$ tautologically implies $\beta$. By Rule T we have $\Gamma\vdash\beta$
    \end{itemize}
    \item If $\Gamma\nvdash\alpha$ $\iff$ $\Gamma;\neg\alpha$ is consistent.
    \begin{itemize}
        \item Equivalently, $\Gamma\vdash\alpha$ iff $\Gamma;\neg\alpha$ is inconsistent. $\Rightarrow$ is trivial.
        \item $\Leftarrow$ If $\Gamma;\neg\alpha$ is inconsistent, then there exists some $\beta$ s.t. $\Gamma;\neg\alpha\vdash\beta$ and $\Gamma;\neg\alpha\vdash\neg\beta$. The rest is similar to the previous proof.
    \end{itemize}
    \item $\Gamma$ is consistent $\iff$ every finite subset of $\Gamma$ is consistent.
    \begin{itemize}
        \item[] Equivalent to $\Gamma$ is inconsistent $\iff$ there is some subset of $\Gamma$ which is inconsistent
        \item[$\Rightarrow$] $\Gamma\vdash\alpha$, $\Gamma\vdash\neg\alpha$. Then there exists some finite subset $\beta_1\vdash\alpha$ and $\beta_2\vdash\neg\alpha$. And the union of $\beta_1$ and $\beta_2$ is the desired subset.
        \item[$\Leftarrow$] Trivial.
    \end{itemize}
    \item If $\Gamma$ is consistent, then for every $\alpha$, either $\Gamma;\alpha$ is consistent or $\Gamma;\neg\alpha$ is consistent.
    \begin{itemize}
        \item Prove by contradiction. Assume for some $\alpha$, $\Gamma;\alpha$ is inconsistent, and $\Gamma;\neg\alpha$ is also inconsistent. Notice that $\Gamma;\neg\alpha$ is inconsistent iff $\Gamma\vdash\alpha$, so we have $\Gamma\vdash\alpha$. We also have $\Gamma\vdash\neg\alpha$ iff $\Gamma;\neg\neg\alpha$\footnote{We are working on pure grammatical level, so if the wffs ``looks'' different, then they are different, so we cannot directly say that $\Gamma;\neg\neg\alpha$ is inconsistent iff $\Gamma;\alpha$ is inconsistent.}. Further, it can be shown that $\Gamma;\neg\neg\alpha$ is inconsistent iff $\Gamma;\alpha$ is inconsistent.
    \end{itemize}
\end{itemize}

\begin{theorem}[Reductio Ad Absurdum]
    \label{thm:ReductioAdAbsurdum}
    If $\Gamma;\alpha$ is inconsistent, then $\Gamma\vdash\neg\alpha$
\end{theorem}

\section{Backward Inference and Prove Strategies}

\subsection{Backward Inference}

With all the properties derived so far, we can formulate a general method to prove theorems.

Assume we are showing $\Gamma\vdash\varphi$

\begin{itemize}
    \item If $\varphi = \alpha\to\beta$, then it suffices to show $\Gamma;\alpha\to\beta$ (Deduction Theorem)
    \item If $\varphi = \forall x \alpha$, and $x$ does not occur free in $\Gamma$, then it suffices to show $\Gamma\vdash\alpha$ (Generalization Theorem)
    \item If $\varphi = \forall x\alpha$, and $x$ occurs free in $\Gamma$, then we pick a variable $y$ that does not occur in $\alpha$ and $\Gamma$, then we have $\forall y \alpha_y^x\vdash\forall x\alpha$ and it suffices to show $\Gamma\vdash\forall y \alpha_y^x$
    \begin{itemize}
        \item To show why we have $\forall y \alpha_y^x\vdash\forall x\alpha$, it suffices to show $\forall y \alpha_y^x\vdash\alpha$. We have $\forall y \alpha_y^x \to \left( \alpha_y^x \right)_x^y = \alpha$ (by Axiom of Substitution and Re-Replacement Lemma).
    \end{itemize}
    \item If $\varphi = \neg\left( \alpha\to\beta \right)$, then it suffices to show $\Gamma\vdash\alpha$ and $\Gamma\vdash\neg\beta$ (Rule T)
    \item If $\varphi=\neg\neg\alpha$, then it suffices to show that $\alpha$ (Rule T)
    \item If $\varphi=\neg\forall x \alpha$, then it suffices to show that $\Gamma\vdash\neg\alpha_t^x$ for some $t$ substitutable for $x$ in $\alpha$
    \begin{itemize}
        \item This is not always possible, because there are cases where $\Gamma\vdash\neg\forall x\alpha$, but $\Gamma\nvdash\neg\alpha_t^x$ for all $t$. In this case, try Contraposition or Reductio Ad Absurdum
    \end{itemize}
    \item For atomic and negations of atomic formula, try contraposition.
\end{itemize}

Note that due to that applying this method will not always lead to a proof, or may lead to something that is not provable, this method is \emph{incomplete}.

\subsubsection{Examples}

\begin{enumerate}
    \item Assume $x$ does not occur free in $\beta$, derive

    \begin{itemize}
        \item $\vdash\forall x \left( \alpha \to \beta \right) \rightarrow \left( \alpha\to\forall x \beta \right)$
        \item[$\Leftarrow$] $\forall x \left( \alpha\to\beta \right)\vdash\exists x\alpha\to\beta$ (Deduction Thm)
        \item[$\Leftarrow$] $\forall x \left( \alpha\to\beta \right), \exists x \alpha \vdash \beta$ (Deduction Thm)
        \item[$\Leftarrow$] $\forall x \left( \alpha\to\beta \right), \exists x \alpha \vdash \neg\neg\beta$ (Rule T)
        \item[$\Leftarrow$] $\forall x \left( \alpha\to\beta \right),\neg\beta \vdash \neg\neg\forall x\neg \alpha$ (Contraposition)
        \item[$\Leftarrow$] $\dots \vdash \forall x\neg \alpha$
        \item[$\Leftarrow$] $\dots\vdash\neg\alpha$ (Generalization, note that $x$ does not occur free in $\beta$)
        \item[$\Leftarrow$] $\forall x \left( \alpha\to\beta \right),\alpha\vdash\neg\neg\beta$ (Contraposition)
        \item[$\Leftarrow$] $\dots\vdash\beta$
    \end{itemize}

    \item Show $\vdash\forall x \forall z Pxz \to \forall{y} Pyy$
    \begin{itemize}
        \item[$\Leftarrow$] $\forall x \forall z Pxz \vdash \forall y Pyy$
        \item[$\Leftarrow$] $\forall x \forall z Pxz \vdash Pyy$
        \item[$\Leftarrow$] $\forall x \forall z Pxz \to \forall z Pyz$ (Substitution)
        \item[] Suffice to show $\forall zPyz \vdash \forall z Pyz$, again can be proved by substitution.
        \item[] However, cannot prove $\vdash\forall x\forall y Pxy \to \forall y Pyy$ like this, although this formula is semantically valid ($\vDash$)
    \end{itemize}

    \item Show $\vdash\forall x \forall y Pxy \to \forall{y} Pyy$
    \begin{itemize}
        \item[] We have shown that Show $\vdash\forall x \forall z Pxz \to \forall{y} Pyy$
        \item[] We will show that $\forall x\forall y Pxy \vdash\forall{x}\forall{z}Pxz$
        \item[$\Leftarrow$] $\forall{x}\forall{y}Pxy \vdash Pxz$ (Generalization)
        \item[] Done. (Substitution)
    \end{itemize}
\end{enumerate}

\subsection{Alphabetic Variants}

\begin{theorem}[Alphabetic Variants]
    \label{thm:AlphabeticVariants}
    Given a wff $\alpha$, a term $t$ and a variable $x$. There is a wff $\alpha'$ s.t. $\alpha'$ differs from $\alpha$ only in quantified variables; $\vdash \alpha\leftrightarrow\alpha'$ and $t$ is substitutable for $x$ in $\alpha'$.
\end{theorem}

Intuitively, we can always find a $\alpha'$ such that we can do substitution whenever there is a conflicting variable in $\alpha$. For the examples mentioned above, $\alpha=\forall{x}\forall{y}Pxy$, and $\alpha'=\forall{x}\forall{z}Pxz$

Can be proved by induction on $\alpha$, and can refer to Enderton.

\subsubsection{Example}

Show $\forall{x}\forall{y}Pxy\vdash\forall{y}Pyy$, and $y$ is \emph{not} substitutable for $x$ in $\forall{y}Pxy$.

Apply the alphabetic variants theorem to $\forall{y}Pxy$ to get $\forall{z}Pxz$.

\begin{itemize}
    \item $\vdash\forall{y}Pxy\leftrightarrow\forall{z}Pxz$
    \item $\forall{x}\forall{y}Pxy\to\forall{x}\forall{z}Pxz$ (Generalization)
    \item $\forall{x}\forall{z}Pxz$ TO BE COMPLETED.
\end{itemize}

\section{The Soundness Theorem}

\begin{lemma}
    Given a set $\Gamma$ of wffs and the wffs $\alpha$ and $\beta$, if $\Gamma\vDash\alpha\to\beta$ and $\Gamma\vDash\alpha$, then $\Gamma\vDash\beta$
\end{lemma}

\begin{theorem}[Soundness Theorem]
    \label{thm:SoundnessTheorem}
    If $\Gamma\vdash\alpha$, then $\Gamma\vDash\alpha$
\end{theorem}
\begin{proof}
    Prove by induction on the deduction. Assume $\Gamma=\{ \gamma_0,\dots,\gamma_n \}$, we show that $\Gamma\vdash\varphi_i$ for all $i$
    \begin{itemize}
        \item[] \textbf{Base.} If $\varphi_i \in \Lambda$, by validity of all axioms, it holds. If $\varphi_i \in \Gamma$, then obviously $\Gamma\vDash\varphi_i$
        \item[] \textbf{Inductive.} Suppose there exists some $\varphi_j$ and $\varphi_k$ s.t. $j,k \le i$. Let $\varphi_k = \varphi_j \to \varphi_i$, and $\Gamma\vdash\varphi_j, \Gamma\vdash\varphi_k$. By inductive hypothesis we have $\Gamma\vDash\varphi_j$ and $\Gamma\vDash\varphi_k=\varphi_j\to\varphi_i$. Therefore by lemma we have $\Gamma\vDash\varphi_i$.
    \end{itemize}
\end{proof}

\begin{corollary}
    If $\vdash\alpha$, then $\vDash\alpha$.
\end{corollary}

\begin{corollary}
    If $\vdash\varphi\leftrightarrow\psi$, then $\varphi$ and $\psi$ are logically equivalent.
\end{corollary}
\begin{proof}
    If $\vdash\varphi\rightarrow\psi$, then $\varphi\vDash\psi$. The converse is similar.
\end{proof}

\begin{corollary}
    If $\varphi'$ is an alphabetic variant of $\varphi$, then $\varphi$ and $\varphi'$ are logically equivalent.
\end{corollary}

\subsection{Soundness and Satisfiability}

\begin{corollary}
    If $\Gamma$ is satisfiable, then $\Gamma$ is consistent.
\end{corollary}

In fact, soundness is equivalent to the above corollary.

\begin{theorem}
    The following two statements are equivalent.
    \begin{itemize}
        \item For any $\Gamma$ and $\alpha$, if $\Gamma\vdash\alpha$, then $\Gamma\vDash\alpha$
        \item For any $\Gamma$, if $\Gamma$ is satisfiable, then $\Gamma$ is consistent.
    \end{itemize}
\end{theorem}
\begin{proof}
    \begin{itemize}
        \item[$\Rightarrow$] If we have soundness, we prove consistency by contradiction. Assume $\Gamma$ is satisfiable, but $\Gamma$ is inconsistent. Then there exists some $\varphi$ s.t. $\Gamma\vdash\varphi$ and $\Gamma\vdash\neg\varphi$. By soundness we have $\Gamma\vDash\varphi$ and $\Gamma\vDash\neg\varphi$. Since $\Gamma$ is satisfiable, there is some structure $\frakA$ and assignment $s$ s.t. $\sat{A}{\varphi}{s}$ and $\sat{A}{\neg\varphi}{s}$. 寄!
        \item[$\Leftarrow$] Conversely, if we have the latter, we prove soundness by contradiction. Assume $\Delta\vdash\alpha$ but $\Delta\nvDash\alpha$. Then $\Delta;\neg\alpha$ is satisfiable. So $\Delta;\neg\alpha$ is consistent. By property of consistency we have $\Delta\nvdash\alpha$. 寄!
    \end{itemize}
\end{proof}

\section{The Completeness Theorem}

\begin{theorem}[G\"{o}del Extended Completeness Theorem]
    If $\Gamma\vDash\alpha$, then $\Gamma\vdash\alpha$.
\end{theorem}

\begin{corollary}[G\"odel Completeness Theorem]
    If $\vDash\alpha$, then $\vdash\alpha$.
\end{corollary}

\subsection{Equivalent Statement for Completeness}

\begin{theorem}
    The following statements are equivalent
    \begin{itemize}
        \item For any $\Gamma$ and $\alpha$, if $\Gamma\vDash\alpha$, then $\Gamma\vdash\alpha$
        \item For any $\Gamma$, if $\Gamma$ is consistent then $\Gamma$ is satisfiable
    \end{itemize}
\end{theorem}
\begin{proof}
    \begin{itemize}
        \item[$\Rightarrow$] Assmue completeness, and assume $\Gamma$ is consistent. Assume for contradiction that $\Gamma$ is unsatisfiable. Then $\Gamma\vDash\alpha$ and $\Gamma\vDash\neg\alpha$. By completeness $\Gamma\vdash\alpha$ and $\Gamma\vdash\neg\alpha$, which means $\Gamma$ is inconsistent.
        \item[$\Leftarrow$] Assume the latter, we prove completeness by constradiction. Assume $\Gamma\vDash\alpha$ but $\Gamma\nvdash\alpha$. Then by property of consistency $\Gamma;\neg\alpha$ is consistent and thus satisfiable. Therefore $\Gamma\vDash\neg\alpha$. 寄!
    \end{itemize}
\end{proof}

\subsection{Proof of Completeness}

\emph{“20世纪逻辑学最重要的发现之一。”}

The proof is similar to that for compactness.

\begin{enumerate}
    \item Extend $\Gamma$ to $\Delta\supseteq\Gamma$ s.t. $\Delta$ is consistent and maximal (for any $\alpha$, either $\alpha\in\Delta$ or $\neg\alpha\in\Delta$)
    \item Define a structure $\frakA$ and an assignment $s$ for $\frakA$ s.t. $\frakA$ satisfies $\Gamma$ with $s$
\end{enumerate}

But the actual proof is more complex, because we have to deal with $\doteq$. The actual proof consists of 6 steps.

\subsubsection{Expanding a Language with Constants}

Let $\Gamma$ be a consistent set of wffs in a \emph{countable} language. We expand the language with a countably infinite set of new constant symbols $c_1,\dots,c_n,\dots$

Assume $\Gamma$ is defined in $\mathbb{L}$, and let
\[ \mathbb{L}' = \mathbb{L}\cup\{ c_1,\dots,c_n,\dots \} \]

We show by contradiction that $\Gamma$ is also consistent in $\mathbb{L}'$.

\begin{proof}
    Assume $\Gamma$ is inconsistent in $\mathbb{L}'$. Then there is some wff $\alpha$ s.t. $\Gamma\vdash\alpha$ and $\Gamma\vdash\neg\alpha$. Therefore $\Gamma\vdash\alpha\wedge\neg\alpha$. $\alpha$ may contain the new introduced constant symbols, but since we have Generalization on Constants, we can show that there is some deduction
    \[ \alpha_1',\dots,\alpha_n' = \alpha'\wedge\neg\alpha' \]
    where $\alpha_i'$ is $\alpha_i$ with all constants not in $\mathbb{L}$ replaced with some variable. Therefore $\Gamma$ is inconsistent in $\mathbb{L}$.
\end{proof}

\subsubsection{Preparing for Satisfiability of Quantified WFFs}

In the new language, for any pair of wff $\varphi$ and variable $x$, we introduce a formula
\[ \neg\forall{x}\varphi \to \neg\varphi_c^x \]

where $c$ is a new constant symbol. Let $\Theta$ be the set of all these formulas. This is essentially (1) $c$ identifies a \emph{counter example} for $\varphi$, and (2) $\Gamma\cup\Theta$ is consistent.

Note that the pairs of $\varphi$ and $x$ are enumerable

\[ (\varphi_1,x_1), (\varphi_2,x_2),\dots \]

and the newly introduced constant are also enumerable

\[ c_1,c_2,\dots \]

Since each $\varphi$ contains finitely many constants, there must be some certain way of enumeration such that $c_{i+1}$ does not occur in $\varphi_1,\dots,\varphi_i$.

We now show that $\Gamma\cup\Theta$ is consistent.

\begin{proof}
    Assume for contradiction that $\Gamma\cup\Theta$ is inconsistent. Then $\Gamma\cup\Theta\vdash\alpha$ and $\Gamma\cup\Theta\vdash\neg\alpha$.

    So there is some finite subset $\Theta_k = \{\theta_1,\dots,\theta_k\} \subseteq \Theta$ s.t. $\Gamma\cup\Theta_k$ is inconsistent. This is guaranteed by the finiteness of deduction. Notice that since $\Gamma$ itself is consistent, $\Theta_k$ cannot be empty. Let $k$ be the minimal number s.t. $\Gamma\cup\Theta_k$ is inconsistent.

    By Reductio Ad Absurdum, we have
    \[ \Gamma:=\Gamma\cup\{ \theta_1,\dots,\theta_{k-1} \} \vdash \neg \theta_k \]
    where
    \[ \theta_k = \neg\forall x \varphi_k \to \neg \varphi_{kc_k}^x \]

    Therefore
    \[ \Gamma' \vdash \neg\forall{x}\varphi_k \wedge \varphi_{kc_k}^k \]
    i.e. $\Gamma' \vdash \neg\forall{x}\varphi_k$ and $\Gamma'\vdash\varphi_{kc_k}^k$

    Note that from a corollary of Generalization on constants \ref{coroll:Corollary24G}. We have $\Gamma'\vdash\varphi_{kc_k}^k\Longrightarrow \Gamma'\vdash\forall{x}\varphi_k$. This shows that $\Gamma'$ is inconsistent, which is contradictory to our assumption that $k$ (not $k-1$) is the smallest number that makes $\Gamma\cup\Theta_k$ inconsistent.
\end{proof}

\subsubsection{Generate Maximally Consistent Set}

We extend $\Gamma\cup\Theta$ to a set $\Delta$ s.t.

\begin{itemize}
    \item $\Delta$ is consistent.
    \item For each wff $\alpha$, either $\alpha\in\Delta$ or $\neg\alpha\in\Delta$
\end{itemize}

\begin{proof}
    $\Gamma\cup\Theta$ is consistent.
    \begin{itemize}
        \item There is no $\beta$ s.t. $\Gamma\cup\Theta\vdash\beta$ and $\Gamma\cup\Theta\vdash\neg\beta$.
        \item[$\Leftrightarrow$] There is no $\beta$ s.t. $\Gamma\cup\Theta\cup\Lambda$ tautologically imply $\beta$ and $\neg\beta$
        \item[$\Rightarrow$] Therefore $\Gamma\cup\Theta\cup\Lambda$ is satisfiable \emph{in the sense of sentential logic}
        \item[$\Rightarrow$] There is some truth assignment $v$ satisfying $\Gamma\cup\Theta\cup\Lambda$. Therefore we pick
        \[ \Delta = \{ \varphi | \bar{v}(\varphi) = T \} \]
    \end{itemize}
\end{proof}

We continue to show some properties of $\Delta$.

\begin{proposition}
    For any $\alpha$, either $\alpha\in\Delta$ or $\neg\alpha\in\Delta$, but not both.
\end{proposition}
\begin{proof}
    If $\bar{v}(\alpha) = T$, then $\alpha\in\Delta$. $\bar{v}(\neg\alpha)=F$. Then $\neg\alpha\notin\Delta$. Conversely, if $\bar{v}(\alpha)=F$, then $\neg\alpha\in\Delta$, and thus $\alpha\notin\Delta$.
\end{proof}

\begin{proposition}
    $\Delta$ is a \emph{theory}. That is,
    \[ \Delta\vdash\alpha \Longrightarrow \alpha\in\Delta \]
\end{proposition}
\begin{proof}
    If $\Delta\vdash\alpha$, then $\Delta\cup\Lambda$ tautologically implies $\alpha$. This is equivalent to $\Delta$ tautologically implies $\alpha$, because by definition of $\Delta$ we have $\Lambda\subseteq\Delta$. Since we know that $v$ satisfies $\Delta$ (in sentential logic), by tautological implication we know that $v$ should also satisfy $\alpha$. Thus $\alpha\in\Delta$
\end{proof}

\begin{proposition}
    $\Delta$ is consistent.
\end{proposition}
\begin{proof}
    If $\Delta$ is inconsistent, then there exists some formula $\beta$ s.t. $\Delta\vdash\beta$ and $\neg\beta$. Then $\beta\in\Delta$ and $\neg\beta\in\Delta$. 噔噔咚
\end{proof}

\subsubsection{Make a Structure for the New Language}

We make a structure $|\frakA|$ from $\Delta$ for the new language where $\doteq$ is replaced by a 2-nary symbol $E$.

sudo make install!

\begin{itemize}
    \item Let $|\frakA|$ be the set of all terms in the new language
    \item $(u,t)\in E^\frakA \iff u\doteq t \in \Delta$
    \item For any n-ary predicate symbol,
    \[ (t_1,\dots,t_n) \in P^\frakA \iff Pt_1\dots t_n \in \Delta \]
    \item For any n-ary predicate symbol,
    \[ f^\frakA (t_1,\dots,t_n) = ft_1\dots t_n \]
    \item For any constant symbol $c$, $c^\frakA=c$
\end{itemize}

Then we make an assignment function $s:V\mapsto|\frakA|$
\[ s(x)=x \]

Then $\bar{s}(t)=t$. For any wff $\varphi$, let $\varphi^*$ be $\varphi$ with $\doteq$ replaced by $E$. We have
\[ \sat{A}{\varphi^*}{s} \iff \varphi\in\Delta \]

Notice that once we prove this, we have already found a structure $\frakA$ and an assignment $s$ that satisfies $\Delta$. That is, if the language does not contain $\doteq$, we have proved completeness.

\begin{proof}
    Assume $n$ is the number of connectives in $\varphi$. Prove by induction on $n$. We do induction on $n$ to avoid $\doteq$'s in $\varphi$ that may cause troubles.
    \begin{itemize}
        \item[] \textbf{Base.} $n=0$. Then $\varphi$ is $u\doteq t$ or $Pt_1\dots t_n$. If $\varphi$ is $u\doteq t$, then $\varphi^*$ is $Eut$.
        \[ \sat{A}{Eut}{s} \iff (\bar{s}(u), \bar{s}(t)) \in E^\frakA \iff (u,t) \in E^\frakA \iff u\doteq t \in \Delta \]
        If $\varphi$ is $Pt_1\dots t_n$, the proof is similar.

        \item[] \textbf{Inductive.} If $\varphi = \neg\alpha$, then $\varphi^*=\neg\alpha^*$.
        \[ \sat{A}{\neg\alpha^*}{s} \iff \unsat{A}{\alpha^*}{s} \iff \alpha\notin\Delta\iff\neg\alpha\in\Delta \]
        The second step uses the inductive hypothesis.

        If $\varphi=\alpha\to\beta$, then $\varphi^*=\alpha^*\to\beta^*$.
        \[ \sat{A}{\alpha^*\to\beta^*}{s} \iff \unsat{A}{\alpha^*}{s}\text{ or }\sat{A}{\beta^*}{s} \iff \alpha\notin\Delta \text{ or }\beta\in\Delta \]
        \begin{itemize}
            \item We first show $\alpha\notin\Delta$ or $\beta\in\Delta$ implies $\alpha\to\beta\in\Delta$. To show this, it suffices to show that $\Delta\vdash\alpha\to\beta$, and it suffices to show that $\Delta;\alpha\vdash\beta$. Then 分类讨论两种前提 and we are done.\footnote{“事实上我也忘了这个怎么证明，我们来证一下。”}
            \item To show $\alpha\to\beta\in\Delta\Rightarrow\alpha\notin\Delta$ or $\beta\in\Delta$.
            \[ \alpha\to\beta\in\Delta\iff\Delta\vdash\alpha\to\beta\Longleftarrow\Delta;\alpha\vdash\beta \]
        \end{itemize}

        If $\varphi=\forall{x}\alpha$. $\varphi^*=\forall{x}\alpha^*$.
        \[ \sat{A}{\forall{x}\alpha^*}{s} \iff \text{For every $t\in|\frakA|$,} \sat{A}{\alpha^*}{s(x|t)} \]
        \begin{itemize}
            \item[$\iff$] $\sat{A}{\alpha^*}{s(x|\bar{s}(t))}$
            \item[$\iff$] $\sat{A}{(\alpha^*)^x_t}{s}$ (Substitution Lemma)
            \item[$\iff$] $\sat{A}{(\alpha^x_t)^*}{s}$
            \item[$\iff$] $\alpha_t^x \in \Delta$ (IH)
            \item[$\Longrightarrow$] $\alpha_c^x\in\Delta$
        \end{itemize}
        Notice that $\neg\forall{x}\alpha\to\neg\alpha_c^x \in \Delta$. Since we have $\alpha_c^x\in\Delta$, the condition $\neg\forall{x}\alpha$ cannot hold, so $\forall{x}\alpha\in\Delta$

        So far we have proved $\Rightarrow$. Now consider the other side $\forall{x}\alpha\in\Delta\Rightarrow\sat{A}{\forall{x}\alpha^*}{s}$. This is equivalent to showing
        \[ \unsat{A}{\forall{x}\alpha^*}{s} \Longrightarrow \forall{x}\alpha\notin\Delta \]
        Assume there is some $t$ s.t. $\unsat{A}{\alpha^*}{s(x|t)}$, equivalent to $\unsat{A}{(\alpha_t^x)^*}{s}$.

        Let $\beta$ be alphabetic equivalent to $\alpha$ s.t. $t$ is substitutable for $x$ in $\beta$. We have $\alpha\vDash\Dashv\beta$ by alphabetical equivalence.

        Therefore previous equation is equivalent to
        \[ \unsat{A}{(\beta_t^x)^*}{s} \iff \beta_t^x \notin \Delta \]

        If $\forall{x}\alpha\in\Delta$, then $\Delta\vdash\forall{x}\alpha$, and thus $\Delta\vdash\forall{x}\beta$. We have axiom $\forall{x}\beta\to\beta_t^x$ since $t$ is substitutable for $x$ in $\beta$. Thus $\Delta\vdash\beta_t^x$ and $\beta_t^x\in\Delta$. Contradiction. So $\forall{x}\alpha\notin\Delta$, we are done.
    \end{itemize}
\end{proof}

Until now, we are done if the language does not contain $\doteq$. But to actually complete the proof for all cases, we need an extra step.

\subsubsection{Deal with Equality}

Not enough time. Pigeoned. Refer to slides or reference books.


\chapter{Sampling with sufficient statistics}
This chapter will go in depth of generating samples from the NHPP given its sufficient statistics. The main focus will be a Gibbs sampler with a Metropolis-Hastings step, and discuss possible modifications.
\section{Gibbs sampler}
The Gibbs sampler is Markov Chain Monte Carlo(MCMC) sampler. A MCMC sampler is a way to sample from a distribution by contructing a Markov Chain with spesific equilibrium. From \cite{casella1992explaining} we have that the Gibbs sampler is used when direct sampling is difficult or cannot be done. This sampler samples from a conditional distribution. This can be done with sufficient statistics.
\section{Metropolis hastings sampler}
This is also a MCMC sampler where direct sampling is difficult. However this is used when the conditional posterior is unknown. Hence this uses a proportional density instead and has an acceptance rejection step to determine if the new sample is from the desired distribution.
\section{Gibbs with hastings step}
A Gibbs sampler with a Metropolis-Hastings step is what we used in this project \cite{gilks1995adaptive}. Since the values of the sufficient statistics cannot differ from the original data set, the sampler must generate new samples with the same values for the sufficient statistics. Given a sample with $N$ points, one can draw three arbitrary points and draw a new value for one of the points given the two others. This is where the Metropolis-Hastings step is used to draw a new value. How to find a new value is shown later. Further when this new value is drawn the two other values must be adjusted to fullfill the sufficient statistics. This is the idea behind our Gibbs sampler.
\subsection{Drawing a new value}
Assume we have chosen three points($X_1, X_2, X_3$) from our sample and that we have the same sufficient statistics as in section 5.2. Another assumption is that all values in the samples must be between 0 and 1. Hence $\tau = 1$. This makes it easier for the uniform distribution. To draw a new value we need to use tranformation of variables. The tranformation is
\begin{equation}
Z = (\sum_{i=1}^{3} X_i, \prod_{i=3}^{2} X_i, X_3),
\end{equation}
where $X_3$ is going to be updated. The determinant of the Jacobian matrix then becomes
\begin{equation}
|J| = X_3 (X_2 - X_1).
\end{equation}
We also have the following relations from the tranformation
\begin{align}
X_1 + X_2 + X_3 = a \\
X_1 + X_2 = a - X_3 \\
X_1 \cdot X_2 \cdot X_3 = b \\
X_1 \cdot X_2 = \frac{b}{X_3}.
\end{align}
Then $X_1$ and $X_3$ are the roots of
\begin{equation}
X^2 - (a - X_3) X^2 + \frac{b}{X_3},
\end{equation}
which is
\begin{equation}
X_{(1,2)} = \frac{(a - X_3) ± \sqrt{(a - X_3)^2 - \frac{4b}{X_3}}}{2}.
\end{equation}
From this the jacobian becomes
\begin{equation}
|J| = X_3\sqrt{(a - X_3)^2 - \frac{4b}{X_3}}
\end{equation}
The density we are going to draw from is then
\begin{equation}
\Pi \propto \frac{1}{X_3\sqrt{(a - X_3)^2 - \frac{4b}{X_3}}}.
\end{equation}
This is where the metropolis hastings step is used. It is imoportant that this density has some requirements. These are listed below.
\begin{align}
0 \leq X_3 \leq \min(\tau, a) \\
0 \leq (a - X_3) ± \sqrt{(a - X_3)^2 - \frac{4b}{X_3}} \leq 1.
\end{align}
So when we draw from a proposal density like a uniform distribution we need to make sure that the new $X_3$ fullfills these requirements. If not the probability of accepting this new value is zero. If the requirements are fullfilled then the acceptance probability is defined as,
\begin{equation}
\alpha = \min\left(1, \frac{\Pi(X_{prop})}{\Pi(X_{curr})}\right)
\end{equation}
An overview of the algorithm is shown below.
\begin{algorithm}
\caption{Generate new samples with sufficient statistics}
\label{alg:simdata}
\begin{algorithmic}
\STATE Draw 3 arbitrary X's
\STATE Caclulate sum and product of these.
\STATE Draw a proposal $X_3$ from $U[0,1]$
\STATE Check requirements
\STATE Calculate $\alpha$
\STATE Accept with probability $\alpha$
\STATE Repeat desired times.
\end{algorithmic}
\end{algorithm}
\subsection{Possible modifications}
In our current version if the sampler generates invalid values for the three new indices in a sample, the algorithm tries over again with three new indices. One possible modification is to not try over again with three new indices, but keep the three indices and draw new random value. However the area for valid values might be small. Hence this modification might not be recommended.
\\
\\
Another modification is to use the indices of the entire sample when generating a new sample. This is done before for another distribution in \cite{lockhart2007use}. The pros of this method is that you new values for all the indices. However the cons is that the values for the sufficient statistics are sensitive to large and small values when the number data points in a sample is large.


% To be compiled by XeLaTeX, preferably under TeX Live.
% LaTeX source for ``Yanqi Lake Lectures on Algebra'' Part III.
% Copyright 2019  李文威 (Wen-Wei Li).
% Permission is granted to copy, distribute and/or modify this
% document under the terms of the Creative Commons
% Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)
% https://creativecommons.org/licenses/by-nc/4.0/

% To be included
\chapter{Some aspects of Koszul complexes}

This lecture is a faithful replay of the relevant sections of \cite{Bour80} and \cite{Bour98}; the main aim here is to complete the proof of Theorem \ref{prop:regular-vs-depth}. In what follows, we work with a chosen ring $R$. We impose no Noetherian or finiteness conditions here.

\section{Preparations in homological algebra}
For any $R$-module $L$, denote its \emph{exterior algebra}\index{exterios algebra} over $R$ by
\[ \bigwedge L := \bigoplus_{n \geq 0} \bigwedge^n L. \]
It is the quotient of the tensor algebra $T(L) = \bigoplus_{n \geq 0} (T^n(L) := L^{\otimes n})$ by the graded ideal generated by the pure tensors
\[ \cdots \otimes x \otimes x \otimes \cdots, \quad x \in L. \]
The multiplication operation in $\bigwedge L$ is written as $\wedge$. Note that $\bigwedge^0 L = T^0(L) = R$ by convention. The traditional notion of exterior algebras encountered in differential geometry is recovered when $\Q \subset R$.

Given $u \in \Hom_R(L, R)$, one can define the corresponding \emph{contractions} $i_u: \bigwedge^{n+1} L \to \bigwedge^n L$, given concretely as
\[ i_u (x_0 \wedge \cdots \wedge x_n) = \sum_{i=0}^n (-1)^i u(x_i) \cdot x_0 \wedge \cdots \widehat{x_i} \cdots \wedge x_n \]
where $\widehat{x_i}$ means $x_i$ is omitted. It is routine to check that $i_u$ satisfies $i_u \circ i_u = 0$, thereby giving rise to a chain complex.

\begin{definition}\label{def:Koszul-general}\index{Koszul complex}
	Let $L$ and $u$ be as above. Define the corresponding \emph{Koszul complex} as $K_\bullet(u) := (\bigwedge^\bullet L, i_u)$. For any $R$-module $M$, put
	\begin{align*}
		K_\bullet(u; M) & := M \dotimes{R} K_\bullet(u), \\
		K^\bullet(u; M) & := \Hom_A(K_\bullet(u), M)
	\end{align*}\index{$K^\bullet(u; M)$}
	which is naturally a chain (resp. cochain) complex in positive degrees; here one regards $M$ as a complex in degree zero. These definitions generalize to the case of any complex $M$, and are functorial in $M$.
\end{definition}

The reader might have encountered the following result in differential geometry.
\begin{proposition}[Homotopy formula]
	For any $x \in L$ and $\omega \in \bigwedge^n L$, we have
	\[ i_u (x \wedge \omega) + x \wedge (i_u(\omega)) = u(x) \omega. \]
\end{proposition}
\begin{proof}
	Consider $\omega = x_1 \wedge \cdots \wedge x_n$. Put $x_0 := x$. The left-hand side equals
	\[ \sum_{i=0}^n (-1)^i \cdot u(x_i) \cdots \wedge \widehat{x_i} \wedge \cdots  \]
	whereas the right-hand side equals
	\[ \sum_{i=1}^n (-1)^{i+1} \cdot u(x_i) x_0 \wedge \cdots \wedge \widehat{x_i} \wedge \cdots. \]
	The terms with a $\widehat{x_i}$ (non-existant --- hopefully this won't generate metaphysical issues) where $i > 0$ cancel out. We are left with $u(x_0) x_1 \wedge \cdots \wedge x_n = u(x) \omega$.
\end{proof}
Obviously, the same formula extends to all $\omega \in \bigwedge L$ by linearity.

\begin{proposition}\label{prop:Koszul-u-vanishing}
	Set $\mathfrak{q} := u(L)$, which is an ideal of $R$. Then $\mathfrak{q}$ annihilates each homology (resp. cohomology) of $K_\bullet(u; M)$ (resp. $K^\bullet(u; M)$). Again, this generalizes to general complexes $M$.
\end{proposition}
\begin{proof}
	Given $t \in \mathfrak{q}$, the homotopy formula implies that the endomorphism $\omega \mapsto t\omega$ of $K_\bullet(u)$ is homotopic to zero, hence so are the induced endomorphisms of $K_\bullet(u; M)$ and $K^\bullet(u; M)$ by standard homological algebra.
\end{proof}

\begin{proposition}\label{prop:Koszul-long-exact-sequence}
	Suppose $L$ is projective over $R$ and $0 \to M' \to M \to M'' \to 0$ is exact. Then there is a natural short exact sequence of complexes
	\[ 0 \to K^\bullet(u; M') \to K^\bullet(u; M) \to K^\bullet(u; M'') \to 0 \]
	which gives rise to a long exact sequence of cohomologies of the Koszul complexes in question.
\end{proposition}
\begin{proof}
	Standard. It suffices to note that $L$ is projective implies each graded piece $\bigwedge^n L$ of $\bigwedge L$ is projective as well.
\end{proof}

\begin{exercise}
	Justify the assertion above concerning the projectivity of $\bigwedge^n L$. Hint: suppose $L$ is a direct summand of a free module $F$, show that $\bigwedge^n L$ is a direct summand of $\bigwedge^n F$ and $\bigwedge^n F$ is free.
\end{exercise}

Similar properties hold for the homological version when $L$ is flat over $R$. One needs the property that $\bigwedge^n L$ is flat if $L$ is.
\begin{exercise}
	Prove the assertion above concerning flatness of $\bigwedge^n L$. Consult the proof in \cite[p.15]{Bour80} if necessary.
\end{exercise}

\section{Auxiliary results on depth}
We fix an ideal $I \subset R$.
\begin{proposition}\label{prop:depth-prod}
	For every family $\{M_\beta\}_{\beta \in \mathcal{B}}$ of $R$-modules, we have
	\[ \mathrm{depth}_I\left(\prod_\beta M_\beta \right) = \inf_\beta \mathrm{depth}_I(M_\beta). \]
\end{proposition}
\begin{proof}
	This follows from $\Ext^n_R(R/I, \prod_i M_i) = \prod_{i \in I} \Ext^n_R(R/I, M_i)$, as is easily seen by taking a projective resolution of $R/I$ and using the fact the $\Hom_R$ preserves direct products in the second variable.
\end{proof}

\begin{remark}
	By stipulation, the empty product is $0$, the zero object of the category $R\dcate{Mod}$. In parallel we define $\inf\emptyset := \infty$, so that Proposition \ref{prop:depth-prod} remains true in this case, since the zero module has infinite depth.
\end{remark}

\begin{proposition}
	Suppose $N$ is an $R$-module annihilated by some $I^m$, where $m \geq 1$. Then $\Ext^i_R(N,M)=0$ whenever $i < \mathrm{depth}_I(M)$.
\end{proposition}
\begin{proof}
	To show $\Ext^i_R(N,M)=0$ for $i < \text{depth}_I(M)$, we begin with the case $m=1$. This case follows by a dimension-shifting argument based on the short exact sequence
	\[ 0 \to K \to (R/I)^{\oplus I} \to M \to 0, \]
	together with induction on $i$ (note that $\Ext^{<0}_R(N,M)=0$ for trivial reasons). The case of general $m$ follows by a standard \emph{dévissage} using
	\[ 0 \to JN \to N \to N/JN \to 0 \]
	and the associated long exact sequence.
\end{proof}

\begin{lemma}\label{prop:depth-power}
	Suppose $J$ is an ideal satisfying $J \supset I^m$ for some $m \geq 1$. Then $\mathrm{depth}_I(M) \leq \mathrm{depth}_J(M)$.
\end{lemma}
\begin{proof}
	From the previous Proposition, we have $\Ext^i_R(R/J, M) = 0$ for $i < \text{depth}_I(M)$ since $I^m$ annihilates $R/J$. The assertion follows upon recalling the definition of depth.
\end{proof}

The following technical results will be invoked in the proof of Theorem \ref{prop:Koszul-depth}.
\begin{proposition}\label{prop:depth-vanishing-aux}
	Let $C^\bullet$ be a cochain complex of $R$-modules such that $n \ll 0 \implies C^n = 0$. Let $h \in \Z$  such that for all integers $n \leq k \leq h$, the depth of $C^n$ with respect to $J_k := \mathrm{ann}(H^k(C^\bullet))$ is $> k-n$, then
	\[ H^{\leq h}(C^\bullet)=0. \]
\end{proposition}
In what follows, we write $H^n = H^n(C^\bullet) = Z^n/B^n$ in the usual notation for homological algebra.
\begin{proof}
	Assume on the contrary that there exists $k \leq h$ with $H^{< k} = 0$ whereas $H^k \neq 0$. Write $J = J_k$. As $J$ annihilates the nonzero $R$-module $H^k$, the criterion of depth-zero modules (Proposition \ref{prop:depth-zero}) implies that $\text{depth}_J(H^k) = 0$. By assumption $\text{depth}_J(C_k) > k-k = 0$, it follows that $\text{depth}_J(Z_k) > 0$ since $Z_k \subset C_k$, by applying the aforementioned criterion of depth-zero. From the short exact sequence
	\[ 0 \to B^k \to Z^k \to H^k \to 0 \]
	we deduce distinguished triangles in $D(R\dcate{Mod})$
	\begin{gather*}
		\mathcal{H}\text{om}(R/J, B^k) \to \mathcal{H}\text{om}(R/J, Z^k) \to \mathcal{H}\text{om}(R/J, H^k) \xrightarrow{+1}, \\
		\mathcal{H}\text{om}(R/J, H^k)[-1] \to \mathcal{H}\text{om}(R/J, B^k) \to \mathcal{H}\text{om}(R/J, Z^k) \xrightarrow{+1}.
	\end{gather*}
	As the leftmost and rightmost terms of the last line are in $D^{\geq 1}(R\dcate{Mod})$, we see
	\[ \text{depth}_J(B^k) \geq 1; \]
	moreover, the piece $\Ext_R^0(R/J, Z^k) \to \Ext_R^0(R/J, H^k) \to \Ext_R^1(R/J, B^k)$ from the long exact sequence shows that $\text{depth}_J(B^k) = 1$. Now for $n < k$ we have short exact sequences
	\[ 0 \to \underbracket{B^n}_{= Z^n} \to C^n \to B^{n+1} \to 0. \]
	Again, one infers from the distinguished triangle
	\[ \mathcal{H}\text{om}(R/J, B^{n+1})[-1] \to \mathcal{H}\text{om}(R/J, B^n) \to \mathcal{H}\text{om}(R/J, C^n) \xrightarrow{+1} , \]
	the assumption $\text{depth}_J(C^n) > k-n$ and descending induction on $n$ that $n < k \implies \text{depth}_J(B_n) = k-n+1$. This is impossible since $B_{\ll 0} = 0$ has infinite depth.
\end{proof}

\begin{corollary}\label{prop:depth-vanishing}
	Let $I \subset R$ be an ideal, $C^\bullet$ be a cochain complex with $n \ll 0 \implies C_n=0$, and $h \in \Z$. Suppose that $n \leq h$ implies $I \cdot H^n(C^\bullet) = 0$ and $\mathrm{depth}_I(C^n) > h-n$. Then $H^{\leq h}(C^\bullet) = 0$.
\end{corollary}
\begin{proof}
	For $k \leq h$ we have $J_k := \text{ann}(H^k(C^\bullet)) \supset I$. Hence Lemma \ref{prop:depth-power} entails that
	\[ n \leq k \leq h \implies \text{depth}_{J_k}(C^n) \geq \text{depth}_I(C^n) > h-n \geq k-n. \]
	Now apply Proposition \ref{prop:depth-vanishing-aux}.
\end{proof}

\section{Koszul complexes and depth}
The simplest Koszul complexes are defined as follows. Given $x \in R$, we form the cochain complex in degrees $\{0,1\}$
\[ K(x) := \left[ R \xrightarrow{x} R \right]. \]
More generally, for any $R$-module $M$, viewed as a complex concentrated in degree zero, and a family $\mathbf{x} = (x_\alpha)_{\alpha \in \mathcal{A}}$ of element of $R$, we define the associated \emph{Koszul complex}\index{Koszul complex} as
\[ K^\bullet(\mathbf{x}; M) := K^\bullet(u; M), \]
where we take $u: R^{\oplus \mathcal{A}} \to R$ corresponding to $\mathbf{x}$ in Definition \ref{def:Koszul-general}. Unfolding definitions, we have \index{$K^\bullet(\mathbf{x};M)$}
\[ K^h(\mathbf{x}; M) = \begin{cases}
	\Hom_R\left( \bigwedge^h (R^{\oplus \mathcal{A}}), M \right), & h \geq 0 \\
	0, & h < 0.
\end{cases} \]
Therefore $K^h(\mathbf{x}; M)$ consists of families $m(\alpha_1, \ldots, \alpha_h) \in M$ that are alternating in the variables $\alpha_1, \ldots, \alpha_h \in \mathcal{A}$. The differential is
\begin{align*}
	\partial^h: K^h(\mathbf{x}; M) & \longrightarrow K^{h+1}(\mathbf{x}; M) \\
	m & \longmapsto \left[ (\alpha_0, \ldots, \alpha_h) \mapsto \sum_{j=0}^h (-1)^j x_{\alpha_j} \cdot m(\ldots, \widehat{\alpha_j}, \ldots) \right].
\end{align*}
We shall abbreviate the cohomologies of $K^\bullet(\mathbf{x}; M)$ as the \emph{Koszul cohomologies}.

When $\mathcal{A} = \{1, \ldots, n\}$ we revert to
\[ K^\bullet(x_1, \ldots, x_n; M) := M \otimes K(x_1) \otimes \cdots \otimes K(x_n) \]
with the well-known sign convention. Also note that $R^{\oplus \mathcal{A}}$ is projective, hence the Proposition \ref{prop:Koszul-long-exact-sequence} can always be applied to Koszul cohomologies.

Let $I$ denote the ideal generated by $\{x_\alpha: \alpha \in \mathcal{A} \}$. The reader is invited to verify that
\begin{itemize}
	\item $K^\bullet\left( \mathbf{x}, \prod_{\beta \in \mathcal{B}} M_\beta \right) = \prod_{\beta \in \mathcal{B}} K^\bullet(\mathbf{x}; M_\beta)$ for any family $\{M_\beta\}_{\beta \in \mathcal{B}}$ of $R$-modules, and same for their cohomologies;
	\item the $0$-th cohomology of $K^\bullet(\mathbf{x}; M)$ is $\Hom_R(R/I, M) = \{x \in M: Ix=0 \}$, the $I$-torsion part of $M$;
	\item if $|A| = n$, the $n$-th cohomology of $K^\bullet(\mathbf{x}; M)$ is $M/IM$.
\end{itemize}
These facts will cast in our later arguments.

\begin{lemma}\label{prop:Koszul-coho-ann}
	Each cohomology of $K^\bullet(\mathbf{x}; M)$ is annihilated by $I$.
\end{lemma}
\begin{proof}
	Apply Proposition \ref{prop:Koszul-u-vanishing} by observing that $\mathfrak{q} = I$ in our setting.
\end{proof}

\begin{theorem}\label{prop:Koszul-depth}\index{depth}
	Let $M$ be an $R$-module and $\mathbf{x} = \{x_\alpha \}_{\alpha \in \mathcal{A}}$ be a family of elements of $R$, which generate an ideal $I$ of $A$. Then $\mathrm{depth}_I(M)$ equals
	\[ \inf \left\{n \geq 0: H^n(K^\bullet(\mathbf{x}; M)) \neq 0 \right\} \in \Z_{\geq 0} \sqcup \{\infty \} . \]
	Consequently, if $I \subset R$ is an ideal generated by elements $x_1, \ldots, x_n$ and $IM \neq M$, then we have
	\[ \mathrm{depth}_I(M) \leq n. \]
\end{theorem}
\begin{proof}
	Let $d := \text{depth}_I(M)$. Since $K^h(\mathbf{x}; M)$ is a direct product of copies of $M$ (possibly the empty product $=0$), by Proposition \ref{prop:depth-prod} its depth equals either $d$ or $\infty$. Combining Lemma \ref{prop:Koszul-coho-ann} and Corollary \ref{prop:depth-vanishing} (with $h = d-1$), we see that $H^{< d}(K^\bullet(\mathbf{x}; M)) = 0$. It remains to show $H^d(K^\bullet(\mathbf{x}; M)) \neq 0$ provided that $d < \infty$, which we assume from now onwards.

	The case $d=0$ is clear since there exists $\mathfrak{p} \in \text{Ass}(M) \cap V(I)$, therefore $R/\mathfrak{p} \hookrightarrow M$ and $\exists x \in M$ with $\mathfrak{p} x \supset Ix = \{0\}$, whence $H^0(K^\bullet(\mathbf{x}; M)) \neq 0$. Now suppose $d \in \Z_{\geq 1}$ and assume $H^d(K^\bullet(\mathbf{x}; M)) = 0$. Take a free resolution $F_\bullet \to R/I \to 0$ and put $C^\bullet := \Hom_R(F_\bullet, M)$, so that
	\[ H^i(C^\bullet) \simeq \Ext^i_R(R/I, M), \quad i \geq 0. \]
	Hence for $i < d$ we have short exact sequences
	\[ 0 \to B^i \to C^i \to B^{i+1} \to 0 \]
	as usual; recall $B^i, Z^i \subset C^i$. Note that each $C^i$ is a direct product of copies of $M$, hence $H^{\leq d}(K^\bullet(\mathbf{x}; C^i)) = 0$. Indeed, $H^{< d}(K^\bullet(\mathbf{x}; M))=0$ has been settled in the first step, whilst $H^d(K^\bullet(\mathbf{x}; M))$ is the hypothesis to be refuted. It then follows from the long exact sequence for Koszul cohomologies (Proposition \ref{prop:Koszul-long-exact-sequence}) that
	\[ (s \leq d) \wedge (i < d) \implies H^s(K^\bullet(\mathbf{x}; B^{i+1})) \hookrightarrow H^{s+1}(K^\bullet(\mathbf{x}; B^i)). \]
	Suppose $i < d$. As $B^0 = 0$, an iteration yields
	\[ H^{d-i}(K^\bullet(\mathbf{x}; B^{i+1})) \hookrightarrow \cdots \hookrightarrow H^{d+1}(K^\bullet(\mathbf{x}; B^0)) = 0. \]
	In particular, $i=d-1$ gives rise to $H^1(K^\bullet(\mathbf{x}; B^d)) = 0$. Hence the short exact sequence $0 \to B^d \to Z^d \to H^d \to 0$ (for $C^\bullet$) together with Proposition \ref{prop:Koszul-long-exact-sequence} give rise to
	\[ H^0(K^\bullet(\mathbf{x}; Z^d)) \twoheadrightarrow H^0(K^\bullet(\mathbf{x}; H^d)). \]
	As $H^d := H^d(C^\bullet) \simeq \Ext^d_R(R/I, M)$ is nonzero and annihilated by $I$, the right-hand side is nonzero. Hence $H^0(K^\bullet(\mathbf{x}; Z^d)) \neq 0$ and then $H^0(K^\bullet(\mathbf{x}; C^d)) \neq 0$, as the zeroth Koszul cohomology is nothing but the $I$-torsion part.

	Finally, recall that $C^d \neq \{0\}$ is a direct product of copies of $M$, consequently
	\[ H^0(K^\bullet(\mathbf{x}; M)) \neq 0. \]
	However, this contradicts the earlier result that $H^{<d}(K^\bullet(\mathbf{x}; M))=0$ since $d \geq 1$.
\end{proof}

This yields an alternative characterization of depth. It also completes the proof of Theorem \ref{prop:regular-vs-depth} as promised in the previous lecture.

% Finalizing...
\vfill
\begin{figure}[h]
	\centering \includegraphics[height=200pt]{WanFeng.jpg} \\ \vspace{1em}
	\begin{minipage}{0.7\textwidth}\begin{center}
			\small \fontspec{Noto Serif CJK SC} 《晚风》, 李桦, 木刻版画, 1963 年.
	\end{center}\end{minipage}
\end{figure}
\vfill


\section*{Problem 2 Solution}

\begin{enumerate}[a)]

\item

The total mass-energy of the excited $^{236}$U atom is the sum of the masses of the reactants: the $^{235}$U atom and the neutron.
\begin{align*}
m(^{236}\text{U}^*)	&= m(^{235}\text{U}) + m_n \\
					&= 235.043930\text{ amu} + 1.00866492\text{ amu} \\
					&= 236.052595\text{ amu}
\end{align*}
The excitation energy of $^{236}$U is the difference between the total mass-energy of the system and the rest-mass of $^{236}$U (multiplied by $c^2$, converting the mass-energy in amu to MeV).
\begin{align*}
E_{\text{ex}}	&= \left[m(^{236}\text{U}^*) - m(^{236}\text{U})\right]c^2 \\
				&= \left[236.052595\text{ amu} - 236.045568\text{ amu}\right]c^2 \\
				&= (0.007027\text{ amu})c^2 \\
E_{\text{ex}}	&= 6.545\text{ MeV}
\end{align*}

\item

The total mass-energy of the excited $^{239}$U atom is the sum of the masses of the reactants: the $^{238}$U atom and the neutron.
\begin{align*}
m(^{239}\text{U}^*)	&= m(^{238}\text{U}) + m_n \\
					&= 238.050788\text{ amu} + 1.00866492\text{ amu} \\
					&= 239.059453\text{ amu}
\end{align*}
The excitation energy of $^{239}$U is the difference between the total mass-energy of the system and the rest-mass of $^{239}$U (multiplied by $c^2$, converting the mass-energy in amu to MeV).
\begin{align*}
E_{\text{ex}}	&= \left[m(^{239}\text{U}^*) - m(^{239}\text{U})\right]c^2 \\
				&= \left[239.059453\text{ amu} - 239.054293\text{ amu}\right]c^2 \\
				&= (0.005160\text{ amu})c^2 \\
E_{\text{ex}}	&= 4.806\text{ MeV}
\end{align*}

\item

The 6.545 MeV excitation energy of the $^{236}$U is greater than the 6.2 MeV activation energy of the fission process for that nucleus. This means that even when a $^{235}$U nucleus absorbs a neutron with zero kinetic energy, fission is possible---$^{235}$U is fissile. $^{238}$U does not exhibit this property. When $^{238}$U absorbs a neutron and forms $^{239}$U, the excitation energy of 4.806 MeV is less than the activation energy of 6.6 MeV for fission to occur. This means that the absorbed neutron must have more than about 1.8 MeV of kinetic energy to trigger fission, and so $^{238}$U is fissionable.

\item

This absorption reaction is given by
$$ ^{238}\text{U} + n + 2\text{ MeV} = ^{132}\text{Sn} + ^{106}\text{Mo} + n + \blacksquare\text{ MeV} .$$
Using the masses provided, we can calculate the mass-energy of the reactants to be
\begin{align*}
E_r	&= \left[m(^{238}\text{U}) + m_n\right]c^2 \\
	&= \left(238.050788\text{ amu} + 1.00866492\text{ amu}\right)c^2 \\
	&= 222.673\text{ GeV},
\end{align*}
and the mass energy of the products to be
\begin{align*}
E_p	&= \left[m(^{132}\text{Sn}) + m(^{106}\text{Mo}) + m_n\right]c^2 \\
	&= \left(131.917816\text{ amu} + 105.918137\text{ amu} + 1.00866492\text{ amu}\right)c^2 \\
	&= 222.473\text{ GeV}
\end{align*}
When we add the 2 MeV of kinetic energy, $T$ of the incoming neutron, we find that the $Q$-value of this fission reaction is
$$ E_r + T - E_p = 222.673\text{ GeV}
 + 0.002\text{ GeV} - 222.473\text{ GeV} = 205\text{ MeV} $$
We are told that the product neutron carries 2.5\% of this energy,
\begin{align*}
E_n	&= 0.025(205\text{ MeV}) \\
	&= 5.125\text{ MeV}.
\end{align*}

\underline{This is more than the 1.8 MeV of kinetic energy required to trigger another fission in $^{238}$U, and fission may occur.}
\end{enumerate}
\-\\
{\small *Remember that not all neutrons are born with this energy, but rather in an energy spectrum. In reality, many neutrons produced from a $^{238}$U fission event will not cause subsequent fission events.}


\section*{Exercise 5.1}
We know that $\bar{x}=26.035$, $s=4.78476$. Here the null hypothesis is
\spl{
    H_0:\ \mu\leq25.
}

Hence,
\spl{
    &P[\overline{X}\geq\overline{x}\big|\mu\leq25]\leq P[\overline{X}\geq\overline{x}\big|\mu=25]\\
    =&P[\frac{\bar{X}-25}{s/\sqrt{n}}\geq\frac{26.035-25}{s/\sqrt{n}}]\\
    =&P[T_{19}\geq\frac{26.035-25}{4.78476}]\\
    =&P[T_{19}\geq0.97]\\
    \approx&0.17>0.05.
}

If we set $\alpha=0.05$, then $t_{0.05}(19)=1.729$.

We can reject $H_0$ if $t=0.97>t_{0.05}(19)$ but we fail, which means we say that we have no significant evidence to reject that $\mu\leq25$. Hence, we support the claim. The P-value is approximately 0.17.

\section*{Exercise 5.2}
\enum{
\item
We denote that $Z=\frac{\overline{X}-\mu_0}{\sigma/\sqrt{n}}$.
\spl{
    \alpha&=P[\text{reject }H_0\big|H_0 \text{ true}]\\
    &=P[\overline{X}\geq\overline{x}\big|\mu\leq4]\\
    &\leq P[\overline{X}\geq\overline{x}\big|\mu=4]\\
    &=P[Z\geq\frac{\overline{x}-4}{0.2/\sqrt{50}}]\\
    &=P[Z\geq z_{\alpha}]\\
    &\overset{!}{=}0.05.
}

Since $z_{0.05}=1.645$, then
\spl{
    \frac{\overline{x}-4}{0.2/\sqrt{50}}&=1.645\\
    \overline{x}&=4.047.
}

Thus the critical region is
\spl{
    \overline{X}\geq4.047.
}

\item
\spl{
    P&=P[\text{reject } H_0\big|H_1 \text{ true}]\\
    &=P[Z\geq z_{\alpha=0.05}\big|\mu\geq4.5]\\
    &\geq P[\frac{\overline{X}-4}{0.2/\sqrt{50}}\geq1.645\big|\mu=4.5]\\
    &=P[\overline{X}\geq4.047\big|\mu=4.5]\\
    &=1-\Phi(\frac{4.047-4.5}{0.2/\sqrt{50}})\\
    &=1.
}

Hence, the power of this test is 1.

\item
Since $z_{1-0.97}=-1.885$, then $z\leq-1.885$, which means
\spl{
    \frac{\overline{X}-4.5}{0.2/\sqrt{n}}=\frac{(z_{0.05}\times0.2/\sqrt{n}+4)-4.5}{0.2/\sqrt{n}}&\leq-1.885,\\
}
which gives $n\geq1.99$.

Hence, the sample size is at least 2.

\item
Since $\bar{x}=4.05>4.047$, we conclude that we can reject $H_0$ at $p<0.05$.

A 95\% confidence interval for $\mu$ is $4.05\pm\frac{z_{0.025}\cdot0.2}{\sqrt{50}}=4.05\pm0.06$
}

\section*{Exercise 5.3}
\enum{
\item
\spl{
    n=8,\quad \sigma=4,\quad \mu_0=100.
}

Thus the normalized statistic is
\spl{
    z=\frac{\bar{x}-100}{4/\sqrt{8}}=1.556
}

Since $z_{0.05}=1.645<1.556$, then we conclude that we reject $H_0$ at $p<0.05$ .

\item
\spl{
    p&=P[\overline{X}\geq\bar{x}\big|\mu\leq100]\\
    &\leq P[Z\geq1.556]\\
    &=(0.0606+0.5904)/2\\
    &=0.06.
}

\item
\spl{
    \text{Power}&=P[\text{reject }H_0\big|\mu=105]=P[Z\geq z_{0.05}\big|\mu=105]\\
    &=P[\frac{\overline{X}-100}{4/\sqrt{8}}\geq z_{0.05}\big|\mu=105]\\
    &=P[\overline{X}\geq102.3\big|\mu=105]\\
    &=1-\Phi(\frac{102.3-105}{4/\sqrt{8}})\\
    &=0.9706.
}

\item
Since $z_{1-0.85}=-1.035$, then $z\leq-1.035$, which means
\spl{
    \frac{\overline{X}-105}{4/\sqrt{n}}=\frac{(z_{0.05}\times4/\sqrt{n}+100)-105}{4/\sqrt{n}}&\leq-1.035,\\
}
which gives $n\geq4.60$.

Hence, the sample size is at least 5.

\item
If we construct a 95\% lower confidence bound for $\mu$, which is
\spl{
    \mu\geq\overline{X}-\frac{z_{0.05}\cdot4}{\sqrt{8}}=102.2-2.3=99.9.
}
Hence, we can say $\mu\geq99.9$ with the confidence of 95\%. It's equivalent to the test in part(i). The results are both rejecting $H_0$.
}

\section*{Exercise 5.4}
Denote that
\spl{
    G&=\sum_{k=1}^nX_k,\\
    K&=\frac{2}{\beta}\sum_{k=1}^nX_k.
}

\spl{
    F_K(y)&=P[K\leq y]=P[G\leq y\cdot\beta/2],\\
    f_K(y)&=F_K'(y)=\frac{\beta}{2}f_G(y\cdot\frac{\beta}{2}).
}

Then when $y>0$,
\spl{
    f_K(y)&=\frac{\beta}{2}\frac{1}{\Gamma(n)\beta^n}\bigg(y\cdot\frac{\beta}{2}\bigg)^{n-1}e^{-\frac{y\beta}{\beta\cdot2}}\\
    &=\bigg(\frac{\beta}{2}\bigg)^n\frac{1}{\Gamma(n)\beta^{n}}y^{n-1}e^{-\frac{y}{2}}\\
    &=\frac{1}{\Gamma(n)2^{n}}y^{n-1}e^{-\frac{y}{2}}\\
    &=\frac{1}{\Gamma(n'/2)2^{n'/2}}y^{n'/2-1}e^{-\frac{y}{2}}\\
    &=f_{\chi_{n'}^2}(y).
}

where $n'=2n$.

Hence, $K$ follows a chi-squared distribution with $2n$ degrees of freedom.

For $H_0:\ \beta=\beta_0$,
\spl{
    \chi_{1-\alpha/2,2n}^2 &\leq K\leq\chi_{\alpha/2,2n}^2\\
    \chi_{1-\alpha/2,2n}^2 &\leq \frac{2}{\beta}\sum_{k=1}^n \overline{X}_k\leq\chi_{\alpha/2,2n}^2\\
    \frac{\chi_{1-\alpha/2,2n}^2}{2\sum_{k=1}^n \overline{X}_k} &\leq \frac{1}{\beta}\leq\frac{\chi_{\alpha/2,2n}^2}{2\sum_{k=1}^n \overline{X_k}}\\
    \frac{2\sum_{k=1}^n \overline{X_k}}{\chi_{\alpha/2,2n}^2}&\leq\beta\leq\frac{2\sum_{k=1}^n \overline{X_k}}{\chi_{1-\alpha/2,2n}^2}.
}

Hence the critical region is
\spl{
    \beta\leq\frac{2\sum_{k=1}^n \overline{X_k}}{\chi_{\alpha/2,2n}^2}\text{ or }\beta\geq\frac{2\sum_{k=1}^n \overline{X_k}}{\chi_{1-\alpha/2,2n}^2}.
}

For $H_0:\ \beta\leq\beta_0$,
\spl{
    K&\geq\chi_{\alpha,2n}^2\\
    \beta&\leq\frac{2\sum_{k=1}^n \overline{X_k}}{\chi_{\alpha,2n}^2}.
}

Hence the critical region is
\spl{
    \beta\geq\frac{2\sum_{k=1}^n \overline{X_k}}{\chi_{\alpha,2n}^2}.
}


%!TEX root =  ../main.tex

\subsection{Definition}

\objective{Describe and predict the general shape of a polynomial graph}

A polynomials is sum of terms, all of which are of the form $a\cdot{}x^n$, where $a$
is any rational number and $n$ is a whole number.

\subsubsection{Degree}
``In the long run, the biggest exponent always wins.''  While it may sound like a odd
cliche from a movie, it is certainly true.  What is more, there are only two possibilities
for polynomials: either larger and larger numbers are being raised to an even degree,
or they are being raised to an odd one.  Positive numbers grow more positive in either case.
Negative numbers become positive, when raised to an even degree.  They grow more
negative when raised to an odd degree.

Of course, in the short, all manner of other behaviors may be manifested, but it is
important to consider limits at the infinities, so that we may be sure of the ultimate
leftward or rightward behavior of a function.

\personfeature[-1in]{\chapdir/pics/1983_CPA_5426}{Muhammad ibn Musa al-Khwarizmi}{
780-850}{was a Persian/Uzbekh mathematician, astronomer, and geographer
during the Abbasid Caliphate, a scholar in the House of Wisdom in Baghdad.
``Algebra'' is derived from al-jabr, one of the two operations he used to solve quadratic equations.
``Algorithm'' stems from the Latin form of his name.
\href{https://en.wikipedia.org/wiki/Muhammad_ibn_Musa_al-Khwarizmi}{Wikipedia}}

\subsubsection{Leading Coefficient}
While it is true that there are only two possibilities for the integer $n$ in the
expression $x^n$ (either it is even, or it is odd), there are two more things that could
happen, when we consider $a\cdot{}x^n$: $a$ could be positive or negative.
In summary, even polynomials go ``up'' at both ``ends'', unless the leading coefficient
is negative, in which case both ends go ``down''.  Odd functions go up to the right and
down to the left, unless the leading coefficient is negative, in which case the opposite
happens.

If we are designing a function ourselves, it can still be useful to consider
end-run behavior.  For example, we might construct a polynomial to have
$x$-intercepts 3, 5, and $-\frac{1}{2}$.  We would write $f(x)=(x-3)(x-5)(2x+1)$,
seemingly obfuscating the leading term.  However, if we multiply only the $x$ terms,
we can still find it quickly.  $x \cdot x \cdot 2x$ is $2x^3$, so we can see this is
a positive, odd polynomial, going up to the right and down to the left.

\subsubsection{$y$-intercept}
Another facet of polynomials that is still quickly discernible even in factored form
is the constant term.  This term (even if it is 0, i.e. absent) provides useful information.
When $x=0$ (which typically corresponds to something like an initial condition), the
only term not to cancel will be the constant one.  In the previous example, we can
easily multiply the plain numbers: $-3 \cdot -5 \cdot 1 = 15$, so we know the
$y$-intercept of the function will be 15.  If we need a different number, the function
can always be scaled by multiplying on the outside by a constant, which could even
be negative.

\subsection{Factored Forms}
When we examined quadratics, there were some cases in which the vertex of the
parabola was found on the $x$-axis.  A similar case can be found in polynomials of
higher degree, and can be seen in their factored forms.  For example,
$f(x)= (x-2)^2(x+5)$ is a cubic equation  (with leading term $x^3$ and constant term
20) which seems to have only two zero's: 2 and -5.  However, the number 2 works as
a solution \emph{twice}, and so we say it has a \textbf{multiplicity} of 2.

Graphically, multiplicity means the graph resembles the exponent of the factored term.
This means $f(x)$ will behave like $x^2$ in the vicinity of 2.  Note that this behavior
may be upside-down, depending on the equation.  Generalizing even more, we can say
that even powers on a factor will result in the graph \emph{not} crossing the $x$-axis
at that zero, but only ``bouncing'' off.  On the other hand, odd powers on a factor
(even including 1!) will appear as the graph proceeding \emph{through} the $x$-axis.


\documentclass{article}
%\usepackage{tikzpicture}
\begin{document}

\tableofcontents

\section{Introduction}

\subsection{Natural numbers}

$N$ - natural numbers, $N = \{1, 2, 3, \dots\}$
Operations: +, *
\paragraph{Properties of addition and multiplication}
\begin{itemize}
    \item $a + b = b + a$ - commutative law for addition
    \item $(a + b) + c = a + (b + c)$ - associative law for addition
    \item $a * b = b * a$ - commutative law for multiplication
    \item $(a * b) * c = a * (b * c)$ - associative law for multiplication
    \item $(a + b) * c = ac + bc$ - distributive law
\end{itemize}

\subsection{Mathematical induction}

\paragraph{Theorem:}\mbox{} \\
If $A \subset N$ and\\
\indent a) $1 \in A$\\
\indent b) If $k \in A$, then $k + 1 \in A$\\
then\\
\indent $A = N$

\paragraph{Theorem: The Principle of Mathematical Induction}\mbox{} \\
If $p_1, p_2, \dots$ are statements (true/false) and\\
\indent a) $p_1$ is true\\
\indent b) If $p_k$ is true, then $p_{k+1}$ is also true\\
then\\
\indent all the statements $p_1, p_2, \dots$ are true\\
\\
\textbf{\textit{Proof:}}\\
$Let A = \{k \in N: p_k\;is\;true\}$\\
Clearly, if $1 \in A$ and if $k \in A$, then $k \neq 1 \in A$\\
So $A = N$\\
\\
\textbf{\textit{Examples:}}
\begin{itemize}
    \item $1 + 2 + \dots + n = \frac{n(n + 1)}{2}$, $n \in N$\\
    a) for $n = 1$: $1 = \frac{1(1 + 1)}{2}$\\
    b) suppose for some $k$: $1 + 2 + \dots = \frac{k(k+1)}{2}$\\
    then\\
    $1 + 2 + \dots + (k + 1) = \frac{(k+1)(k+2)}{2}$\\
    $1 + 2 + \dots + k + (k + 1) = \frac{(k+1)(k+2)}{2}$\\
    so
    $1 + 2 + \dots + n = \frac{n(n+1)}{2}$ for all $n \in N$\\
\end{itemize}

\paragraph{Theorem: The Generalized Principle of Mathematical Induction}\mbox{} \\
If $p_1, p_2, \dots$ are statements (true/false) and\\
\indent a) $p_{k_o}$ is true\\
\indent b) If $p_k$ is true, then $p_{k+1}$ is also true\\
then\\
\indent all the statements $p_{k_o}, p_{k_o+1}, \dots$ are true\\
\\

\subsection{Sigma notation}
\paragraph{Definition}\mbox{} \\
If $a_1, a_2, \dots, a_n \in R$\\
then their sum $a_1 + a_2 + \dots + a_n$ will be denoted by $\sum^{n}_{k=1}a_k$.
\paragraph{Properties}
\begin{itemize}
    \item $\sum^{n}_{k=1}(a_k + b_k) = \sum^{n}_{k=1}(a_k) + \sum^{n}_{k=1}(b_k)$
    \item $\sum^{n}_{k=1}(c*a_k) =  c*\sum^{n}_{k=1}(a_k)$
\end{itemize}

\subsection{Binomial expansion}
\paragraph{Pascal's triangle}\mbox{}\\\\
\newcommand{\ap}{\ensuremath{\swarrow\,\searrow}}
\setlength{\tabcolsep}{0pt}
\begin{tabular}{ccccccccc}
  &     &     &      & 1   &      &      &     & \\
  &     &     &      & \ap &      &      &     & \\
  &     &     & 1    &     &  1   &      &     & \\
  &     &     & \ap  &     &  \ap &      &     & \\
  &     & 1   &      & 2   &      & 1    &     & \\
  &     & \ap &      & \ap &      & \ap  &     & \\
  & 1   &     & 3    &     &  3   &      & 1   & \\
  &\ap  &     & \ap  &     &  \ap &      & \ap & \\
1 &     & 4   &      & 6   &      & 4    &     & 1
\end{tabular}
\paragraph{Binomial coefficient (Newton's symbol)}\mbox{}\\
${n \choose k} = \frac{n!}{k!(n-k)!}$ - number of k-element subsets of n-element set\\
\paragraph{Theorem: Newton's binomial expansion formula}\mbox{}\\
$(a+b)^n={n \choose 0}a^{n}b^{0}+{n \choose 1}a^{n-1}b^{1}+\dots+{n \choose k}a^{n-k}b^{k}+\dots+{n \choose n}a^{0}b^{n}$\\
$(a+b)^n=\sum^n_{k=0} {n \choose k}a^{n-k}b^k$

\subsection{Logic}
%TODO here
\begin{displaymath}
    \begin{array}{|c c|c|}
        p & q & p \land q\\ % Use & to separate the columns
        \hline % Put a horizontal line between the table header and the rest.
        T & T & T\\
        T & F & F\\
        F & T & F\\
        F & F & F\\
    \end{array}
\end{displaymath}

\subsection{Quantifiers}
%TODO here
todo

\section{Complex numbers}
\subsection{Introduction}
\paragraph{Definition}
Complex number is a pair of real numbers.\\
Examples: $(2,3)$, $\sqrt{2},5)$\\
We will usually denote them by $z, w, \dots$.\\
If $z=(x,y)$ ($x, y \in R$ - we can skip it)\\
then $x$ is called the \textbf{real part} of \textbf{z} and it will be denoted by $Re z$, $y$ will be called the \textbf{imaginary part} of \textbf{z} and denoted by $Im z$.\\
The set of all complex numbers will be denoted by $C$, so\\
$C = \{(x, y): x, y \in R\}$
\paragraph{Geometric interpretation of C}\mbox{}\\
% \begin{tikzpicture}
%     \begin{axis}[
%     axis x line=center,
%     axis y line=none,
%     xmin=-3,xmax=3,
%     ]
%     \end{axis}
% \end{tikzpicture}
\begin{itemize}
    \item points on a plane
    \item vectors on a plane
    \item free vectors (not starting at origin, but at any point)
\end{itemize}
\subsection{Algebra on complex numbers}\mbox{}\\
\begin{itemize}
    \item addition\\
    If $z = (x_1, y_1), w = (x_2, y_2)$ then $z + w$ is defined as\\
    $(x_1 + x_2, y_1 + y_2)$ (just like addition of vectors).
    %TODO theorem
    \item multiplication\\
    If $z = (x_1, y_1), w = (x_2, y_2)$ then $z * w$ is defined as\\
    $(x_1 x_2 - y_1 y_2,\;x_1 y_2 + x_2 y_1)$
\end{itemize}
\end{document}


\chapter{Predicate Formulas}
\label{chapter:predicate-formulas}
\marginurl{%
  Predicate Formulas:\\\noindent
  Introduction to Mathematical Logic \#5
}{youtu.be/yb9NvmXyFfg}

In the previous chapters we studied propositional logic. But in real mathematics
there are many non propositional formulas. For example, we may wish to prove
that if a relation $R$ on $M$ is transitive, then
\[
  (R(w, x) \land R(x, y) \land R(y, z)) \implies R(w, z)
\]
is true for any $w, x, y, z \in M$. This chapter defines a logical
system that allows us to formally prove such statements.

Let us write the previous statement in a formula-like form:
\begin{multline*}
    \overbrace{
        \left(
          \forall x, y, z \in M ~ (R(x, y) \land R(y, z)) \implies R(x, z)
         \right)
    }^{R \text{ is transitive}} \implies \\
    \underbrace{(
        \forall w, x, y, z \in M ~ (R(w, x) \land R(x, y) \land R(y, z))
        \implies
        R(x, z)
    )}_\text{the desired conclusion}.
\end{multline*}
Note that there are several things we need to explain if we wish to define
formally formulas like this:
\begin{itemize}
    \item we need to explain what kind of sets we can use (in this case we need
        to define $M$),
    \item we need to explain what kind of relations we can use (in this case we
        need to define $R$),
\end{itemize}

Another example of a statement we may wish to prove is saying that if
$f : M \to M$ is an inverse of itself (i.e. $f(f(x)) = f(x)$ for any $x \in M$),
then $f(f(f(x))) = f(x)$ for any $x \in M$; more formally, we may wish to prove
the statement
\[
    \underbrace{
        (\forall x \in M ~ f(f(x)) = x)}_{f \text{is an inverse of itself}}
    \implies
    \underbrace{
        (\forall x \in M ~ f(f(f(x))) = f(x))
    }_\text{the desired conclusion}.
\]
In order to explain what we mean by such formulas
\begin{itemize}
  \item we need to explain what kind functions we can use (in this case we need
    to define $f$).
\end{itemize}

\paragraph{Signature.}
In predicate logic, formula uses just symbols for all these objects. We specify
these symbols only when we wish to compute actual truth value of the formula.
We also assume that all the quantifiers are over the same set so we do not need
a symbol for the set $M$.

Signature is the way to define the list of all these symbols, it consists of
three objects:
\begin{itemize}
  \item a set (possibly empty) of symbols for relations,
  \item a set (possibly empty) of symbols for functions,
  \item arities of these functions and relations (i.e. how many arguments they
    may take).
\end{itemize}
An example of a signature is a triple $(\set{\text{``R''}}, \set{\text{``f''}},
a)$, where
\[
  a(s) = \begin{cases}
    2 & \text{if } s =  \text{``R''} \\
    1 & \text{if } s =  \text{``f''}
  \end{cases}.
\]
This signature is enough to define the formulas we discussed. Now we are ready
to define the predicate formulas.

\begin{definition}
  Let $\signature{S} = (S_\mathrm{rel}, S_\mathrm{fun}, a)$ be a signature and $V$
  be some set.

  We say that $t$ is a \emph{term} in the signature $\signature{S}$ over the variables
  from $V$ if
  \begin{itemize}
    \item either $t$ is equal to $x$ for $x \in V$,
    \item or $t$ is equal to $f(t_1, \dots, t_{a(f)})$, where
      $f \in S_\mathrm{fun}$ and $t_1$, \dots, $t_{a(f)}$ are terms in the
      signature $\mathcal{S}$ over the variables from $V$.
  \end{itemize}

  We say that $\phi$ is a \emph{predicate formula} in the signature
  $\signature{S}$ over the variables from $V$ if
  \begin{itemize}
    \item either $\phi$ is equal to $R(t_1, \dots, t_{a(R)})$, where
      $R \in S_\mathrm{rel}$ and $t_1$, \dots, $t_{a(R)}$ are terms in the
      signature $\signature{S}$ over the variables from $V$,
    \item or $\phi$ is equal to $(\psi_1 \land \psi_2)$, or
      $(\psi_1 \lor \psi_2)$, or $(\psi_1 \limplies \psi_2)$, where $\psi_1$ and
      $\psi_2$ are predicate formulas in the signature $\signature{S}$ over the
      variables from $V$,
    \item or $\phi$ is equal to $(\lnot \psi)$, where $\psi$ is a predicate
      formula in the signature $\signature{S}$ over the variables from $V$,
    \item or $\phi$ is equal to $\left( \exists x_i~\psi \right)$ or
      $\left( \forall x_i~\psi \right)$ where
      $\psi$ is a predicate formula in the signature $\signature{S}$ over the
      variables from $V$.
  \end{itemize}

  We denote by $\pred{V}{\signature{S}}$ the set of all predicate formulas in
  $\signature{S}$ over the variables from $V$; we also denote by
  $\term{V}{\signature{S}}$ the set of all terms in $\signature{S}$ over the
  variables from $V$.
\end{definition}

In order to compute the truth value of a predicate formula, we need to specify
the values of all the free variables and all the symbols from the signature.
The specification of the symbols from the signature is called structure; i.e.
a structure for a signature
$\signature{S} = (S_\mathrm{rel}, S_\mathrm{fun}, a)$
is a triple $\structure{M} = (M, F_\mathrm{rel}, F_\mathrm{fun})$ such that
\begin{itemize}
  \item $F_\mathrm{rel}$ assigns a relation to every relation symbol in
    $\signature{S}$; i.e.,
    $F_\mathrm{rel} : S_\mathrm{rel} \to \bigcup_{i = 0}^\infty 2^{M^i}$
    such that $F_\mathrm{rel}(R) \in 2^{M^{a(R)}}$ and
  \item $F_\mathrm{fun}$ assignes a function to every function symbol in
    $\signature{S}$; i.e.,
    $F_\mathrm{fun} : S_\mathrm{fun} \to \bigcup_{i = 0}^\infty M^{M^i}$
    such that $F_\mathrm{fun}(f) \in M^{M^{a(f)}}$.
\end{itemize}
The set $M$ in the structure is called the \emph{domain} of the structure and
denoted by $\domain{\structure{M}}$. We also denote $F_\mathrm{rel}(R)$ by
$R^{\structure{M}}$ and $F_\mathrm{fun}(f)$ by $f^{\structure{M}}$.

\begin{definition}
  Let $\signature{S} = (S_\mathrm{rel}, S_\mathrm{fun}, a)$ be a signature and
  $\structure{M}$ be a structure for $\signature{S}$.

  Let $\rho : V \to \domain{\structure{M}}$.
  The valuation of terms in $\signature{S}$ over the variables from $V$
  corresponding to the assignement $\rho$ is the function
  $\substitute{\cdot}{\rho, \structure{M}} :
    \term{V}{\signature{S}} \to \domain{\structure{M}}$ such that
  \begin{itemize}
    \item $\substitute{x}{\rho, \structure{M}} = \rho(x)$ for any $x \in V$,
    \item $\substitute{f(t_1, \dots, t_\ell)}{\rho, \structure{M}} =
      f^{\structure{M}}(
        \substitute{t_1}{\rho, \structure{M}},
        \dots,
        \substitute{t_\ell}{\rho, \structure{M}}
      )$ \,
      for any function symbol $f \in S_\mathrm{fun}$ and
      terms $t_1, \dots, t_\ell \in \term{V}{\signature{S}}$.
  \end{itemize}

  Similarly we may define valuations of predicate formulas (we will abuse the
  notation and denote them by $\substitute{\cdot}{\rho, \structure{M}}$ too).
  The valuation of predicate formulas in $\signature{S}$ over the variables from
  $V$ corresponding to the assignement $\rho$ is the function
  $\substitute{\cdot}{\rho, \structure{M}} :
    \pred{V}{\signature{S}} \to \domain{\structure{M}}$ such that
  \begin{itemize}
    \item $\substitute{R(t_1, \dots, t_\ell)}{\rho, \structure{M}} =
      R^{\structure{M}}(
        \substitute{t_1}{\rho, \structure{M}}),
        \dots,
        \substitute{t_\ell}{\rho, \structure{M}})
      $\,
      for any relation symbol $R \in S_\mathrm{rel}$
      and terms $t_1, \dots, t_\ell \in \term{V}{\signature{S}}$,
    \item
      $
        \substitute{\psi_1 \land \psi_2}{\rho, \structure{M}} =
        \substitute{\psi_1}{\rho, \structure{M}} \land
        \substitute{\psi_2}{\rho, \structure{M}}
      $\,
      for any $\psi_1, \psi_2 \in \pred{V}{\signature{S}}$,
    \item
      $
        \substitute{\psi_1 \lor \psi_2}{\rho, \structure{M}} =
        \substitute{\psi_1}{\rho, \structure{M}} \lor
        \substitute{\psi_2}{\rho, \structure{M}}
      $\,
      for any $\psi_1, \psi_2 \in \pred{V}{\signature{S}}$,
    \item
      $
        \substitute{\psi_1 \limplies \psi_2}{\rho, \structure{M}} =
        \substitute{\psi_1}{\rho, \structure{M}} \limplies
        \substitute{\psi_2}{\rho, \structure{M}}
      $\,
      for any $\psi_1, \psi_2 \in \pred{V}{\signature{S}}$,
    \item
      $
        \substitute{\lnot \psi}{\rho, \structure{M}} =
        \lnot \substitute{\psi_1}{\rho, \structure{M}}
      $\,
      for any $\psi \in \pred{V}{\signature{S}}$,
    \item for $\psi \in \pred{V}{\signature{S}}$,
      $
        \substitute{\exists x \  \psi}{\rho, \structure{M}} =
        \ltrue
      $ iff
      $\substitute{\psi}{\assign{\rho}{x}{v}, \structure{M}} = \ltrue$
      for some $v \in \domain{\structure{M}}$, and
    \item for $\psi \in \pred{V}{\signature{S}}$,
      $
        \substitute{\forall x \  \psi}{\rho, \structure{M}} =
        \ltrue
      $ iff
      $\substitute{\psi}{\assign{\rho}{x}{v}, \structure{M}} = \ltrue$
      for all $v \in \domain{\structure{M}}$.
  \end{itemize}

  Let $V = \set{x_1, \dots, x_n}$ then we denote the assignement $\rho : V \to
  \domain{\structure{M}}$ such that $\rho(x_i) = v_i$ by
  $x_1 = v_1, \dots, x_n = v_n$.
\end{definition}

We say that $\structure{M}$ is a model of a formula $\phi$
(written $\structure{M} \models \phi$)\footnote{
  Sometimes ``$\structure{M}$ is a model of $\phi$'' is written
  as $\models_{\structure{M}} \phi$.
} over the variables
from $V$ iff $\substitute{\psi}{\rho, \structure{M}} = \ltrue$ for all $\rho : V
\to \domain{\structure{M}}$.

We also say that $\phi$ is true in $\structure{M}$ if $\structure{M} \models \phi$,
and we say that $\phi$ is false in $\structure{M}$ if $\structure{M} \not\models \phi$.

Let us consider an example:
\begin{itemize}
  \item First, we define a signature
    $\signature{S} = (\set{=, <}, \set{+, \cdot}, a)$ (we write
    $S = (=, <; +, \cdot)$ when arities of the functions and relations are clear
    from the context),
    where $a(x) = 2$ for any $x \in \set{<, =, +, \cdot}$.
  \item After this we define the structure $\structure{M}$ in signature
    $\signature{S}$ such that
    \begin{gather*}
      f^{\structure{M}}(x, y) = \begin{cases}
        x \cdot y & \text{if } f \text{ is } \cdot \\
        x + y & \text{if } f \text{ is } +
      \end{cases} \\
      \text{and} \\
      R^{\structure{M}}(x, y) \begin{cases}
        x = y & \text{if } R \text{ is } = \\
        x < y & \text{if } R \text{ is } <
      \end{cases}
    \end{gather*}
    Note that such a definition is pretty cumbersome, especially considering the
    fact that we use standard $+$ instead of the symbol $+$,
    standard $=$ instead of the symbol $=$ etc. So in similar cases we
    write $\structure{M} = (\R; =, <; +, \cdot)$.
  \item Finally, we consider the formulas in $\signature{S}$ over the set of
    variables $\set{x, y, z}$:
    \begin{gather*}
      \forall x \ \forall y \ x + y = y + x \\
      \text{and} \\
      \forall x \ \forall y \ \forall z \ (x < y \implies x + z < y + z).
    \end{gather*}
    (Note that we write $a = b$ instead of $=(a, b)$ and $a + b$ instead of
    $a + b$, this is a common notation when the standard mathematical
    operations and relations are used in the signature.)
\end{itemize}
The first formula says that addition is commutative, which is true in $\R$, so
the value of the formula is true in $\structure{M}$.
(Note that we do not mention the values of the variables $x$ and $y$
since both of them are not free.) Indeed, consider $a, b \in \R$, note that
$\substitute{x + y}{x = a, y = b, \structure{M}} = a + b = b + a =
  \substitute{y + x}{x = a, y = b, \structure{M}}$. Hence,
$\substitute{\forall x \ \forall y \ x + y = y + x}{\rho, \structure{M}} = \ltrue$
for all $\rho$.

The second formula says that the inequalities are additive, so it should be also
true with respect to the structure $\structure{M}$.

\begin{chapterendexercises}
  \exercise
    Show that the second formula is true in $\structure{M}$.
  \exercise
    Let us consider a signature $(=; +, \cdot, 0, 1)$ and two models with this
    signature: $\structure{R} = (\R; =; +, \cdot, 0, 1)$, and
    $\structure{Q} = (\Q; =; +, \cdot, 0, 1)$.
    Find a predicate formula $\phi$ in this signature such that
    $\structure{R} \models \phi$ but $\structure{Q} \not\models \phi$.
\end{chapterendexercises}


\section{Fuzzy Inference}

A simplified Sugeno-style fuzzy inference system is developed with functionalities for fuzzy rule / set parsing, input fuzzification, fuzzy inferencing and output defuzzification.

\subsection{Determining Fuzzy sets}

As mentioned in the previous section, the noises in the image inevitably introduces uncertainties to our system. Therefore \textbf{the feature values referenced in our rules have to be represented in fuzzy sets}.

To determine the \textbf{best fuzzy sets capturing the appropriate degree of uncertainties}, we use the statistics from the training set to find the optimal threshold.

For $Thinness$, we use 3 fuzzy sets for ellipse-like, rectangle-like and triangle-like $Extent$ ratio. For $Extent$, we use 2 fuzzy sets for circle-like and square-like $Thinness$ ratio. The shapes and boundries of these fuzzy sets are determined based on the \textbf{distribution and percentiles of feature values} in the training set. We consider the area between 25th percentile and 75th percentile as the \textbf{high confidence range}, while the remaining area plus a small portion of area adjacent to the extreme values (to tolerate some outliners) are considered as the \textbf{low confidence range}. The corresponding polygons are then constructed based on these two ranges to represent the fuzzy sets. The final fuzzy sets are illustrated in Figure 3 (Figures not drawn to scale).

\begin{figure}

\begin{subfigure}[b]{\columnwidth}
\includegraphics[width=\columnwidth]{Figure_3_Fuzzy_Sets_1.png}
\caption{Fuzzy Set of Extent}\end{subfigure}

\begin{subfigure}[b]{\columnwidth}
\includegraphics[width=\columnwidth]{Figure_4_Fuzzy_Sets_2.png}
\caption{Fuzzy Set of Thinness}
\end{subfigure}

\caption{Fuzzy Sets}

\end{figure}

\subsection{Implementing Inference Engine}

Now that the fuzzy sets are determined, we have to implement the inference engine. But the specific nature of our task requires us to make some adaptations to the the usual fuzzy inference engine.

Firstly, unlike the example from the textbook, the set of possible outputs in our task is \textbf{NOT} a set of linguistic values which can be represented by a series of contiguous intervals. We can't compute a COG-like output and decide which category it falls in, because it's not possible to assign a reasonable order to the shapes.

Secondly, rules derived in the previous section are all of the format:

\textit{IF X IS LIKE A AND Y IS NOT LIKE B THEN Shape IS C}

Note that the consequent of each rule is an assertion about the shape of the input figure and that \textbf{each rule corresponds to the recognition process for one specific shape}. In fact a fuzzy inference system of this kind is essentially \textbf{a flattened decision tree}, with \textbf{each rule acting as a filter to calculate the possibility of one specific shape}. The only difference is that in our expert system rules are separated from inference, making it easily maintainable.

Thus, in order to gain the advantages brought by fuzzy inference without over-complicating our task, we choose to implement \textbf{a simplified Sugeno-style inference engine}. In the final defuzzification stage, we don't compute a weighted average of the rule outputs. Instead we examine all the possible values for the target variable $Shape$ and \textbf{choose the one with the highest possibility as the output}.


\section{The classifying type of a group}

\subsection{The classifying type of a group}

\begin{thm}
  For every group $G$ there is a pointed connected $1$-type $BG$ equipped with group isomorphism
  \begin{equation*}
    \loopspace{BG} \simeq G
  \end{equation*}
\end{thm}