\lipsum[4] See Section\ \ref{sec:headings}.

\subsection{Headings: second level}
\lipsum[5]
\begin{equation}
	\xi _{ij}(t)=P(x_{t}=i,x_{t+1}=j|y,v,w;\theta)= {\frac {\alpha _{i}(t)a^{w_t}_{ij}\beta _{j}(t+1)b^{v_{t+1}}_{j}(y_{t+1})}{\sum _{i=1}^{N} \sum _{j=1}^{N} \alpha _{i}(t)a^{w_t}_{ij}\beta _{j}(t+1)b^{v_{t+1}}_{j}(y_{t+1})}}
\end{equation}

\subsubsection{Headings: third level}
\lipsum[6]

\paragraph{Paragraph}
\lipsum[7]


\subsection{Solo Calculus}

    Developed by Cosimo Laneve and Bj{\"o}rn Victor in the early 2000s, the solo calculus aims to be an improvement of the Fusion calculus.
    As such, there exists an encoding of the Fusion calculus within the solo calculus (and hence an encoding of the $\pi$-calculus).
    The name comes from the strong distinction between the components of the calculus: \textit{solos} and \textit{agents}.
    These are roughly analogous to input/output actions and a calculus syntax similar to the $\lambda$-calculus.
    Through some clever design choices, the solo calculus is found to have some interesting properties over other process calculi.

    \begin{definition}{(Syntax)\\}
        \label{solo-calculus-syntax}
        As defined by~\cite{solo-calculus}, the solo calculus is constructed from \textit{solos} ranged over by $\alpha, \beta \ldots$ and \textit{agents} ranged over by $P, Q \ldots$ as such:
        \begin{center}
            \begin{tabular}{ l l l }
                $\alpha \quad \defeq$   & $u \, \tilde{x}$          & (input) \\
                                        & $\bar{u} \, \tilde{x}$    & (output)~\footnotemark\\ \\
                $P \quad \defeq$        & $0$                       & (inaction) \\
                                        & $\alpha$                  & (solo) \\
                                        & $Q \, | \, R$             & (composition) \\
                                        & $(x) \, Q$                & (scope) \\
                                        & $[x=y] \, Q$              & (match) \\
                                        & $!\,P$                    & (replication)
            \end{tabular}
        \end{center}\footnotetext{$\tilde{x}$ is used as shorthand for any tuple $(x_1 \ldots x_n)$.}
        where the scope operator $(x) \, P $ is a declaration of the named variable $x$ in $P$.
        This ensures that $x$ is local to $P$, even if it assigned outside of $P$ (ex. $(x \, y | (x) \, P)$ will never have $x \defeq y$ unless explicitly assigned such in P).
    \end{definition}
    This is a much more minimal syntax when compared to CCS (and certainly Higher-Order CCS as described by~\cite{pi-calculus-in-ccs}).
    It will further be seen that the reduction rules retain this simplicity.
    It should be noted that the names $u, x, etc\ldots$ within a solo may be treated as both channel names and as values.

    \begin{definition}{(Match Operator)\\}
        The match operator $[x \, = \, y] \, P$ computes $P$ if $x$ and $y$ are the same name, otherwise computes \textbf{0}.
        These match operators are iterated over here by $M, N$, with sequences of match operators iterated over by $\tilde{M}, \tilde{N}$.
        Each name occurring in $M$ is a \textit{labelled node} of $M$.
    \end{definition}

    \begin{definition}{(Structural Congruence)\\}
        The structural congruence relation $\equiv$ in the solo calculus is exactly that defined in Definition~\ref{fusion-calculus-structural-congruence}.
    \end{definition}

    \begin{definition}{(Reduction)\\}
        Reduction semantics on solo expressions are defined as:
        \begin{align}
            (x)(\bar{u} \, x \, | \, u \, y \, | \, P) \rightarrow & \, P\{y / x\} \\
            P \rightarrow P' \implies &
            \begin{cases}
                P \, | \, Q \rightarrow P' \, | \, Q \\
                (x) \, P \rightarrow (x) \, P' \\
                P \equiv Q \text{ and } P' \equiv Q' \implies Q \rightarrow Q'
            \end{cases}
        \end{align}
        where $P\{y / x\}$ is $\alpha$-substitution of the name $x$ to the name $y$.
    \end{definition}
    It is interesting to note here the asynchronous behaviour of the solo calculus.
    Where in the $\pi$-calculus and CCS input/output actions where synchronised and preceded processes as guards, the solo calculus naturally treats all agents as unguarded and names may be substituted whenever is desired.

    \begin{remark*}
        There exists an encoding of the Fusion calculus within the solo calculus.
        This can most easily be seen as an encoding of the choice-free Fusion calculus as a combination of the above syntax and semantics of the solo calculus and also the prefix operator $\alpha \, . \, P$\footnote{For further details,~\cite{solo-calculus} discuss this implementation in Section 3}.
        Hence there exists an encoding of the $\pi$-calculus also, complete with the same style of guarded input/output communication.
    \end{remark*}

\subsection{Solo Diagrams}
    The solo calculus was further developed by~\cite{solo-diagrams} to provide a one-to-one correspondence between these  expressions and `diagram-like' objects.
    This provides a strong analog to real-world systems and an applicability to be used as a modelling tool for groups of communicating systems.
    Furthermore, as discussed by~\cite{learning-styles}, a visual output of information is often found to be preferable for cognition than verbal or textual information.

    \begin{definition}{(Edge)\\}
        An edge is defined to be:
        \begin{align}
            E \defeq \, \langle a, a_1 \ldots a_k\rangle_t \quad \text{for } t \in \{ i, o \}
        \end{align}
        where $a, a_i$ are \textit{nodes}, $\langle \ldots \rangle_i$ is an \textit{input edge}, $\langle \ldots \rangle_o$ is an \textit{output edge} and $k$ the edge's \textit{arity}.
    \end{definition}

    \begin{figure}[H]
        \centering
        \begin{subfigure}{0.4\linewidth}
            \centering
            \begin{tikzpicture}[transform shape, every node/.style={circle, fill=black!100, inner sep=0.05cm}]
                \node[anchor=center, label=below:{$a$}](a){};
                \coordinate[right=1cm of a](ax);
                \node[above right=1cm of ax, label=below:{$a_1$}](a1){};
                \node[below right=1cm of ax, label=below:{$a_2$}](a2){};
                \draw[-{>[scale=2]}] (ax) -- (a);
                \draw[-] (a1) -- (ax) -- (a2);
            \end{tikzpicture}
            \caption*{Output edge $\langle a, a_1, a_2\rangle_o$}
        \end{subfigure}
        \begin{subfigure}{0.4\linewidth}
            \centering
            \begin{tikzpicture}[transform shape, every node/.style={circle, fill=black!100, inner sep=0.05cm}]
                \node[anchor=center, label=below:{a}](a){};
                \coordinate[right=1cm of a](ax);
                \node[above right=1cm of ax, label=below:{$a_1$}](a1){};
                \node[below right=1cm of ax, label=below:{$a_2$}](a2){};
                \draw[-{<[scale=2]}] (ax) -- (a);
                \draw[-] (a1) -- (ax) -- (a2);
            \end{tikzpicture}
            \caption*{Input edge $\langle a, a_1, a_2\rangle_i$}
        \end{subfigure}
    \end{figure}

    This is analogous to an input or output solo in the calculus, where $a$ is $u$ or $\bar{u}$ and $a_1 \ldots a_n$ is $\tilde{x}$ as written in Definition~\ref{solo-calculus-syntax}.
    Note that inputs and outputs must have matching arity --- a 2-arity input cannot communicate with a 3-arity output for obvious reasons.

    \begin{definition}{(Box)\\}
        A box is defined to be:
        \begin{align}
            B \defeq \, \langle G, S \rangle \quad \text{for } S \subset nodes(G)\footnotemark
        \end{align}\footnotetext{This is written as shorthand for all nodes contained within a given object, in this case $\{a \text{ s.t. } a \in nodes(S),\, S \in G \}$}
        where G is a \textit{graph} (or multiset of \textit{edges}) and S is a set of \textit{nodes}, referred to as the \textit{internal nodes} of $B$.
        The \textit{principal nodes} of $B$ are then $nodes(G) \setminus S$.
    \end{definition}

    \begin{figure}[H]
        \centering
        \begin{subfigure}{0.4\linewidth}
            \centering
            \begin{tikzpicture}[transform shape, every node/.style={circle, fill=black!100, inner sep=0.05cm}]
                \coordinate[anchor=center](nw);
                \coordinate[below=3cm of nw](sw);
                \coordinate[right=3cm of nw](ne);
                \coordinate[below right=3cm and 3cm of nw](se);
                \draw[-] (nw) -- (ne) -- (se) -- (sw) -- (nw);
                \node[right=1.5cm of sw, label=below:{$x$}](x){};
                \node[below=1.5cm of ne, label=right:{$y$}](y){};
                \node[below right=1.5cm of nw, label=above left:{$w$}](w){};
                \coordinate[below right=1cm of w](wxyx){};
                \draw[-] (y) -- (wxyx) -- (w);
                \draw[-{<[scale=2]}] (wxyx) -- (x);
                \draw[-{>[scale=2]}] (y) to [out=140, in=10] (w);
            \end{tikzpicture}
            \caption*{Box representing $!(w)(x \, w y \, | \, \tilde{w} \, y)$}
        \end{subfigure}\footnotemark
    \end{figure}\footnotetext{Usually the $w$ in the diagram would be excluded, but is included here for illustration purposes only.}

    This can then be seen to be analogous to the replication operator, with the idea being that the principal nodes form the perimeter of a box and cannot be replicated --- they serve as the interface to the internals of the box.

    \begin{definition}{(Diagram)\\}
        A solo diagram is defined to be:
        \begin{align}
            SD \defeq (G, M, \ell)
        \end{align}
        where $G$ is a finite multiset of \textit{edges}, $M$ is a finite multiset of \textit{boxes} and $\ell$ a labelling of the $nodes(G)$ and of $principals(M)$.
    \end{definition}
    From here, we can convert solo calculus to diagrams, where composition is intuitively just including two separate diagrams together and scope is simply any connected nodes labelled by $\ell$.
    There are then four required reduction cases (edge-edge, edge-box, box-box and box internals) which can be deduced from the definition of the calculus.

    \begin{definition}{(Diagram Reduction)\\}
        Let $G, G_1, G_2 \ldots$ be arbitrary graphs, $M, M'$ arbitrary box multisets, $\alpha \defeq \langle a, a_1 \ldots a_k \rangle_i$, $\beta \defeq \langle a, a_1' \ldots a_k' \rangle_o$, $\sigma \defeq a_i \mapsto a_i'$, $\rho$ a arbitrary but fresh relabelling and $G\sigma$ shorthand for $G[\sigma]$ the application of the renaming $\sigma$ on the edges of G.
        $\alpha$ and $\beta$ need not be fixed to input and output respectively, but must be opposite polarity.
        Then, the following reductions may be made:
        \begin{align}
            (G \cup \{\alpha, \beta\}, M, \ell)                                                                                & \rightarrow (G\sigma, M\sigma, \ell') \\
            (G_1 \cup \{\alpha\}, M \defeq \langle G_2 \cup \{\beta\}, S \rangle, \ell)                                             & \rightarrow ((G_1 \cup G_2\rho)\sigma, M\sigma, \ell') \\
            (G, M \defeq \{\langle \{ \alpha \} \cup G_1, S_1 \rangle,\langle \{ \beta \} \cup G_2, S_2 \rangle\} \cup M', \ell)    & \rightarrow ((G \cup G_1\rho \cup G_2\rho)\sigma, M\sigma, \ell') \\
            (G, M \defeq \langle \{ \alpha, \beta \} \cup G_1, S \rangle \cup M', \ell)                                             & \rightarrow ((G \cup G_1\rho)\sigma, M\sigma, \ell')
        \end{align}
        where each represents reduction of an edge-edge, edge-box, box-box and of box internals respectively.
    \end{definition}

    \begin{example*}
        \begin{figure}[H]
            \centering
            \begin{subfigure}{0.4\linewidth}
                \centering
                \begin{tikzpicture}[transform shape, every node/.style={circle, fill=black!100, inner sep=0.05cm}]
                    \coordinate[anchor=center](nw);
                    \coordinate[below=3cm of nw](sw);
                    \coordinate[right=3cm of nw](ne);
                    \coordinate[below right=3cm and 3cm of nw](se);
                    \draw[-] (nw) -- (ne) -- (se) -- (sw) -- (nw);
                    \node[right=1.5cm of sw, label=below left:{$x$}](x){};
                    \coordinate[below=1cm of x](xba){};
                    \coordinate[above=1cm of x](xaa){};
                    \node[above left=1cm of xaa](v){};
                    \node[above right=1cm of xaa](w){};
                    \node[below left=1cm of xba, label=left:{$y$}](y){};
                    \node[below right=1cm of xba, label=right:{$z$}](z){};
                    \draw[-{<[scale=2]}] (xaa) -- (x);
                    \draw[-{>[scale=2]}] (xba) -- (x);
                    \draw[-{>[scale=2]}] (w) to [out=140, in=40] (v);
                    \draw[-] (w) -- (xaa) -- (v);
                    \draw[-] (y) -- (xba) -- (z);
                \end{tikzpicture}
                \caption*{$\bar{x}\, y z \, | \, !(uv)(x \, u v \, | \, \bar{u} \, v)$}
            \end{subfigure}
            $\longrightarrow$
            \begin{subfigure}{0.4\linewidth}
                \centering
                \begin{tikzpicture}[transform shape, every node/.style={circle, fill=black!100, inner sep=0.05cm}]
                    \coordinate[anchor=center](nw);
                    \coordinate[below=3cm of nw](sw);
                    \coordinate[right=3cm of nw](ne);
                    \coordinate[below right=3cm and 3cm of nw](se);
                    \draw[-] (nw) -- (ne) -- (se) -- (sw) -- (nw);
                    \node[right=1.5cm of sw, label=below left:{$x$}](x){};
                    \coordinate[below=1cm of x](xba){};
                    \coordinate[above=1cm of x](xaa){};
                    \node[above left=1cm of xaa](v){};
                    \node[above right=1cm of xaa](w){};
                    \node[below left=1cm of xba, label=left:{$y$}](y){};
                    \node[below right=1cm of xba, label=right:{$z$}](z){};
                    \draw[-{<[scale=2]}] (xaa) -- (x);
                    \draw[-{>[scale=2]}] (w) to [out=140, in=40] (v);
                    \draw[-] (w) -- (xaa) -- (v);
                    \draw[-{>[scale=2]}] (z) -- (y);
                \end{tikzpicture}
                \caption*{$\bar{y} \, z \, | \, !(uv)(x \, u v \, | \, \bar{u} \, v)$}
            \end{subfigure}
        \end{figure}
    \end{example*}

    \begin{remarks}
        The solo calculus is found to be simple, expressive and remarkable in its capability to be visualised as a diagram.
        For further reading,~\cite{acyclic-solos} present in great detail the topics of the $\pi$ and solo calculus, solo diagrams and furthermore differential interaction nets.
    \end{remarks}


\documentclass[nofootinbib,amssymb,amsmath]{revtex4}
\usepackage{mathtools}
\usepackage{amsthm}
\usepackage{algorithm}
\usepackage{algpseudocode}
\usepackage{lmodern}
\usepackage{graphicx}
\usepackage{color}
\usepackage{bm}

%Put an averaged random variable between brackets
\newcommand{\ave}[1]{\left\langle #1 \right\rangle}

\newcommand{\vzero}{{\bf 0}}
\newcommand{\vI}{{\bf I}}
\newcommand{\vb}{{\bf b}}
\newcommand{\vd}{{\bf d}}
\newcommand{\vf}{{\bf f}}
\newcommand{\vc}{{\bf c}}
\newcommand{\vv}{{\bf v}}
\newcommand{\vz}{{\bf z}}
\newcommand{\vn}{{\bf n}}
\newcommand{\vm}{{\bf m}}
\newcommand{\vG}{{\bf G}}
\newcommand{\vQ}{{\bf Q}}
\newcommand{\vM}{{\bf M}}
\newcommand{\vW}{{\bf W}}
\newcommand{\vX}{{\bf X}}
\newcommand{\vPsi}{{\bf \Psi}}
\newcommand{\vSigma}{{\bf \Sigma}}
\newcommand{\vlambda}{{\bf \lambda}}
\newcommand{\vpi}{{\bf \pi}}
\newcommand{\valpha}{{\bm{\alpha}}}
\newcommand{\vbeta}{{\bm{\beta}}}
\newcommand{\vomega}{{\bm{\omega}}}
\newcommand{\vLambda}{{\bf \Lambda}}
\newcommand{\vA}{{\bf A}}

\newcommand{\code}[1]{\texttt{#1}}

\newtheorem{lemma}{Lemma}
\newtheorem{corollary}{Corollary}

\def\SL#1{{\color [rgb]{0,0,0.8} [SL: #1]}}
\def\DB#1{{\color [rgb]{0,0.8,0} [DB: #1]}}

\newcommand{\HOM}{$\mathsf{Hom}$}
\newcommand{\HET}{$\mathsf{Het}$}
\newcommand{\REF}{$\mathsf{Ref}$}
\newcommand{\epss}{\varepsilon}

\begin{document}

\title{Mathematical Notes on Mutect}
\author{David Benjamin}
\email{davidben@broadinstitute.org}
\affiliation{Broad Institute, 75 Ames Street, Cambridge, MA 02142}
\author{Takuto Sato}
\email{tsato@broadinstitute.org}
\affiliation{Broad Institute, 75 Ames Street, Cambridge, MA 02142}

\date{\today}

\maketitle

\section{Somatic Likelihoods Model}\label{introduction}

We have a set of potential somatic alleles and read-allele likelihoods $\ell_{ra} \equiv P({\rm read~}r|{\rm allele~}a)$.  We don't know which alleles are real somatic alleles and so we must compute, for each subset $\mathbb{A}$ of alleles, the likelihood that the reads come from $\mathbb{A}$.  A simple model for this likelihood is as follows: each read $r$ is associated with a latent indicator vector $\vz_r$ with one-hot encoding $z_{ra} = 1$ iff read $r$ came from allele $a \in \mathbb{A}$.  The conditional probability of the reads $\mathbb{R}$ given their allele assignments is
\begin{equation}
P( \mathbb{R} | \vz, \mathbb{A}) = \prod_{r \in \mathbb{R}} \prod_a \ell_{ra}^{z_{ra}}.
\end{equation}
The alleles are not equally likely because there is a latent vector $\vf$ of allele fractions -- $f_a$ is the allele fraction of allele $a$.  Since the components of $\vf$ sum to one it is a categorical distribution and can be given a Dirichlet prior,
\begin{equation}
P(\vf) = {\rm Dir}(\vf | \valpha).
\end{equation}
Then $f_a$ is the prior probability that a read comes from allele $a$ and thus the conditional probability of the indicators $\vz$ given the allele fractions $\vf$ is
\begin{equation}
P(\vz | \vf) = \prod_r \prod_a f_a^{z_{ra}}.
\end{equation}
The full-model likelihood is therefore
\begin{equation}
\mathbb{L}(\mathbb{A}) = P(\mathbb{R}, \vz, \vf | \mathbb{A}) = {\rm Dir}(\vf | \valpha) \prod_a  \prod_r \left( f_a \ell_{ra}\right)^{z_{ra}}.
\label{full_likelihood}
\end{equation}
And the marginalized likelihood of $\mathbb{A}$, that is, the model evidence for allele subset $\mathbb{A}$, is
\begin{equation}
P(\mathbb{R} | \mathbb{A}) = \sum_\vz \int d \vf \, {\rm Dir}(\vf | \valpha) \prod_a  \prod_r \left( f_a \ell_{ra}\right)^{z_{ra}},
\label{evidence}
\end{equation}
where the integral is over the probability simplex $\sum_a f_a = 1$.

The integral over $\vf$ is the normalization constant of a Dirichlet distribution and as such we can simply look up its formula.  However, the sum over all values of $\vz$ for all reads has exponentially many terms.  We will get around this difficulty by handling $\vz$ with a mean-field approximation in which we factorize the likelihood as $\mathbb{L} \approx q(\vz) q(\vf)$.  This approximation is exact in two limits: first, if there are many reads, each allele is associated with many reads and therefore the Law of Large Numbers causes $\vf$ and $\vz$ to become uncorrelated.  Second, if the allele assignments of reads are obvious $\vz_r$ is effectively not a random variable at all (there is no uncertainty as to which of component is non-zero) and also becomes uncorrelated with $\vf$.

In the variational Bayesian mean-field formalism the value of $\vf$ that $\vz$ ``sees'' is the expectation of $\log \mathbb{L}$ with respect to $q(\vf)$ and vice versa.  That is,
\begin{equation}
q(\vf) \propto {\rm Dir}(\vf | \valpha) \prod_a  \prod_r f_a^{\bar{z}_{ra}} \propto {\rm Dir}(\vf | \valpha + \sum_r \bar{\vz}_r),
\label{qf}
\end{equation}
where $\bar{z}_{ra} \equiv E_q \left[ z_{ra} \right]$, and
\begin{equation}
q(\vz_r) = \prod_a \left( \tilde{f}_a \ell_{ra}\right)^{z_{ra}}, \tilde{f}_a = \exp E[\ln f_a]
\end{equation}
Because $q(\vz)$ is categorical and $q(\vf)$ is Dirichlet\footnote{Note that we didn't \textit{impose} this in any way.  It simply falls out of the mean field equations.} the necessary mean fields are easily obtained and we have
\begin{equation}
\bar{z}_{ra} = \frac{\tilde{f}_a \ell_{ra}}{\sum_{a^\prime} \tilde{f}_{a^\prime} \ell_{ra^\prime}}
\label{z_mean_field}
\end{equation}
and
\begin{equation}
\ln \tilde{f}_a = \psi(\alpha_a + \sum_r \bar{z}_{ra}) - \psi(\sum_{a^\prime} \alpha_{a^\prime} + N)
\label{f_mean_field}
\end{equation}
where $\psi$ is the digamma function and $N$ is the number of reads.  To obtain $q(\vz)$ and $q(\vf)$ we iterate Equations \ref{z_mean_field} and \ref{f_mean_field} until convergence.  A very reasonable initialization is to set $\bar{z}_{ra} = 1$ if $a$ is the most likely allele for read $r$, 0 otherwise.  Having obtained the mean field of $\vz$, we would like to plug it into Eq \ref{evidence}.  We can't do this directly, of course, because Eq \ref{evidence} says nothing about our mean field factorization.  Rather, we need the variational approximation (Bishop's Eq 10.3) to the model evidence, which is
\begin{align}
\ln P(\mathbb{R} | \mathbb{A}) \approx& \sum_{\vz} \int d \vf q(\vz) q(\vf) \left[ \ln P(\mathbb{R}, \vz, \vf | \mathbb{A}) - \ln q(\vz) - \ln q(\vf) \right] \\
=& E_q \left[ \ln P(\mathbb{R}, \vz, \vf | \mathbb{A}) \right] - E_q \left[ \ln q(\vz) \right] - E_q \left[ \ln q(\vf) \right]. \label{lagrangian}
\end{align}
Before we proceed, let's introduce some notation.  First, from Eq \ref{qf} the posterior $q(\vf)$ is
\begin{equation}
q(\vf) = {\rm Dir}(\vf | \vbeta), \quad \vbeta = \valpha + \sum_r \bar{\vz}_r.
\end{equation}
Second, let's define the log normalization constant of a Dirichlet distribution as $g$ so that
\begin{equation}
\ln {\rm Dir}(\vf | \vomega) = g(\vomega) + \sum_a (\omega_a - 1) \ln f_a, \quad g(\vomega) = \ln \Gamma(\sum_a \omega_a) - \sum_a \ln \Gamma(\omega_a).
\end{equation}
Finally, define the Dirichlet mean log (aka ``that digamma stuff") as $h$:
\begin{equation}
E_{\rm Dir(\vf | \vomega)} \left[ \ln f_a \right] = \psi(\omega_a) - \psi(\sum_{a^\prime} \omega_{a^\prime}) \equiv h_a(\vomega).
\end{equation}

The log of Eq \ref{full_likelihood} is
\begin{equation}
\ln P(\mathbb{R}, \vz, \vf | \mathbb{A}) = g(\valpha) + \sum_a (\alpha_a - 1) \ln f_a + \sum_{ra} z_{ra} (\ln f_a + \ln \ell_{ra}).
\end{equation}
and thus the first term in Eq \ref{lagrangian} is
\begin{align}
E_q \left[ \ln P(\mathbb{R}, \vz, \vf | \mathbb{A}) \right] =& g(\valpha) + \sum_a (\alpha_a - 1) h_a(\vbeta) + \sum_{ra}\bar{z}_{ra} \left( h_a(\vbeta) + \ln \ell_{ra} \right) \\
=& g(\valpha) + \sum_a (\beta_a - 1) h_a(\vbeta) + \sum_{ra}\bar{z}_{ra} \ln \ell_{ra}, \label{first_term}
\end{align}
where we used the relationship $\vbeta = \valpha + \sum_r \bar{\vz}_r$.

The second term in Eq \ref{lagrangian} is
\begin{align}
- E_q \left[ \ln q(\vz) \right] = - \sum_{ra} \bar{z}_{ra} \ln \bar{z}_{ra} \label{second_term}.
\end{align}

The third term in Eq \ref{lagrangian} is
\begin{align}
- E_q \left[ \ln q(\vf) \right] = -g(\vbeta) - \sum_a (\beta_a - 1) E_q [\ln f_a] = -g(\vbeta) - \sum_a (\beta_a - 1) h_a(\vbeta) \label{third_term}.
\end{align}

Adding Eqs \ref{first_term}, \ref{second_term}, and \ref{third_term} and noting the cancellation between parts of Eqs \ref{first_term} and \ref{third_term} we obtain
\begin{equation}
\ln P(\mathbb{R} | \mathbb{A}) \approx g(\valpha) - g(\vbeta) +  \sum_{ra} \bar{z}_{ra} \left( \ln \ell_{ra} - \ln \bar{z}_{ra} \right).
\end{equation}

We now have the model evidence for allele subset $\mathbb{A}$.  This lets us choose which alleles are true somatic variants.  It also lets us make calls on somatic loss of heterozygosity events.  Furthermore, instead of reporting max-likelihood allele fractions as before, we may emit the parameters of the Dirichlet posterior $q(\vf)$, which encode both the maximum likelihood allele fractions and their uncertainty.

\section{Strand Artifact Model}
\begin{figure}
\centering
\includegraphics[width=0.3\textwidth]{strand_artifact_pgm.png}
\caption{\label{fig:frog}The probabilistic graphical model}
\end{figure}

\begin{itemize}
	\item $\vz$ is a latent random variable having a 1-of-K representation. For each variant locus, $\vz$ encodes the presence of strand artifact in forward reads ($[1, 0, 0]$), artifact in reverse reads ($[0, 1, 0]$), or no artifact ($[0, 0, 1]$)
	\item $f \sim \text{Unif}(0, 1)$ is a prior distribution over alt allele fraction $f$
	\item $\epsilon \sim \text{Beta}(\alpha, \beta)$ is a prior distribution over the error probability on a read on the artifact strand. For instance, if we have strand artifact on the reverse strand (i.e. $z = [0, 1, 0]$), $\epsilon$ is the probability that the sequencer reads a ref allele on a reverse read as alt
	\item $x^+ | f, \epsilon, z$ is the number of forward reads with the alt allele. It's a mixture of binomials, defined as follows:
	\begin{equation}
	x^+ | f, \epsilon, z \sim
		\begin{cases}
			\text{Bin} (n^+, f + \epsilon(1-f)) & z = \mathrm{Art+}\\
			 \text{Bin} (n^+, f) 			& z = \mathrm{Art-} \\
			 \text{Bin} (n^+, f)			& z = \mathrm{noArt}
		\end{cases}
	\end{equation}
\end{itemize}

We compute the conditional distributions of $x^-$  analogously.

Having observed the read counts in the forward and reverse directions, we can compute the posterior probabilities of the latent variable $z$.  Below we derive the unnormalized posterior probability of strand artifact in forward reads ($z = art+$), given that we observed $x^+$ forward alt reads and $x^-$ reverse alt reads. We use a shorthand $z_0$ to denote $z=art+$ for conciseness.

First we will derive the likelihood $p(x^+, x^-, f, \epsilon | z_0)$
\begin{align}
p(x^+, x^- | z_0)  &= \iint_{f, \epsilon}  p(x^+, x^-, f, \epsilon | z_0) \,df\,d\epsilon \\
			  &= \iint_{f, \epsilon}  p(f) p(\epsilon) p(x^+, x^- | z_0, f, \epsilon) \,df\,d\epsilon \\
			  &= \iint_{f, \epsilon}  p(f) p(\epsilon) p(x^+ | z_0, f, \epsilon) p(x^- | z_0, f, \epsilon) \,df\,d\epsilon \\
			  &= \iint_{f, \epsilon}  p(\epsilon) p(x^+ | z_0, f, \epsilon) p(x^- | z_0, f, \epsilon) \,df\,d\epsilon \label{watershed} \\
			  &= \iint_{f, \epsilon}  \mathrm{Beta}(\epsilon|\alpha, \beta) \mathrm{Bin}(x^+ | f + \epsilon(1-f), n^+) \mathrm{Bin}(x^- | f, n^-) \,df\,d\epsilon
\end{align}

The posterior probability of strand artifact in forward reads is therefore

\begin{align}
p(z_0 |x^+, x^-) & \propto p(z_0) p(x^+, x^- | z_0) \\
			 & = p(z_0) \iint_{f, \epsilon}  \mathrm{Beta}(\epsilon|\alpha, \beta) \mathrm{Bin}(x^+ | f + \epsilon(1-f), n^+) \mathrm{Bin}(x^- | f, n^-) \,df\,d\epsilon
\end{align}

The derivation for the probability of strand artifact on reverse strand is analogous.

For the case of no strand artifact, the derivation of likelihoods is identical up to (\ref{watershed}). Here we can simply the equation to a single integral over $f$ because the conditional probabilities of $x^+$ and $x^-$ do not depend on $\epsilon$. We use a shorthand $z_2$ for $z=\mathrm{noArt}$

\begin{align}
p(x^+, x^- | z_2)  &= \iint_{f, \epsilon}  p(\epsilon) p(x^+ | z_2, f, \epsilon) p(x^- | z_2, f, \epsilon) \,df\,d\epsilon \nonumber \\
			  &= \int_{f}  p(x^+ | z_2, f) p(x^- | z_2, f) \,df \int_{\epsilon}  p(\epsilon) d\epsilon \\
			  &= \int_{f}  \mathrm{Bin}(x^+ | f, n^+) \mathrm{Bin}(x^- | f, n^-) \,df
\end{align}

And the posterior probability is

\begin{align}
p(z_2 |x^+, x^-) & \propto p(z_2) p(x^+, x^- | z_2) \\
                         & = p(z_2) \int_{f}  \mathrm{Bin}(x^+ | f, n^+) \mathrm{Bin}(x^- | f, n^-) \,df
\end{align}

\section{Germline Filter}\label{germline-filter}
Suppose we have detected an allele such that its (somatic) likelihood in the tumor is $\ell_t$ and its (diploid) likelihood in the normal is $\ell_n$.  By convention, both of these are relative to a likelihood of $1$ for the allele \textit{not} to be found.  If we have no matched normal, $\ell_n = 1$.  Suppose we also have the population allele frequency $f$ of this allele.  Then the prior probability for the normal to be heterozygous or homozygous alt for the allele is $2f(1-f) + f^2$ and the prior probability for the normal genotype not to contain the allele is $(1-f)^2$.  Finally, suppose that the prior for this allele to arise as a somatic variant is $\pi$.

We can determine the posterior probability that the variant exists in the normal genotype by calculating the unnormalized probabilities of four possibilities:
\begin{enumerate}
\item The variant exists is both the normal and the tumor samples.  This has unnormalized probability $\left(2f(1-f) + f^2 \right) \ell_n \ell_t (1 - \pi)$.
\item The variant exists in the tumor but not the normal.  This has unnormalized probability $(1-f)^2 \ell_t \pi$.
\item The variant exists in neither the tumor nor the normal.  This has unnormalized probability $(1-f)^2 (1 - \pi)$.
\item The variants exists in the normal but not the tumor.  This is biologically very unlikely.  Furthermore, if it \textit{did} occur we wouldn't care about filtering the variant as a germline event because we wouldn't call it as a somatic event.  Thus we neglect this possibility.
\end{enumerate}

Normalizing, we obtain the following posterior probability that an allele is a germline variant:
\begin{equation}
P({\rm germline}) = \frac{(1)}{(1) + (2) + (3)} = \frac{\left(2f(1-f) + f^2 \right) \ell_n \ell_t (1 - \pi)}{\left(2f(1-f) + f^2 \right) \ell_n \ell_t (1 - \pi) + (1-f)^2 \ell_t \pi + (1-f)^2 (1 - \pi)}.
\end{equation}

To filter, we set a threshold on this posterior probability.

\end{document}


\section{Introduction}

Consider classic supervised binary classification.  A learner is fit to a training set consisting of labeled examples from the positive and negative classes.  In contrast, \textit{Positive-unlabeled} (PU) \textit{learning} is a form of \textit{partially-supervised} binary classification where labeled data exists for only one class, i.e.,~positive.  The training set consists of a (positive) labeled set,~$\Pos$, and an unlabeled set,~$\Unlabel$, which is composed of both positive and negatives examples all of whose labels are unknown.  By convention, ${\Pos\cap\Unlabel=\emptyset}$.  PU~learning is applicable to both the \textit{transductive} setting, where the goal is to label~$\Unlabel$ as accurately as possible, as well as the \textit{inductive} setting, where the objective is to maximize the generalization performance on an unseen test set.

There are many applications where labeled data for one class may be unavailable or prohibitively expensive/difficult to collect. Domains where PU~learning has been applied include: land-cover classification~\cite{Li:2011}, protein similarity prediction~\cite{Elkan:2008}, disease gene identification~\cite{Yang:2012}, deceptive/incentivized review identification~\cite{Ren:2014}, targeted marketing~\cite{Yi:2017}, and prescription drug interaction analysis~\cite{Liu:2017}. The PU~learning framework can also be used for outlier detection given a set of inlier examples.~\cite{Scott:2009}

State-of-the-art PU learning algorithms generally rely on a cost-sensitive learning framework where each unlabeled example is simultaneously treated as both positive \textit{and} negative valued with each instance's class weights proportional to its specific label confidence.~\cite{Elkan:2008}  These previous works generally made one or more assumptions about the composition of~$\Pos$ and~$\Unlabel$. Most PU~learning paradigms use more traditional machine learning algorithms such as support vector machines~\cite{Elkan:2008} and logistic regression~\cite{Lee:2003}.  We are not aware of any previous PU~research that specifically focused on leveraging the significant, recent advances in deep learning.

The primary contribution of this work is \textit{\toolname}, a new autoencoder-based positive-unlabeled learner.  The remainder of this document is structured as follows.  Section~\ref{sec:Siamese} describes the Siamese neural network, which inspired this work.  Section~\ref{sec:Toolname} describes \toolname's neural architecture and reviews the novel learner's training algorithm including our custom PU~loss functions.  Section~\ref{sec:Experiments} describes our experimental setup and compares the performance of our algorithm against a~PU and supervised baseline.  Section~\ref{sec:FutureWork} outlines deficiencies identified with our algorithm and proposes future work that may address them.


\chapter{Background}
\label{chapter:background}
Information retrieval (IR) is the activity of obtaining information resources which are relevant to a given query. In terms of semantic similarity, the task of information retrieval is to find documents which are semantically similar to a given query. One of the easiest approaches for finding relevant documents given a query is TFIDF.
TFIDF, short for term frequency inverse document frequency, is an information retrieval technique that shows how important a word is to a document. Another information retrieval algorithm known as BM25 (BM stands for Best Match) \cite{robertson2009probabilistic} can be used to retrieve matching documents according to their relevance to a given query. \cite{dumais2004latent} introduced a word embedding method as an extension of TDIDF known as Latent Semantic Analysis (LSA). \cite{mikolov2013efficient}, \cite{pennington2014glove} and \cite{shazeer2016swivel} have introduced neural network based word embedding models which have now become benchmarks in semantic similarity extraction.

\section{Summary of relevant approaches}
The most naive and intuitive way of finding relevant documents given a query term is to use term frequency (TF). Term frequency assumes that the more frequently the given query term appears in a document the more relevant that document is to the given query. Term frequency, however, suffers from a critical problem: all words are considered equally important when it comes to assessing relevancy on a query. In fact, though some words appear multiple times in a document they have very little discriminating power  in determining the relevancy. For instance, a collection of documents on football is likely to have the word football or soccer in almost every document. To circumvent this issue a technique known as inverse document frequency (IDF) is used to  attenuate the effect of words that occur too often in the collection of documents to be meaningful for relevance determination.
We define the inverse frequency of a word $w$ as follows:

\begin{displaymath}
\mbox{IDF}_w = \log {N\over \mbox{DF}_w}.
\end{displaymath}

where $DF_{w}$ is the document frequency and is defined as the number of documents in the collection that contain a word $w$ and N is the total number of documents. Thus the inverse document frequency of a rare word is high, whereas the inverse document frequency of a frequent word is likely to be low. Each word has its own term frequency and inverse document frequency score, the product of the two scores is called the TFIDF weight of that word. \citet{ramos2003using} provide evidence that TFIDF returns
documents highly correlated to the given query. Different weight schemes for these counts lead to a variety of TFIDF ranking features. One very successful TFIDF
formulation is known as BM25 \citep{robertson2009probabilistic}.
BM25  is a bag-of-words information retrieval function that ranks a set of documents based on their relevance to the query terms. Tweaking different components and parameters produce different variations of the BM25. \citet{mitra2016dual} have shown BM25 to be effective in information retrieval and have also proposed using it in an ensemble model along with another embedding model, namely Word2Vec.

<<<<<<< Updated upstream
Another extension introduced by \citet{dumais2004latent} is known as Latent Semantic Analysis(LSA). It uses Singular Value Decomposition (SVD) to perform dimensionality reduction on the TFIDF vectors resulting in smaller and better features. In Latent Semantic Analysis documents are represented as bags-of-words, where the order of the words in a document is not important, only how many times each word appears in a document. Furthermore, it assumes that words which are close in meaning will appear in similar pieces of text, a concept known as the distributional hypothesis. \citet{boling2014semantic} have shown that Latent Semantic Analysis performs very well in finding semantic similarity between documents. Other popular models which use distributional hypothesis are Word2Vec \citep{mikolov2013efficient} and Swivel \citep{shazeer2016swivel}.
=======
Another extension introduced by \citet{dumais2004latent} is known as Latent Semantic Analysis(LSA). It uses Singular Value Decomposition (SVD) to perform dimensionality reduction on the TFIDF vectors resulting in smaller and better features. In Latent Semantic Analysis documents are represented as bags-of-words, where the order of the words in a document is not important, only how many times each word appears in a document. Furthermore, It assumes that words which are close in meaning will appear in similar pieces of text, a concept known as the distributional hypothesis. \citet{boling2014semantic} have shown that Latent Semantic Analysis performs very well in finding semantic similarity between documents. Other popular models which use distributional hypothesis are Word2Vec \citep{mikolov2013efficient} and Swivel \citep{shazeer2016swivel}.
>>>>>>> Stashed changes
\subsection{Word and Document Embeddings}
Word embedding is a language modeling and feature learning technique in natural language processing(NLP) which maps words and phrases to the real number vector space of the desired dimension.

\paragraph{Word2Vec} Word2Vec model was  introduced by \citet{mikolov2013efficient}. It uses distributed vector representation of words, a well-known framework for learning word vectors as shown in the Figure \ref{fig:Word2Vec model}. The task is to learn to predict a word given other words in the context.
More formally, given a sequence of training words
$w_{1}, w_{2}, w_{3}, ..., w_{T} $, the objective of the word vector model is to maximize the average log probability
\\
\begin{equation}
\frac{1}{T} \sum_{t=K}^{T-K} \log p(w_{t} \mid w_{t-1},....,w_{t+1})
\end{equation}

\begin{figure}[h]
	\centering
	\includegraphics[width=8cm, height=5cm]{w2v.png}
	\caption{A framework for learning word vectors. Context of
		three words (“the,” “cat,” and “sat”) is used to predict the fourth
		word (“on”). The input words are mapped to columns of the matrix
		W to predict the output word.}
	\label{fig:Word2Vec model}
\end{figure}

 \citet{bojanowski2016enriching} propose an extention of Word2Vec model known as FastText. It learns word
 representations while taking into account morphology.
 FastText models morphology by considering subword
 units, and representing words by a sum of its character
 n-grams.  Since FastText exploits subword information, it can also
 compute valid representations for out-of-vocabulary
 words. FastText obtains representations for out-of-vocabulary words by summing the vectors of character
 n-grams.

 \citet{ghosh2016characterizing} introduce a vocabulary driven Word2Vec method known as Dis2Vec which is
 used to generate disease specific word embeddings from unstructured health
 related news corpus. The input corpus D consists of a collection of word context pairs. Based on the vocabulary $V$, we can categorize the word context pairs into three types as shown
 below:
 \\
 \begin{itemize}
 	\item $ D(d) = {(w, c): w \in V ∧c \in V }$, i.e. both the word w and the context c are in V
 	\item $D(\rightharpoondown d) = {(w, c): w \notin V ∧c \notin V }$, i.e. neither the word w nor the context c are in V
 	\item $D(d)(\rightharpoondown d) = {(w, c): w \in V \oplus c \in V }$, i.e. either the word w is in V or the context c is in V but both cannot be in V
 \end{itemize}

 Each of these categories of (w, c) pairs
 needs special consideration while generating disease specific embeddings.

 All the above mentioned word embedding models learn word embeddings from co-occurrence information in corpora.
 One drawback of learning word embeddings by this approach is that such methods will generally fail to tell synonyms from
 antonyms \citep{mohammad2008computing}. For example, words like east and west
 or expensive and inexpensive appear in near-identical contexts, which means
 that distributional models produce very similar word vectors for such words. Such embedding is very undesirable when the goal is to find semantic similarity between documents.
 \citet{mrksic:2016:naacl} proposed a novel counter-fitting method which injects antonym and
 synonymy constraints into vector space representations in order to circumvent this issue. Table \ref{tab:counter_fitting} shows the results \citet{mrksic:2016:naacl} achieved using their counter-fitting technique.

 \begin{table}[h]
 	\begin{center}
 		\begin{tabular}{ c c c c }
 			\hline
 			& east & expensive & British \\
 			\hline
 			\multirow{5}{4em}{Before} &

 			west & pricey &  American
 			\\
 			& north & cheaper & Australian\\
 			& south & costly & Britain\\
 			& southeast & overpriced & European\\
 			& northeast & inexpensive & England\\
 			\hline
 			\multirow{5}{4em}{After} &
 			eastward & costly & Brits\\
 			& eastern & pricy & London\\
 			& easterly & overpriced & BBC\\
 			& - & pricey & UK\\
 			& - & afford & Britain\\
 			\hline
 		\end{tabular}

 	\end{center}

 	\caption{Nearest neighbours for target words using GloVe
 		vectors before and after counter-fitting} \label{tab:counter_fitting}
 \end{table}

 An alternative to
 the bag-of-words approach is to derive contexts
 based on the syntactic relations the word participates
 in as proposed by \citet{levy2014dependency}.

All the above mentioned approaches are word embedding models and do not generalize to sentences and documents. \citet{jsnior2017nilc} propose two methods for obtaining sentence and document level embeddings. The first approach obtains vector embeddings for documents by averaging the word embeddings of all the words in a document. The second approach also averages word embeddings, but each embedding vector is now weighted (multiplied) by the TFIDF
of the word it represents.
\paragraph{Doc2Vec} An extension to the Word2Vec model known as Doc2Vec was introduced by \citet{le2014distributed}. Doc2Vec is capable of constructing representations of input sequences of
variable length. Unlike some of the previous approaches, it is general and
applicable to texts of any length: sentences, paragraphs, and documents. In
Doc2Vec framework (see Figure \ref{fig:doc2vec model}), every document is mapped to a unique
vector, and every word is also mapped to a unique vector. The document
vector and word vectors are averaged or concatenated to predict the next
word in a context. The only difference to a Word2Vec model is the additional
document token. It acts as a memory that remembers what is missing from
the current context or the topic of the document. The document vectors and
word vectors are trained using stochastic gradient descent and the gradient
is obtained via back-propagation.

\begin{figure}[h]
	\centering
	\includegraphics[width=8cm, height=5cm]{para}
	\caption[]{A framework for learning paragraph vector. This framework
		is similar to the framework presented in Figure 1; the only
		change is the additional paragraph token that is mapped to a vector
		via matrix D.}
	\label{fig:doc2vec model}
\end{figure}


\documentclass[a4paper]{article}

\usepackage{INTERSPEECH2021}

\usepackage{cases}
\usepackage{listings}
\usepackage[framemethod=tikz]{mdframed}

\hyphenpenalty=10000 % Disables hyphenation.

\title{Procedural terrain generation}
\name{Willy Jacquet$^1$, Parra Yoan$^2$}
%The maximum number of authors in the author list is twenty. If the number of contributing authors is more than twenty, they should be listed in a footnote or in acknowledgement section, as appropriate.
\address{
  $^1$Author Affiliation\\
  $^2$Co-author Affiliation}
\email{author@university.edu, coauthor@company.com}

\begin{document}

\maketitle
%
\begin{abstract}
%Thus we propose a model base on mathematical logic and it's resulting C++ implementation. The model is comprehensive by extensibility and therefore faster development of procedural terrain by inferring basic function properties and automatically optimizing code.

\textit{Symbolic computation} is seen as a trade-off between expressiveness and performance.
Metaprogramming can minimize that overhead and enable further optimization.

\end{abstract}

%\noindent\textbf{Index Terms}: speech recognition, human-computer interaction, computational paralinguistics

\section{Introduction}

In this paper we use procedural terrain generation as our primary application and thus tweaked the model for vector manipulation.

\section{State of the art}

\section{Symbolic computation}

Manipulation of \textit{functions} instead of \textit{values}.
\textit{Functions} are \textit{first class citizens}.

The following model is designed to take advantage of \textit{template metaprogramming} facilities.

\begin{mdframed}
This document is thus extended with insight about making a \textit{C++} implementation out of it.
\end{mdframed}

\subsection{Generic functions}

A \textbf{reduction} predicate is introduced, noted $a \rightarrow b$ ($a$ can be \textit{reduced} to $b$). This predicate is \textit{transitive}.

\begin{mdframed}
A \textit{function} can be represented as a \textit{type} that holds no \textit{non-static data member}.
In particular, if two instances of functions share the same type then they refer to the same function.
\end{mdframed}

A \textbf{tuple} is defined as a \textit{sequence of objects} $(x_1,...,x_n)$.

The \textbf{application} of a function $f$ with $x_1,...,x_n$ is noted $f(x_1,...,x_n)$.
Generally, the \textit{application of a tuple} $(f_1,...,f_n)$ is defined as:
\begin{equation}
(f_1,...,f_n)(x_1,...,x_n) \rightarrow (f_1(x_1,...,x_n),...,f_n(x_1,...,x_n))
\end{equation}
If $f(x)$ cannot be further \textit{reduced} ($\neg\exists a, f(x) \rightarrow a$) then for $y_1,...y_n$:
\begin{equation}
f(x)(y_1,...y_n) \rightarrow f(x(y_1,...y_n))
\end{equation}

A \textbf{constant} $c$ is defined as a function that returns itself:
\begin{equation}
	c(x_1,...,x_n) \rightarrow c
\end{equation}

\begin{mdframed}
A \textit{compile-time constant} can be represented as:
\begin{lstlisting}
template<typename Type, Type Value>
struct constant {};
\end{lstlisting}
Special constants ($0$, $1$, $\pi$, $e$, ...) can be introduced as specific symbols since they appear in many (number) sets.
\begin{lstlisting}
struct zero {}; struct one {}; ...
\end{lstlisting}
\end{mdframed}

Given $i \in \mathbb{N}^*$, the \textbf{$i$-th projection} is a function that returns its $i$-th parameter:
\begin{numcases}{proj_i(x_1,...,x_n) \rightarrow}
	x_i          & \text{if $i \leq n$} \\
	proj_{i - n} & \text{otherwise} \label{partial_projection}
\end{numcases}
When the $i$-th projection is applied to the $i$-th parameter of each function, the notation is abbreviated:
\begin{equation}
	f(p_0,p_1,...) = f
\end{equation}

The \textbf{arity} (function) can be defined as follows:
\begin{equation}
\begin{split}
	arity(c) &= 0 \\
	arity(proj_i) &= i \\
	arity(f(g_1,...,g_n)) &= max\{arity(g_i)\}_{i \leq n}
\end{split}
\end{equation}

\subsection{Arithmetic functions}

\textit{Elementary arithmetic} functions are introduced with their usual definition:
\begin{equation}
\begin{split}
	(f + g)(x)      &\rightarrow f(x) + g(x) \\
	(f - g)(x)      &\rightarrow f(x) - g(x) \\
	(f \times g)(x) &\rightarrow f(x) \times g(x) \\
	(f / g)(x)      &\rightarrow f(x) / g(x)
\end{split}
\end{equation}
\textit{Partial application} is made possible by (\ref{partial_projection}). For example:
\begin{equation}
\begin{split}
	(proj_0 + proj_1)(x) &\rightarrow proj_0(x) + proj_1(x) \\
	                     &\rightarrow x + proj_0
\end{split}
\end{equation}
\textit{Lazy evaluation} is introduced by establishing:
\begin{equation}
\begin{split}
	0 \times g &\rightarrow 0\\
	f \times 0 &\rightarrow 0\\
	0 / g      &\rightarrow 0
\end{split}
\end{equation}
\begin{mdframed}
This can be implemented by providing \textit{function overloads} having the first or second operand with the zero type.
\end{mdframed}

The current model can be extended with any usual function.
Any function that is not mentioned in this section is defined with its usual meaning.

\subsection{Differential calculus}

The \textbf{partial derivative} \textit{with respect to $x$} is introduced for generic functions:
\begin{align}
\frac{\partial}{\partial x}(c) &\rightarrow 0 \text{ with $c$ a constant} \\
\frac{\partial}{\partial proj_j}(proj_i) &\rightarrow
\begin{cases}{}
1 & \text{if $i = j$} \\
0 & \text{otherwise}
\end{cases} \\
\frac{\partial}{\partial x}(f(y)) &\rightarrow \frac{\partial}{\partial x}(y) \times \frac{\partial}{\partial x}(f)(y)
\end{align}
For arithmetic functions:
\begin{align}
\frac{\partial}{\partial x}(f + g) &\rightarrow \frac{\partial}{\partial x}(f) + \frac{\partial}{\partial x}(g) \\
\frac{\partial}{\partial x}(f - g) &\rightarrow \frac{\partial}{\partial x}(f) - \frac{\partial}{\partial x}(g) \\
\frac{\partial}{\partial x}(f \times g) &\rightarrow \frac{\partial}{\partial x}(f) \times g + f \times \frac{\partial}{\partial x}(g) \\
\frac{\partial}{\partial x}(f / g) &\rightarrow \frac{\frac{\partial}{\partial x}(f) \times g - f \times \frac{\partial}{\partial x}(g)}{g \times g}
\end{align}

\subsection{Computational redundancy}

Suppose that, provided a function $f \times g$, you wish to compute its value \emph{and} partial derivative for an argument $x$.

\subsection{Parallelization}

\subsection{Compilation}

\subsection{Performance}

\section{Common functions}

\subsection{Interpolation}

Provided an interpolant $t \in [0, 1]$.
Polynomial interpolation with null derivatives is defined as follows:
\begin{itemize}
\item linea:r $t$
\item cubic: $-2t^3 + 3t^2$
\item quintic: $6t^5 - 15t^4 + 10t^3$
\end{itemize}

\subsection{Noise functions}

\textit{Coherent noise} can be used to model many natural phenomena and is characterized as follow \cite{libnoise_coherent_noise}:
\begin{itemize}
\item \textit{Referential transparency} i.e.\ the same input produces the same output
\item A small change in the input value will produce a small change in the output value.
\item A large change in the input value will produce a random change in the output value
\end{itemize}

\textbf{Value noise} is achieved by interpolating random values on a regular grid along each axis until it is reduced to a single number.

\begin{figure}[h]
\includegraphics[width=\linewidth]{img/value_noise_2d}
\caption{2D value noise with quintic interpolation.}
\end{figure}

\begin{lstlisting}[caption=Pseudo code implementation of 2D value noise]
float noise(v:float[2]):
  i=floor(v)
  t=fract(v)
  return interp(
    interp(hash(i+[0,0]),hash(i+[0,1]),t[0]),
    interp(hash(i+[1,0]),hash(i+[1,1]),t[0]),
    t[1])
\end{lstlisting}
With $interp$ being an interpolation function.

\subsection{Worley noise}

\section{Terrain representation}

A \textbf{terrain} can be defined as a $C^2$ function $\mathbb{R}^2 \rightarrow \mathbb{R}$.
Therefore it is interpreted

\subsection{Coloration}

\subsection{Rendering}

\subsection{Implicit representation}

\section{Results}

\section{Appendix A: Derived functions}

Many usual functions can be defined in terms of those previously introduced.
Doing so makes inferring most properties (derivative, domain, ...) automatically possible.

\subsection{Generic functions}
\begin{flalign*}
swap = (proj_1, proj_0)
\end{flalign*}

\subsection{Arithmetic functions}
\begin{equation*}
\begin{array}{ll}
opposite    &= 0 - proj_0 \\
inverse     &= 1 / proj_0 \\
\\
translation &= p1(p0 + p2) \\
scaling     &= p1(p0 \times p2)
\end{array}
\end{equation*}

\subsection{Calculus functions}
\begin{equation*}
\begin{array}{ll}
\partial(cos) &= - sin \\
\partial(sin) &= cos \\
\end{array}
\end{equation*}

\bibliographystyle{IEEEtran}
\bibliography{mybib}

\end{document}


\documentclass[bigger]{beamer}

\input{header-beam} % change to header-handout for handouts

% ====================
\title[Lecture 22]{Logic I F13 Lecture 22}
\date{December 3, 2013}
% ====================

\input{header}

\setlength{\fitchprfwidth}{14em}

\section{Logical Metatheory}

\subsec{Truth-functional Completeness}{

\bit
\item Every truth function can be expressed by a sentence containing only
\ben
\item $\{\land, \lnot\}$,
\item $\{\lor, \lnot\}$,
\item $\{\to, \lnot\}$,
\item $\{\to, \bot\}$,
\item $\{\mid\}$: not-both, truth table: \F\T\T\T
\item $\{\downarrow\}$: neither-nor, truth table: \F\F\F\T
\een

\item Others, e.g., $\{\land, \lor\}$, $\{\liff, \lnot\}$ are not
  truth-functionally complete

\eit

}

\subsec{Semantics}{

\bit
\item A \emph{truth-value assignment} is an assignment of \T{} or \F{} to the atomic sentences (schematic letters in the truth-functional form)
\item A \emph{FO structure} is a non-empty domain together with
\bit
\item extensions for each predicate symbol
\item objects in the domain for each constant symbol
\item functions for each function symbol
\eit
\item A tautology is a sentence (the truth-functional form of) which is true in all truth-value assignments
\item A FO validity is a sentence that's true in all FO interpretations
\eit

}

\subsec{Soundness and Completeness}{

\bit
\item Soundness

Arguments have formal proofs only if they are valid\\
If there is a proof of B from premises $\sf A_1, \dots A_n$, then B is a consequence of $\sf A_1, \dots A_n$

\item Completeness

Arguments have formal proofs if they are valid\\
If B is a FO consequence of $\sf A_1, \dots A_n$, then there is a proof of B from premises $\sf A_1, \dots A_n$\\[2ex]
Proved by Kurt G\"odel (1929)
\eit
}

\subsec{Other Proof Systems: Resolution}{

\bit
\item Fitch is a proof system for \emph{validity/tautologies}
\item Also possible to design proof systems for dual notion: \emph{unsatisfiability}\\
(A is a tautology $\Leftrightarrow$ $\sf\lnot A$ is unsatisfiable)
\item A \emph{clause} is a disjunction $\sf A_1 \lor \ldots \lor A_n$
where each $\sf A_i$ is atomic or negated atomic.
\item The resolution rule:
\[
\fitchctx{\pline{A_1 \lor \ldots \lor A_n \lor C}\\
\pline{B_1 \lor \ldots \lor B_m \lor \lnot C}\\
\fpline{A_1 \lor \ldots \lor A_n \lor B_1 \lor \ldots \lor B_m}}
\]
\item Preserves \emph{joint satisfiability}
\item If you can prove the empty clause $\bot$ from a set of clauses, they can't be
  jointly satisfiable.
\eit

}

\section{Theories and Decidability}

\subsec{Church-Turing Theorem}{

\begin{block}{Instance: Sentence A of any FO language\\
Problem: Is A a FO validity?}

\bit
\item Undecidable
\item Proved independently by Alonzo Church and Alan Turing in 1935
\eit
\end{block}
}

\subsec{Decidable Classes}{

\bit
\item The decision problem \emph{in general} is undecidable
\item But special cases \emph{can} be decided, e.g.:
\eit
\begin{block}{Instance: Sentence A with only 1-place predicate symbols\\
Problem: Is A a FO validity?}

\bit
\item Decidable
\item Proved by Leopold L\"owenheim (1915)
\eit
\end{block}
}

\subsec{Theories}{

\bit
\item A set of sentences of FOL also called a \emph{theory}, and the sentences in it \emph{axioms}
\item Some (types of) FO structures can be characterized
as the models of a theory
\item Examples:
\bit
\item Mathematical theories (theory of orders, group theory, arithmetic)
\item KR classification systems, e.g., SNOMED-CT
\item Mereology, theories of truth, scientific theories
\eit
\eit

}

\subsec{The Axiomatic Method}{

\bit
\item Theories + logic: what follows from axioms?
\item Axiomatic method: do science by investigating what follows from
  the axioms of a theory
\item Logic can also determine:
\bit
\item Are axioms (in)consistent?
\item Are axioms independent, or is one superfluous?
\eit
\item Paradigm of axiomatic method: geometry (Euclid)
\eit
}

\subsec{Examples of Theories: Linear Orders}{

A relation $\preceq$ on set $O$ is a \emph{linear order} if
it is a model of the theory LO with axioms:
\begin{align*}
& \forall x\forall y((x \preceq y \land y \preceq x) \to x = y) & \text{Antisymmetry}\\
& \forall x\forall y\forall z((x \preceq y \land y \preceq z) \to x \preceq z) &\text{Transitivity}\\
& \forall x\forall y(x \preceq y \lor y \preceq x) & \text{Totality}
\end{align*}
Every total relation is reflexive:
\[
{\sf LO} \models \forall x\, x \preceq x
\]
}

\subsec{Examples of Theories: Robinson's Q}{

Theories of arithmetic, such as Robinson's theory Q:
\begin{align*}
 \lnot\exists x\, (x + 1) & = 0\\
 \forall x(x \neq 0  \to \exists y\, (y+1) & = x)\\
 \forall x\forall y((x + 1) = (y + 1)  \to x & = y)\\
 \forall x\,(x + 0) & = x\\
 \forall x\forall y\, (x + (y+1)) & = ((x + y) +1) \\
 \forall x\,(x \times 0) & = 0\\
 \forall x\forall y\, (x \times (y + 1)) & = ((x \times y) + x)
\end{align*}
}

\subsec{Examples of Theories: SNOMED-CT}{

\begin{align*}
\texttt{bacterial } & \texttt{pneumonia} = \\
& \texttt{is-a|bacterial infectious disease}\\
& \texttt{is-a|infective pneumonia}\\
& \texttt{causative agent|bacteria}\\
& \texttt{finding site|lung structure}\\[2ex]
%\begin{align*}
\forall x(& BacterialPneumonia(x) \liff  \\
& BacterialInfectiousDisease(x) \land\\
& InfectivePneumonia(x) \land \\
& \exists y(HasCausativeAgent(x, y) \land Bacteria(y)) \land\\
&\exists y(HasFindingSite(x, y) \land LungStructure(y)))
\end{align*}
}

\subsec{Examples of Theories: SNOMED-CT}{

\bit
\item Over 300,000 concepts (predicate symbols), e.g.,\\
\bit
\item 1-place predicates:\\
parts of body, findings, organisms, physical objects, procedures, substances, diseases, \dots
\item 2-place predicates:\\
has finding site, has causative agent, with method, has active ingredient, laterality is, using device, \dots
\eit
\item About 1,000,000 descriptions (axioms)
\item SNOMED-CT is decidable
\eit

}

\subsec{Examples of Theories: Mereology}{

\bit
\item Mereology: the theory of the part-whole relation (\emph{metaphysics})
\item Primitive relation: Pt(x, y), ``x is a part of y''
\item Some axioms:
\begin{align*}
& \forall x\, Pt(x, x) & \text{Reflexivity}\\
& \forall x\forall y\forall z((Pt(x,y) \land Pt(y, z)) \to Pt(x,z)) & \text{Transitivity}\\
& \forall x\forall y((Pt(x, y) \land Pt(y, x)) \to x = y) & \text{Antisymmetry}
\end{align*}
\item Defined properties and relations
\[
PP(x, y) \dots Pt(x, y) \land x \neq y \qquad At(x) \dots \lnot\exists y\,PP(y, x)
\]
\item Different theories settle questions differently, e.g.,
\bit
\item Are there atoms?
\item Does everything comprise at least one atom?
\item Is everything made of atomless ``gunk''?
\eit
\eit

}

\subsec{Property Theories and Grelling's Paradox}{

\bit
\item Primitive relation: App(x ,y), ``x applies to y''
\item Proposed axiom (``comprehension''): For any wff P(y),
\[\sf
\exists x\forall y(App(x, y) \liff P(y))
\]
\item Axiom is \emph{inconsistent} (contradictory)
\item A property is \emph{heterological} if it does not apply to itself, i.e., $
\sf \lnot Ap(x, x)$
\item Is the property of being heterological itself heterological?
\eit
\fitchprf{\pline{\forall y(Ap(h, y) \liff \lnot Ap(y, y))}}{
\pline{Ap(h, h) \land \lnot Ap(h, h)}}
}

\subsec{Completeness of Theories}{

\bit
\item A theory T is \emph{complete} if for every sentence A in its language,
either $\sf T \models A$ or $\sf T \models \lnot A$
\item Every complete theory is decidable!
\item Some incomplete theories are still decidable (e.g., LO)
\item Some incomplete theories are incomplete\emph{able}: no consistent extension is complete
\item Philosophical upshot of this: truth in the intended model(s) of the theory outstrips provability from the theory
\eit
\textbf{G\"odel's Incompleteness Theorem (1930)}\\
Arithmetic, set theory, mereology are incompleteable

}

\section{Philosophy and Nonstandard Logics}

\subsec{Logical Consequence and FO Consequence}{

\bit
\item Philosophers interested in \emph{valid arguments}, i.e.,
those where the conclusion is a \emph{logical consequence} of the premises
\item Definition: There is no logically possible circumstance where the
premises are true and the conclusion is false
\bit
\item \emph{Important:} It does not say ``it \emph{isn't the case} that the premises
are true and the conclusion is false''
\item That would make every argument with
\bit
\item true premises, true conclusion
\item false premises, true conclusion
\item false premises, false conclusion
\eit
logically valid.  But that's not the case.
\item It says ``it is \emph{impossible} that the premises could be true and the conclusion false!''
\eit
\item Difficulty: What logically possible circumstances are there?
\eit

}

\subsec{What Logic Does For Logical Validity}{

\bit
\item Truth-tables, FO interpretations, Fitch give \emph{sufficient conditions}
for validity, i.e.,
\bit
\item Every tautologically valid argument is logically valid
\item Every FO valid argument is logically valid
\item Every argument with a formal proof in Fitch is logically valid (soundness!)
\eit
\item Also provide tools to do more:
\bit
\item FO interpretations show why consequence fails
\item Show which relations between predicate symbols would suffice for logical validity
\item Provide ways to symbolize these relations
\item Formal theories about informal concepts can capture these relations, or elucidate consequences of assumptions about them (axiomatic method)
\eit
\eit

}

\subsec{Nonstandard Logics}{

\bit
\item Formal models of logical consequence make a number of simplifying assumptions:
\bit
\item Only \emph{determinate} properties allowed, e.g, no vague properties
\item Every (atomic) sentence either \T{} or \F; not both and nothing in between
\item Every constant must refer, i.e., no empty names
\item Only truth-functional connectives, e.g., no subjunctive contionals, ``because'', or tenses
\eit
\item Non-standard logics: expand FO logic to deal with these
\eit

}

\subsec{Many-valued Logic}{
\def\I{{\color{blue}\mathsf{U}}}
\bit
\item Add to the truth-values \T{} and \F, e.g.,
\bit
\item ``Undetermined'': neither true nor false
\[
\begin{array}{c|c}
P & \lnot P\\
\hline
\T & \F \\
\I & \I\\
\F & \T
\end{array}
\qquad
\begin{array}{cc|c}
P & Q & (P \land Q)\\
\hline
\T & \T & \T\\
\T & \I & \I\\
\T & \F & \F\\
\I & \T & \I\\
\I & \I & \I\\
\I & \F & \F\\
\F & \T & \F\\
\F & \I & \F\\
\F & \F & \F
\end{array}
\qquad
\begin{array}{cc|c}
P & Q & (P \lor Q)\\
\hline
\T & \T & \T\\
\T & \I & \T\\
\T & \F & \T\\
\I & \T & \T\\
\I & \I & \I\\
\I & \F & \I\\
\F & \T & \T\\
\F & \I & \I\\
\F & \F & \F
\end{array}
\]
\item ``Inconsistent'': both true and false
\item Fuzzy truth values: any number between 0 and 1
\eit
\eit

}

\subsec{Modal Logic}{

\bit
\item Alethic logic\\
``It is possible that'' ($\Diamond$), ``it is necessary that'' ($\Box$)
\[
\Box A \to A \qquad \Diamond\Box A \to \Box A
\]
\item Epistemic logic:
``It is known that'' (K)
\[
KK A \to K A
\]
\item Conditional logic\\
Subjunctive conditionals, ``if it were true that \dots, then\\ \quad it would be true that --- ---'' ($\strictif$)
\[
(A \strictif B) \to (A \to B)
\]
\item Temporal logic\\
``It was true that'' (P), ``It will be true that'' (F)
\[
F\,P\, A \to (P\, A \lor A \lor F\, A)
\]
\eit

}

\subsec{Christmas Trick}{

If the first sentence on this slide is true, \\\qquad then Santa Claus exists.

Proof that Santa Claus exists:
\bens
\item The first sentence on this slide is true\\\qquad (Assumption for
  conditional proof).
\item If the first sentence on this slide is true, then Santa Claus exists (from 1, since if S is true, then S).
\item Santa Claus exists (from (1) and (2), by modus ponens)
\item If the first sentence on this slide is true, Santa Claus exists\\\qquad(from (1)--(3), by conditional proof).
\item We've just proved (4) [= the first sentence on this slide], so the first sentence on this slide is true.
\item Santa Claus exists (from (4) and (5), by modus ponens)
\een
}

\end{document}


\section{Conclusion}

\begin{frame}{Conclusion}
	\begin{itemize}
		\item The understanding of the thermalization process of gluons is key to find an appropriate description of the complex physical processes during RHICs.
		\item Using \alert{kinetic theory} and \alert{statistical transport equations} we can estimate important quantities such as the \alert{equilibration time} and understand the importance and differences of \alert{elastic and inelastic collisions}.
		\item The role of \alert{Bose-Einstein condensation} during the thermalization process
		\item It is possible to find \alert{analytic solutions for a Nonlinear Boson Diffusion equation} providing further insights into the thermalization process.

	\end{itemize}
\end{frame}