% !TEX root = ../zeth-protocol-specification.tex

\section{\zeth~statement}\label{zeth-protocol:statement}

As explained in~\cite{zethpaper}, the $\mix$ function of $\mixer$ verifies the validity of $\zkp$ on the given primary inputs in order to determine whether the state transition is valid. As such, $\mixer$ verifies whether for $\zkp$, and primary input $\priminputs$, there exists an auxiliary input $\auxinputs$, such that the tuple $\smalltuple{\priminputs, \auxinputs}$ satisfies the $\npol$-relation $\RELCIRC$, consisting of the following constraints:

\begin{itemize}
    \item For each $i \in [\jsin]$:
    \begin{enumerate}
        \item $\auxinputs.\jsins{i}.\znote.\apk = \prfaddr{\auxinputs.\jsins{i}.\ask}{0}$
        \item $\auxinputs.\jsins{i}.\cm{} = \comm.\commit{\auxinputs.\jsins{i}.\znote.\apk,\allowbreak \auxinputs.\jsins{i}.\znote.\rrho,\allowbreak \auxinputs.\jsins{i}.\znote.\notev}{\auxinputs.\jsins{i}.\znote.\noter{}}$
        \item $\auxinputs.\jsins{i}.\nf{} = \prfnf{\auxinputs.\jsins{i}.\ask}{\auxinputs.\jsins{i}.\znote.\rrho}$
        \item $\auxinputs.\htags{i} = \prfpk{\auxinputs.\jsins{i}.\ask}{i, \auxinputs.\hsig}$ (non-malleability, see~\cref{appendix:trnm})
        \item $(\auxinputs.\jsins{i}.\znote.\notev) \cdot (1 - e)  =  0$ is satisfied for the boolean value $e$ set such that if $\auxinputs.\jsins{i}.\znote.\notev > 0$ then $e = 1$.
        \item The Merkle root $\mkroot'$ obtained after checking the Merkle authentication path $\auxinputs.\jsins{i}.\mkpath$ of commitment $\auxinputs.\jsins{i}.\cm{}$, with $\mkhash$, equals to $\priminputs.\mkroot$ if $e = 1$.
        \item $\priminputs.\nfs{i}$ \\ $=  \indexedset{\pack{\slice{\auxinputs.\jsins{i}.\nf{}}{k \cdot \fieldBitCap}{(k+1) \cdot \fieldBitCap}}{\FFx{\rCURVE}}}{k \in [\floor{\prfNfOutLen/\fieldBitCap}]}$ (see~\cref{instantiation:statement:pack} for definition of $\pack{}{}$)
        \item $\priminputs.\htags{i}$ \\ $= \indexedset{\pack{\slice{\auxinputs.\htags{i}}{k \cdot \fieldBitCap}{(k+1) \cdot \fieldBitCap}}{\FFx{\rCURVE}}}{k \in [\floor{\prfPkOutLen/\fieldBitCap}]}$ (see~\cref{instantiation:statement:pack} for definition of $\pack{}{}$)
    \end{enumerate}
    \item For each $j \in [\jsout]$:
        \begin{enumerate}
            \item $\auxinputs.\znotes{j}.\rrho = \prfrho{\auxinputs.\pphi}{j, \auxinputs.\hsig}$ (non-malleability, see~\cref{appendix:trnm})
            \item $\priminputs.\cms{j} =\comm.\commit{\auxinputs.\znotes{j}.\apk,\allowbreak \auxinputs.\znotes{j}.\rrho,\allowbreak \auxinputs.\znotes{j}.\notev}{\auxinputs.\znotes{j}.\noter{}}$
        \end{enumerate}
    \item $\priminputs.\hsig = \indexedset{\pack{\slice{\auxinputs.\hsig}{k \cdot \fieldBitCap}{(k+1) \cdot \fieldBitCap}}{\FFx{\rCURVE}}}{k \in [\floor{\crhhsigOutLen/\fieldBitCap}]}$ (see~\cref{instantiation:statement:pack} for definition of $\pack{}{}$)
    \item $\priminputs.\resbits = \packResBits{\indexedset{\auxinputs.\jsins{i}.\nf{}}{i \in [\jsin]}, \auxinputs.\vin, \auxinputs.\vout, \auxinputs.\hsig, \indexedset{\auxinputs.\htags{i}}{i \in [\jsin]}}$ (see~\cref{instantiation:statement:pack} for definition of $\packResBits{}$)
    \item Check that the ``\gls{joinsplit} is balanced'', i.e.~check that the \gls{joinsplit-eq} holds:\footnote{where $\pack{x}{\FFx{\rCURVE}}$ outputs the numerical value of $x$ in $\FFx{\rCURVE}$.  We rely on the fact that $\zvalueLen < \fieldBitCap$ to perform this sum.}
    \begin{align*}
        &\pack{\auxinputs.\vin}{\FFx{\rCURVE}} + \sum_{i \in [\jsin]} \pack{\auxinputs.\jsins{i}.\znote.\notev}{\FFx{\rCURVE}} \\
        & = \sum_{j \in [\jsout]} \pack{\auxinputs.\znotes{j}.\notev}{\FFx{\rCURVE}} + \pack{\auxinputs.\vout}{\FFx{\rCURVE}}
    \end{align*}
\end{itemize}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS630: Database Management Systems
% Copyright 2014 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/beacon
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 2}

Write \textbf{SQL queries} for the following:

\begin{enumerate}

\item
Write a statement to create the table \texttt{Works}.
You do NOT need to provide create table statements for the other tables.
You may choose at your discretion the details of the exact SQL type for columns, as long as the category matches (e.g. integer, fractional or character type).

\item
Find the names of employees who work under the supervision of a manager named \textit{Steve Smith}.

\item
Find the ages of employees who do not work in any department with budget below 20000.

\item
Find the age(s) of the employee(s) with the highest salary.

\item
Find the \texttt{did} and average salary over employees younger than 45 years old for each department with at least 10 employees of any age.

\item
Find the names of employees who work in all departments.

\item
Find the name(s) of the department(s) with the highest average salary.

\end{enumerate}

\subsection*{Solution}

\lstset{language=sql}
\lstinputlisting[firstline=8]{
	\topDirectory/src/sql/m01/m01q02.sql
}


\subsection{\texorpdfstring{The 0\textsuperscript{th} Law of Thermodynamics}{The 0th Law of Thermodynamics}}
The 0\textsuperscript{th} law of Thermodynamics states that:
\begin{center}
    If systems A and B are in thermal equilibrium, and system B and C are in equilibrium, then systems A and C are in equilibrium.
\end{center}
This law may seem extremely silly to even bother stating, but there's actually a lot of things we can do with it! For example, this law allows us to construct thermometers as we know them (object B in the above definition is exactly a thermometer)!


\documentclass{article}

% - style template
\usepackage{base}

% - title, author, etc.
\title{PHYS4000 - Workshop 3}
\author{Tom Ross - 1834 2884}
\date{\today}

% - headers
\pagestyle{fancy}
\fancyhf{}
\rhead{\theauthor}
\chead{}
\lhead{\thetitle}
\rfoot{\thepage}
\cfoot{}
\lfoot{}

% - output files
\newcommand{\wffile}[2]{../output/qho/wf.n_x-#1.#2.txt}
\newcommand{\enfile}[1]{../output/qho/en.#1.txt}
\newcommand{\itfile}[1]{../output/qho/it.#1.txt}

\pgfplotstableread{\enfile{sb}}{\ensb}
\pgfplotstableread{\enfile{nc}}{\ennc}

\pgfplotstableread{\itfile{sb}}{\itsb}
\pgfplotstableread{\itfile{nc}}{\itnc}

% - document
\begin{document}

\tableofcontents

\listoffigures

\listoftables

\clearpage

The entire code repository used to calculate the data for this report, and the
\lilf{tex} file used to produce this \lilf{pdf} document, can be found at
\url{https://github.com/dgsaf/acqm-workshop-3}.

% Note that at this point \autoref{sec:dissociative} has not yet been attempted -
% it may be at a later date, time permitting.

\section{Quantum Harmonic Oscillator}
\label{sec:qho}

\subsection*{Discussion of Theory}
\label{sec:qho-theory}

\subsection*{Discussion of Implementation}
\label{sec:qho-implementation}

The vibrational wavefunctions were calculated using both the shooting-bisection
and the Numerov-Cooley methods, on a grid
$X_{N} = \lrset{x_{1}, \dotsc, x_{N}}$.
The grid $X_{N}$ was constructed such that $x_{1} = -5$, $x_{N} = 5$, with
varying $N_{k} = 2^{k}$, and hence with correspondingly varying step size
$\lr{\delta x}_{k} = 10 \times 2^{-k}$, for $k = 10, \dotsc 17$.

\subsection*{Wavefunctions}
\label{sec:qho-wavefunctions}

It was observed that the wavefunctions had converged indistinguishably close to
the analytic functions by the first data point $k = 10$, corresponding to
$N = 1024$ and $\lr{\delta x} = 0.0097656$.
Hence, we simply present the wavefunctions for one data point, $k = 15$.

The $n = 0, \dotsc, 3$ wavefunctions, calculated using the shooting-bisection
method with, are compared with the analytic wavefunctions in
\autoref{fig:qho-wf-sb}.

\begin{figure}[h]
  \begin{center}
    \input{wf_sb.tex}
  \end{center}
  \caption[S-B Wavefunctions]{
    The $n = 0, \dotsc, 3$ vibrational wavefunctions, calculated using the
    shooting-bisection method for $k = 15$ (shown in blue), are compared with
    the analytic wavefunctions (shown in red).
    The potential is also presented for clarity (shown in black).
    Note that the calculated wavefunctions have been shifted and scaled to
    $\psi_{i} \to \tfrac{1}{2}\psi_{i} + E_{i}$, while the analytic
    wavefunctions have been shifted and scaled to
    $\psi_{i} \to \tfrac{1}{2}\psi_{i} + E_{i} + 0.05$.
  }
  \label{fig:qho-wf-sb}
\end{figure}

The $n = 0, \dotsc, 3$ wavefunctions, calculated using the Numerov-Cooley
method, are compared with the analytic wavefunctions in \autoref{fig:qho-wf-nc}.

\begin{figure}[h]
  \begin{center}
    \input{wf_nc.tex}
  \end{center}
  \caption[N-C Wavefunctions]{
    The $n = 0, \dotsc, 3$ vibrational wavefunctions, calculated using the
    Numerov-Cooley method for $k = 15$ (shown in blue), are compared with the
    analytic wavefunctions (shown in red).
    The potential is also presented for clarity (shown in black).
    Note that the calculated wavefunctions have been shifted and scaled to
    $\psi_{i} \to \tfrac{1}{2}\psi_{i} + E_{i}$, while the analytic
    wavefunctions have been shifted and scaled to
    $\psi_{i} \to \tfrac{1}{2}\psi_{i} + E_{i} + 0.05$.
    Note also that $n = 1$ calculated wavefunction has a phase shift of $-1$,
    which is an arbitrary artifact of the calculation.
  }
  \label{fig:qho-wf-nc}
\end{figure}

\subsection*{Energies}
\label{sec:qho-energies}

The $n = 0, \dotsc, 3$ vibrational energies, calculated using the
shooting-bisection method, are compared with the analytic energies in
\autoref{tab:qho-en-sb}.

\begin{table}[h]
  \begin{center}
    \pgfplotstabletypeset
    [ multicolumn names
    , col sep=space
    , display columns/0/.style={
      column name={$k = 10$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/1/.style={
      column name={$k = 11$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/2/.style={
      column name={$k = 12$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/3/.style={
      column name={$k = 13$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/4/.style={
      column name={$k = 14$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/5/.style={
      column name={$k = 15$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/6/.style={
      column name={$k = 16$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/7/.style={
      column name={$k = 17$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/8/.style={
      column name={Analytic}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , every head row/.style={
      before row={\toprule}
      , after row={\midrule}
    }
    , every last row/.style={
      after row=\bottomrule
    }
    ]{\enfile{sb}}
  \end{center}
  \caption[S-B Energies]{
    The $n = 0, \dotsc, 3$ vibrational energies, calculated using the
    shooting-bisection method for $k = 10, \dotsc, 17$, are presented and
    compared with the analytic energies, to 6 significant figures.
    It can be seen that they are already convergent to the analytic energies by
    $k = 12$ - perhaps smaller grid sizes should be investigated further.
  }
  \label{tab:qho-en-sb}
\end{table}

The $n = 0, \dotsc, 3$ vibrational energies, calculated using the
Numerov-Cooley method, are compared with the analytic energies in
\autoref{tab:qho-en-nc}.

\begin{table}[h]
  \begin{center}
    \pgfplotstabletypeset
    [ multicolumn names
    , col sep=space
    , display columns/0/.style={
      column name={$k = 10$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/1/.style={
      column name={$k = 11$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/2/.style={
      column name={$k = 12$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/3/.style={
      column name={$k = 13$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/4/.style={
      column name={$k = 14$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/5/.style={
      column name={$k = 15$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/6/.style={
      column name={$k = 16$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/7/.style={
      column name={$k = 17$}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , display columns/8/.style={
      column name={Analytic}
      , fixed
      , fixed zerofill
      , dec sep align
      , precision=5
    }
    , every head row/.style={
      before row={\toprule}
      , after row={\midrule}
    }
    , every last row/.style={
      after row=\bottomrule
    }
    ]{\enfile{nc}}
  \end{center}
  \caption[N-C Energies]{
    The $n = 0, \dotsc, 3$ vibrational energies, calculated using the
    Numerov-Cooley method for $k = 10, \dotsc, 17$, are presented and
    compared with the analytic energies, to 6 significant figures.
    It can be seen that they are already convergent to the analytic energies by
    $k = 10$ - perhaps smaller grid sizes should be investigated further.
  }
  \label{tab:qho-en-nc}
\end{table}

\subsection*{Iterations}
\label{sec:qho-iterations}

The number of iterations required to calculate the $n = 0, \dotsc, 3$
vibrational energies and wavefunctions using the shooting-bisection method, for
$k = 10, \dotsc, 17$, is presented in \autoref{tab:qho-it-sb}.

\begin{table}[h]
  \begin{center}
    \pgfplotstabletypeset
    [ multicolumn names
    , col sep=space
    , display columns/0/.style={
      column name={$k = 10$}
      , fixed
    }
    , display columns/1/.style={
      column name={$k = 11$}
      , fixed
    }
    , display columns/2/.style={
      column name={$k = 12$}
      , fixed
    }
    , display columns/3/.style={
      column name={$k = 13$}
      , fixed
    }
    , display columns/4/.style={
      column name={$k = 14$}
      , fixed
    }
    , display columns/5/.style={
      column name={$k = 15$}
      , fixed
    }
    , display columns/6/.style={
      column name={$k = 16$}
      , fixed
    }
    , display columns/7/.style={
      column name={$k = 17$}
      , fixed
    }
    , every head row/.style={
      before row={\toprule}
      , after row={\midrule}
    }
    , every last row/.style={
      after row=\bottomrule
    }
    ]{\itfile{sb}}
  \end{center}
  \caption[S-B Iterations]{
    The number of iterations required to calculate the $n = 0, \dotsc, 3$
    vibrational energies and wavefunctions using the shooting-bisection method,
    for $k = 10, \dotsc, 17$, are presented.
  }
  \label{tab:qho-it-sb}
\end{table}

The number of iterations required to calculate the $n = 0, \dotsc, 3$
vibrational energies and wavefunctions using the Numerov-Cooley method, for
$k = 10, \dotsc, 17$, is presented in \autoref{tab:qho-it-nc}.

\begin{table}[h]
  \begin{center}
    \pgfplotstabletypeset
    [ multicolumn names
    , col sep=space
    , display columns/0/.style={
      column name={$k = 10$}
      , fixed
    }
    , display columns/1/.style={
      column name={$k = 11$}
      , fixed
    }
    , display columns/2/.style={
      column name={$k = 12$}
      , fixed
    }
    , display columns/3/.style={
      column name={$k = 13$}
      , fixed
    }
    , display columns/4/.style={
      column name={$k = 14$}
      , fixed
    }
    , display columns/5/.style={
      column name={$k = 15$}
      , fixed
    }
    , display columns/6/.style={
      column name={$k = 16$}
      , fixed
    }
    , display columns/7/.style={
      column name={$k = 17$}
      , fixed
    }
    , every head row/.style={
      before row={\toprule}
      , after row={\midrule}
    }
    , every last row/.style={
      after row=\bottomrule
    }
    ]{\itfile{nc}}
  \end{center}
  \caption[N-C Iterations]{
    The number of iterations required to calculate the $n = 0, \dotsc, 3$
    vibrational energies and wavefunctions using the Numerov-Cooley method,
    for $k = 10, \dotsc, 17$, are presented.
  }
  \label{tab:qho-it-nc}
\end{table}

\clearpage

\section{Dissociative Wave Functions for $\rm{H}_{2}^{+}$}
\label{sec:dissociative}

\begin{figure}[h]
  \begin{center}
    \input{figure_1ssg.tex}
  \end{center}
  \caption[Dissociative Wavefunctions for $1s\sigma_{g}$ PEC]{
    A set of dissociative wavefunctions, for the $1s\sigma_{g}$
    $\rm{H}_{2}^{+}$ potential-energy-curve, (shown in red-to-blue) are
    presented across a range of inter-nuclear distances from
    \SIrange{0}{10}{\bohr}.
  }
  \label{fig:1ssg}
\end{figure}

\begin{figure}[h]
  \begin{center}
    \input{figure_2psu.tex}
  \end{center}
  \caption[Dissociative Wavefunctions for $2p\sigma_{u}$ PEC]{
    A set of dissociative wavefunctions, for the $2p\sigma_{u}$
    $\rm{H}_{2}^{+}$ potential-energy-curve, (shown in red-to-blue) are
    presented across a range of inter-nuclear distances from
    \SIrange{0}{10}{\bohr}.
  }
  \label{fig:2psu}
\end{figure}

\begin{figure}[h]
  \begin{center}
    \input{figure_fc_1ssg.tex}
  \end{center}
  \caption[Kinetic Energy Release Distributions for $1s\sigma_{g}$]{
    The kinetic energy release distributions, for $1s\sigma_{g}$, calculated
    using the Franck-Condon approximation (shown in red-to-blue), are presented
    for the 0, 3, 6, 9 vibrational states of $1s\sigma_{g}$ $\rm{H}_{2}^{+}$,
    across a range of kinetic energy release from \SIrange{0}{30}{\eV}.
    Note that the scaling is very funky due to the scaling of the dissociative
    wavefunctions.
  }
  \label{fig:fc-1ssg}
\end{figure}

\end{document}


%!TEX root=report.tex
\subsection{OLS}

The first part of the OLS analysis will examine the EWH at two specific locations. The second part will then focus on the entire world. The main purpose of the OLS analysis is to estimate the velocity and acceleration of the mass changes.

\subsubsection{Selected locations}

The two selected locations are:
\begin{itemize}
\item Greenland ($63.5^\circ$ N, $49.5^\circ$ W)
\item Western Antartica ($74.5^\circ$ S, $87.5^\circ$ W)
\end{itemize}
\begin{figure}[H]
	\centering
	\includegraphics[height=6cm]{figures/ols-selected-map}
	\caption{The two selected locations, marked with a red star}
\end{figure}

\paragraph{Greenland}

At the west coast of Greenland a strong period trend can be observed, probably caused by the the ice melting over the summer and reappearing during the winter. This trend is seen in Figure \ref{fig:ols-selected-0-fit}. In order to estimate the velocity and acceleration of the EWH accurately, it is important that the periodic trend is caught by the $\sin(\cdot)$ and $\cos(\cdot)$ terms so that seasonal patterns do not interfere with the estimates.
\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{figures/ols-selected-0-fit}
	\caption{Measurements are blue, the OLS fit is red.}
	\label{fig:ols-selected-0-fit}
\end{figure}

From Figure \ref{fig:ols-selected-0-fit} it's seen that the period trend is caught by the model, however for some seasons (particularly between 2008 and 2010) the fit is not very good. This is even more clear when looking at the residuals in Figure \ref{fig:ols-selected-0-residual}. From this it is clear that the residuals are far from white noise, which was one of the OLS assumptions.
\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{figures/ols-selected-0-residual}
	\caption{The OLS residuals are blue.}
	\label{fig:ols-selected-0-residual}
\end{figure}

In addition to the yearly season, when looking at the estimated OLS coefficients (Table \ref{table:ols-selected-0-paramters}) there also seems to be a half yearly season. Though strangely enough it is not very visible on the OLS fit (Figure \ref{fig:ols-selected-0-fit}). The remaining seasons are with 95\% confidence not significantly different from zero and for the purpose of getting a better estimate, it might be worth iteratively removing these parameters.
\begin{table}[H]
\centering
\centerline{\input{figures/ols-selected-0-paramters}}
\caption{Parameter estimates $\hat{\beta}$ and their p-values for Greenland.}
\label{table:ols-selected-0-paramters}
\end{table}

\paragraph{Western Antarctica}

Just like with Greenland there is a clear drop in mass as seen in Figure \ref{fig:ols-selected-1-fit}. Interestingly enough, there is no seasonal trend or at least it is not very strong. Also the existence of high frequency terms in the OLS regression, is more apparent. Though from the actual data it does not look like such periodic trends exist. This suggests that too many seasonal parameters have been used, thus causing some overfitting.
\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{figures/ols-selected-1-fit}
	\caption{Measurements are blue, the OLS fit is red.}
	\label{fig:ols-selected-1-fit}
\end{figure}

When looking at the residuals in Figure \ref{fig:ols-selected-1-residual}, it is again clear that the residuals are far from being white noise. There also seems to be some periodic trend in the residuals; that is the residuals appear to have a seasonal pattern. However, the beginnings and the endings of these seasons are quite hard to make out.

\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{figures/ols-selected-1-residual}
	\caption{The OLS residuals are blue.}
	\label{fig:ols-selected-1-residual}
\end{figure}

Just as with Greenland many of the OLS terms are with 95\% confidence not significantly different from zero. Interestingly enough there is a yearly season when looking at Table \ref{table:ols-selected-1-paramters}, though it's very hard to make out from the OLS fit in Figure \ref{fig:ols-selected-1-fit}.

\begin{table}[H]
\centering
\centerline{\input{figures/ols-selected-1-paramters}}
\caption{Parameter esimates $\hat{\beta}$ and their p-values for Western Antarctica. }
\label{table:ols-selected-1-paramters}
\end{table}

\pagebreak
\subsubsection{World view}

\paragraph{Parameters}

As mentioned earlier, the velocity and acceleration of the mass changes are the most important parameters in the OLS regression.
\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{figures/ols-world-parameter-vel}
	\caption{Estimated velocity ($t$) parameters}
	\label{fig:ols-world-parameter-vel}
\end{figure}

\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{figures/ols-world-parameter-acc}
	\caption{Estimated acceleration ($\frac{1}{2} t^2$) parameters}
	\label{fig:ols-world-parameter-acc}
\end{figure}

The first two figures (Figure \ref{fig:ols-world-parameter-vel} and Figure \ref{fig:ols-world-parameter-acc}) show that, generally there is a mass loss at both Greenland and the Western Antarctica and that it is accelerating. However interestingly enough the East Coast of Greenland do not show any acceleration, maybe even a slight deceleration of the mass loss (a positive value).

On both figures there is also a circular pattern at Thailand (10 N, 95 E), which is caused by the infamous earthquake of the 26th December 2004, resulting in significant changes in the mass distribution, thus affecting the data from GRACE.
\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{figures/ols-world-parameter-year}
	\caption{Intensity-Hue plot. Estimated phase controls the hue while the amplitude sets the intensity.}
	\label{fig:ols-world-parameter-year}
\end{figure}

The seasonal trends do not reveal any information about mass loss due to ice melting, since this is something which will continue to have effect for many years. However it does affect the estimation of the velocity and acceleration. Also having more season related terms in the OLS regression than necessary, will result in fewer degrees of freedom.

In Figure \ref{fig:ols-world-parameter-year} it is seen from the intensity, that the Western Antarctica do not have a strong yearly seasonal trend unlike Greenland. It also appears that it is primarily the East Coast of Greenland which has a yearly seasonal trend. The strongest seasonal trend appears in South America (more specifically the Amazon Basin), and this is due to the rain season. This rain also takes quite some time to reach the ocean, thus causing the phase gradient.

\paragraph{Performance}

As a measure of  model performance the root mean squared error has been calculated for each position.
\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{figures/ols-world-performance-rmse}
	\caption{RMSE for each position.}
	\label{fig:ols-world-performance-rmse}
\end{figure}

From \ref{fig:ols-world-performance-rmse} it is seen that the hardest places to fit are South America and Thailand. In South America it is especially around the Amazon River that the RMSE is high. This suggests that it is caused by changes in the rain seasons, since this is also the place with the highest amplitude of the yearly seasonal term. It's a bit odd that this is so hard to fit, since the model does support seasonal trends. One explanation could be that this OLS model assumes that the season are equally long and with identical phases each year, which is not very likely. The circular pattern around Thailand is no surprise, since the OLS model was never meant to fit this distortion.

Neither of these issues are related to the ice melting, which makes them less important. However the East Coast of Greenland and the Western Antarctica also show a relative higher RMSE, when comparing to the ocean or the nearby land, which is less than ideal. The fact that it is primarily the East Coast of Greenland and not the west coast, suggests that the issues are related to the seasons, just like with the Amazon Basin. This was also something observed, when looking at the specific location's time series.

\paragraph{Standard diagnostics} Recalling that $\mathrm{Var}[\hat{Y}_i] = \sigma^{2} H_{ii}$ and $\mathrm{D}[\hat{\beta}] = \sigma^{2} V \Sigma^{-2} V^T$ the diagonal of $H$ and $V \Sigma^{-2} V^T$ are both important to examine. This will identify any days and parameters with high variance (the reader should remember that is applies to all positions).

\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{figures/ols-world-diagnostics-diagH}
	\caption{The diagonal of $H$ will reveal any days with a high variance.}
	\label{fig:ols-world-performance-diagH}
\end{figure}

The variances shown in Figure \ref{fig:ols-world-performance-diagH} are not very big, the highest is $0.16$ and recalling that the 95\% confidence interval can be approximated by $\hat{Y}_i \pm 2 \cdot \hat{\sigma} \sqrt{\mathrm{Var}[\hat{Y}_i]}$. The maximal confidence interval becomes $\hat{Y}_i \pm 2 \cdot 0.135 \cdot \sqrt{0.16} = \hat{Y}_i \pm 0.108$, which is small when comparing to the EWH values, which range from $1$ to $-3$. What is odd about $\mathrm{diag}(H)$ is that there appears to be some parts in each year which are more difficult to predict than others.

\begin{figure}[H]
	\centering
	\includegraphics[height=9cm]{figures/ols-world-diagnostics-cov}
	\caption{Colleration matrix reveals correlated parameter.}
	\label{fig:ols-world-performance-cov}
\end{figure}

From Figure \ref{fig:ols-world-performance-cov} it is seen that the parameters are nicely uncorrelated and while the seasonal parameters have a slightly high variance when comparing to their values; the variation of the velocity and the acceleration parameters are practically zero.
\begin{table}[H]
\centering
\begin{tabular}{r|c c c}
	name & $\mathrm{Var}[\hat{\beta}_i]$ \\ \hline
	$vel. (t)$ & $2.91390457 \cdot 10^{-09} [\frac{m}{day^2}]$  \\
	$acc. (0.5 \cdot t^2)$ & $1.29525397 \cdot 19^{-14} [\frac{m^2}{day^4}]$
\end{tabular}
\caption{Variance of the velocity and acceleration parameters for all positions .}
\end{table}
\paragraph{p-values} Just as for the selected positions, inspecting the p-values can tell which OLS terms are important.

\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{figures/ols-world-diagnostics-pvalue-1}
	\caption{p-values for the velocity parameter.}
	\label{fig:ols-world-diagnostics-pvalue-1}
\end{figure}

\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{figures/ols-world-diagnostics-pvalue-2}
	\caption{p-values for the acceleration parameter.}
	\label{fig:ols-world-diagnostics-pvalue-2}
\end{figure}

There is not much surprise in either Figure \ref{fig:ols-world-diagnostics-pvalue-1} or Figure \ref{fig:ols-world-diagnostics-pvalue-2}. Both the velocity and the acceleration appears to be significant pretty much everywhere. The places where it is not, is where there is a sign shift thus causing the p-value gradient. This is particularly apparent with the acceleration. One odd detail is that also on the ocean, the velocity and acceleration parameters appears to be significantly different from zero. In the ocean there is no reason to expect a steady mass change, though when looking at actual values it's seen that there is a slight slope. This quite small slope then becomes significant, due to the very small variance of the velocity and acceleration parameters.

\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{figures/ols-world-diagnostics-pvalue-3}
	\caption{p-values for the yearly seasonal cosine parameter.}
	\label{fig:ols-world-diagnostics-pvalue-3}
\end{figure}

\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{figures/ols-world-diagnostics-pvalue-4}
	\caption{p-values for the yearly seasonal sine parameter.}
	\label{fig:ols-world-diagnostics-pvalue-4}
\end{figure}

Also the yearly seasonal parameters are significant pretty much everywhere. For these parameters it makes more sense that they are significant in the ocean.

\paragraph{PCA of residuals}

By computing the principal component for the OLS residuals, it is some times possible to discover patterns in the residuals. As with all PCA one the first couple of principal components are valuable to look at.
\begin{figure}[H]
\centering
\includegraphics[width=1.0\textwidth]{figures/ols-pca-explained}
\caption{Variance explained by each principal component}
\label{fig:ols-pca-explained}
\end{figure}

If the PCA had been used on the original GRACE data, first couple of PCs would likely have explained most of the variance. However since this PCA is done on the residuals, the primary data is hopefully noise, thus no particularly direction shows a lot of variance. The shape of Figure \ref{fig:ols-pca-explained} indicates that there is a lot of noise, but that there also exists residuals there isn't just noise.

Plotting the first principal component score and loading, one gets:
\begin{figure}[H]
\centering
\includegraphics[width=1.0\textwidth]{figures/ols-pca-score-0}
\caption{PC score for the first principal component}
\label{fig:ols-pca-score-0}
\end{figure}

From Figure \ref{fig:ols-pca-score-0} its seen that there is some residuals in South America which are unlikely to be caused by noise. This could be because of the rainy seasons, since those are poorly modeled using this OLS regression. If that is the case one should expect Figure \ref{fig:ols-pca-loading-0} to show some yearly periodic trend. Unfortunately this does not seem to be the case.

\begin{figure}[H]
\centering
\includegraphics[width=1.0\textwidth]{figures/ols-pca-loading-0}
\caption{PC loadings for the first principal component}
\label{fig:ols-pca-loading-0}
\end{figure}

Plotting the second principal component score and loading, one gets:
\begin{figure}[H]
\centering
\includegraphics[width=1.0\textwidth]{figures/ols-pca-score-1}
\caption{PC scores for the second principal component}
\label{fig:ols-pca-score-1}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=1.0\textwidth]{figures/ols-pca-loading-1}
\caption{PC loading for the second principal component}
\label{fig:ols-pca-loading-1}
\end{figure}

In Figure \ref{fig:ols-pca-score-1} a strong circular pattern appears. This is the caused by the earthquake near Thailand in 2004 and can't be modeled by this OLS regression, so those residuals are expected. It is a bit strange that Fig. \ref{fig:ols-pca-loading-1} indicates that the residuals not only comes from 2002 to 2004 but also around 2006 and 2009.

More importantly Fig. \ref{fig:ols-pca-score-1} also show some residuals near west Greenland. This is not ideal since this is one of the areas this report attempts to analyze.


\documentclass[12pt]{article}
\usepackage{fullpage,enumitem,amsmath,amssymb,graphicx,amsthm,geometry,graphicx}

\geometry{left=2cm, right=2cm}

\begin{document}

\begin{center}
{\Large CS221 Fall 2017 Homework [Foundations]}

\begin{tabular}{rl}
Name: & yf-yang
\end{tabular}
\end{center}

By turning in this assignment, I agree by the Stanford honor code and declare
that all of this is my own work.

\section*{Problem 1}

\begin{enumerate}[label=(\alph*)]
  \item
    \small
      \begin{tabular}{||c|c|c|c|c|c||}
        \hline
          Input Vector & Weight & Out & Label & Loss & Grad w.r.t. Input \\
        \hline
        \hline
          $\begin{bmatrix}
            1 & 0 & 1 & 0 & 0 & 0
          \end{bmatrix}$
          &
          $\begin{bmatrix}
            0 & 0 & 0 & 0 & 0 & 0
          \end{bmatrix}$
          & 0 & -1 & 1 &
          $\begin{bmatrix}
            1 & 0 & 1 & 0 & 0 & 0
          \end{bmatrix}$ \\
        \hline
          $\begin{bmatrix}
            0 & 1 & 0 & 1 & 0 & 0
          \end{bmatrix}$
          &
          $\begin{bmatrix}
            -1 & 0 & -1 & 0 & 0 & 0
          \end{bmatrix}$
          & 0 & 1 & 1 &
          $\begin{bmatrix}
            0 & -1 & 0 & -1 & 0 & 0
          \end{bmatrix}$ \\
        \hline
          $\begin{bmatrix}
            0 & 1 & 0 & 0 & 1 & 0
          \end{bmatrix}$
          &
          $\begin{bmatrix}
            -1 & 1 & -1 & 1 & 0 & 0
          \end{bmatrix}$
          & 1 & -1 & 2 &
          $\begin{bmatrix}
            0 & 1 & 0 & 0 & 1 & 0
          \end{bmatrix}$ \\
        \hline
          $\begin{bmatrix}
            1 & 0 & 0 & 0 & 0 & 1
          \end{bmatrix}$
          &
          $\begin{bmatrix}
            -1 & 0 & -1 & 1 & -1 & 0
          \end{bmatrix}$
          & -1 & 1 & 2 &
          $\begin{bmatrix}
            -1 & 0 & 0 & 0 & 0 & -1
          \end{bmatrix}$ \\
        \hline
      \end{tabular}\\
    After the 4th backward propagation, we get a weight vector
    $ \mathbf{w} =
    \begin{bmatrix}
      0 & 0 & -1 & 1 & -1 & 1
    \end{bmatrix}$. \qed

  \item
    Here are the four mini-reviews:\\
    \begin{center}
    \begin{tabular}{||c|c|c||}
      \hline
        Review & Vector & Label \\
      \hline
      \hline
        good &
        $\begin{bmatrix}
            1 & 0 & 0
        \end{bmatrix}$
        & 1 \\
      \hline
        bad &
        $\begin{bmatrix}
            0 & 1 & 0
        \end{bmatrix}$
        & -1 \\
      \hline
        not good &
        $\begin{bmatrix}
            1 & 0 & 1
        \end{bmatrix}$
        & -1 \\
      \hline
        not bad &
        $\begin{bmatrix}
            0 & 1 & 1
        \end{bmatrix}$
        & 1 \\
      \hline
    \end{tabular}
    \end{center}
    Thus, we can generate a system of four inequalities:
    \begin{equation}
      \begin{cases}
        w_1 \ge 1 \\
        w_2 \le -1 \\
        w_1 + w_3 \le -1 \\
        w_2 + w_3 \ge 1
      \end{cases}
    \end{equation}
    From which we have\\
      \begin{equation}
        \begin{cases}
          1 \le w_1 \le -1 - w_3 \\
          1 - w_3 \le w_2 \le -1
        \end{cases}
      \end{equation}
    As a result
      \begin{equation}
        2 \le w_3 \le -2
      \end{equation}
    Which is impossible. The problem arises from interaction between not and other features. In order to fix this, we can add a feature "not good", then a weight vector like $
      \begin{bmatrix}
            1 & -1 & 2 & -4
      \end{bmatrix}$
    can achieve 0 error on the tiny dataset. \qed

\end{enumerate}

\section*{Problem 2}
\begin{enumerate}[label=(\alph*)]
  \item
    \begin{equation}
        L(x, y, \mathbf{w}) = (y - \frac{1}{1+\exp({-\mathbf{w}\cdot\phi(x)})})^2
    \end{equation} \qed

  \item
    \begin{align}
      \frac{\partial L}{\partial\mathbf{w}} &= \frac{\partial L}{\partial p}\frac{\partial p}{\partial \mathbf{w}} \\
      &= -2(y-p)\phi(x)p^2\exp({-\mathbf{w}\cdot\phi(x)}) \\
      &= -2p(1-p)(y-p)\phi(x)
    \end{align} \qed

  \item
    The gradient is nearly 0 when p approaches 0 or 1, which means
    \begin{equation}
      \exp{(-\mathbf{w}\cdot\phi(x))} = +\infty \: or \: 0
    \end{equation}
    i.e.
    \begin{equation}
      \mathbf{w}\cdot\phi(x) = -\infty \: or \: +\infty
    \end{equation}
    Intuitively, when weights of $+1$ in $x$ approach $+\infty/-\infty$ and weights of $-1$ in $x$ approach $-\infty/+\infty$, the gradient would be very small. \qed

  \item
    When $p=\frac{1}{3}$, the gradient has the largest magnitude of $\frac{8}{27}\phi(x)$ \qed

  \item
    If I understand the problem correctly, then we have some $\mathbf{w}$ which satisfies
    \begin{equation}
      y = \frac{1}{1+\exp{(-\mathbf{w}\cdot\phi(x))}}
    \end{equation}
    for each (x, y) pairs in D.\\
    Thus.
    \begin{equation}
      \exp{(-\mathbf{w}\cdot\phi(x))} = \frac{1-y}{y}
    \end{equation}
    \begin{equation}
      \mathbf{w}\cdot\phi(x) = -\log{\frac{1-y}{y}}=log{y}-\log{(1-y)}
    \end{equation}
    Then, we can let
    \begin{equation}
      y'=log{y}-\log{(1-y)}
    \end{equation}
    The new label could be easily fitted with least squares regression. \qed

\end{enumerate}

\section*{Problem 3}
\begin{enumerate}[label=(\alph*)]
  \setcounter{enumi}{3}
  \item
    \begin{enumerate}[label=\arabic*.]
    \item
      \begin{quote}
      screenwriter dan schneider and director shawn levy substitute volume and primary colors for humor and bite . \\
Truth: -1, Prediction: 1 [WRONG] \\
and                         3 * 0.17 = 0.51 \\
substitute                    1 * 0.08 = 0.08 \\
.                             1 * 0.08 = 0.08 \\
screenwriter                  1 * 0.03 = 0.03 \\
volume                        1 * 0 = 0 \\
shawn                         1 * 0 = 0 \\
primary                       1 * 0 = 0 \\
dan                           1 * 0 = 0 \\
humor                         1 * -0.02 = -0.02 \\
director                      1 * -0.02 = -0.02 \\
for                           1 * -0.04 = -0.04 \\
levy                          1 * -0.09 = -0.09 \\
bite                          1 * -0.09 = -0.09 \\
colors                        1 * -0.1 = -0.1 \\
schneider                     1 * -0.26 = -0.26
      \end{quote}
    Analysis: Some neutral words, especially "and", has a positive weight as high as .17, that may stem from our biased data set with more positive examples than negative examples. Besides, the occurrence frequency of "and" shouldn't be linear with out final output. It's problematic to multiply the weight of and directly by 3.
    \item
      \begin{quote}
        a heady , biting , be-bop ride through nighttime manhattan , a loquacious videologue of the modern male and the lengths to which he'll go to weave a protective cocoon around his own ego . \\
Truth: 1, Prediction: -1 [WRONG] \\
ride                          1 * 0.88 = 0.88 \\
modern                        1 * 0.23 = 0.23 \\
ego                           1 * 0.19 = 0.19 \\
and                           1 * 0.17 = 0.17 \\
.                             1 * 0.08 = 0.08 \\
he'll                         1 * 0.07 = 0.07 \\
,                             3 * 0.02 = 0.06 \\
of                            1 * 0.05 = 0.05 \\
a                             3 * 9.95731275211e-16 = 2.98719382563e-15 \\
cocoon                        1 * 0 = 0 \\
videologue                    1 * 0 = 0 \\
loquacious                    1 * 0 = 0 \\
lengths                       1 * 0 = 0 \\
protective                    1 * 0 = 0 \\
weave                         1 * 0 = 0 \\
nighttime                     1 * 0 = 0 \\
be-bop                        1 * 0 = 0 \\
heady                         1 * -0.01 = -0.01 \\
biting                        1 * -0.06 = -0.06 \\
own                           1 * -0.08 = -0.08 \\
manhattan                     1 * -0.08 = -0.08 \\
through                       1 * -0.1 = -0.1 \\
go                            1 * -0.15 = -0.15 \\
his                           1 * -0.17 = -0.17 \\
which                         1 * -0.21 = -0.21 \\
the                           2 * -0.11 = -0.22 \\
to                            2 * -0.11 = -0.22 \\
male                          1 * -0.31 = -0.31 \\
around                        1 * -0.38 = -0.38
      \end{quote}
    Analysis: Some bias in out data set makes the weight of ride exceptionally high.
    \item
      \begin{quote}
        'it's painful to watch witherspoon's talents wasting away inside unnecessary films like legally blonde and sweet home abomination , i mean , alabama . ' \\
Truth: -1, Prediction: 1 [WRONG] \\
sweet                         1 * 0.56 = 0.56 \\
painful                       1 * 0.3 = 0.3 \\
home                          1 * 0.22 = 0.22 \\
films                         1 * 0.21 = 0.21 \\
and                           1 * 0.17 = 0.17 \\
.                             1 * 0.08 = 0.08 \\
blonde                        1 * 0.07 = 0.07 \\
legally                       1 * 0.07 = 0.07 \\
inside                        1 * 0.05 = 0.05 \\
i                             1 * 0.05 = 0.05 \\
alabama                       1 * 0.04 = 0.04 \\
,                             2 * 0.02 = 0.04 \\
wasting                       1 * 0 = 0 \\
abomination                   1 * 0 = 0 \\
'it's                         1 * 0 = 0 \\
witherspoon's                 1 * 0 = 0 \\
away                          1 * -0.04 = -0.04 \\
mean                          1 * -0.06 = -0.06 \\
talents                       1 * -0.07 = -0.07 \\
to                            1 * -0.11 = -0.11 \\
like                          1 * -0.15 = -0.15 \\
unnecessary                   1 * -0.15 = -0.15 \\
watch                         1 * -0.15 = -0.15 \\
'                             1 * -0.17 = -0.17
      \end{quote}
    Analysis: The word "like" has interaction with examples behind, so words like "sweet" should not affect our result. Besides, I don't know why "painful" has a positive weight.
    \item
      \begin{quote}
        wickedly funny , visually engrossing , never boring , this movie challenges us to think about the ways we consume pop culture . \\
Truth: 1, Prediction: -1 [WRONG] \\
culture                       1 * 0.43 = 0.43 \\
funny                         1 * 0.42 = 0.42 \\
ways                          1 * 0.3 = 0.3 \\
us                            1 * 0.23 = 0.23 \\
wickedly                      1 * 0.22 = 0.22 \\
engrossing                    1 * 0.22 = 0.22 \\
visually                      1 * 0.1 = 0.1 \\
.                             1 * 0.08 = 0.08 \\
think                         1 * 0.07 = 0.07 \\
,                             3 * 0.02 = 0.06 \\
challenges                    1 * 0.05 = 0.05 \\
we                            1 * 0.05 = 0.05 \\
consume                       1 * 0 = 0 \\
this                          1 * -5.89805981832e-17 = -5.89805981832e-17 \\
movie                         1 * -0.09 = -0.09 \\
the                           1 * -0.11 = -0.11 \\
about                         1 * -0.11 = -0.11 \\
to                            1 * -0.11 = -0.11 \\
pop                           1 * -0.24 = -0.24 \\
never                         1 * -0.37 = -0.37 \\
boring                        1 * -1.26 = -1.26
      \end{quote}
    Analysis: The stupid linear predictor can't understand interaction between "never" and "boring".
    \item
      \begin{quote}
        atchy combination of soap opera , low-tech magic realism and , at times , ploddingly sociological commentary . \\
Truth: -1, Prediction: 1 [WRONG] \\
magic                         1 * 0.54 = 0.54 \\
realism                       1 * 0.42 = 0.42 \\
times                         1 * 0.24 = 0.24 \\
and                           1 * 0.17 = 0.17 \\
at                            1 * 0.08 = 0.08 \\
.                             1 * 0.08 = 0.08 \\
,                             3 * 0.02 = 0.06 \\
of                            1 * 0.05 = 0.05 \\
sociological                  1 * 0.02 = 0.02 \\
patchy                        1 * 0 = 0 \\
ploddingly                    1 * 0 = 0 \\
low-tech                      1 * 0 = 0 \\
combination                   1 * -0.11 = -0.11 \\
commentary                    1 * -0.2 = -0.2 \\
opera                         1 * -0.3 = -0.3 \\
      \end{quote}
    Analysis: Our predictor haven't encountered "low-tech" before and can't understand relations between "low-tech" and "magic realism".
    \end{enumerate} \qed
    \setcounter{enumi}{5}
    \item
      See prob3-6.py for the code, the result is demonstrated in Fig 1. \\
      \begin{figure}
          \centering
          \includegraphics{fig3_6.png}
          \caption{Character Features(blue) vs Word Features(red)}
      \end{figure}
      Analysis: When $n \approx 7$, our predictor achieves the best result. That's because ~7 characters could contain a word and some interaction between the word and previous/next one, while the noise introduced by bilateral words are not so high. \qed

\end{enumerate}

\section*{Problem 4}
\begin{enumerate}[label=(\alph*)]
  \item
    \begin{enumerate}[label=\arabic*.]
        \item
        \# assign point 2, 4 to center 1, assign point 1, 3 to center 2 \\
        \# get new centers $ \mu_1 = [.5, 2], \mu_2=[1, 0]$ \\
        \# assign point 2, 4 to center 1, assign point 1, 3 to center 2 \\
        \# convergence
        \item
        \# assign point 1, 2 to center 1, assign point 3, 4 to center 2 \\
        \# get new centers $ \mu_1 = [0, 1], \mu_2=[1.5, 1]$ \\
        \# assign point 1, 2 to center 1, assign point 3, 4 to center 2 \\
        \# convergence
    \end{enumerate}\qed
  \setcounter{enumi}{2}
  \item
    \begin{enumerate}[label=\arabic*.]
      \item Get M connected components based on S (note that a single vertex is also an connected component), assure $M \ge K$.
      \item If a connected component has more than 1 vertices, we use it's center of mass as it's coordinate. Then, randomly sample K connected components and use it's coordinate as initial centers.
      \item If a connected component has more than 1 vertices, we use the vertices' average distance from a point as the distance between the connected component and that point. Then, assign each connected components to it's nearest center.
      \item stop if the process converges, otherwise go back to step 2. \qed
    \end{enumerate}
\end{enumerate}

\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{graphicx}
\usepackage{fancyhdr}
\usepackage{listings}
\usepackage{color}
\usepackage{hyperref}
\usepackage{amsmath}
\usepackage{amssymb}

\graphicspath{ {images/} }

\pagestyle{fancy}

\title{%
    Uncertainty Modelling for Intelligent Systems \\
    \vspace{0.4cm}
    \large Lecture Notes
}

\author{jerakrs}
\date{January 2018}

\begin{document}

\maketitle

\begin{abstract}
    This document is the lecture note of \textit{Uncertainty Modelling for Intelligent Systems}. The course code is \textit{EMATM1120} and the unit director is \textit{Jonathan Lawry}.
\end{abstract}

\section{Quantitative Measures of Uncertainty}

\textbf{Certainty vs. Uncertainty}

\begin{itemize}
    \item Certainty: Given a number of inference rules we are then able to deduce other propositions as being certainty true.
    \item Uncertainty: Our knowledge consists of proposition in which we have some level of belief but about which we are not certain.
\end{itemize}

Aim: investigate frameworks according to which a rational intelligent agent could reason under uncertainty.

\subsection{Inductive Logic Rudolf Carnap}

There are $N$ unary predicates $Q_1, Q_2, \dots, Q_N$ which can be applied to $M$ objects $a_1, a_2, \dots, a_M$, any object must satisfy exactly one of the formula:

\begin{equation}
    \alpha(x) = \pm Q_1(x) \wedge \pm Q_2(x) \wedge \dots \wedge \pm Q_n(x)
\end{equation}

and $\pm Q_i(x) = Q_i(x)$ or $\neg Q_i(x)$, then for any event $p$, have:

\begin{equation}
    p = \alpha(a_1) \wedge \alpha(a_2) \wedge .... \wedge \alpha(a_M)
\end{equation}

\subsection{Uncertainty Measures}

\begin{itemize}
    \item $W$: the set of all possible worlds which is assumed be finite.
    \item $\mu: 2^W \rightarrow [0, 1]$ is a function mapping subsets of $W$ into the interval.
\end{itemize}

e.g. four student, each bit represent that it is present or not:
$$\mu(\{<1, 0, 0, 1>\}) = 1$$

\subsection{Random Variables}

$X: W \rightarrow \Omega$ is a function from $W$ into some measurement domain $\Omega$.

\begin{itemize}
    \item $\mu(X=x) = \mu(\{ w: X(w) = x\})$
    \item For $S \subseteq \Omega, \mu(X \in S) = \mu(\{ w: X(w) \in S\})$
\end{itemize}

e.g. $X$ is the number of male students present, then:
$$\mu(X=2) = \mu(\{ <1,0,0,1>, <1,0,1,1>, <1,1,0,1>, <1,1,1,1>\})$$

\subsection{Random Sets}

$R:W \rightarrow 2^{\Omega}$: measurements which are imprecise so that $R(w)=S$ when $S \subseteq \Omega$.

\begin{equation}
    \mu(R=S) = \mu(\{ w: R(w)=S\})
\end{equation}

e.g. Let $R$ denote the lecturer’s estimate of the number of people present:

$$R(<1,0,1,1>) = \{3, 4\}$$

\subsection{Attribute of Measures}

\begin{itemize}
    \item \textbf{Additive}
\begin{itemize}
    \item For uncertainty measures: If $A \cap B = \varnothing$ then $\mu( A \cup B) = \mu(A) + \mu(B)$
    \item General form: For any $A$ and $B$, $\mu( A \cup B) = \mu(A) + \mu(B) - \mu(A \cap B)$
\end{itemize}
    \item \textbf{Sub-Additive}
\begin{itemize}
    \item For uncertainty measures: If $A \cap B = \varnothing$ then $\mu( A \cup B) \leq \mu(A) + \mu(B)$
    \item General form: For any $A$ and $B$, $\mu( A \cup B) \leq \mu(A) + \mu(B) - \mu(A \cap B)$
\end{itemize}
    \item \textbf{Super-Additive}
\begin{itemize}
    \item For uncertainty measures: If $A \cap B = \varnothing$ then $\mu( A \cup B) \geq \mu(A) + \mu(B)$
    \item General form: For any $A$ and $B$, $\mu( A \cup B) \geq \mu(A) + \mu(B) - \mu(A \cap B)$
\end{itemize}
    \item \textbf{Maxitive}
\begin{itemize}
    \item For any $A$ and $B$, $\mu( A \cup B) = max(\mu(A), \mu(B))$: Special case of sub-additive measure.
\end{itemize}
    \item \textbf{Minimal}
\begin{itemize}
    \item For any $A$ and $B$, $\mu( A \cup B) = min(\mu(A), \mu(B))$: Special case of super-additive measure.
\end{itemize}
\end{itemize}

\section{Probability Theory}

\subsection{Axioms}

For probability measures we denote the uncertainty measure $\mu = P$

\begin{itemize}
    \item \textbf{P1}: $P(W) = 1, P(\varnothing) = 0$, $W$ is the all possible world.
    \item \textbf{P2}: $A \cap B = \varnothing$ then $P(A \cup B) = P(A) + P(B)$ to general additivity $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
\end{itemize}

\textbf{Complement}: $P(A^c) = 1 - P(A)$

\textit{Proof}:

$$\because  P1, A \cup A^c=W$$
$$\therefore 1 = P(W) = P(A \cup A^c)$$
$$\because P2, A \cap A^c = \varnothing$$
$$\therefore P(A \cup A^c) = P(A) \cup P(A^c)$$
$$Therefore: P(A) \cup P(A^c) = 1$$

\textbf{Probability Distributions}: $A = \{w_1, ...., w_k\}$ so that $A = \{w_1\} \cup ....\cup \{w_k\}$, then $P(A) = P(w_1) + P(w_2) + .... + P(w_k)$

\begin{itemize}
    \item For uncertainty measure: if $W$ has $n$ elements, agent needs to specify $2^n - 2$ values ($\mu(W) = 1, \mu(\varnothing) = 0$).
    \item For probability measure: if $W$ has $n$ elements, agent needs to specify $n-1$ values (Probability distribution must sum to $1$).
\end{itemize}

\subsection{Probabilities Distributions}

\begin{itemize}
    \item \textbf{Joint Probability}: $P(X_1, X_2, \dots X_n)$, if $X_i$ has $k_i$ values then specifying the joint distribution requires $(\prod_{i=1}^{n}k_i)-1$.
    \item \textbf{Marginal Probability}: $P(X_1) = \sum_{x_2 \in \Omega} \dots \sum_{x_n \in \Omega}P(X_2=x_2, \dots, X_n=x_n)$
    \item \textbf{Conditional Distribution}: $P(X_1=x_1 | X_2=x_2) = \frac{P(X_1=x_1, X_2=x_2)}{P(X_2=x_2)}$
\end{itemize}

\textbf{Independence}

\begin{itemize}
    \item Random variable $X_1$ is independent of $X_2$ if $P(X_1 =x_1 | X_2 = x_2)=P(X_1 = x_1)$
    \item Random variables $X_1, \dots, X_n$ are independent if $P(X_1=x_1, \dots, X_n=x_n) = \prod_{i=1}^{n}P(X_i=x_i)$
\end{itemize}

The number of values which must be specified in order to define the joint distribution is:

\begin{itemize}
    \item the independent case requires $\sum^n_{i=1}(k_i - 1)$ values.
    \item the fully dependent case requires $\prod^n_{i=1}(k_i)-1$ values.
\end{itemize}

\textbf{Conditional Independence}: Let $U, V, W$ be exclusive subsets of $\{X_1, ..., X_n\}$, then the variables in $U$ are said to be conditionally independent of variables in $V$ given the variables in $P(U|V, W) = P(U | W)$.

\subsection{Conditional Probability}

For $A, B \subseteq W$ the conditional probability of $A$ given $B$ is define by:

\begin{equation}
    P(A|B) = \frac{P(A \cap B)}{P(B)}
\end{equation}

Notice that if $P(B) = 0$ then $P(A|B)$ is undefined.

\textbf{Bayes Theorem}

\begin{equation}
    P(H|E) = \frac{P(H \cap E)}{P(E)} = \frac{P(E|H)P(H)}{P(E)}
\end{equation}

$P(E|H)$ is called the likelihood, $P(H)$ is called a prior probability, and

\begin{equation}
    P(E) = P(E|H)P(H) + P(E|H^c)P(H^c)
\end{equation}

\textbf{Laplace’s principle of insufficient reason}

In the absence of any other information all possible worlds should be assumed to be equally probable, i.e. the probability distribution should be uniform.

Sometimes assuming a uniform prior gives you different answers if you transform the problem:

\begin{itemize}
    \item Let $W = \{w1, w2, w3\}$ and $X$ be a random variable with values in $\{1, 2\}$
    \item By insufficient reason, then $p(w1) = p(w2) = p(w3) = 1$ and $P(X=1)=P(X=2)= 1$
    \item However, $P(X=1)=P(\{w:X(w)=1\})=0$ or $=\frac{1}{3}$ or $=\frac{2}{3}$ or $= 1$
\end{itemize}

\subsection{Betting Justification}

\textbf{Notation}: Let $w^*$ denote the true world, so $\chi_A(w^*) = 1$ when $A$ is true and $\chi_A(w^*) = 0$ when $A$ is false.

\begin{equation}
    \chi_A: W \rightarrow \{0, 1\} \left\{\begin{matrix} &1, w^* \in A  & \\  & 0, w^* \notin A  &\end{matrix}\right.
\end{equation}

\textbf{Betting Rules}:

\begin{itemize}
    \item $0 \leq p \leq 1$ and $S > 0$
    \item \textbf{Bet 1}: Gain $S(1-p)$ if $A$ is true and lose $Sp$ if $A$ is false.
    \item \textbf{Bet 2}: Lose $S(1-p)$ if $A$ is false and gain $Sp$ if $A$ is ture.
\end{itemize}

\textbf{Gain for bet 1 and bet 2}

\begin{itemize}
    \item gain for bet 1 $= S(1-p)\chi_A(w^*) - Sp(1-\chi_A(w^*)) = S(\chi_A(w^*)-p)$
    \item gain for bet 2 $= -S(1-p)\chi_A(w^*) + Sp(1-\chi_A(w^*)) = -S(\chi_A(w^*)-p)$
\end{itemize}

When the expectation of gain for bet 1 and bet 2 is equal, we have:

$$\mu(A)S(1-p) = -(1-\mu(A))S(0-p)$$
$$\mu(A)(1-p) + (\mu(A)-1)p = 0$$
$$\mu(A) = p$$

therefore when $p < \mu$, the agent pick bet 1 and when $p > \mu$, the agent pick bet 2.

\textbf{Dutch Book Problem}: A Dutch book is a sequence of bets the outcome of which is a sure loss no matter what the actual state of the world. The gain from two bets is $S(\chi_A(w^{\ast}) - p) + S(\chi_{A^c}(w^{\ast}) - p)$ where $w^{\ast}$ is the actual state of the world. Simplifying gives

$$S(\chi_A(w^{\ast}) - p) + S(\chi_{A^c}(w^{\ast}) - p)$$
$$= S(\chi_A(w^{\ast}) + \chi_{A^c}(w^{\ast}) - 2p) = S(1 - 2p) < 0 $$

since $S > 0$ and $p > \frac{1}{2}$.

\subsection{Bayesian Networks}

A Bayesian network is a directed graph $(V, E)$ where $V = \{ X_1, ..., X_n \}$ enumerated such that $(X_j, X_i) \in E$ only if $j < i$ together with a probability distribution on $V$ satisfying:

\begin{equation}
    P(X_i | X_1, ..., X_{i-1}) = P(X_i | \sqcap(X_i))
\end{equation}

where $\sqcap(X_i) = \{ X_j : (X_j, X_i) \in E \}$ are the parents of $X_i$.

\begin{itemize}
    \item Full dependent: for any $1 \leq i \leq N$ and $\sqcap(X_i) = \{X_j | j < i \}$.
    \item Full independent: for any $1 \leq i \leq N$, $\sqcap(X_i) = \varnothing$.
\end{itemize}

\begin{figure}[h]
    \centering
    \includegraphics[scale=0.3]{images/bayesian-network-example.png}
    \caption{The example of Bayesian Network}
    \label{fig:my_label}
\end{figure}

\textit{PS: The graph not exist cycles since it is illegal for one thing dependent its self.}

\textbf{The Chain Rule}:

\begin{multline}
    P(X_1, ..., X_n) \\
    = P(X_1) \frac{P(X_1, X_2)}{P(X_1)} \frac{P(X_1, X_2, X_3)}{P(X_1, X_2)} \dots \frac{P(X_1, ..., X_n)}{P(X_1, \dots, X_{n-1})} \\
    = P(X_1)P(X_2|X_1)\dots P(X_n|X_1, ..., X_{n-1}) = \prod^n_{n=1}P(X_i| \sqcap(X_i))
\end{multline}

\textbf{Possible Values}: Suppose that $X_1, X_2, \dots, X_n$ is a binary distribution, the total number of possible values is $\sum^n_{i=1}2^{|\sqcap(x_i)|}$, since $0 \leq |\sqcap(x_i)| \leq i-1$, we have

\begin{equation}
    (\sum^n_{i=1}2^0=n) \leq \sum^n_{i=1}2^{|\sqcap(x_i)|} \leq (\sum^n_{i=1}2^{i-1} = 2^n-1)
\end{equation}

which means the possible values satisfy $independent \leq Bayesian Network \leq fully dependent$.

\section{Information and Inference}

\subsection{Linear Knowledge}

A base knowledge can be represented by a set of linear equations on a probability measure $P$:

\begin{equation}
    K = \sum^{n_j}_{i=1}a_{i,j}P(A_{i,j}) = b_j: j = 1, \dots,m
\end{equation}

\textit{Example of Knowledge}: Supposing there are three sections, red(r), blue(b) and green(g). And our knowledge base is the probability of red is twice to probability of blue, $K = \{ P(r) = 2P(b)\}$.

\begin{itemize}
    \item Let denotes $P(r) = p_1, P(b) = p_2, P(g) = p_3$.
    \item From $K$ we have that $p_1 = 2p_2$ also we know hint $p_1 + p_2 + p_3 = 1 \rightarrow p_3 = 1 - 3p_2$
    \item The $V(K) \subseteq V$ corresponding to all probability distributions which satisfy $K$
    \item $V(K) = \{ <2p_2, p_2, 1-3p_2> : 0 \leq p_2 \leq \frac{1}{3} \}$
    \item Notice $p_2 \leq \frac{1}{3}$ since we require $3p_2 \leq 1$ and $2p_2 \leq 1$
\end{itemize}

\textit{Recall some basic maths}:

\begin{itemize}
    \item $\log_2{x} = \frac{\ln{x}}{\ln{2}}$, let $y = log_a{x} \Rightarrow a^y = x \Rightarrow ln{a^y} = ln{x} \Rightarrow y = \frac{\ln{x}}{\ln{a}}$
    \item $\frac{d\ln{x}}{dx} = \frac{1}{x}$
    \item $\frac{d(-x\log_2{x})}{dx} = \frac{ d(-x \frac{\ln{x}}{\ln{2}})}{dx} = -\frac{1}{\ln{2}} \frac{d(x\ln{x})}{dx} = -\frac{1}{\ln{2}}(\ln{x} + x\frac{1}{x}) = -\log_2{x} - \frac{1}{\ln{2}}$
\end{itemize}

\subsection{Information and Entropy}

\textbf{Shannon’s Entropy Measure}: Let $W = \{ w_1, ..., w_n\}$ and $P(w_i) = p_i$ then the information content of this distribution is

\begin{equation}
    H := -\sum^N_{i=1}{p_{i}log_2(p_i)}
\end{equation}

\begin{itemize}
    \item $H$ is minimal when $p_j = 1$ and for some $j \in \{ 1, ..., n\}$ and $p_i = 0$ for all $i \neq j$
    \item $H$ is maximal when $p_i = \frac{1}{n}$ for $i = \{1, \dots, n\}$
\end{itemize}

\textit{Proof Maximal Entropy}: Let $p_n = 1 - \sum^{n-1}_{i=1}p_i$ and $H = \sum^{n}_{i=1} - p_{i}log_2{p_i}$

$$\frac{dH}{dp_i} = \frac{d(-p_{i}log_{2}p_i)}{dp_i} + \frac{d(-p_{n}log_{2}p_n)}{dp_i} = -log_{2}p_i - \frac{1}{\ln2} + log_{2}(1 - \sum^{n-1}_{i=1}p_i) + \frac{1}{\ln2}$$
$$\frac{dH}{dp_i}  = -log_{2}p_i + log_{2}(1 - \sum^{n-1}_{i=1}p_i)$$
$$\frac{dH}{dp_i} = 0 \Rightarrow log_2{p_i} = log_2{1 - \sum^{n-1}_{i=1}p_i} \Rightarrow p_i = p_n$$
$$1 = \sum^{n}_{i=1}p_i \Rightarrow p_i = \frac{1}{n}$$

\textbf{Center of Mass}: Suppose we give every element of $V(K)$ equal probability, then we obtain the probability distribution

\begin{equation}
    \hat{p_i} = \frac{\int_{V(K)}p_{i}dV(K)}{\int_{V(K)}dV(K)}
\end{equation}

$<\hat{p_1}, \dots, \hat{p_n}>$ is the centre of mass of $V(K)$.

\textit{Example of CM}: For $V(K) = \{<p_1, 0.8-p_1, 0.7-p_1, p_1-0.5>: 0.5 \leq p_1 \leq 0.7\}$,

$$\hat{p_1} = \int_{V(K)}p_{1}dV(K) = \frac{\int_{0.5}^{0.7}p_{1}dp_1}{\int_{0.5}^{0.7}dp_1} = \frac{0.12}{0.2} = 0.6$$

\section{Dempster-Shafer Theory}

\subsection{Mass Assignments}

Shafer-Dempster belief functions are defined by a probability distirbution on the power set of $W$.

\begin{equation}
    m: 2^w \rightarrow [0, 1], m(\varnothing) = 0, \sum_{A \subseteq W}{ m(A) = 1}
\end{equation}

\textbf{Belief and Plausibility}

\begin{itemize}
    \item Belief $Bel: 2^W \rightarrow [0, 1]$ is a measure of the total amount of evidence in favor of a proposition
    \begin{equation}
        Bel(A) = \sum_{B \subseteq A}{m(B)}
    \end{equation}
    \item Plausibility $Pl: 2^W \rightarrow [0, 1]$ is a measure of the total amount of evidence not against a proposition.
    \begin{equation}
        Pl(A) = \sum_{B: B\cap A \neq \varnothing}m(B)
    \end{equation}
    \item $\forall A \subseteq W$, $Bel(A) \leq Pl(A)$ and $Pl(A) - Bel(A)$ provides a direct measure of ignorance about the proposition $A$
    \item $Bel(A) + Bel(A^c) \leq 1$
    \item $Pl(A) = 1 - Bel(A^c)$
    \item Belief is sub-additive: $Bel(A \cup B) \geq Bel(A) + Bel(B)$
    $$Bel(A \cup B) = \sum_{C \subseteq A \cup B}{m(C)} = \sum_{C \subseteq A }{m(C)} + \sum_{C \subseteq B}{m(C)} + \sum_{C \subseteq A \cup B, C \nsubseteq A, C \nsubseteq B}{m(C)}$$
    $$\geq \sum_{C \subset A}{m(C)} + \sum_{C \subset B}{m(C)} = Bel(A) + Bel(B)$$
    \item Plausibility is super-additive: $Pl(A \cup B) \leq Pl(A) + Pl(B)$
    $$Pl(A \cup B) = \sum_{C \cap (A \cup B) \neq \varnothing}m(C) = \sum_{C \cap A \neq \varnothing}m(C) + \sum_{C \cap B \neq \varnothing}m(C) $$
    $$- \sum_{C \cap A \neq \varnothing, C \cap B \neq \varnothing}m(C) \leq \sum_{C \cap A \neq \varnothing}m(C) + \sum_{C \cap B \neq \varnothing}m(C) = Pl(A) + Pl(B)$$
\end{itemize}

\textbf{Belief Inversion}: If we already have the Belief and we want to know the Mass, we can consider different levels of each elements.

\begin{equation}
    m(A) = \sum_{B \subseteq A}(-1)^{|A - B|}Bel(B)
\end{equation}

\begin{equation}
    m(A) = \sum_{B \subseteq A}(-1)^{|A - B|}(1 - Pl(B^c))
\end{equation}

\textbf{Lower and Upper Probabilities}: A lower probability $\underline{P}: 2^W \rightarrow [0, 1]$ and an upper probability $\overline{P}: 2^W \rightarrow [0, 1]$
\begin{itemize}
    \item $\forall A \subseteq W\ \ \underline{P}(A) \leq \overline{P}(A)$
    \item $\forall A \subseteq W\ \ \overline{P}(A^c) = 1 - \underline{P}(A)$
\end{itemize}

$\mathbb{P}$ is a set of probability measures on $2^W$, then $\underline{P}(A) = inf \{ P(A): P \in \mathbb{P} \}$ and $\overline{P}(A) = sup\{ P(A): P \in \mathbb{P} \}$.

\subsection{Updating Probabilities}

Suppose we have a prior probability measure $P$ and evidence about the true state of the world is given by mass assignment $m$. To get a single probability measure we could then take the expected value:

\begin{equation}
    P( \bullet | m) = \sum_{B \subseteq W}P(\bullet | B)m(B)
\end{equation}

\textit{Example for Updating Probabilities}:

$$P(w_1)=0.5,\ P(w_2)=0.3,\ P(w_3)=0.2$$
$$P(w_1 | \{w_1, w_2\}) = \frac{P(w_1)}{P(w_1) + P(w_2)} = \frac{5}{8}$$
$$P(w_2 | \{w_1, w_2\}) = \frac{P(w_2)}{P(w_1) + P(w_2)} = \frac{3}{8}$$
$$P(w_3 | \{w_1, w_2\}) = \frac{P(w_3)}{P(w_1) + P(w_2)} = 0 \ since\ \{w_3\} \cap \{w_1, w_2\} = \varnothing$$

\textbf{Pignistic Distribution}: Take  to be the uniform measure so that

\begin{equation}
    P_u(A | m) = \sum_{B \subseteq W}\frac{|A \cap B|}{|B|}m(B)
\end{equation}

where $|A|$ means the number of elements in set $A$.

\subsection{Nested Evidence}

\textbf{Nested hierarchy}: There exists $F_1 \subseteq F_2 \subseteq \dots \subseteq F_n \subseteq W$ such that $\sum^n_{i=1}m(F_i)=1$, $F_1 \subseteq F_2 \subseteq \dots \subseteq F_n \subseteq W$ are called the \textbf{focal sets} of $m$.

\textit{Example for Nested hierarchy}: There are $W=\{d_1, d_2, h\}$, suppose all doctors agree that the patient may have disease $d_2$, a subset of doctors think that he may have disease $d_1$, some doctors in this subset also believe it is possible that he is healthy.

\textbf{Possibility and Necessity Measures}: For a nested mass assignment the corresponding belief measure is called a \textbf{necessity measure} and the plausibility measure is called a \textbf{possibility measure}.

\begin{itemize}
    \item Let $F_j$ be the largest focal set for which $F_j \subseteq A$ then:
    \begin{equation}
        Bel(A) = Nec(A) = \sum_{B \subseteq A}m(B) = \sum^{j}_{i=1}m(F_i)
    \end{equation}
    \item Let $F_k$ be the smallest focal set for which $A \cap F_k \neq \varnothing$ then:
    \begin{equation}
        Pl(A) = Pos(A) = \sum_{B:B \cap A \neq \varnothing}m(B) = \sum^{n}_{i=k}m(F_k)
    \end{equation}
\end{itemize}

\textbf{Possibility and Necessity Axioms}

\begin{itemize}
    \item \textbf{Pos1}: $Pos(W) = 1, Pos(\varnothing)=0$
    \item \textbf{Pos2}: $Pos(A \cup B) = max(Pos(A), Pos(B))$
    \item \textbf{Nec1}: $Nec(W) = 1, Nec(\varnothing)=0$
    \item \textbf{Nec2}: $Nec(A \cap B) = min(Nec(A), Nec(B))$
    \item \textbf{Consequences1}: $\forall A \subseteq W$, $Pos(A) = 1$ or $ Pos(A^c) = 1$. Notice that $$1 = Pos(W) = Pos(A \cup A^c) = max(Pos(A), Pos(A^c))$$
    \item \textbf{Consequences2}: $\forall A \subseteq W$, $Nec(A) = 0$ or $Nec(A^c) = 0$. Notice that $$0 = Nec(\varnothing) = Nec(A \cap A^c) = min(Nec(A), Nec(A^c))$$
    \item Either $[Nec(A), Pos(A)] = [x, 1]$ or $[0, y]$ where $0 < x \leq 1$ and $0 \leq y < 1$.
\end{itemize}

\textit{Proof if $Nec(A) > 0$ then $Pos(A) = 1$}:

\begin{itemize}
    \item $Pos(A^c) = 1 - Nec(A)$ therefore $Nec(A) > 0 \Rightarrow Pos(A^c) < 1$.
    \item Now $1 = Pos(W) = Pos(A \cup A^c) = max(Pos(A), Pos(A^c))$.
    \item Hence, since $Pos(A^c) < 1$ so that $Pos(A) = 1$.
\end{itemize}

\textbf{Possibility Distribution}: For any $A \subseteq W$, $Pos(A) = max\{ Pos(\{w_i\}): w_i \in A \}$. Let $Pos(\{w_i\}) = \pi(w_i)$ and $max \{ \pi(w_i): w_i \in W \} = 1$.

\textbf{Possibility Inversion}: Suppose $W = \{ w_1, \dots, w_n\}$ ordered, then

$$1 = \pi(w_1) \geq \pi(w_2) \geq \dots \geq \pi(w_n)$$
$$m(\{w_1, \dots, w_n \}) = \pi(w_n)$$
$$m(\{w_1, \dots, w_{n-1} \}) = \pi(w_{n-1}) - \pi(w_n)$$
$$m(\{w_1, \dots, w_i \}) = \pi(w_i) - \pi(w_{i+1}) $$
$$m(\{w_1\}) = 1 - \pi(w_2)$$

\subsection{Demspter’s Rule of Combination}

Suppose we have two mass assignments provided by two independent sources of evidence.

\begin{equation}
    \forall A \subseteq W\ \ m_1 \bigoplus m_2 (A) = \frac{\sum_{(B, C: B \cap C = A)} m_1(B)m_2(C)}{1 - \sum_{(B, C): B \cap C = \varnothing}m_1(B)m_2(C)}
\end{equation}

\textbf{Conditional Belief Functions}: Suppose that $Pl$ is a plausibility:

\begin{equation}
    Pl(A | B) = \sum_{C \subseteq W; C \cap A \neq \varnothing} m \bigoplus m_B(C) = \frac{Pl(A \cap B)}{Pl(B)}
\end{equation}

Notice that $m \bigoplus m_B(C) = \frac{\sum_{C: C \cap A \neq \varnothing}m(D)}{1-\sum_{D:D\cap B = \varnothing}m(D)}$

$$Pl(A | B) = \frac{\sum_{D:(D \cap B) \cap A \neq \varnothing} m(D)}{1 - \sum_{D: D \cap B = \varnothing}m(D)}=\frac{\sum_{D:(D\cap B)\cap A \neq \varnothing}m(D)}{\sum_{D:D\cap B \neq \varnothing}m(D)} = \frac{Pl(A\cap B)}{Pl(B)}$$

Also

\begin{multline}
    Bel(A | B) = 1 - Pl(A^c | B) = 1 - \frac{Pl(A^c \cap B)}{Pl(B)} = \frac{Pl(B) - Pl(A^c \cap B)}{Pl(B)} = \\ \frac{(1 - Bel(B^c)) - (1 - Bel(A \cup B^c))}{1 - Bel(B^c)} = \frac{Bel(A \cup B^c) - Bel(B^c)}{1 - Bel(B^c)}
\end{multline}

\section{Fuzzy Set Theory}

Fuzzy sets are sets to which elements can belong to some partial degree. A (crisp) set $A \subseteq W$ can be characterised by a membership function $\chi_A: W \rightarrow \{0, 1\}$ so that

\begin{equation}
    \chi_A = \left\{\begin{matrix} 1: & w \in A \\  0: & w \notin  A \end{matrix}\right.
\end{equation}

The support of a fuzzy set $\tilde{A}$ is given by $Sup(\tilde{A}) = \{ w \in W: \chi_{\tilde{A}}(w)>0 \}$ and the fuzzy set is

\begin{equation}
    \tilde{A} = \sum_{w \in Supp(\tilde{A})} w / \chi_{\tilde{A}} (w)
\end{equation}

\textbf{Sorites Paradox}: Consider a man with a lot of hair on his head. One hair is plucked at a time until he is bald. It follows that there must be a hair pluck at which the man switches from being not bald to being bald. However, this is counter intuitive since the difference of one hair cannot induce a category change from not bald to bald. This or an alternative version of sorites.

\subsection{T-norm}

\textbf{T-norm} are function $T: [0, 1]^2 \rightarrow [0, 1]$ which satisfy the following properties:

\begin{itemize}
    \item \textbf{T1}: $\forall x \in [0, 1]\ \ T(x, 1) = x$
    \item \textbf{T2}: $\forall x, y, z \in [0, 1]$, if $y \leq z$ then $T(x, y) \leq T(x, z)$
    \item \textbf{T3}: $\forall x, y \in [0, 1], T(x, y) = T(y, x)$
    \item \textbf{T4}: $\forall x, y, z \in [0, 1], T(x, T(y, z)) = T(T(x, y), z)$
    $$\varphi_{(A \cap B) \cap C} = \varphi_{A \cap (B \cap C)} \Rightarrow T(T(A, B), C) = T(A, T(B, C))$$
    \item \textbf{T5}: $\forall x \in [0, 1]$ $T(x, x) = x$
\end{itemize}

\textbf{Theorem}: For any t-norm $T$, $drastic \leq T \leq min$, where

\begin{equation}
    drastic(x, y) = \left\{\begin{matrix} x & : & y = 1\\ y & : & x = 1\\ 0 & : & otherwise \end{matrix}\right.
\end{equation}

\begin{itemize}
    \item Show that $T \leq min(x, y)$
    $$T(x, y) \leq T(x, 1) = x$$
    $$T(x, y) = T(y, x) \leq T(y, 1) = y $$
    $$T(x, y) \leq min(x, y)$$
    \item Show that $T \geq drastic(x, y)$
    $$T(x, 1) = x$$
    $$T(1, y) = T(y, 1) = y$$
    $$T(x, y) \geq 0$$
    $$T(x, y) \geq drastic$$
\end{itemize}

\textbf{Families of T-norms}

\begin{itemize}
    \item \textbf{Frank’s t-norms}: $T_s(x, y) = log_s[1 + \frac{(s^x-1)(s^y-1)}{s-1}]$ where $s > 0$ and $s \neq 1$
    \item \textbf{Dombi Family}: $T_{\lambda}(x, y) = \{ 1 + [(\frac{1}{x} - 1)^{\lambda} + (\frac{1}{y} - 1)^{\lambda}]^{\frac{1}{\lambda}} \}^{-1}$, where $\lambda > 0$
\end{itemize}

\subsection{T-conorms}

\textbf{T-conorms} are function $S: [0, 1]^2 \rightarrow [0, 1]$, it equal to $1 - T(1-x, 1-y)$ and satisfy the following properties:

\begin{itemize}
    \item \textbf{S1}: $\forall x \in [0, 1]$ $S(x, 0) = x$
    \item \textbf{S2}: $\forall x, y, z \in [0, 1]$ if $y \leq z$ then $S(x, y) \leq S(x, z)$
    \item \textbf{S3}: $\forall x, y \in [0, 1]$ $S(x, y) = S(y, x)$
    \item \textbf{S4}: $\forall x, y, z \in [0, 1]$ $S(x, S(y, z)) = S(S(x, y), z)$
    \item \textbf{S5}: $\forall x \in [0, 1]$ $S(x, x) = x$
\end{itemize}

\subsection{Fuzzy Set}

\textbf{$\alpha$-cuts} provide a way of representing fuzzy sets as a nested sequence of crisp sets, Fuzzy set $\tilde{A}_{\alpha} = \{ w \in W: \chi_{\tilde{A}}(w) \geq \alpha \}$ so $\chi_{\tilde{A}}(w) = \int_{\alpha:w\in\tilde{A}_{\alpha}}d\alpha$.

Notice that $w \in \tilde{A}_{\alpha}$ if and only if $\varphi_{\tilde{A}}(w) \geq \alpha$, therefore $\{ \alpha: w \in \tilde{A}_{\alpha} \} = (0, \chi_{\tilde{A}}(w)]$.

\textit{Example for Fuzzy Set}: There are $\tilde{A} = 5/1 + 6/0.8 + 7/05$ and $\tilde{B} = 1/1+ 2/0.5 + 3/0.2$, use the $\alpha$-cut method to determine $\frac{\tilde{A}}{\tilde{B}}$.

\begin{table}[h]
    \centering
    \begin{tabular}{|c||c|c|c|}
        \hline
        $\frac{\tilde{A}}{\tilde{B}}$ & $\{5, 6, 7\}:(0, 0.5]$ & $\{5, 6\}:(0.5, 0.8]$ & $\{5\}:(0.8, 1]$\\
        \hline
        \hline
        $\{1, 2, 3\}:(0, 0.2]$ & $\{5,6,7,\frac{5}{2},3,\frac{7}{2},\frac{5}{3},2,\frac{7}{3}\}:(0,0.2]$ & $\varnothing$ & $\varnothing$ \\
        \hline
        $\{1, 2\}:(0.2, 0.5]$ & $\{5,6,7,\frac{5}{2},3,\frac{7}{2}:(0.2,0.5]$ & $\varnothing$ & $\varnothing$ \\
        \hline
        $\{1\}:(0.5, 1]$ & $\varnothing$ & $\{5,6\}:(0.5,0.8]$ & $\{5\}:(0.8,1]$ \\
        \hline
    \end{tabular}
    \label{tab:my_label}
\end{table}

$$\tilde{A}_{\alpha} = \left\{\begin{matrix} \{5, 6, 7\} & : & \alpha \in (0, 0.5] \\  \{5, 6 \}  & : & \alpha \in (0.5, 0.8] \\ \{5 \} & : & \alpha \in (0.8, 1] \end{matrix}\right.\ and\ \tilde{B}_{\alpha} = \left\{\begin{matrix} \{1, 2, 3\} & : & \alpha \in (0, 0.2] \\  \{1, 2 \}  & : & \alpha \in (0.2, 0.5] \\  \{1 \} & : & \alpha \in (0.5, 1] \end{matrix}\right. $$

$$(\frac{\tilde{A}}{\tilde{B}})_{\alpha} = \left\{\begin{matrix} \{5,6,7,\frac{5}{2},3,\frac{7}{2},\frac{5}{3},2,\frac{7}{3}\} & : & \alpha \in (0,0.2] \\  \{5,6,7,\frac{5}{2},3,\frac{7}{2} & : & \alpha \in (0.2,0.5] \\ \{5,6\} & : & \alpha \in (0.5,0.8] \\ \{5\} & : & \alpha \in (0.8,1] \end{matrix}\right.$$

$$\frac{\tilde{A}}{\tilde{B}} = 5/1 + 6/0.8 + 7/0.5 + \frac{5}{2}/0.5 + 3/0.5 + \frac{7}{2}/0.5 + 5/0.2 + 6/0.2 + 7/0.2$$

\section{Model Logic}

The idea is to directly incorporate the notions of possibly the case and necessarily the case directly in to a formal logic.

\textbf{Proposition}: $w \models A$ denote that proposition $A$ is true in world $w$.

\begin{equation}
    W(A) = \{ w \in W: w \models A \}
\end{equation}

\textbf{Well-Formed Formula}

\begin{itemize}
    \item standard connectives: $\neg$ (not), $\wedge$ (and), $\vee$ (or)
    \item modal operators: $\square$ (necessarily), $\diamond$ (possibly)
    \begin{itemize}
        \item $\square \theta$ means $\theta$ is necessarily the case.
        \item $\diamond \theta$ means $\theta$ is possibly the case.
    \end{itemize}
\end{itemize}

\textbf{Semantics of  Modal Logic}

\begin{itemize}
    \item $w \models \square \theta$: for all $w'$ such that $Edge(w, w'), w' \models \theta$
    \item $w \models \diamond \theta$: if $\theta$ is true in every world accessible from $w$, there exists $w'$ such that $Edge(w, w'), w' \models \theta$
    \item $W(\square B) \subseteq W(B) \subseteq W( \diamond B)$
    \item $\theta \models \varphi$ ($\theta$ entails $\varphi$ or $\varphi$ follows $\theta$) if and only if $w \models \theta$ implies that $w \models \varphi$
    \item $\neg \square \neg \theta \equiv \diamond \theta$ and $\neg \diamond \neg \theta \equiv \square \theta$
\end{itemize}

\textbf{Properties of Relations}

\begin{itemize}
    \item Reflexive: $E(w, w)$ for all $w \in W$
    \item Symmetric: If $E(w, w')$ then $E(w', w)$
    \item Transitive: If $E(w_1, w_2)$ and $E(w_2, w_3)$ then $E(w_1, w_3)$
    \item Dense: If $E(w_1, w_3)$ then there exists $w_2$ such that $E(w_1, w_2)$ and $E(w_2, w_3)$
    \item Serial: For all $w$, there exists $w'$ such that $E(w, w')$
    \item Identity Relation: $E(w, w')$ if and only if $w = w'$
\end{itemize}

\textbf{Probability, Model Logic and DS-Theory}: $P$ is a probability measure on $2^W$; $R(w)$ is the set of worlds which are reachable from $w$ denoted as $R(w) = \{ w' : E(w, w') \}$. Notice that $w \models \theta$ if and only if $R(w) \subseteq W(\theta)$, $w \models \diamond \theta$ if and only if $R(w) \cap W(\theta) \neq \varnothing$.

Mass function: $m(F) = P(\{ w: R(w) = F \}) = \sum_{w:R(w)=F}P(w)$

If $E$ is serial then for all $w$, $R(w) \neq \varnothing$ and hence $m(\varnothing) = 0$, $P(\square \theta) = Bel(\theta)$ and $P(\diamond\theta) = Pl(\theta)$.

$$P(\square \theta) = P(W(\square \theta)) = \sum_{w: R(w)\subseteq W(\theta)}P(w)$$
$$= \sum_{F \subseteq W(\theta)}\sum_{w:R(w)=F}P(w) = \sum_{F\subseteq W(\theta)}m(F) = Bel(W(\theta)) = Bel(\theta)$$

and also $P(\diamond \theta) = P(\neg \square \neg \theta)  = 1- P(\square \neg \theta) = 1 - Bel(\neg \theta) = Pl(\theta)$.

\textit{Web page Example}: Consider the probability distribution $\{ w_1, w_2, w_3, w_4, w_5 \}$

\begin{figure}[h]
    \centering
    \includegraphics[scale=1]{images/example-web-page.png}
    \caption{Web Page}
    \label{fig:my_label}
\end{figure}

$$P(w_1) = 0.1, P(w_2) = 0.2, P(w_3) = 0.2, P(w_4) = 0.4, P(w_5) = 0.1$$
$$R(w_1) = \{w_1, w_2, w_3, w_5\}, R(w_2) = \{ w_1, w_2 \}, R(w_3) = \{ w_3 \}$$
$$R(w_4) = \{w_1, w_4, w_5 \}, R(w_5) = \{ w_2, w_5 \}$$
$$W(B) = \{w_1, w_2, w_3, w_5 \}$$
$$Bel(B) = m(\{w_1, w_2, w_3, w_5 \}) + m(\{ w_1, w_2 \}) + m(\{ w_3 \}) + m(\{ w_2, w_5 \}) = 0.6$$
\end{document}

src/Uncertainty Modelling for Intelligent Systems/src/main.tex


    \filetitle{hh}{IRIS serial date number for half-yearly date}{dates/hh}

	\paragraph{Syntax}\label{syntax}

\begin{verbatim}
Dat = hh(Y)
Dat = hh(Y,H)
\end{verbatim}

\paragraph{Input arguments}\label{input-arguments}

\begin{itemize}
\item
  \texttt{Y} {[} numeric {]} - Year.
\item
  \texttt{H} {[} numeric {]} - Half-year; if missing, first half-year
  (January to June) is assumed.
\end{itemize}

\paragraph{Output arguments}\label{output-arguments}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{Dat} {[} numeric {]} - IRIS serial date numbers representing
  the half-yearly date.
\end{itemize}

\paragraph{Description}\label{description}

\paragraph{Example}\label{example}


\subsection{Continuum equations}

\begin{frame}{Large deformations framework -- Balance equations}
  \begin{footnotesize}
    \begin{block}{Kinematics}
      \begin{columns}
        \begin{column}{0.45\textwidth}
          \input{section2/pgfFigures/deformationFunction}
        \end{column}
        \begin{column}{0.5\textwidth}
          \begin{itemize}
          \item Reference mass density $\rho(\vect{X},t=0)=\rho_0(\vect{X})$
          \item Deformation gradient $\tens{F}(\vect{X},t)=\nablav_{\!0}\vect{\varphi}(\vect{X},t)$
          \item Hyperelastic material $\dot{\tens{\Pi}}=\Hbb(\tens{F}) : \dot{\tens{F}}$
          \item Cartesian coordinates system
          \end{itemize}
        \end{column}
      \end{columns}
    \end{block}
    \begin{block}{Balance equations \cite{Plohr}}
      \begin{equation*}
        \left.\begin{aligned}
            & \rho_0 \dot{\vect{v}} - \nablav_{\!0} \cdot \tens{\Pi} = \vect{0} \\
            & \dot{\tens{F}} - \nablav_{\!0} \cdot (\vect{v} \otimes \tens{I})= \tens{0}
          \end{aligned}\right\rbrace \Rightarrow \text{Conservative form: } \drond{\Ucb}{t} + \drond{\Fcb\cdot\vect{E}_\alpha}{X_\alpha} = \vect{0}
      \end{equation*}
      with $\Ucb = \matrice{\rho_0\vect{v} \\ \tens{F}} \: ; \: \Fcb\cdot\vect{E}_\alpha=\Fcb_\alpha=-\matrice{\tens{\Pi}\cdot\vect{E}_\alpha\\ \vect{v} \otimes\vect{E}_\alpha}$
    \end{block}
  \end{footnotesize}
  \footnoteCite{Plohr}
\end{frame}

% \begin{frame}{Large deformations framework -- Constitutive equations}
%   \begin{footnotesize}
%     \begin{block}{Hyperelastic materials}
%       %% Polyconvex, hyperbolicity ??
%       From the thermodynamics: $\tens{\Pi}=\rho_0\drond{\psi}{\tens{F}}\: ; \: \dot{\tens{\Pi}}=\Hbb : \dot{\tens{F}}$ \\
%       $\rho_0\psi$: Stored energy function, polyconvex $\Rightarrow$ hyperbolicity ensured \cite{Kluth}
%     \end{block}
%     \begin{block}{Quasi-linear form}
%       Auxiliary vector $\Qcb$ \cite{Trangenstein91}: %$\Qcb=\matrice{\vect{v}\\\tens{\Pi}}$
%       \begin{gather*}
%         % \dot{\tens{\Pi}} = \Hbb(\tens{F}) : \dot{\tens{F}} \Rightarrow \text{Quasi-linear form: } \drond{\Qcb}{t} + \Absf^\alpha \drond{\Qcb}{X_\alpha} = \vect{0}
%         \drond{\Ucb}{t} + \drond{\Fcb\cdot\vect{E}_\alpha}{X_\alpha} = \vect{0} \quad \Leftrightarrow \quad \drond{\Ucb}{\alert{\Qcb}}\drond{\alert{\Qcb}}{t} + \drond{\Fcb\cdot\vect{E}_\alpha}{\alert{\Qcb}}\drond{\alert{\Qcb}}{X_\alpha}=\vect{0} \\
%         \drond{\Qcb}{t} + \Absf^\alpha \drond{\Qcb}{X_\alpha} = \vect{0}
%       \end{gather*}
%       where $\Qcb=\matrice{\vect{v}\\\tens{\Pi}} \: ; \: \Absf^\alpha = -\matrice{\tens{0}^2 & \frac{1}{\rho_0}\tens{I}\otimes\vect{E}_\alpha \\ \Hbb\cdot\vect{E}_\alpha & \tens{0}^4}$
%     \end{block}
%   \end{footnotesize}
%   \footnoteCite{Kluth,Trangenstein91}
% \end{frame}

\subsection{Discrete system}

\begin{frame}{Weak formulation}
  \begin{columns}
    \begin{column}{0.425\textwidth}
      \begin{block}{MPM space discretization}
        $N_p$ particles in $E$ elements
        \input{section2/pgfFigures/continuum}
      \end{block}
    \end{column}
    \begin{column}{0.55\textwidth}
      \begin{block}{DG approximation \cite{DiPietro}}
        \vspace{1cm}
        %Broken polynomial space $\Vscr^k_h = \{\Vcb \in \Pscr^k(\Omega^e)\}$\\
        %Approximate solution $\Ucb(\vect{X},t) \in \Vscr^k_h$ \\
        $\Ucb(\vect{X},t) = \sum_{i=1}^{N_{nodes}} S_i(\vect{X}) \Ucb^i(t)$
        \vspace{1cm}
      \end{block}
    \end{column}
  \end{columns}
  \begin{block}{Element-wise weak form}
    \begin{equation*}
      \begin{aligned}
        & \text{Find $\Ucb$ such that $\forall \:e,t \in  [1, E]\times \tau$:}\\
        & \int_{\Omega^e} \[\drond{\Ucb}{t} \Vcb - \Fcb_\alpha \drond{\Vcb}{X_\alpha}\]d\Omega + \int_{\partial \Omega^e} (\Fcb\cdot \vect{N}) d\Gamma = 0 \quad \forall \Vcb%\int_{\Omega^e} \Scb\Vcb d\Omega
      \end{aligned}
    \end{equation*}
  \end{block}
  \footnoteCite{DiPietro}
\end{frame}

\begin{frame}
  \begin{block}{Specific quantities}
    Approximation of the reference mass density \cite{Sulsky94}: $\rho_0(\vect{X})=\sum_{p=1}^{N_p}m_p \delta(\vect{X}^p-\vect{X})$ \\
    $\Rightarrow \: \Ucb = \rho_0 \bar{\Ucb} \:; \: \Fcb_\alpha = \rho_0 \bar{\Fcb}_\alpha$%\:; \: \Scb = \rho_0 \bar{\Scb}$
  \end{block}

  \begin{block}{Discrete weak form}
    \begin{equation*}
      \begin{aligned}
        & \text{Find $\Ucb$ such that $\forall \:e,t \in  [1, E]\times \tau$:}\\
        & \sum_{p=1}^{N_p} m_p\[\ddroit{\bar{\Ucb}}{t} \Vcb - \bar{\Fcb}_\alpha \drond{\Vcb}{X_\alpha}\]_{\lvert_{\vect{X}=\vect{X}^p}} + \int_{\partial \Omega^e} (\Fcb\cdot \vect{N}) d\Gamma = \vect{0} \quad \forall \Vcb
      \end{aligned}
    \end{equation*}
  \end{block}
  \footnoteCite{Sulsky94}
\end{frame}

\begin{frame}
  \begin{footnotesize}
    \begin{block}{Semi-discrete system}
      Introduction of polynomial approximation %$\Rightarrow  M_{ij} \ddroit{\bar{\Ucb}^j}{t}  - K_{ij}^\alpha \bar{\Fcb}^j_{\alpha} - \Scb^i + \hat{\Fcb}^i=  \vect{0} $
      \begin{equation*}
        M_{ij} \ddroit{\bar{\Ucb}^j}{t}  - K_{ij}^\alpha \bar{\Fcb}^j_{\alpha}  + \hat{\Fcb}^i=  \vect{0}
      \end{equation*}
      Diagonally lumped mass matrix $M^L_i=\sum_j M_{ij}$ \cite{Love}
    \end{block}
    \metroset{block=fill}
    \begin{block}{Explicit time discretization}
      \metroset{block=transparent}
      \begin{columns}
        \begin{column}{.4\textwidth}
          \begin{block}{\footnotesize Forward Euler}
            \vskip 7pt
            \begin{flalign*}
              M^L_i \frac{\bar{\Ucb}^{i,n+1} - \bar{\Ucb}^{i,n}}{\Delta t^n}=K_{ij}^\alpha \bar{\Fcb}^{j,n}_{\alpha}  - \hat{\Fcb}^{i,n}
            \end{flalign*}
            \vskip 14pt
          \end{block}
        \end{column}
        \begin{column}{.55\textwidth}
          \begin{block}{\footnotesize Second-order Runge-Kutta (RK2)}
            \vskip -5pt
            \begin{align*}
              & M^L_i \frac{\bar{\Ucb}^{i,n+1/2} - \bar{\Ucb}^{i,n}}{\Delta t^n}=\frac{1}{2}\(K_{ij}^\alpha \bar{\Fcb}^{j,n}_{\alpha}  - \hat{\Fcb}^{i,n}\) \\
              &M^L_i \frac{\bar{\Ucb}^{i,n+1} - \bar{\Ucb}^{i,n}}{\Delta t^n}=K_{ij}^\alpha \bar{\Fcb}^{j,n+1/2}_{\alpha}  - \hat{\Fcb}^{i,n+1/2}
            \end{align*}
          \end{block}
        \end{column}
      \end{columns}
    \end{block}
  \end{footnotesize}
  \footnoteCite{Love}
\end{frame}

\subsection{Interface fluxes}
\begin{frame}
  \begin{block}{Riemann problem}
    \begin{columns}
      \begin{column}{0.5\textwidth}
        \begin{footnotesize}
          \begin{align*}
          (\Rc\Pc): \quad &\drond{\Ucb}{t} + \drond{\Fcb_N}{\Ucb} \drond{\Ucb}{\xi }= \vect{0} \\
                          & \Ucb(\xi,0) = \left\lbrace
                            \begin{aligned}
                              & \Ucb_L \: \text{if } \xi < 0 \\
                              & \Ucb_R \: \text{if } \xi > 0
                            \end{aligned} \right.
          \end{align*}
        \end{footnotesize}
      \end{column}
      \begin{column}{0.3\textwidth}
        \input{section2/pgfFigures/2DEdge}
      \end{column}
    \end{columns}
  \end{block}
  \vspace{-0.4cm}
  \begin{footnotesize}
    \begin{block}{Approximate-state Riemann solver \cite{Toro}}
      \begin{columns}
        \begin{column}{0.3\textwidth}
          \begin{itemize}
          \item Characteristic speeds $\lambda^i(\tens{F})$
          \item Nonlinear waves
          \end{itemize}
        \end{column}
        \begin{column}{0.55\textwidth}
          \input{section2/pgfFigures/approximate_RS}
        \end{column}
      \end{columns}
    \end{block}
    Computation of $\Ucb^* \Rightarrow$ corresponding flux: $\vect{\Fc}_N(\Ucb^*)$ \cite{Godunov_method}\\
    \alert{Integration of constitutive equations}

  \end{footnotesize}
  \vspace{-0.4cm}
  \footnoteCite{Toro,Godunov_method}
 \end{frame}

\begin{frame}%{\normalsize Boundary conditions}
  \begin{block}{\footnotesize Boundary Conditions: ghost nodes}
    \begin{columns}
      \begin{column}{0.3\textwidth}
        \input{section2/pgfFigures/BCs}
      \end{column}
      \begin{column}{0.6\textwidth}
        \begin{scriptsize}
          \begin{itemize}
          \item[] Enforce $\Ucb_L$ such that $\Ucb^*=\Ucb^{imp}$
          \item[] Problem: apply strain rather than stress
          \end{itemize}
        \end{scriptsize}
      \end{column}
    \end{columns}
  \end{block}
  \begin{footnotesize}
    %Auxiliary vector $\Qcb^T=\[\vect{v}\:,\: \tens{\Pi}\]$
    $\Rightarrow $ Quasi-linear form with $\Qcb^T=\[\vect{v} \:,\: \tens{\Pi}\]$ \cite{Trangenstein91}:
    \begin{gather*}
      % \dot{\tens{\Pi}} = \Hbb(\tens{F}) : \dot{\tens{F}} \Rightarrow \text{Quasi-linear form: } \drond{\Qcb}{t} + \Absf^\alpha \drond{\Qcb}{X_\alpha} = \vect{0}
      \drond{\Ucb}{t} + \drond{\Fcb\cdot\vect{E}_\alpha}{X_\alpha} = \vect{0} \quad \Leftrightarrow \quad \drond{\Ucb}{\alert{\Qcb}}\drond{\alert{\Qcb}}{t} + \drond{\Fcb\cdot\vect{E}_\alpha}{\alert{\Qcb}}\drond{\alert{\Qcb}}{X_\alpha}=\vect{0}
      %\\      \drond{\Qcb}{t} + \Absf^\alpha \drond{\Qcb}{X_\alpha} = \vect{0}
    \end{gather*}
    \alert{Also avoids the integration of constitutive equations at element interfaces!}
  \end{footnotesize}
  \begin{block}{\footnotesize Transverse corrections: Adaptation of the Corner Transport Upwind method \cite{Colella_CTU}}
    \centering
    \input{section2/pgfFigures/transverse}
  \end{block}
  \vspace{-0.2cm}
  \footnoteCite{Trangenstein91,Colella_CTU}
\end{frame}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "../presentation"
%%% End:


\newpage
\appendix
%redefine figure and table counters
\renewcommand\thefigure{S\arabic{figure}}
\setcounter{figure}{0}
\renewcommand\thetable{S\arabic{table}}
\setcounter{table}{0}

\section{Appendix: Sensitivity of the optimisation procedure to initial values}
\subsection{Approach}
\label{sec:appendix}
The sensitivity of parameter estimates for the cumulative model with constant variance (Eqn.~\ref{eq:candy_cm_count_form}) obtained using the direct likelihood optimisation with \verb+optim+ was assessed by simulation.
Simulations consisted of drawing a set of random initial values. To preserve the ordering of the cut-point parameters $\alpha_i$, their initial values $\alpha^{\circ}_i$ were assembled as the cumulative sum of six independent draws $$x_j \sim \mathrm{Uniform}(1,20), \qquad j = 1,\ldots, 6$$ i.e. $$\alpha^{\circ}_i = \sum_{j=1}^i x_j.$$ The initial value for $\beta$ was drawn as $$\beta^{\circ}\sim\mathrm{Uniform}(-1,1).$$

Optimisation proceeded using the likelihood outlined in Section~\ref{sec:const_var_cm} with a numerical threshold at the machine's precision to prevent the Poisson likelihood from approaching negative infinity. The optimisation routine used the BFGS method and parameter scaling value for $\beta$ of 0.01 relative to the cut-point parameters $\alpha_i$. This improved overall convergence rates but allowed convergence at an infinite likelihood, the likelihood was therefore evaluated one final time without thresholding at the converged parameter values and parameters were only retained when the unthresholded likelihood was finite.
Parameters were further filtered to remove parameter sets where convergence was achieved at a finite likelihood, but with a substantially larger overall log-likelihood than the best model fit. Thresholds for this were determined empirically and set at 93 for the logit link and 100 for the cloglog model.

\subsection{Results}
The optimisation was reasonably robust for the logit link model. 73\% of simulations achieved convergence at a finite likelihood and of these over 99\% met the filtering threshold (Figure~\ref{fig:figS1}). Among the filtered parameter sets variability in the final estimates was very small (para\-meter-wise coefficients of variation were all smaller than 0.01\%) and there was no correlation between initial values and parameter estimates when taking into account that multiple comparisons were made (Table~\ref{tab:tabS2}). Convergence failed for initial values $\beta^{\circ}>0.7$, and convergence failures were likely, but not guaranteed for $\beta^{\circ}\approx 0$ (Figure~\ref{fig:figS2}).

The optimisation of the cloglog model was less stable. Only 47\% of simulations resulted in convergence, and of those 92\% met the filtering threshold (Figure~\ref{fig:figS1}). Variability in the filtered parameter estimates was larger than for the logit model, but still small in absolute terms (parameter-wise coefficients of variation were all below 0.01\%) and there was no correlation between initial values and parameter estimates (Table~\ref{tab:tabS2}). Convergence failed for many $0<\beta^{\circ}<0.2$, and always for $\beta^{\circ}>0.2$ (Figure~\ref{fig:figS3}).

\begin{table}[bph]
  \small
    \centering
    \caption{Summary statistics of parameter sensitivities to starting values for the cumulative model with constant variance (Eqn.~\ref{eq:candy_cm_count_form}). The coefficients of variation (CV), correlation coefficients between initial and converged values $\rho^{\circ}$, and their corresponding p-values $P_\rho$ were calculated for the filtered parameter estimates only and adjusted for multiple comparisons using the Bonferroni correction.
    }
  \input{../outputs/parameter_sens_table}
  \label{tab:tabS2}
\end{table}

\begin{figure}[htbp]
  \centering
  \includegraphics[width=\textwidth]{../figures/figS1_initial_value_sensitivity_unfiltered.pdf}
  \caption{Optimisation results for randomly drawn initial values for the cumulative model with constant variance. Parameter estimates are coloured by the log of the negative log-likelihood of the corresponding model fit. Yellow dashed lines correspond to parameter values reported in \citep{candy1991modeling}. The majority of initial values resulted in estimates close to the original publication for the logit model (bottom). Convergence to local optima far from the published parameters was more common in the cloglog link model (top).}
  \label{fig:figS1}
\end{figure}

\begin{figure}[p]
  \centering
  \includegraphics[width=\textwidth]{../figures/figS2_initial_value_sensitivity_logit.pdf}
  \caption{Sensitivity of optimisation results to initial values for the logit-link cumulative model with constant variance. The lower triangle and diagonal elements of the scatterplot matrix (grey symbols) show pairwise plots of filtered parameter estimates $\hat{\alpha}_i, \hat{\beta}$  against corresponding initial values $\alpha^{\circ}_i, \beta^{\circ}$. The upper triangle shows pairwise plots of the initial values for the same model parameters against each other. Green dots indicate successful convergence of the optimisation, yellow dots indicate convergence failures. Optimisation failed whenever $\beta^{\circ}>0.7$, furthermore convergence failures were more likely when $\beta^{\circ}\approx 0$.}
  \label{fig:figS2}
\end{figure}

\begin{figure}[p]
  \centering
  \includegraphics[width=\textwidth]{../figures/figS3_initial_value_sensitivity_cloglog.pdf}
  \caption{Sensitivity of optimisation results to initial values for the cloglog-link cumulative model with constant variance. The lower triangle and diagonal elements of the scatterplot matrix (grey symbols) show pairwise plots of filtered parameter estimates $\hat{\alpha}_i, \hat{\beta}$ against corresponding initial values $\alpha^{\circ}_i, \beta^{\circ}$. The upper triangle shows pairwise plots of the initial values for the same model parameters against each other. Green dots indicate successful convergence of the optimisation, yellow dots indicate convergence failures. Optimisation failed for most $\beta^{\circ}>0$.}
  \label{fig:figS3}
\end{figure}


\section{Probabilistic/Approximate design}
\label{sect:related-work-prob-approx}

While the shrinkage of device's feature size according to Moore's law~\cite{Moore1965}
has driven the prosperity of microelectronic industry,
the variability and uncertainty of devices at the atomic level
pose serious challenges on the continuation of the law.

New computational paradigms are proposed in response to the challenges in the post-Moore's era.
Among many other efforts,
\textit{approximate design} allows deterministic deviation of an implemented circuit from its specification.
A real-world example of approximate design is the deployment of neural networks to edge devices.
Quantization is often applied to convert floating-point parameters to fixed-point ones to reduce hardware cost.
Along this direction, circuit architectures for reconfigurable adders whose accuracy can be adjusted based on application scenarios~\cite{Kahng2012,Ye2013} and energy-efficient adders with a moderate error rate~\cite{Kim2013} are proposed.
The automatic synthesis and analysis of approximate circuits have also been studied~\cite{Venkatesan2011ApproxDesign,Venkataramani2012,Miao2013,Miao2014,Li2014,Mrazek2016,Rehman2016}.
On the other hand, \textit{probabilistic design} allows nondeterministic deviation from the specification.
It can be applied to, for example, low-power video decoding, where correctness can be sacrificed for power efficiency.
Chakrapani et al.~\cite{Chakrapani2006ProbDesign} consider CMOS devices with probabilistic behavior,
establish the relation between energy consumption and probability of correct switching,
and exploit it to trade power efficiency against correctness.

Despite the advancements made by prior endeavors, most of them focus on approximate design.
The analysis and synthesis of probabilistic design have gained relatively less attention.
Nevertheless, the study of probabilistic design is important in the following respects.
First, randomness is a valuable resource to trade for computation efficiency.
For example, there exist problems that can be solved efficiently by randomized algorithms
but not deterministic algorithms.
Second, there are applications, such as data mining, compressive sensing, etc.,
that may not be sensitive to minor random fluctuations.
Third, devices at their quantum foundation or systems in their biological nature are intrinsically probabilistic.
We aim at providing a formalism for the analysis and verification of probabilistic design.
Notice that in the design automation process,
verification is essential to the entire design flow.
For example, equivalence checking~\cite{Kuehlmann1997,Mishchenko2006}
should be applied to validate the correctness of synthesized circuits.
Hence, establishing a verification framework is a crucial step in automated synthesis of probabilistic design.

Probabilistic behavior of a design has also been studied along the research of circuit reliability analysis,
which focuses on analyzing the robustness of a circuit against permanent defects or transient faults.
The reliability of a circuit is characterized by the probability of the occurrence of an error at the primary outputs.
Therefore, the study of circuit reliability is very related to the evaluation of probabilistic design.
Classical approaches to reliability analysis apply fault injection and Monte Carlo simulation~\cite{Mohanram2003}.
Symbolic analysis methods exploiting mathematical tools,
such as Markov random fields~\cite{Bahar2003},
probability transfer matrices~\cite{Krishnaswamy2005},
Bayesian networks~\cite{Rejimon2005},
and algebraic decision diagrams~\cite{Miskov-Zivanov2006},
have also been investigated.
Choudhury and Mohanram~\cite{Choudhury2009} propose three accurate and scalable algorithms to address the scalability issue of symbolic methods.

However, prior methods for circuit reliability analysis are inadequate to probabilistic circuits
for the following two reasons.
First, most prior approaches assume single-gate failures.
This assumption makes prior methods inapplicable to probabilistic design,
where multiple probabilistic gates may be commonly present.
Second, most prior efforts consider only the average error rate of a design.
The necessity of analyzing the maximum error rate comes from the increasing demand of safety-centric systems,
e.g., utilized in health care or automotive industries~\cite{Lingasubramanian2007,Lingasubramanian2011}.
Unfortunately, its computational scalability is much limited due to the underlying symbolic modeling via Bayesian network~\cite{Jensen1996}.