\subsection{Vectors and physics}

Suppose you push
on something. Then, your push is made up of two components, how hard you push and the direction you push. This illustrates the concept of force.

\begin{definition}{Force}{force}
\textbf{Force}\index{force} is a vector. The magnitude of this vector is a measure of how
hard it is pushing. It is measured in Newtons\index{Newton}. The direction of this vector is the direction in which the push is taking place.
\end{definition}

Vectors are used to model force and other physical vectors like velocity.
 As with all vectors, a vector modelling force has two essential
ingredients, its magnitude and its direction.

Recall the special vectors which point along the coordinate axes.
These are given by
\begin{equation*}
\vect{e}_{i} = \mat{0,\ldots,0,1,0,\ldots,0}^T
\end{equation*}
where the $1$ is in the $i\th$ slot and there are zeros in all the other
spaces. The direction of $\vect{e}_{i}$ is referred to as the $i\th$ direction.

Consider the following picture which illustrates the case of $\R^{3}$.
Recall that in $\R^3$, we may refer to these vectors as $\vect{i}, \vect{j}$, and $\vect{k}$.

\begin{center}
\begin{tikzpicture}
\draw(0,0,2.5)--(0,0,0)--(0,0,-2);
\draw(-2,0,0)--(0,0,0)--(2,0,0);
\draw(0,-2,0)--(0,0,0)--(0,2,0);
\draw[ultra thick, ->, red](0,0,0)--(0,0,1.5);
\draw[ultra thick, ->, blue](0,0,0)--(0,1,0);
\draw[ultra thick, ->, green](0,0,0)--(1,0,0);
\node[left] at (0,0,2.5){$x$};
\node[below] at (2,0,0){$y$};
\node[above] at (0,2,0){$z$};
\node[left] at (-0.2,0,1){$\vect{e}_1$};
\node[below] at (1,0,0){$\vect{e}_2$};
\node[left] at (0,1,0){$\vect{e}_3$};
\end{tikzpicture}
\end{center}

Given a vector $\vect{u}=\mat{u_{1},\ldots,u_{n}}^T$, it follows
that
\begin{equation*}
\vect{u}=u_{1}\vect{e}_{1}+\ldots +u_{n}\vect{e}_{n}=
\sum_{k=1}^{n}u_{i}\vect{e}_{i}
\end{equation*}

What does addition of vectors mean physically? Suppose two forces are
applied to some object. Each of these would be represented by a force vector
and the two forces acting together would yield an overall force acting on
the object which would also be a force vector known as the
resultant\index{resultant}. Suppose the two vectors are $\vect{u}=\sum_{k=1}^{n}u_{i}\vect{e}_{i}$ and $\vect{v}=\sum_{k=1}^{n}v_{i}\vect{e}_{i}$. Then the vector $\vect{u}$ involves a component in the $
i\th$ direction given by $u_{i}\vect{e}_{i}$, while the component in the $i\th$
direction of $\vect{v}$ is $v_{i}\vect{e}_{i}$. Then the vector $\vect{u} + \vect{v}$ should have a component in the $i\th$
direction equal to $(u_{i}+v_{i}) \vect{e}_{i}$. This is
exactly what is obtained when the vectors, $\vect{u}$ and $\vect{v}$ are
added.
\begin{eqnarray*}
\vect{u}+\vect{v}& =&\mat{u_{1}+v_{1},\ldots,u_{n}+v_{n}}^T  \\
& =&\sum_{i=1}^{n}(u_{i}+v_{i}) \vect{e}_{i}
\end{eqnarray*}

Thus the addition of vectors according to the rules of addition in $\R^{n}$ which were presented earlier, yields the appropriate vector which
duplicates the cumulative effect of all the vectors in the sum.

Consider now some examples of vector addition.

\begin{example}{The resultant of three forces}{resultant-three-forces}
There are three ropes attached to a car and three people pull on these
ropes. The first exerts a force of $\vect{F}_1 =
2\vect{i} + 3\vect{j} -2 \vect{k}$ Newtons, the second exerts a force of $\vect{F}_2
=
3\vect{i}+5\vect{j}+\vect{k}$ Newtons
and the third exerts a force of $5\vect{i}-\vect{j}+2\vect{k}$ Newtons. Find
the total force in the direction of $\vect{i}$.
\end{example}

\begin{solution}
To find the total force, we add the vectors as described above.
This is given by
\begin{eqnarray*}
&&(2\vect{i}+3\vect{j}-2\vect{k}) + (3\vect{i}+5\vect{j}+\vect{k}) + (5\vect{i}-\vect{j}+2\vect{k})\\
&=&
(2  + 3 + 5) \vect{i} + (3 + 5 + -1) \vect{j} + (-2+1+2) \vect{k} \\
&=&
10 \vect{i} + 7 \vect{j} + \vect{k}
\end{eqnarray*}
Hence, the total force is  $10\vect{i}+7\vect{j}+\vect{k}$ Newtons. Therefore, the force in the $
\vect{i}$ direction is $10$ Newtons.
\end{solution}

Consider another example.

\begin{example}{Finding a vector from geometric description}{vector-from-geometric-description}
An airplane flies North East at $100\textrm{km}/\textrm{h}$. Write this as a vector.
\end{example}

\begin{solution}
A picture of this situation follows.

\begin{center}
\begin{tikzpicture}[scale=0.5]
\draw(-2,0)--(2,0);
\draw(0,-2)--(0,2);
\draw[ultra thick, ->, blue](0,0)--(2.5,2.5);
\end{tikzpicture}
\end{center}

Therefore, we need to find the vector $\vect{u}$ which has length 100 and direction as shown in this diagram.
We can consider the vector $\vect{u}$ as the hypotenuse of a
right triangle having equal sides, since the direction of $\vect{u}$ corresponds with the $45 ^{\circ}$ line.
The sides, corresponding to the $\vect{i}$ and $\vect{j}$ directions,  should be each of length 100/$
\sqrt{2}$. Therefore, the vector is given by
\[
 \vect{u} = \frac{100}{\sqrt{2}} \vect{i}+ \frac{100}{\sqrt{2
}}\vect{j}
=
\begin{mymatrix}{rr}
\vspace{0.05in} \frac{100}{\sqrt{2}} & \vspace{0.05in}\frac{100}{\sqrt{2}}
\end{mymatrix}^T\
\]
\end{solution}

This example also motivates the concept of \textbf{velocity}, defined below.

\begin{definition}{Speed and velocity}{speed-velocity}
The \textbf{speed}\index{speed} of an object is a measure of how fast it is going. It is
measured in units of length per unit time. For example, kilometers per
hour, meters per second. The
\textbf{velocity}\index{velocity} is a vector having the speed as the
magnitude but also specifying the direction.
\end{definition}

Thus the velocity vector in the above example is $\vspace{0.05in}\frac{100}{\sqrt{2}}\vect{i}+
\vspace{0.05in}\frac{100}{\sqrt{2}}\vect{j}$, while the speed is $100\textrm{km}/\textrm{h}$.

Consider the following example.

\begin{example}{Position from velocity and time}{position-velocity-time}
The velocity of an airplane is $100\vect{i}+\vect{j}+\vect{k}$
measured in kilometers per hour and at a certain instant of time its
position is $(1,2,1)$.

Find the position of this airplane one minute later.
\end{example}

\begin{solution}
Here imagine a Cartesian coordinate
system in which the third component is altitude and the first and second
components are measured on a line from West to East and a line from South to
North.

Consider the vector $
\begin{mymatrix}{rrr}
1 & 2 & 1
\end{mymatrix}^T$, which is the initial position vector
of the airplane. As the plane moves, the position vector changes according to the velocity vector.
After one minute (considered as $\frac{1}{60}$ of an hour)
the airplane has moved in the $\vect{i}$ direction a distance of
$100\times \frac{1}{60}= \frac{5}{3}$ kilometer. In the $\vect{j}
$ direction it has moved $\frac{1}{60}$ kilometer during this same time,
while it moves $\frac{1}{60}$ kilometer in the $\vect{k}$ direction.
%\begin{picture}(1,425)\end{picture}
Therefore, the new displacement vector for the airplane is
\begin{equation*}
\begin{mymatrix}{rrr}
1 & 2 & 1
\end{mymatrix}^T +
\begin{mymatrix}{rrr}
\frac{5}{3} & \frac{1}{60} & \frac{1}{60}
\end{mymatrix}^T
=\begin{mymatrix}{rrr}
\frac{8}{3} & \frac{121}{60} & \frac{121}{60}
\end{mymatrix}^T
\end{equation*}
\end{solution}

Now consider an example which involves combining two velocities.

\begin{example}{Sum of two velocities}{sum-of-two-velocities}
A certain river is one half kilometer wide with a current flowing at 4 kilometers per
hour from East to West. A man swims directly toward the opposite shore from
the South bank of the river at a speed of 3 kilometers per hour. How far down the
river does he find himself when he has swam across? How far does he end up
swimming?
\end{example}

\begin{solution}
Consider the following picture which demonstrates the above scenario.

\begin{center}
\begin{tikzpicture}[scale=0.5]
\draw(-3,1)--(2,1);
\draw(-3,-1)--(2,-1);
\draw[ultra thick, blue, ->](1,0)--(-3,0);
\draw[ultra thick, blue, ->](1,0)--(1,3);
\node[above] at (-1,0){$4$};
\node[right] at (1,2){$3$};
\end{tikzpicture}
\end{center}

First we want to know the total time of the swim across the river.
The velocity in the direction across the river is
$3$ kilometers per hour, and the river is $\frac{1}{2}$ kilometer wide. It follows the trip
takes $1/6$ hour or $10$ minutes.

Now, we can compute how far downstream he will end up. Since the river runs at a rate of
$4$ kilometers per hour, and the trip takes $1/6$ hour, the distance travelled downstream
is given by $4 \paren{\frac{1}{6}} = \frac{2}{3}$ kilometers.

The distance travelled by the swimmer is given by the hypotenuse of a right triangle.
The two arms of the triangle are given by the distance across the river, $\frac{1}{2}$km, and
the distance travelled downstream, $\frac{2}{3}$ km. Then, using the Pythagorean Theorem, we can calculate
the total distance $d$ travelled.
\begin{equation*}
d
=
\sqrt{ \paren{\frac{2}{3}}^2 + \paren{\frac{1}{2}} ^2 }
=
\frac{5}{6} \mbox{km}
\end{equation*}

Therefore, the swimmer travels a total distance of $\frac{5}{6}$ kilometers.
\end{solution}


\chapter{Propositional Formulas}
\label{chapter:propositional-formulas}
\marginurl{%
  Propositional Formulas:\\\noindent
  Introduction to Mathematical Logic \#1
}{youtu.be/X0797bVFf3Y}

This part, as it follows from the title, is devoted to mathematical logic,
a mathematical approach to a branch of philosophy called logic. Logic studies
reasoning and mathematical logic studies mathematical reasoning. As we have
mentioned in \Cref{chapter:proofs} proofs in mathematics consists of
\emph{sentences} of a certain structure that are connected by implications.
In addition,  as we discussed in \Cref{chapter:predicates}, we can build larger
sentences from smaller ones using connectives.

Note that in real life the sentences are written using common English which is
ambiguous and therefore hard for analysis.
So to create a formal description of mathematics we need to create an
artificial formal language for mathematics.

First
(\Cref{chapter:propositional-formulas,chapter:propositional-truth,chapter:propositional-deduction})
we will define a language for propositional (sentential) logic; i.e. the logic
which deals only with propositions. Later (\Cref{chapter:predicate-formulas}) we
extend it to a logic which also takes properties of individuals into account.

The process of formalization of propositional logic consists of two main parts:
\begin{itemize}
  \item present a formal language,
  \item specify a procedure for obtaining valid or true propositions.
\end{itemize}

\section{Definition of Formulas}

Statements in  propositional logic are either some independent atomic
statements, or are formed from the atomic one using connectives.

In other words, statements in propositional logic can be defined using
propositional formulas (also known as sentential formulas or Boolean formulas).
\begin{definition}
  We say that a finite sequence $\phi$ of elements of the set
  $V \cup \set{\lnot, \lor, \land, \limplies, \text{``(''}, \text{``)''}}$
  is a propositional formula on the variables from $V$ if
  \begin{itemize}
    \item either $\phi$ is equal to $x$ for some $x \in V$,
    \item or $\phi$ is equal to $(\psi_1 \land \psi_2)$, or
      $(\psi_1 \lor \psi_2)$, or $(\psi_1 \limplies \psi_2)$,\footnote{%
        The symbol $\limplies$ is used to denote the implication.
        Due to historical reasons the standard symbol $\implies$ is rarely
        used as a connective in mathematical logic; hence, we will use
        $\limplies$ instead of $\implies$ in this part of the book.
        It is important to note that, sometimes the symbol $\supset$ is also
        used instead of $\implies$.
      }
      where $\psi_1$ and $\psi_2$ are propositional formulas on the variables
      from $V$,
    \item or $\phi$ is equal to $(\lnot \psi)$, where $\psi$ is a propositional
      formula on the variables from $V$.
  \end{itemize}

  We denote the set of all propositional formulas by $\prop{V}$.
\end{definition}

For example, $((x_1 \lor (\lnot x_2)) \land x_3)$ is a propositional formula on
the variables from $\set{x_1, x_2, x_3}$ (we also say that it is a formula on
$x_1$, $x_2$, $x_3$).

\begin{exercise}
  Write the definition of propositional formulas using the terminology
  ``the set generated by \dots from \dots'' (see \Cref{chapter:functions}).
\end{exercise}

Hereafter when naming formulas, we will not mention explicitly all the
parenthesis. To establish a more compact notation, we adopt the following
conventions.
\begin{itemize}
  \item The outermost parentheses do not need to be explicitly mentioned; e.g.,
    we write ``$A \land B$'' to refer to $(A \land B)$.
  \item The negation symbol applies to as little as possible.
    For example, $\lnot A \land B$ denotes $(\lnot A) \land B$;
    i.e., $((\lnot A) \land B)$. Which is not the same as
    $(\lnot (A \land B))$.
  \item The conjunction and disjunction symbols apply to as little as possible,
    given that convention 2 is to be observed. For example,
    $A \land B \limplies \lnot C \lor D$ is
    $((A \land B) \limplies ((\lnot C) \lor D))$.
  \item Where one connective symbol is used repeatedly, grouping is to the
    right: $A \land B \land C$ is $A \land (B \land C)$,
    $A \limplies B \limplies C$ is $A \limplies (B \limplies C)$.
\end{itemize}

Interpreting propositional logic is not difficult since the considered entities
have a simple structure. The propositions are built up from rough blocks by
adding connectives. The simplest parts (atoms) are of the form ``cows are
animals'', ``Earth is flat'', ``$2 \times 2 = 2$'', which are simply true or
false. We extend this assignment of truth values to composite propositions, by
reflection on the meaning of the logical connectives.

\begin{definition}
  A function $v : \prop{V} \to \set{\ltrue, \lfalse}$ is a valuation if
  \begin{itemize}
    \item $v(\lnot \psi) = \lnot v(\psi)$,
    \item $v(\psi_1 \land \psi_2) = v(\psi_1) \land v(\psi_2)$,
    \item $v(\psi_1 \lor \psi_2) = v(\psi_1) \lor v(\psi_2)$, and
    \item $v(\psi_1 \limplies \psi_2) = v(\psi_1) \limplies v(\psi_2)$.
  \end{itemize}
\end{definition}

We may note that all the valuations are actually can be defined by the values
of variables.
\begin{theorem}
  Let $\rho : V \to \set{\ltrue, \lfalse}$ be a function (we say that $\rho$ is
  a propositional assignement). Then there is a unique valuation
  $\substitute{\cdot}{\rho} : \prop{V} \to \set{\ltrue, \lfalse}$ such that
  $\substitute{x}{\rho} = \rho(x)$ for any $x \in V$.
\end{theorem}

Since any valuation can be defined by the values assigned to variables, we need
to introduce the following notation.
If $V = \set{x_1, \dots, x_n}$ and $v_1, \dots, v_n \in \set{\ltrue, \lfalse}$,
then $\substitute{\cdot}{x_1 = v_1, \dots, x_n = v_n}$ denotes the valuation
such that $\substitute{x_i}{x_1 = v_1, \dots, x_n = v_n} = v_i$ for each
$i \in \range{n}$.

For example, the value of a formula $(x_1 \land \lnot x_2) \lor x_3$ when
$\ltrue$ is substituted as the value of $x_1$, $\ltrue$ is substituted as the
value of $x_2$, and $\lfalse$ is substituted as the value of $x_3$ is equal to
$(\ltrue \land \lfalse) \lor \lfalse = \lfalse$.

Note that if $\phi$ is a formula on the variables from $V$ it does not mean that
all the variables from $V$ have to be used.
For example, $x_1$ is a formula on the variables from $\set{x_1, x_2}$; however,
$x_2$ is not used in the formula.

\begin{exercise}
  Give a formal definition (using structural induction) of the set of all the
  variables that are used in a propositional formula $\phi$ on variables from a
  set $V$.
\end{exercise}

Let $\phi$ be a formula on the variables from a set $V$. The definition
of a value of a formula requires us to specify all the values of all the
variables from $V$. However, the following theorem shows that in
fact we need to specify only the variables that are actually used in $\phi$.
\begin{theorem}
  Let $\phi$ be a formula $\phi$ on the variables from a set $V$,
  and $U$ be the set of the variables used in $\phi$.

  Consider $\rho_1, \rho_2 : V \to \set{\ltrue, \lfalse}$ such that
  $\rho_1(x) = \rho_2(x)$ for any $x \in U$.
  Then $\substitute{\phi}{\rho_1} = \substitute{\phi}{\rho_2}$.
\end{theorem}
\begin{proof}
  We prove the statement using the structural induction.
  \begin{description}
    \item[(base case)] Let $\phi = x$ for some $x \in V$.
      Note that $x \in U$ and $\substitute{\phi}{\rho_1} = \rho_1(x) =
      \rho_2(x) = \substitute{\phi}{\rho_2}$.
    \item[(induction step)] We need to consider the following three cases.
      \begin{itemize}
        \item Let $\phi$ be equal to $\psi_1 \land \psi_2$ such that
          $\substitute{\psi_1}{\rho_1} = \substitute{\psi_1}{\rho_2}$ and
          $\substitute{\psi_1}{\rho_2} = \substitute{\psi_2}{\rho_2}$.
          In this case,
          \[
            \substitute{\phi}{\rho_1} =
            (\substitute{\psi_1}{\rho_1} \land \substitute{\psi_2}{\rho_1} )=
            (\substitute{\psi_1}{\rho_2} \land \substitute{\psi_2}{\rho_2}) =
            \substitute{\phi}{\rho_2}.
          \]
        \item Let $\phi$ be equal to $\psi_1 \lor \psi_2$ such that
          $\substitute{\psi_1}{\rho_1} = \substitute{\psi_1}{\rho_2}$ and
          $\substitute{\psi_1}{\rho_2} = \substitute{\psi_2}{\rho_2}$.
          In this case,
          \[
            \substitute{\phi}{\rho_1} =
            (\substitute{\psi_1}{\rho_1} \lor \substitute{\psi_2}{\rho_1}) =
            (\substitute{\psi_1}{\rho_2} \lor \substitute{\psi_2}{\rho_2}) =
            \substitute{\phi}{\rho_2}.
          \]
        \item Let $\phi$ be equal to $\psi_1 \limplies \psi_2$ such that
          $\substitute{\psi_1}{\rho_1} = \substitute{\psi_1}{\rho_2}$ and
          $\substitute{\psi_1}{\rho_2} = \substitute{\psi_2}{\rho_2}$.
          In this case,
          \[
            \substitute{\phi}{\rho_1} =
            (\substitute{\psi_1}{\rho_1} \limplies \substitute{\psi_2}{\rho_1}) =
            (\substitute{\psi_1}{\rho_2} \limplies \substitute{\psi_2}{\rho_2}) =
            \substitute{\phi}{\rho_2}.
          \]
      \end{itemize}
  \end{description}
\end{proof}

\begin{exercise}
  Let $\phi_1$, $\phi_2$, and $\phi_3$ be propositional formulas on the
  variables from a set $V$. Show that for any propositional assignement
  $\rho$ to $V$,
  $\substitute{\phi_1 \land (\phi_2 \land \phi_3)}{\rho} =
   \substitute{(\phi_1 \land \phi_2) \land \phi_3}{\rho}$.
\end{exercise}

\section{Conjunctive and Disjuctive Normal Form}

Let $\phi_1$, \dots, $\phi_n$ be some propositional formulas. Then
\begin{itemize}
  \item $\bigland_{i = 1}^1 \phi_i = \phi_1$ and
    $\biglor_{i = 1}^1 \phi_i = \phi_1$, and
  \item $\bigland_{i = 1}^{k + 1} \phi_i =
    (\bigland_{i = 1}^{k} \phi_i) \land \phi_{k + 1}$ and
    $\biglor_{i = 1}^{k + 1} \phi_i =
      (\biglor_{i = 1}^{k} \phi_i) \lor \phi_{k + 1}$.
\end{itemize}
In other words $\bigland_{i = 1}^n \phi_i$ and $\biglor_{i = 1}^n \phi_i$
denotes the conjunction of the formulas $\phi_1$, \dots, $\phi_n$, and
$\biglor_{i = 1}^n \phi_i$ denotes the disjunction of them.

\begin{exercise}
  Let $\phi_1$, \dots, $\phi_n$, $\psi_1$, \dots, $\psi_m$, $\chi_1$, \dots,
  $\chi_{n + m}$ be some propositional formulas on the variables from $V$
  such that $\chi_i = \phi_i$ for $i \le n$ and $\chi_i = \psi_{i - n}$ for
  $n < i \le m$. Show that
  $\substitute{\left(\bigland_{i = 1}^n \phi_i\right) \land
    \left(\bigland_{i = 1}^n \psi_i\right)}{\rho} =
  \substitute{\left(\bigland_{i = 1}^{n + m} \chi_i\right)}{\rho}$
  for any propositional assignement $\rho$ to $V$.
\end{exercise}

Using this notation we may show that propositional formulas can represent all
the Boolean functions (functions from $\set{\ltrue, \lfalse}^n$ to
$\set{\ltrue, \lfalse}$).
\begin{theorem}
\label{theorem:function-to-formula}
  For any function $f : \set{\ltrue, \lfalse}^n \to
  \set{\ltrue, \lfalse}$ there is a
  formula $\phi$ on the variables $x_1$, \dots, $x_n$ such that
  $\substitute{\phi}{x_1 = v_1, \dots, x_n = v_n} = f(v_1, \dots, v_n)$ for all
  $v_1, \dots, v_n \in \set{\ltrue, \lfalse}$.
\end{theorem}

Let $u \in \set{\ltrue, \lfalse}$ and $x \in V$. Then $x^u$ denotes a
formula on the variables from $V$ such that $x^u = x$ if $u = \ltrue$ and
$x^u = \lnot x$ if $u = \lfalse$. Note that $\substitute{x^u}{\rho} = \ltrue$
iff $\rho(x) = u$, for any propositional assignement $\rho$ to $V$.
Indeed, if $u = \ltrue$, then $x^u = x$ and
$\ltrue = \substitute{x^u}{\rho} = \substitute{x}{\rho} = \rho(x)$ so
$\rho(x) = \ltrue = u$;
if $u = \lfalse$, then $x^u = \lnot x$ and
$\ltrue = \substitute{x^u}{\rho} = \substitute{\left(\lnot x\right)}{\rho} =
\lnot \rho(x)$ so $\rho(x) = \lfalse = u$.

\begin{exercise}
  Let $\phi_1$, \dots, $\phi_k$ are propositional formulas
  on the variables from $V$.
  \begin{itemize}
    \item Show that
      $\substitute{\left(\biglor_{i = 1}^k \phi_i\right)}{\rho} = \ltrue$ iff
      $\substitute{\phi}{\rho} = \ltrue$ for some $i \in \range{k}$.
    \item Show that
      $\substitute{\left(\bigland_{i = 1}^k \phi_i\right)}{\rho} = \ltrue$ iff
      $\substitute{\phi}{\rho} = \ltrue$ for all $i \in \range{k}$.
  \end{itemize}
\end{exercise}

Using this observation and the exercise we can
prove~\Cref{theorem:function-to-formula}.
\begin{proof}
  Let $S = \set[f(u_1, \dots, u_n) = \ltrue]{(u_1, \dots, u_n) \in
    \set{\ltrue, \lfalse}^n}$.
  Assume that
  $S = \set{(u_{1, 1}, \dots, u_{1, n}), \dots, (u_{k, 1}, \dots, u_{k, n})}$.
  By the previous observations
  \[
    \substitute{
      \left(
        \biglor_{i = 1}^k
          \bigland_{j = 1}^n x_j^{u_{i, j}}
      \right)
    }{x_1 = v_1, \dots, x_n = v_n}
    =
    f(v_1, \dots, v_n)
  \]
  for all $v_1, \dots, v_n \in \set{\ltrue, \lfalse}$.
  (Note that we have not considered the case when $S = \emptyset$, in this
  case $f$ is a constant $\lfalse$ function and it is equal to
  $x_1 \land \lnot x_1$.)
\end{proof}

One may notice that the formulas we constructed have very specific form,
such a form is called disjunctive normal form (DNF).
\begin{definition}
  We say that a propositional formula $\lambda$ on the variables from $V$
  is a \emph{literal} if it is equal to $x$ or to $\lnot x$ for some
  $x \in V$.

  We say that a propositional formula $\psi$ on the variables from $V$ is
  a \emph{term} if $\psi$ is equal to $\bigland_{i = 1}^\ell \lambda_i$, where $\lambda_1$, \dots, $\lambda_\ell$ are literals.

  Finally, we say that a propositional formula $\phi$ on the variables from
  $V$ is in \emph{disjunctive normal form} (DNF) if $\phi$ is equal to
  $\biglor_{i = 1}^k \psi_i$, where $\psi_1$, \dots, $\psi_k$ are
  terms.
\end{definition}

However, there is nothing special in this order of operations (disjunction of
conjunctions). So we can define conjunctive normal form (CNF) too.
\begin{definition}
  We say that a propositional formula $\psi$ on the variables from $V$ is
  a \emph{clause} if $\psi$ is equal to $\biglor_{i = 1}^\ell \lambda_i$, where
  $\lambda_1$, \dots, $\lambda_\ell$ are literals.

  Finally, we say that a propositional formula $\phi$ on the variables from
  $V$ is in \emph{conjunctive normal form} (DNF) if $\phi$ is equal to
  $\bigland_{i = 1}^k \psi_i$, where $\psi_1$, \dots, $\psi_k$ are
  clauses.
\end{definition}

Using the following simple trick we can prove that any function
has a representation in CNF. First, we define a function
$g(x_1, \dots, x_n) = \lnot f(x_1, \dots, x_n)$. Secondly, we may notice that
\[
  \substitute{
    \left(
      \lnot
      \left(
        \bigland_{i = 1}^k \biglor_{j = 1}^n \phi_{i, j}
      \right)
    \right)
  }{x_1 = v_1, \dots, x_n = v_n}
  =
  \substitute{
    \left(
      \biglor_{i = 1}^k \bigland_{j = 1}^n \lnot \phi_{i, j}
    \right)
  }{x_1 = v_1, \dots, x_n = v_n}
\]
for all $v_1, \dots, v_n \in \set{\ltrue, \lfalse}$
% THE REFERENCE TO THE EXERCISE
(see Exercise 15.7). Therefore the negation
of a formula in DNF can be easily transformed into a formula in CNF.
Finally, we know that the function
$g$ has a representation in DNF, which implies that $f$ has a representation
in CNF.

\begin{chapterendexercises}
  \exercise % CAREFUL, DONT MOVE! THERE IS A REFERENCE TO THIS NUMBER
    Let $\phi_1$ and $\phi_2$ be some propositional formulas on
    the variables from $V$. Show that for any propositional assignement
    $\rho$ to $V$,
    \begin{itemize}
      \item
        $\substitute{
        \lnot\left(
          \phi_1 \land \phi_2
        \right)}{\rho} =
        \substitute{
          \left(
            \lnot \phi_1 \lor \lnot \phi_2
          \right)
         }{\rho}$ and
    \item
    $\substitute{
    \lnot\left(
      \phi_1 \lor \phi_2
    \right)}{\rho} =
    \substitute{
      \left(
        \lnot \phi_1 \land \lnot \phi_2
      \right)
     }{\rho}$.
  \end{itemize}
  \exercise % CAREFUL, DONT MOVE! THERE IS A REFERENCE TO THIS NUMBER
    Let $\phi_1$, \dots, $\phi_n$ be some propositional formulas on
    the variables from $V$. Show that for any propositional assignement
    $\rho$ to $V$,
    \begin{itemize}
      \item
        $\substitute{
          \left(
            \lnot \left(
                    \bigland_{i = 1}^n \phi_i
                  \right)
          \right)}{\rho} =
          \substitute{
            \left(
              \biglor_{i = 1}^n \phi_i
            \right)
           }{\rho}$ and
      \item
        $\substitute{
         \left(
           \lnot \left(
                   \biglor_{i = 1}^n \phi_i
                 \right)
         \right)}{\rho} =
         \substitute{
           \left(
             \bigland_{i = 1}^n \phi_i
           \right)
          }{\rho}$.
    \end{itemize}
  \exercise Let $\phi = \bigvee_{i = 1}^m \lambda_i$ be a clause; we say that
    the width of the clause is equal to $m$.
    Let $\phi = \bigwedge_{i = 1}^\ell \chi_i$ be a formula in CNF
    ($\chi_i$'s are clauses'); we say that the width of $\phi$ is equal to
    the maximal width of $\chi_i$ for $i \in \range{\ell}$.

    Let $p_n : \set{\ltrue, \lfalse}^n \to \set{\ltrue, \lfalse}$ such that
    $p_n(x_1, \dots, x_n) = \ltrue$
    iff the set $\set[x_i = \ltrue]{i}$ has an odd number of elements.
    Show that any CNF representation of $p_n$ has width $n$.
\end{chapterendexercises}


\documentclass{article}

\usepackage{fullpage}
\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage[colorinlistoftodos]{todonotes}
%\usepackage[]{algorithm2e}
\usepackage[linesnumbered]{algorithm2e}
\usepackage{enumerate,url}
\usepackage{hyperref}
\hypersetup{colorlinks=true}
\usepackage{enumitem}

\title{CSE-6140/CX-4140 Algorithm Problems\\by\\Shahrokh Shahi }
\author{}
\date{}
\begin{document}
\maketitle

\noindent\fbox{\parbox{\dimexpr\textwidth-2\fboxsep-2\fboxrule\relax}{
\vspace{2mm}
Please upload two files:
\begin{enumerate}
 \item A single PDF named \texttt{assignment.pdf} containing your solutions.
 \item
\end{enumerate}
}}
\section*{ASSIGNMENT PROBLEMS}
\section{Dynamic Programming: Atlanta MARTA [xx pts]}
The  Metropolitan  Atlanta  Rapid Transit  Authority  (MARTA) is  the  principal  public  transportoperator in the Atlanta metropolitan area. It was Formed in 1971 as strictly a bus system, and today, it is transporting almost 450,000 passengers a day (bus and train). Currently, MARTA Passes are the cheapest option for those who regularly use MARTA for transportation. Assume the MARTA Passes are sold in three following forms:

\begin{itemize}
\item Daily: A 1-day pass sold for $tickets[0]$ dollars;
\item Weekly: A 7-day pass sold for $tickets[1]$ dollars;
\item Monthly: A 30-day pass sold for $tickets[2]$ dollars.
\end{itemize}

For instance, $tickets = {2, 7, 20}$ means we need to pay $\$2$, $\$7$, and $\$20$ for each daily, weekly,and monthly pass, respectively. The passes allow consecutive days of travel.  For example, if we get a weekly pass on day 5, then we can travel for 7 consecutive days which are: day 5, 6, 7, 8, 9,10, and 11. George P. Burdell is a student at Georgia Tech and he has already organized his commuting plan for  the  upcoming  year.  In  his  plan,  each   day of  year  is  specified   by an  integer   identification number from 1 to 365. Therefore, he can represent his commuting plan as an array of integers. For instance, days = {8, 9, 10, 11, 14, 17, 18, ...} means George needs to commute to the school on the 8th, 9th, 10th, 11th, ... days of year. He asked you to help him find the minimum amount of money that he should spend to purchase MARTA Passes for commuting to school in the next year.

\begin{itemize}
\item[a)] (3 pts) Prove the optimal substructure.
\item[b)] (4 pts) Write a recursive expression for calculating min Cost including the base case.
\item[c)] (6 pts) Give the pseudocode of  a   linear   Dynamic   Programming   algorithm   to   return   the   minimum   cost   of commuting  every day in the array “days”, if the cost of MARTA passes is given in a three-element array “tickets”.	Analyze the space and time complexity of your algorithm.
\end{itemize}
%\clearpage
% -----------------------------------------------------------------------
\color{blue}
\subsection*{Solution}

\subsubsection*{Optimal Substructure}

We are given an array of integer numbers representing the days that George wants to commute to school. $days = \{d_1, d_2, \ldots, d_n \}$, and a three-element array $tickets = \{t_1, t_2, t_3\}$. we want to calculate the minimum possible value that George has to pay for buying MARTA passes. Let $OPT[i]$ denotes the minimum amount of money George needs to pay to fulfill the plan from day $i$ to the end of the plan. Therefore, the minimum cost of commuting for the entire plan is $OPT[1]$.
There can be two approaches to solve this problem: \\
\begin{itemize}
\item\textit{Approach 1: Iterate over all days}\\
Starting from day one, if George doesn't want to travel today, there is no need to buy a MARTA pass today. Therefore, it is strictly better to wait until the day that he needs to travel, say day $i$. Then, he will have three options: buying a 1-day, 7-day, or 30-day MARTA pass:
\begin{itemize}
	\item If he buys a 1-day pass: he needs to pay $tickets[0]$ dollars and the next subproblem is $OPT[i+1]$ because the 1-day pass is only valid for one day and for the next day he has to pay $OPT[i+1]$ dollars for commuting from day $i+1$ to the end of the plan.
	\item If he buys a 7-day pass: he needs to pay $tickets[1]$ dollars and the next subproblem is $OPT[i+7]$ since the 7-day pass is valid for 7 days (from the day of purchase), and thus after that 7 days, he has to pay $OPT[i+7]$ dollars for commuting from day $i+7$ to the end of the plan.
	\item Similarly, if he buys a 30-day pass: he needs to pay $tickets[2]$ dollars and the next subproblem is $OPT[i+30]$
\end{itemize}

Therefore, if George needs to travel in day $i$, the optimum amount of money can be obtained by taking the minimum values of these three:
$$
OPT[i] = \min\{ tickets[0] + OPT[i+1], tickets[1] + OPT[i+7] ,  tickets[2] + OPT[i+30]\}
$$

The solution of these three subproblems, $OPT[i+1]$, $OPT[i+7]$, and $OPT[i+30]$ are minimum. For the sake of contradiction, assume these three values are not the minimum solutions. Therefor, there must exist other optimum solutions with the less ticket cost. Then, we can substitute the solution(s) of subproblem(s) with these minimum values. That implies we obtain less value for $OPT[i]$, the cost of tickets for the commuting plan from day $i$ which is a contradiction because we started with this assumption that $OPT[i]$ is the minimum ticket cost. Thus, this problem has optimal substructures. $\blacksquare$

%But we can only consider the days that their identification number is in the commuting plan.

\item\textit{Approach 2: Iterate over the days of the commuting plan}\\
In the first approach, we iterate over all days of the year (starting from day 1 to the last possible day in the plan e.g. 365 , regardless of their presence in the commuting plan.
However, instead of all days, we can only check the days that their identification numbers are in the commuting plan. This approach will be slightly faster than the first one. The second approach will particularly be more preferable when the number of days in the plan is much less than the maximum day index, i.e. $|days| << \max\{days\}$. In the worst case, number of days of the commuting plan will be equal to the maximum day index which happens when George wants to commute everyday.
In this case, the subproblem $OPT[i]$ denotes the minimum amount of money George needs to pay to fulfill the plan from day $days[i]$ to the end of the plan. (In approach 1, $i$ denotes identification number of each day. In approach 2, $i$ denotes the index of each day in $days$ array). Now, to pass the unnecessary checks, we need to define three additional indices. Let $j1$ be the largest index such that
$days[j1] < days[i] + 1$, $j7$ be the largest index such that
$days[j7] < days[i] + 7$, $j30$ be the largest index such that
$days[j30] < days[i] + 30$. In this way, if George needs to travel in day $i$, the optimum amount of money can be obtained by taking the minimum values of these three:
$$
OPT[i] = \min\{ tickets[0] + OPT[j1], tickets[1] + OPT[j7] ,  tickets[2] + OPT[j30]\}
$$
The proof of optimal substructure is similar to the first approach.
\end{itemize}

\subsubsection*{Recursive Expression}
\begin{itemize}

%$$
% OPT[i] =
%    \begin{cases}
%      0 & i > \max\{days\}\\
%      \min\{ tickets[0] + OPT[i+1],tickets[1] + OPT[i+7] ,  tickets[2] + OPT[i+30]\} & i \leq \max\{days\}
%    \end{cases}
%$$

\item\textit{Approach 1}
$$
 OPT[i] =
    \begin{cases}
      0 & i > \max\{days\}\\
      \min
      \begin{cases}
      	tickets[0] + OPT[i+1], \\
      	tickets[1] + OPT[i+7] ,\\
      	tickets[2] + OPT[i+30]
      \end{cases}
       & i \leq \max\{days\}
    \end{cases}
$$

\item\textit{Approach 2}
$$
 OPT[i] =
    \begin{cases}
      0 & i > |days|\\
      \min
      \begin{cases}
      	tickets[0] + OPT[j1], \\
      	tickets[1] + OPT[j7] ,\\
      	tickets[2] + OPT[j30]
      \end{cases}
       & i \leq |days|
    \end{cases}
$$
where $j1$, $j7$, and $j30$ are the largest indices such that $days[j1] < days[i] + 1$, $days[j7] < days[i] + 7$, and $days[j30] < days[i] + 30$, respectively.

\end{itemize}

\subsubsection*{Pseudocode}
\begin{itemize}

\item \textit{The driver code for Top-Down algorithm}
\begin{center}
	\begin{minipage}{0.8\linewidth} % Adjust the minipage width to accomodate for the length of algorithm lines
	\scalebox{0.88}{
		\begin{algorithm}[H]
			\KwIn{$days = \{d_1, d_2, \ldots, d_n \}$, $tickets = \{t_1, t_2, t_3\}$}  % Algorithm inputs
			\KwResult{$minCost$ (the minimum amount to pay for MARTA passes)} % Algorithm outputs/results

			\bigskip

			$memo \leftarrow \{\}$\\
			$minCost \leftarrow \texttt{minCostRecur}(days, tickets, memo, 1)$

			\bigskip

			{\bf return} $minCost$
			\caption{\texttt{MinMARTACost}} % Algorithm name
			\label{alg1}   % optional label to refer to
		\end{algorithm}
		}
	\end{minipage}
\end{center}

\item \textit{Approach 1}
\begin{center}
	\begin{minipage}{0.8\linewidth} % Adjust the minipage width to accomodate for the length of algorithm lines
	\scalebox{0.88}{
		\begin{algorithm}[H]
			\KwIn{$days = \{d_1, d_2, \ldots, d_n \}$, $tickets = \{t_1, t_2, t_3\}, memo, i$}  % Algorithm inputs
			\KwResult{$minCost$ (the minimum amount to pay for MARTA passes starting from day $i$)} % Algorithm outputs/results

			\bigskip

			\If(\tcp*[h]{base case}) {($i >\max\{days\}$)}{
				{\bf return} $0$
			}

			\bigskip

			\If{($memo[i] \neq \phi $)}{

				{\bf return} $memo[i]$

			}

			\bigskip

			\uIf(\tcp*[h]{$i$ is in the commuting plan}) {($i \in days$)}{
				\begin{equation*}
				\begin{split}
					memo[i] = \min\{ & tickets[0] + \texttt{minCostRecur}(days, tickets, memo, i+1),\\
					 & tickets[1] + \texttt{minCostRecur}(days, tickets, memo, i+7),\\
					 & tickets[2] + \texttt{minCostRecur}(days, tickets, memo, i+30)\}
				\end{split}
				\end{equation*}
			} \Else{

				$memo[i] =\texttt{minCostRecur}(days, tickets, memo, i+1)$
			}

			\bigskip

			{\bf return} $memo[i]$
			\caption{\texttt{minCostRecur}} % Algorithm name
			\label{alg1-rec}   % optional label to refer to
		\end{algorithm}
	}
	\end{minipage}
\end{center}

\item \textit{Approach 2}
\begin{center}
	\begin{minipage}{0.8\linewidth} % Adjust the minipage width to accomodate for the length of algorithm lines
	\scalebox{0.90}{
		\begin{algorithm}[H]
			\KwIn{$days = \{d_1, d_2, \ldots, d_n \}$, $tickets = \{t_1, t_2, t_3\}, memo, i$}  % Algorithm inputs
			\KwResult{$minCost$ (the minimum amount to pay for MARTA passes starting from day $i$)} % Algorithm outputs/results

			\bigskip

			\If(\tcp*[h]{base case}) {($i > length(\{days\})$)}{
				{\bf return} $0$
			}

			\bigskip

			\If{($memo[i] \neq \phi $)}{

				{\bf return} $memo[i]$

			}

			\bigskip

%			$minFromToday \leftarrow \infty$
			\tcp{Finding the largest indices $j1$, $j7$, and $j30$}
			$j1 \leftarrow i$\\
			\lWhile {($j1 < |days|$ \textsf{\&\&} $days[j1] < days[i] + 1$)} {$j1++$}

			$j7 \leftarrow j1$\\
			\lWhile {($j7 < |days|$ \textsf{\&\&} $days[j7] < days[i] + 7$)} {$j7++$}

			$j30 \leftarrow j7$\\
			\lWhile {($j30 < |days|$ \textsf{\&\&} $days[j30] < days[i] + 30$)} {$j30++$}

			\begin{equation*}
			\begin{split}
				memo[i] = \min\{ & tickets[0] + \texttt{minCostRecur}(days, tickets, memo, j1),\\
				 & tickets[1] + \texttt{minCostRecur}(days, tickets, memo, j7),\\
				 & tickets[2] + \texttt{minCostRecur}(days, tickets, memo, j30)\}
			\end{split}
			\end{equation*}

			\bigskip

			{\bf return} $memo[i]$
			\caption{\texttt{minCostRecur}} % Algorithm name
			\label{alg2-rec}   % optional label to refer to
		\end{algorithm}
	}
	\end{minipage}
\end{center}
\end{itemize}

\subsubsection*{Time and Space Complexity}
In approach 1, the recursive calls is executed for the maximum number of days identifiers. For instance, if the commuting plan is written for one year, the algorithm will be executed 365 times. Therefore, the time complexity is $O(\max\{days\})$. The space complexity is the same as the time complexity due to the memory required for the memoization.

\noindent In approach 2, as explained earlier, the algorithm is executed for the number of days of the commuting plan. Therefore, both time and space complexity are $O(|days|)$.

\subsubsection*{Note}
In the presented solution, we solved the subproblems from the end of the commuting plan (the last day of the plan) to obtain the minimum cost estimation in the first day of the plan. Therefore, $OPT[i]$ is defined as the minimum amount of money George needs to pay for commuting from day $i$ to the end of the plan, and the minimum value for the entire plan is $OPT[1]$. It is also correct and accepted to start from the beginning of the plan and solve the subplroblems as the minimum amount of money spent on MARTA passes from the first day of the plan until day $i$. In this case, the recursive expression can be written as,

$$
 OPT[i] =
    \begin{cases}
      0 & i \leq 0\\
      \min
      \begin{cases}
      	tickets[0] + OPT[i-1], \\
      	tickets[1] + OPT[i-7] ,\\
      	tickets[2] + OPT[i-30]
      \end{cases}
       & 0 < i \leq \max\{days\}
    \end{cases}
$$
Thus, $OPT[\max\{days\}]$ gives the minimum amount of money spent on MARTA passes.

% -----------------------------------------------------------------------
\color{black}
\section{Frenemies [xx pts]}

Assume you are planning a dinner party and going to invite a set of friends. However, among them, there are some pairs of persons who are enemies. You need to create a seating plan and you are wondering if it is possible to arrange this set of $n$ friends of yours around a round table such that none of the two enemies will seat next to each other. Given the set of the $n$ friends and the set of the pairs of enemies, prove that this problem is NP-Complete. Remember to follow the steps from lecture to prove NP-completeness.

You can use the fact that \texttt{Hamiltonian Cycle (HC)} is NP-complete.

\color{blue}
\subsection*{Solution}
The three steps to prove that \texttt{Frenemies} is an NP-Complete problem. In this solution, we will show that the \texttt{Frenemies} problem is poly-time reducible from \texttt{Hamiltonian Cycle (HC)} problem.
\begin{itemize}
	\item \textbf{Step 1} \texttt{Frenemies} is in NP\\
	Certificate: A  potential solution can be represented as a set of friends names in which each two consecutive member of the set will sit next to each other. (Because the table is round the last member also sits next to the first person) \\
	Certifier: Trivially, we can check each two consecutive members to make sure that they are not a member of the set of enemies, and this can be done in polynomial time. In a naive implementation it takes $O(n^2)$ with two nested loops. In a better implementation, it can be done in $O(n)$ time using a hashset data structure.

	\item \textbf{Step 2} Choosing an NP-complete problem: \texttt{HC}

	\item \textbf{Step 3} Proving that $\texttt{HC} \leq_p \texttt{Frenemies}$:\\
	\begin{itemize}
		\item Let $I_1$ be an instance of \texttt{HC}. Then, $I_1$ is represented as a graph $G=(V,E)$. From this, we can build an instance $I_2$ of the \texttt{Frenemies} in which we define a person corresponding to each vertex of $I_1$. Two persons are enemies if and only if there are no edges in $E$ between the two corresponding vertices in $V$. We can show that there is Hamiltonian cycle in $I_1$ if and only if there is a valid sitting for friends of instance $I_2$.

		\item $(\Rightarrow)$ Assume there is a Hamiltonian cycle in $I_1$. Then, we arrange the friends in the order of their corresponding vertices along the Hamiltonian cycle. As there are no edge in $E$ between two vertices corresponding to two enemies, two vertices of $I_1$ corresponding to two enemies cannot be consecutive in the Hamiltonian cycle. Therefore, the arrangement if the friends is valid.

		\item$(\Leftarrow)$ Assume that there is a valid friends sitting for instance $I_2$. Then, the ordering of the friends sittings around the table defines a Hamiltonian cycle among the corresponding vertices. Thus, it will be a valid answer for instance $I_1$.

	\end{itemize}
\end{itemize}
% -----------------------------------------------------------------------
\newpage
\color{black}
\section*{EXAM PROBLEMS}
% -----------------------------------------------------------------------
\color{black}
% \newpage
\section{Dynamic Programming [xx pts]} % By Shahrokh Shahi
Let $A$ be the set of all integers in the range of $1$ to $n$. For each pair of $(i,j)$, a cost function $C[i,j] > 0$ is defined and the task is to find an increasing sequence of cutpoints $i_1, i_2, \ldots, i_k \in \{1,2,\ldots,n-1\}$ to minimize the total cost $\sum_{t=0}^k C[i_t + 1, i_{t+1}]$, where $i_0=0$ and $i_{k+1} = n$.

\begin{itemize}
\item[a)] (3 pts) Prove optimal substructure for this problem
\item[b)] (3 pts) Explain a Dynamic Programming algorithm to find the minimum cost of the segmentation and provide the recurrence (including the base case(s)).
\item[c)] (5 pts) Give the pseudocode of the bottom-up approach, and the back-tracing step to give start and end position of each sequence of numbers.
\item[d)] (2 pts) What are the time and space requirements in terms of $n$?
\end{itemize}

\subsection*{Solution}
\color{blue}
\begin{itemize}
\item[a)] Prove optimal substructure for this problem.
Let $i_1, i_2, \ldots, \i_k$ be the cutpoints of an optimal segmentation of range $[1,n]$. Now the cutpoints $i_1, i_2, \ldots, i_{k-1}$ form the optimal segmentation of range $[1,i_k]$. to prove the optimal substructure, assume this is not true, that is, there should be another increasing sequence of cutpoints $j_1,j_2, \ldots, j_h$ with less total cost for range $[1,i_k]$. But that implies we have
$\sum_{t=0}^h C[j_t + 1, j_{t+1}] + C[i_k + 1, n]  <  \sum_{t=0}^k C[i_t + 1, i_{t+1}]$, where $j_0 = 0$ and $j_{h+1} = i_k$. Thus, $i_1, i_2, \ldots, \i_k$ cannot be an optimal solution, which is a contraction.
\item[b)] (3 pts) The DP recurrence:\\
Let $OPT[j]$ be the (minimum) total cost of the optimum segmentation of range $[1,j]$. The cost of an optimal solution for a complete problem is $OPT[n]$. Because of the optimal substructure property, we can write the following recurrence:
$$
 OPT[j] =
    \begin{cases}
      0 & if\ j=0 \\
      \min\limits_{0 \leq i < j} \{ OPT[i] + C[i+1,j] \}  & if\ j>0
    \end{cases}
$$

Using this recurrence, it is easy to implement the DP algorithm, which computes $OPT[j]$ for $j=0,\ldots,n$ and a backtracking array to construct the optimal solution.

\item[c)] (5 pts) Give the pseudocode of the bottom-up approach.

\begin{center}
	\begin{minipage}{0.8\linewidth} % Adjust the minipage width to accomodate for the length of algorithm lines
	\scalebox{0.88}{
		\begin{algorithm}[H]
			\KwIn{$n, C_{n\times n}$}  % Algorithm inputs
			\KwResult{$minCost$ (the minimum total cost of the segmentation)} % Algorithm outputs/results

			\bigskip

			$opt[0:n] \leftarrow 0$\\
			$track[0:n] \leftarrow 0$\\

			\bigskip

			\For{$j=1$ to $n$ do}{
				$c \leftarrow \infty$\\

				\For{$i=1$ to $j-1$ do}{
					\If{$opt[i]+C[i+1,j] < c$}{
						$c \leftarrow opt[i]+C[i+1,j] $\\
						$track[j] \leftarrow i$
					}
				}
				$opt[j] \leftarrow c$
			}

			\bigskip
			$minCost \leftarrow opt[n]$
			\bigskip

			{\bf return} $minCost, track$
			\caption{\texttt{Optimal Segmentation}} % Algorithm name
			\label{alg1}   % optional label to refer to
		\end{algorithm}
		}
	\end{minipage}
\end{center}

Give the pseudocode of the back-tracing step to print the optimal solution.
\begin{center}
	\begin{minipage}{0.7\linewidth} % Adjust the minipage width to accomodate for the length of algorithm lines
		\begin{algorithm}[H]
			\KwIn{$n, track$}  % Algorithm inputs
%			\KwResult{} % Algorithm outputs/results

			\bigskip

			\While{$track[n] > 0$}{
				print $track[n]$\\
				$n \leftarrow track[n]$
			}

%			{\bf return} $minCost, track$
			\caption{\texttt{Print Optimal Solution}} % Algorithm name
			\label{alg1}   % optional label to refer to
		\end{algorithm}
	\end{minipage}
\end{center}

\item[d)] (2 pts) What are the time and space requirements in terms of $n$?

Time complexity: $O[n^2]$ (There are two nested loop in the implementation.)\\
Space complexity: $O[2n] = O[n]$.
\end{itemize}
% -----------------------------------------------------------------------
\color{black}
\newpage
\section{NP-Completeness -- Campus Tour[xx pts]} % Shahrokh Shahi (2020)
The office of Administration at Georgia Tech is planning for the Campus Visit and Tour Experience Program in which various Campus visit tours are offered to groups of prospective students and their families to become more familiar with the Georgia Tech buildings and amenities. Each tour starts from Georgia Tech Student Center and after visiting some (not necessarily all) buildings and landmarks, in a particular order, returns to the first point. Therefore, if we represent the map of the campus with a directed graph G, each node represents a building or landmark, and each edge shows a directed path from one point to another. Accordingly, the set of the various tours can be represented as a set $S = \{R_1, R_2, \ldots, R_n\}$, where $n$ is the number of the various tours and $R_i$ is the route (circuit) that the visitors pass along tour $i$, starting from and ending at the Student Center. Now the \texttt{Campus Tours} problem is defined as the following:\\

Given the Directed Graph $G=(V,E)$, the set of the campus tours $S = \{R_1, R_2, \ldots, R_n\}$, and an integer number $k$, is there a subset of $k$  tour(s) in $S$ such that a visitor group can be sure that they have seen all the buildings and landmarks of Georgia Tech?\\
(\textit{Note: It is possible to visit a node more than once during these $k$ tours.)}\\

\begin{itemize}
	\item[] Prove that the \texttt{Campus Tours} problem is NP Complete. Remember to follow the steps from lecture to prove NP-completeness. (\textit{Hint: You can use the fact that \texttt{Vertex Cover} problem is NP-complete.)}
\end{itemize}

\subsection*{Solution}
\color{blue}
%NOT COMPLETE YET -- Shahrokh\\

The three steps to prove that \texttt{Campus Tours} is an NP-Complete problem. In this solution, we will show that the \texttt{Campus Tours (CT)} problem is poly-time reducible from \texttt{Vertex Cover (VC)} problem.
\begin{itemize}
	\item \textbf{Step 1} \texttt{Campus Tours} is in NP. The certificate comprises $k$ routes that between them include all the vertices in the graph. We can verify a certificate by checking that each route is indeed a path from the first node $s$ and back to $s$, and via a sort, that the collection of vertices in these path includes all the vertices in $G$. Clearly, this verification takes polynomial time in the size of the given graph $G$ and the certificate, and further the certificate has size at most $kn$, where $n$ is the number of vertices in $G$.

	\item \textbf{Step 2} Choosing an NP-complete problem: \texttt{VC}

	\item \textbf{Step 3} Proving that $\texttt{VC} \leq_p \texttt{CT}$:\\
	\begin{itemize}
		\item Let $I_1 = (G=(V,E), k)$ be an instance of \texttt{VC} with $k \leq |V|$. Then we can build an instance $I_2 = (G', S, k')$ of the \texttt{Campus Tours} such that $Sol(I_1) \Leftrightarrow Sol(I_2)$. \\
		Recall $G=(V,E)$. $G'$ has the set of vertices $\{s\} \cup V_E$ where $V_E$ has a vertex $v_e$ for each edge $e \in E$, and $s$ is an additional vertex not in $V_E$. We call $V_E$ the set of edge vertices. For simplicity, we make $G'$ a complete graph, i.e. every possible edge in both directions. Afterwards, for each vertex $u \in V$, we create a route $R_u$ in $S$. This route starts and ends at $s$ and includes in addition exactly those vertices $v_e$ for which $e$ is incident on $u$. In this way, the order is the same as the order in the adjacency list of graph $G$. Finally, we set $k'=k$. Clearly, this instance of \texttt{Campus Tours} can be constructed in polynomial time.

		\item $(\Rightarrow)$ Now, if $G$ has a vertex cover $\{u_1, u_2, \ldots, u_k\}$, then the routes that solve the \texttt{Campus Tours} instance are $R_{u_1}, R_{u_2}, \ldots, R_{u_k} $ which all of them include $s$ plus all vertices corresponding to edges covered by $u_1, u_2, \ldots, u_k$, i.e. $\{s\} \cup V_E$.

		\item$(\Leftarrow)$ If the \texttt{Campus Tours} instance can be solved by $R_{u_1}, R_{u_2}, \ldots, R_{u_k}$, then these routes include all the edge vertices in $V_E$, and correspondingly $\{u_1, u_2, \ldots, u_k\}$ forms a Vertex Cover of $G$.

		This completes the proof. \\
		Therefore $I_1 = (G,k) \in \texttt{Vertex Cover} \Leftrightarrow I_2 = (G',S,k) \in \texttt{Campus Tours}$ and \texttt{Campus Tours} is and NP-Complete problem.

	\end{itemize}
\end{itemize}

% ------------------------------------------------------------------ %
\color{black}
\newpage
\section{Dr. Frankenstein [xx pts]}  %  Shahrokh Shahi (2020)
Let $X = \{x_1, x_2, \ldots, x_m\}$ and $Y = \{y_1, y_2, \ldots, y_n\}$ be two genomic sequence represented by strings of letters and $C[i,j]$ be the cost function defined on pairs of letters, one from $X$ and one from $Y$.
Dr. Frankenstein is trying to merge these two sequences to create a new creature with genomic sequence $Z$ and he needs to find the cheapest merge of $X$ and $Y$, while maintaining the order of letters from both $X$ and $Y$. Therefore, for instance, if $X = \{a,b,a,c\}$ and $Y=\{d,a,e,b\}$, then $Z = \{ a,d,a,b,a,e,c,b \}$ and $Z = \{a,d,a,e,b,b,a,c \}$ are valid merges, but $Z=\{a,b,a,e,c,d,a,b\}$ is not because $e$ from the second sequence is used before $d$ and $a$.\\
Total cost of the merge is the sum of the merging costs of the adjacent letters from different sequence. So if $Z$ includes $x_i$ and $x_{i+1}$ as consecutive letters, there is no cost, but if $Z$ has $x_sy_t$ as consecutive letters, then there is a cost $C[s,t]$ and it should be added to the total cost of the merge.\\
\textit{Note: You can assume that the cost function is symmetric}.

\begin{itemize}
	\item[] (4 pts) Give a Dynamic Programming algorithm to find the minimum cost of merging $X$ and $Y$.  A high level recursive relation (including the base case) will suffice. Pseudocode is not required.
\end{itemize}

\subsection*{Solution}
\color{blue}
Let $Merge1[i,j]$ be the min cost of merging the first $i$ letters in $X$ with the first $j$ letters in $Y$ with $x_i$ as the last letter in the merged sequence $Z$.\\
Similarly, let $Merge2[i,j]$ be the min cost of merging the first $i$ letters in $X$ with the first $j$ letters in $Y$ with $y_j$ as the last letter in the merged sequence $Z$. \\
The recursion relation is as follows:

$$
 Merge1[i,j] =
    \begin{cases}
      0 & i =0\ or\ j = 0\\
      \min
      \begin{cases}
      	Merge1[i-1,j] \\
      	Merge2[i-1,j]+ C[j,i]
      \end{cases}
       & otherwise
    \end{cases}
$$

$$
 Merge2[i,j] =
    \begin{cases}
      0 & i =0\ or\ j = 0\\
      \min
      \begin{cases}
      	Merge1[i,j-1]+ C[i,j], \\
      	Merge2[i,j-1]
      \end{cases}
       & otherwise
    \end{cases}
$$

Accordingly, the minimum cost of merging two genemic sequence is $\min\{Merge1[m,n],Merge2[m,n]\}$

% -----------------------------------------------------------------------
\color{black}
\newpage
\section{Branch and Bound [xx pts]}
\begin{itemize}
\item[(a)] (3 pts) In general, what are the three different orders of exploring the search tree during the Branch-and-Bound algorithm?  For each order, what kind of data structure should be used for the Queue $F$ of partial solutions to keep the order for exploring the search tree?
%\item[(b)] (2 pts) In a Branch-and-Bound algorithm, what is the purpose of each of these components in \underline{"minimization"} and \underline{"maximization"} problems?
%
%$\boxed{\texttt{Choose}(F)}$,
%$\boxed{\texttt{Expand}(X_i, Y_i)}$,
%$\boxed{\texttt{lower\_bound}(X_i)}$,
%$\boxed{\texttt{upper\_bound}(X_i)}$
\end{itemize}

\vspace{5pt}
\noindent Let $\Phi = C_1 \wedge \ldots \wedge C_p $ be a boolean formula with $p$ clauses and $n$ boolean variables $x_1,\ldots, x_n$, where each clause $C_i$ has the following form,
$C_i = (x_{j_1} \vee \ldots \vee x_{j_n} )$, $1 \leq j_k \leq n$. The optimization version of $\texttt{SAT}$ problem aims at finding the optimum assignment of boolean variables such that the number of satisfied clauses is maximized. Consider the following boolean formula with 10 clauses, and answer parts (b)-(d):
$$ \boxed{
(x_1  \vee  \overline{x_2}) \wedge
(x_1  \vee x_3  \vee \overline{x_4}) \wedge
(\overline{x_1}  \vee x_2) \wedge
(x_1  \vee \overline{x_3}  \vee  x_4) \wedge
(x_2  \vee x_3  \vee  \overline{x_4}) \wedge
(x_1  \vee \overline{x_3}  \vee  \overline{x_4}) \wedge
x_3 \wedge
(x_1  \vee  x_4) \wedge
(\overline{x_1}  \vee  \overline{x_3}) \wedge
x_1
}$$

Note that this is a maximization problem, whereas when we introduced Branch-and-Bound algorithms in class we assumed we were working with minimization problems. To solve the following questions, you can either convert this problem into a minimization problem, or answer the questions in the context of a maximization problem.
\begin{itemize}
\item[(b)] (3 pts) Draw the complete search tree for assignments of $x_i=0/1$, $i \in \{ 1,2,3,4 \}$ and find all possible solutions (which is the number of satisfied clauses) of the given boolean formula. Note that here we are not asking you to perform bounding or pruning operations. The tree you will draw will correspond to an exhaustive search of all assignments. When choosing a variable to expand, please use the order $x_1, x_2, x_3, x_4$. That is, you first consider the values of $x_1$, then $x_2$, etc. And please use the the left subtree to represent $x_i=1$, and right subtree for $x_i=0$.

\item[(c)] (1 pts) Can this formula be satisfied? What is the maximum number of satisfied clause(s)?

\item[(d)] (3 pts) When a feasible solution is found, how the branches of this search tree can be pruned? Describe the steps of the corresponding Branch-and-Bound algorithm if the tree is explored by depth-first-search approach. You can use the tree you draw for subquestion (b). It is not required to find the best order of $x_i$ for expanding. You can describe in words that during your DFS search, which subtrees will be pruned and why. You can mark the pruned subtrees on the tree you draw in (b).

%\item[(e)] (3 pts) Prove that the following algorithm is a $\frac{1}{2}$-approximation algorithm for finding the maximum number of satisfied clauses:
%\begin{center}
%	\begin{minipage}{0.8\linewidth} % Adjust the minipage width to accomodate for the length of algorithm lines
%	\scalebox{0.88}{
%		\begin{algorithm}[H]
%			%\KwIn{}  % Algorithm inputs
%			%\KwResult{} % Algorithm outputs/results
%
%			\bigskip
%
%			\For{$i=1$ to $n$}{
%				\uIf{($x_i$ appears in more clauses than $\bar{x_i}$)}{
%					Set $x_i = 1$
%				}\Else{
%					Set $x_i = 0$
%				}
%				Remove all satisfied clauses from $\Phi$ and remove $x_i$ and $\bar{x_i}$ from all other clauses
%			}
%
%			\bigskip
%
%		%	{\bf return} $minCost$
%			\caption{\texttt{Approximation Algorithm for Maximum SAT}} % Algorithm name
%			\label{alg1}   % optional label to refer to
%		\end{algorithm}
%		}
%	\end{minipage}
%\end{center}

\end{itemize}

%\newpage
\color{blue}
\subsection*{Solution}
\begin{itemize}
\item[(a)] \ \\
– Depth-first search using a stack (LIFO)\\
– Breadth-first search using a queue (FIFO)\\
– Best-first search using a priority queue\\
%\item[(b)] \ \\
%\begin{itemize}
%\item Both minimization and maximization problem:\\
%$\boxed{\texttt{Choose}(F)}$: chooses the most promising partial solution to expand next from the queue $F$.\\
%$\boxed{\texttt{Expand}(X_i, Y_i)}$: expands the partial solution $X_i$ by setting values to variables that are not already set and adding them to $X_i$, creating new partial solutions.\\
%
%\item Just in minimization problems:\\
%$\boxed{\texttt{lower\_bound}(X_i)}$: computes a lower bound on the cost of the partial solution $X_i$ according to some bounding function. The lower bound is then compared against the global upper bound $B$, and $X_i$ is pruned if $lb(X_i) > B$.
%
%\item Just in maximization problems:\\
%$\boxed{\texttt{upper\_bound}(X_i)}$: computes an upper bound on the cost of the partial solution $X_i$ according to some bounding function. The upper bound is then compared against the global lower bound $B$, and $X_i$ is pruned if $ub(X_i) < B$.
%\end{itemize}

%\item[(c)]  Three outcomes of $\boxed{\texttt{Check}(X_i, Y_i)}$:\\

\item[(b)]
\begin{figure}[h]
    \centering
	\includegraphics[width=0.55\textwidth]{SearchTree.png}
	\caption{Part (b)}
\end{figure}

\item[(c)] No, it cannot be satisfied. The maximum number of satisfied clauses is 9.

\item[(d)] Once a partial assignment is found, the idea is to check how many clauses cannot be satisfied and to cut the branches once a solution with more clauses is found. The algorithm explores the tree with a DFS strategy and obtains the first solution with $x_i = 1,~ \forall i$, for which $n$ clauses cannot be satisfied. In the first branch ($ x_1 = x_2 = x_3 = x_4 = 1$), 9 clauses are satisfied, so $n = 1$ .
Pursuing with DFS approach, we cut branches that cannot lead to a better solution, i.e., partial assignments with at least $n$ clauses that cannot be satisfied. If a better solution is found, we update $n$ before continuing the tree exploration.
With the first branch explored, for the partial assignment $x1 = x2 = x3 = 1$, clause $\overline{x_1} \vee  \overline{x_3}$ cannot be satisfied by any choice of the variable $x_4$, and therefore, we do not explore the branch $x_4 = 0$. Then, for the partial assignment $x_1 = x_2 = 1$, all clauses can be satisfied; we explore the branch with $x_3 = 0$, but then clause $x_3$ cannot be satisfied and we do not go further. Similarly, we stop the exploration at $x1 = 1$ and $x2 = 0$ because clause $\overline{x_1} \vee x_2$ cannot be satisfied, and then we stop at $x_1 = 0$ because clause $x1$ cannot be satisfied.

%\item[(e)] Let $t_i$ be the number of satisfied clauses when we set the values of $x_i$, and $f_i$ be the number of false clauses at the same time (all the terms of the clause must be already set to false). It is clear that the algorithm guarantees $t_i \ge f_i$.
%
%On the other hand, after the algorithms terminates, $\sum t_i$ is the total number of satisfied clauses in the formula and $\sum f_i$ is the total number of false clauses. Therefore,  $\sum t_i \geq \sum f_i$ which means at least half of the clauses are satisfied, i.e. the algorithm is $\frac{1}{2}$-approximation.
\end{itemize}
% -----------------------------------------------------------------------
\end{document}


\section{What's going on?}
We get a lot of mileage by looking at the following correspondence.
\begin{mdframed}
  \adjustbox{scale=1,center}{%
    \begin{tikzcd}
      \set{\mbox{2-cocycles}}\ar[rrr, dashed, leftrightarrow]& & & \set{\mbox{Group extensions}}
    \end{tikzcd}
  }
\end{mdframed}
There is no direct way to go from a group extension to a 2-cocycle, we need to first write the group $G$ as 2-digit numbers $\tens{a}\units{b}$ where $a \in H$ and $b \in K$.
The 2-cocycle then arises as the carry function.

This correspondence is not a one-to-one correspondence. There are many more cocycles than there are group extensions.
To get a 1-1 correspondence, we need to take the quotient of the left-hand side by the set of 2-coboundaries and the right-hand side by isomorphisms.
\begin{mdframed}
  \adjustbox{scale=1,center}{%
    \begin{tikzcd}
      \dfrac{\set{\mbox{2-cocycles}}}{\set{\mbox{2-coboundaries}}}
      \ar[rrr, dashed, leftrightarrow, "1-1"]
      & & &
      \dfrac{\set{\mbox{Group extensions}}}{\set{\mbox{isomorphisms}}} \\
      \dfrac{\calz^2(K;H)}{\calb^2(K;H)} \ar[u, equal]
      & & & \\
      \calh^2(K;H)
        \ar[u, equal]
        \ar[rrr, leftrightarrow, "1-1"]
      & & &
      \ext^1(K;H) \ar[uu, equal]
    \end{tikzcd}
  }
\end{mdframed}
Let us try to understand this correspondence by an example.

\subsection{Example}
Consider the case that we started with $H = K = \bbz/10$ so that the group extensions are groups consisting of 2-digit numbers and hence have sizes $100$.
Let $c_k = k \left \lfloor \dfrac{b_1 + b_2}{10}  \right \rfloor$ be the standard carry function multiplied by $k$. By repeatedly adding $\tens{0}\units{1}$ to itself, one can easily find the isomorphism types of the resulting group extensions.
\begin{center}
  \begin{tabular}{|r|l|}
    \hline
    carry  & isomorphism\\
    function & class \\\hline
    $c_1$ & $\bbz/100$ \\
    $c_2$ & $\bbz/50 \times \bbz/2$ \\
    $c_3$ & $\bbz/100$ \\
    $c_4$ & $\bbz/50 \times \bbz/2$ \\
    $c_5$ & $\bbz/20 \times \bbz/5$ \\
    $c_6$ & $\bbz/50 \times \bbz/2$ \\
    $c_7$ & $\bbz/100$ \\
    $c_8$ & $\bbz/50 \times \bbz/2$ \\
    $c_9$ & $\bbz/100$ \\
    $c_0$ & $\bbz/10 \times \bbz/10$ \\\hline
  \end{tabular}
\end{center}
Turns out, these are all the group extensions upto isomorphism.
The following is a theorem from algebraic topology,
\begin{theorem}
  The second group cohomology of $\bbz/10$ with coefficients in $\bbz/10$ is given by \label{theorem:cohomologyOfZ10}
    \begin{align*}
      H^2(\bbz/10; \bbz/10) \cong \bbz/10.
    \end{align*}
    A generator for this cohomology group is given by $c_1$.
\end{theorem}
We can use this theorem to make rapid computions of isomorphism classes.
\begin{proposition}
  The group extension corresponding to the carry function $d(b_1, b_2) = 2 b_1 b_2$ is given by $\bbz/10 \times \bbz/10$.
\end{proposition}
\begin{proof}
  We know that $c_0$ gives rise to the group extension $\bbz/10 \times \bbz/10$.
  It suffices to show that there exists a 2-coboundary $\alpha:\bbz/10 \rightarrow \bbz/10$ such that
  \begin{align*}
    d(b_1) - c_0(b_1) &= \alpha(b_1) + \alpha(b_2) - \alpha(b_1 + b_2) \\
    \Longleftrightarrow 2 b_1 b_2 &= \alpha(b_1) + \alpha(b_2) - \alpha(b_1 + b_2)
  \end{align*}
  $\alpha(x) = -x^2$ works.
\end{proof}
\begin{proposition}
  The group extension corresponding to the carry function $e(b_1, b_2) = b_1 b_2$ is either $\bbz/20 \times \bbz/5$ or $\bbz/10 \times \bbz/10$.
\end{proposition}
\begin{proof}
  This is because $d = 2 e$ where $d$ is as in the previous proposition.
  And $c_0 = 2 c_5$ and $c_0 = 2 c_0$. Hence, $e$ must correspond to either $\bbz/20 \times \bbz/5$ or $\bbz/10 \times \bbz/10$.
\end{proof}
Thus cohomology allows us to reduce questions about reduce questions about isomorphism classes of group extensions to solving identities among functions.
However, many a times it only provides partial information and we still need to put in more effort to find the final answer.

\subsection{Group cohomology}
  For each positive integer $n$, let $\calc^n(K;H)$ be the set of functions from $K^{\times n}$ to $H$.
  \begin{align*}
    \calc^n(K;H) = \{ \underbrace{K \times \dots \times K}_{n-\text{times}} \rightarrow H \}
  \end{align*}

  There is a differential function $d^n: \calc^n(K;H) \rightarrow \calc^{n+1}(K;H)$ which takes a function that has $n$ inputs and produces a function that has $n+1$ inputs, defined as follows
  \begin{align*}
    (d^n \varphi) (k_1, k_2, k_3, \dots, k_n, k_{n+1})
    &:=
    \varphi (k_2, k_3, \dots, k_n, k_{n+1}) \\
    & -
    \varphi (k_1 + k_2, k_3, \dots, k_n, k_{n+1}) \\
    & +
    \varphi (k_1, k_2 + k_3, \dots, k_n, k_{n+1}) \\
    & \mp \dots \pm \\
    & (-1)^n \varphi (k_1, k_2 + k_3, \dots, k_n + k_{n+1}) \\
    & (-1)^{n+1} \varphi (k_1, k_2 + k_3, \dots, k_n) \\
  \end{align*}
  where $\varphi$ is a function $K^{\times n} \rightarrow H$.

  \begin{ex}
    For $c:K \times K \rightarrow H$
    \begin{align*}
      (d^2 \varphi) (k_1, k_2, k_3)
      &=
      \varphi(k_2, k_3)
      -
      \varphi(k_1 + k_2, k_3)
      +
      \varphi(k_1, k_2 + k_3)
      -
      \varphi(k_1, k_2).
    \end{align*}
  \end{ex}
  \begin{ex}
    For $\alpha: K \rightarrow H$
    \begin{align*}
      (d^1 \alpha)(k_1, k_2)
      &=
      \alpha(k_1) - \alpha(k_1 + k_2) + \alpha(k_2)
    \end{align*}
  \end{ex}

  From the above examples, one can see that $c$ is a 2-cocycle precisely when $d^2 c = 0$ and $c$ is a 2-coboundary when $c = d^1 \alpha$ for some $\alpha$.

  \begin{definition}
    The set of $n$-cocycles is the kernel of $ d^n$ and the set of $n$-coboundaries is the image of $d^{n-1}$.
    \begin{align*}
      \calz^n(K;H) := \set{n-\mbox{cocycles}} &= \ker d^n \\
      \calb^n(K;H) := \set{(n-1)-\mbox{cocycles}} &= \im d^{n-1}
    \end{align*}
    The $n^{th}$ group cohomology group of the group $K$ with coefficients in $H$ is the quotient
    \begin{align*}
      \calh^n(K;H) := \calz^n(K;H) / \calb^n(K;H) = \ker d^n / \im d^{n-1}
    \end{align*}
  \end{definition}

  The other group cohomologies also have different meanings.
  \begin{ex}
    A 1-coboundary is a map $\alpha: K \rightarrow H$ such that $d^1 \alpha = 0$.
    Expanding this out, we get
    \begin{align*}
      \alpha(k_1) - \alpha(k_1 + k_2) + \alpha(k_2) &= 0 \\
      \alpha(k_1) + \alpha(k_2) &= \alpha(k_1 + k_2)
    \end{align*}
    But this means that $\alpha$ is a group homomorphism!
    Thus, 1-cocycles are exactly group homomorphisms.
    Further, there are no 1-coboundaries and hence
    \begin{align*}
      \calh^1(K;H) := \{ \mbox{group homomorphisms }K \rightarrow H \}
    \end{align*}
  \end{ex}

  Thus we have
  \begin{align*}
    \mbox{first group cohomology} & \longleftrightarrow \mbox{ group homomorphisms }\\
    \mbox{second group cohomology} & \longleftrightarrow \mbox{ group extensions }\\ \mbox{higher group cohomology} & \longleftrightarrow \mbox{ ?? }
  \end{align*}

What the higher group cohomologies mean is a very subtle question and computing and studying this forms a branch of mathematics called \emph{homological algebra}.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS671: Machine Learning
% Copyright 2015 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/beacon
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 3}

Consider the hypothesis family of sin functions of the form $f_\omega(x) = \sin \omega x$.
These functions can be used to classify the points in $\mathbb{R}$ as follows.
A point is labeled as positive if it is above the curve, and negative otherwise.
\begin{enumerate}[label=(\alph*)]
\item For $m > 0$, consider the set of points $S = \{x_1, x_2, ..., x_m\}$ with arbitrary labels $y_1, y_2, ... y_m \in \{-1, 1\}$.
A subset of $S$ is defined by a choice of the parameters $y_i$ and it consists of those $x_i$ such that $y_i = 1$.
Define
\begin{equation}\label{eq31}
\omega = \pi (1 + \sum_{i=1}^{m} 2^{i}y_{i}^{\prime})
\end{equation}
where $y_i^\prime = \frac{1-y}{2}$.
Prove that with this choice of $\omega$ the set $S$ is shattered, that is, for every subset $T$ of $S$ there would be an $\omega$ such that $T$ equals the set of positive examples.

\item What is the Vapnik-Chervonenkis dimension of this classifier?
\end{enumerate}

\subsection*{Solution}

\begin{enumerate}
\item As $S$ is defined based on $y_i$ which in turn depend on $x_i$ based on the function $F_w(x)$, we can simplify the problem by using the function.
By substituting $y_i^\prime$ in Equation \ref{eq31}, we'll get

\begin{equation}\label{eq32}
S = \{2 ^ {-j} | 1 \leq j \leq m\}
\end{equation}

Since we can choose $w$ as we like, we can move any arbitrary point that we choose to below or above the sine curve.
Therefore, the set $S$ can be shattered for all subsets of $S$ with arbitrary size.

\item The class of $f_w(x)$ functions can shatter any arbitrary number of points.
Therefore, Vapnik-Chervonenkis Dimension for this class is infinite.
\end{enumerate}


\chapter{Linear time invariant DT systems}

\section{DT system representations}

We can mathematically represent, or model, DT systems multiple ways.

\begin{itemize}
\item purely mathematically - in time domain we will use

  \begin{itemize}
  \item linear, constant coefficient difference equations, e.g.
    \[
    y[n] = a y[n-1] + b y[n-2] + x[n]
    \]
  \item DT impulse response $h[n]$
  \end{itemize}
\item purely mathematically - in frequency domain we will use
  \begin{itemize}
  \item frequency response
  \item transfer function (complex frequency, covered in ECE 3704)
  \end{itemize}
\item graphically, using a mixture of math and block diagrams
\end{itemize}

\section{System properties and classification}

Choosing the right kind of system model is important. Here are some important properties that allow us to broadly classify systems.
\begin{itemize}
\item Memory
\item Invertability
\item Causality
\item Stability
\item Time-invariance
\item Linearity
\end{itemize}

Let's define each it turn.

\subsection{Memory}
The output of a DT system with memory depends on previous or future inputs and is said to be {\it dynamic}. Otherwise the system is memoryless or {\it instantaneous}, and the output $y[n]$ at index $n$ depends only on $x[n]$.
For example:
\[
y[n] = 2x[n]
\]
is a memoryless system, while
\[
y[n+1] + y[n] = x[n]
\]
has memory. To  see this, write the difference equation in recursive form
\[
y[n] = -y[n-1] + x[n-1]
\]
and we see explicitly the current output $y[n]$ depends on past values of output and input.

\subsection{Invertability}

A system is invertible if there exists a system that when placed in series with the original recovers the input.
\[
x[n] \mapsto{T} y[n] \mapsto{T^{-1}} x[n]
\]
where $T^{-1}$ is the inverse system of $T$. For example, consider a system
\[
x[n] \mapsto y[n] = \sum\limits_{m=-\infty}^{n} x[m]
\]
and a system
\[
y[n] \mapsto z[n] = y[n] - y[n-1]
\]
The combination in series $x[n] \mapsto y[n] \mapsto z[n] = x[n]$, since
\[
z[n] = y[n] - y[n-1] = \sum\limits_{m=-\infty}^{n} x[m] - \sum\limits_{m=-\infty}^{n-1} x[m] = x[n]
\]
i.e. the difference undoes the accumulation.

\subsection{Causality}
A DT system is causal if the output at index $n$ depends on the input for index values at or before $n$:
\[
y[n] \;\text{depends on}\; x[m] \;\text{for} \; m \leq n
\]
While all physical CT systems are causal, practical DT systems may not be since we can used memory to "shift time". For CT systems we cannot store the infinite number of values between two time points $t_1$ and $t_2$, but we can store the $n_2-n_1$ values of a DT system between between two indices $n_1$ and $n_2$ (assuming infinite precision).

\begin{example}
Consider a DT system whose difference equation is
\[
y[n] = -x[n-1] + 2x[n] - x[n+1]
\]
We see the current output $y[n]$ depends on a "future" value of the input $x[n+1]$. Thus the system \textbf{is not} causal. In practice we can shift the difference equation to
\[
y[n-1] = -x[n-2] + 2x[n-1] - x[n]
\]
and then delay the output by one sample to get $y[n]$.
\end{example}

\begin{example}
Consider a DT system whose difference equation is
\[
y[n] = -y[n-1] + 2x[n]
\]
We see the current output $y[n]$ depends on a "past" value of the output $y[n-1]$ and the current input $x[n]$. Thus the system \textbf{is} causal. In practice we can immediately compute $y[n]$ with no delay.
\end{example}

\subsection{Stability}

A DT system is (BIBO) stable if applying a bounded-input
\[
x[n] < \infty \; \forall \; n
\]
results in a bounded-output $x[n] \mapsto y[n]$ and
\[
y[n] < \infty \; \forall \; n
\]
Note, bounded in practice is limited by the physical situation, e.g. the number of bits used to store values.

For example, a DT system described by the LCCDE
\[
y[n+1] - 2 y[n] = x[n+1]
\]
is unstable because the solution $y[n]$ will have one term of the form $\left( 2\right)^n$, for most non-zero inputs $x[n]$ or any non-zero initial condition, that grows unbounded as $n$ increases.

\subsection{Time-invariance}
A DT system is time(index)-invariant if, given
\[
x[n] \mapsto y[n]
\]
then an index-shift of the input leads to the same index-shift in the output
\[
x[n-m] \mapsto y[n-m]
\]

An important example is a DT system described by a LCCDE, e.g.
\[
y[n+1] - \frac{1}{2} y[n] = x[n+1]
\]
or in recursive form
\[
y[n] = \frac{1}{2} y[n-1] + x[n]
\]

If we index shift the input $x[n - m]$ we replace $n$ by $n-m$ and the difference equation becomes
\[
y[n-m+1] - \frac{1}{2} y[n-m] = x[n-m+1]
\]
which has the same solution shifted by $m$
\[
y[n-m] = \frac{1}{2} y[n-m -1] + x[n-m]
\]

If a coefficient depends on $n$ however, e.g
\[
y[n+1] - \frac{n}{2} y[n] = x[n+1]
\]
so that it is no longer LCC then the solution depends on $m$ and the system is no longer time-invariant.

\subsection{Linearity}

A DT system is linear if the output due to a sum of scaled individual inputs is the same as the scaled sum of the individual outputs with respect to those inputs. In other words given
\[
x_1[n] \mapsto y_1[n] \;\text{and}\; x_2[n] \mapsto y_2[n]
\]
then
\[
a x_1[n] + b x_2[n] \mapsto a y_1[n] + b y_2[n]
\]
for constants $a$ and $b$.
Note this property extends to sums of arbitrary signals, e.g. if
\[
x_i[n] \mapsto y_i[n] \; \forall\; i \in [1 \cdots N]
\]
then given $N$ constants $a_i$, if the system is linear
\[
\sum\limits_{i = 1}^N a_i x_i[n] \mapsto \sum\limits_{i = 1}^N a_i y_i[n]
\]
This is a very important property, called {\it superposition}, and it simplifies the analysis of systems greatly.

An important non-linear system is that is described by a LCCDE with non-zero auxiliary conditions at some $n_0$, $y[n_0] = y_0$. As in CT, such systems will have a term in it's solution that depends on $y_0$. Given two inputs, each individual response will have that term in it, so their sum has double that term. However the response due to the sum of the inputs would again only have one and the sum of the responses would not be the same as the response of the sum. Such a system cannot be linear. Thus the system must be "at rest" before applying the input in order to be a linear system.

\section{Stable LTI Systems}

The remainder of this course is about stable, linear, time-invariant (LTI) systems. As we have seen in DT such systems can be described by a LCCDE with zero auxiliary (initial) conditions (the system is \emph{at rest}).

We have seen previously how to find the impulse response, $h[n]$, of such systems. We now note some relationships between the impulse response and the system properties described above.

\begin{itemize}
\item If a system is memoryless then $h[n] = C \delta[n]$ for some constant $C$.
\item If a system is causal then  $h[n] = 0$ for $n < 0$.
\item If a system is BIBO stable then
  \[
  \sum\limits_{-\infty}^{\infty} |h[n]| < \infty
  \]
\end{itemize}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS622: Theory of Formal Languages
% Copyright 2014 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/beacon
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 6}

Let $A$ be an alphabet and let $a \in A$.

\begin{enumerate}[label=(\alph*)]

	\item
	Prove that $a^{-1}A^* = a^{-1}A^+ = A^*$.

	\item
	Prove that $a^{-1}A^n = A^{n-1}$.

\end{enumerate}

\subsection*{Solution}

\begin{enumerate}[label=(\alph*)]

\item
It is assumed that $z \in a^{-1}A^*$.
It is shown that $z \in a^{-1}A^*$ and $z \in A*$.
To change inclusion to equality, it is also shown for $z\in A*$ that $z \in a^{-1}A^*$.

\begin{equation}\label{6eq1}
z \in a^{-1}A^* \Rightarrow az \in A^*
\end{equation}

\begin{equation}\label{6eq2}
A^* = A^+ \cup \lambda
\end{equation}

Substituting \eqref{6eq2} in \eqref{6eq1},

\begin{equation}
az \in A^+ \cup \lambda \label{6eq3}
\end{equation}

\begin{equation}
a \in az \Rightarrow az \neq \lambda \Rightarrow az \notin \lambda \label{6eq4}
\end{equation}

Using \eqref{6eq4} in \eqref{6eq3}

\begin{equation}
az \in A^+ \Rightarrow z \in a^{-1}A^+
\end{equation}

Thus $a^{-1}A^*\subseteq a^{-1}A^+$ is proven so far.

\begin{equation}\label{6eq6}
z \in a^{-1}A^+ \Rightarrow az \in A^+
\end{equation}

\begin{equation}\label{6eq7}
a \in A \Rightarrow a \in A^+
\end{equation}

Using \eqref{6eq7} in \eqref{6eq6} concludes

\begin{equation}
z \in A^+ \Rightarrow z \in A^*
\end{equation}

Thus $a^{-1}A^*\subseteq a^{-1}A^+ \subseteq A^*$ is proven so far.

\begin{equation}
z \in A^* \Rightarrow az \in A^* \Rightarrow z \in a^{-1}A^*
\end{equation}

Which shows that $A^* \subseteq a^{-1}A^*$. Therefore

\begin{equation}
a^{-1}A^* = a^{-1}A^+ = A* \nonumber
\end{equation}

\item
Statement is proven by induction on \textit{n}.

Initial step: let $n = 1: a^{-1}A = \emptyset $.

\begin{equation}
z \in a^{-1}A \rightarrow az \in A
\end{equation}

As $A$ is the alphabet, any element in $A$ is a symbol.

\begin{equation}
|az| = 1 \Rightarrow |a| + |z| = 1 \Rightarrow |z| = 0 \Rightarrow z = \lambda \in \emptyset
\end{equation}

Thus, $a^{-1}A \subseteq \emptyset$.

\begin{equation}
z \in \emptyset \Rightarrow z = \lambda
\end{equation}

Since $a \in A$,

\begin{equation}
az = a\lambda \in A \Rightarrow z \in a^{-1}A
\end{equation}

Thus $\emptyset \subseteq a^{-1}A$ and initial induction step is proven.
$$ a^{-1}A = \emptyset $$
Assumption step:

\begin{equation}\label{6eq14}
a^{-1}A^n = A^{n-1}
\end{equation}

Induction step: $a^{-1}A^{n+1} = A^n$.

\begin{equation}\label{6eq15}
z \in a^{-1}A^n+1 \Rightarrow z \in a^{-1}A^nA
\end{equation}

Substituting \eqref{6eq14} in \eqref{6eq15},

\begin{equation}
z \in a^{-1}A^nA \Rightarrow z \in A^{n-1}A \Rightarrow z \in A^n
\end{equation}

Thus $a^{-1}A{n+1}\subseteq A^n$. Similarly,

\begin{equation}
z \in A^n \Rightarrow z \in A^{n-1}A
\end{equation}

From \eqref{6eq14},

\begin{equation}
z \in a^{-1}A^nA \Rightarrow z \in a^{-1}A^{n+1}
\end{equation}

Therefore, $a^{-1}A^{n+1}=A^n$.
And the statement is proven by induction.

\end{enumerate}


\section{Oriented PCA}

\begin{align*}
	\mb{u} = \mb{A x}
\end{align*}
with $\mb{A}^{-1}$ exists and %and $\mb{A}$ is a whitening matrix.
\begin{align*}
	\mb{y} = \mb{w\TT x}
\end{align*}
is a dimensionality reducing tranformation.
The  Rayleigh quotient in the $\mb{x}$-space is given by
\begin{align}
	R_{ab} = \frac{\mb{w}\TT \mb{C}_a \mb{w}}{\mb{w}\TT \mb{C}_b \mb{w}}
\end{align}
where $\mb{C}_a$ and $\mb{C}_b$ are the covariance matrices of two random variables. The Rayleigh
quotient is maximized by the first principle component of
$\mb{C}_b^{-\frac{1}{2}} \mb{C}_b \mb{C}_b^{-\frac{1}{2}}$ (Rayleigh-Ritz theorem).
\begin{align}
	\mb{R}_{ab} = \frac{\mb{v}\TT \mb{A} \mb{C}_a \mb{A}\TT \mb{v}}{\mb{v}\TT \mb{A} \mb{C}_b \mb{A}\TT \mb{v}}
	\stackrel{!}{=} \imat
\end{align}

We want this to be the identity matrix. $\mb{C}_b$ has to be positive definite.

\begin{align}
	\begin{split}
	\mb{v}\TT \mb{C}_b \mb{v}
		&= \mb{v}\TT \mb{C}_b^{-\frac{1}{2}} \mb{C}_b \mb{C}_b^{-\frac{1}{2}} \mb{v} \\
		&= \mb{v}\TT \, \mb{U} \mb{D}^{-\frac{1}{2}} \mb{U}\TT \, \mb{U} \mb{D}
		   \mb{U}\TT \, \mb{U} \mb{D}^{-\frac{1}{2}} \mb{U}\TT \, \mb{v} \\
 		&= \mb{v}\TT \mb{U} \, \mb{D}^{-\frac{1}{2}} \mb{D} \mb{D}^{-\frac{1}{2}} \, \mb{U}\TT \mb{v} \\
 		&= \mb{v}\TT \mb{U} \imat \mb{U}\TT \mb{v} \\
 		&= \mb{v}\TT \imat \mb{v} \\
 		&= \mb{v}\TT \mb{v}
	\end{split}
\end{align}

\subsection{Whitening transform}
\begin{align}
	\mb{A} := \mb{Q} \mb{C}_b^{-\frac{1}{2}}
\end{align}
where $\mb{Q}$ is an arbitrary orthogonal matrix. For now we pick $\mb{Q} = \imat$

\begin{align}
	\Rightarrow \frac{\mb{v}\TT \mb{C}_b^{-\frac{1}{2}} \mb{C}_a \mb{C}_b^{-\frac{1}{2}} \mb{v}}{\mb{v}\TT \mb{v}}
\end{align}

\begin{align*}
	\mb{x} \stackrel{\mb{A}}{\longrightarrow} \mb{u} \\
	\mb{w} \stackrel{\text{?}}{\longleftarrow} \mb{v}
\end{align*}

\begin{align}
	\frac{\mb{w}\TT \mb{C}_a \mb{w}}{\mb{w}\TT \mb{C}_b \mb{w}} \leftrightarrow
	\frac{\mb{v}\TT \mb{C}_b^{-\frac{1}{2}} \mb{C}_a \mb{C}_b^{-\frac{1}{2}} \mb{v}}{\mb{v}\TT \mb{v}}
\end{align}

\begin{align}
	\mb{w} = \mb{A}^{-1} \mb{v}
\end{align}

\begin{align}
	\mb{C}_b^{-\frac{1}{2}} \mb{C}_a \mb{C}_b^{-\frac{1}{2}} = \mb{V} \tilde{\mb{D}} \mb{V}\TT
\end{align}

if $\mb{D} = \mathrm{diag}(\mb{D}_{11}, \dots, \mb{D}_{MM})$ such that
$\mb{D}_{11} \geq \dots \geq \mb{D}_{MM}$
then
\begin{align}
	\mb{v} = \mb{V}(:,1) = \mb{v}_1 \qquad ; \qquad \mb{V} = (\mb{v}_1, \dots, \mb{v}_M)
\end{align}

\subsection{Signal to noise ratio}
\begin{align}
	\mb{x} = \left( \mb{A}\mb{s} + \mb{B}\mb{n} \right)
\end{align}
where $\mb{s}$ is the signal and $\mb{n}$ is noise.
\begin{align}
	{\mb{s} \choose \mb{n}} \sim \mathcal{N}(0,\imat)
\end{align}
using plain PCA implies the assumption that $\mathrm{Cov}[\mb{n}] = \imat$.
It is assumed that $\mb{s}$ and $\mb{n}$ are statistically independent such that
\begin{align}
	\mathrm{Cov} \left[ \begin{matrix}
		\mb{As} \\ \mb{Bn}
	\end{matrix} \right]
	= \left( \begin{array}{c c}
		\mb{A A\TT} & 0 \\
		0 & \mb{B B\TT}
	  \end{array} \right)
\end{align}
and therefore the data convariance is given by
\begin{align*}
	\mathrm{Cov}[\mb{x}] = \mb{A A\TT} + \mb{B B\TT}
\end{align*}

Noise covariance:
\begin{align*}
	\mathrm{Cov}[\mb{x} | \mb{s}] = \mb{B B\TT}
\end{align*}

\subsubsection{How large is the mutual information between $\mb{x}$ and $\mb{s}$?}

\begin{align}
\begin{split}
	\mathrm{I}[\mb{x}:\mb{s}] &= \mathrm{h}(\mb{x}) \mathrm{h}(\mb{x} | \mb{s}) \\
					&= \frac{1}{2} \log \left(2 \pi \mathrm{e} \right)^M | \mb{A A\TT} + \mb{B B\TT} |
					   -\frac{1}{2} \log \left(2 \pi \mathrm{e} \right)^M | \mb{B B\TT} | \\
					&= \frac{1}{2} \log \frac{| \mb{A A\TT} + \mb{B B\TT} |}{| \mb{B B\TT} |} \\
					&= \frac{1}{2} \log |\underbrace{(\mb{B B\TT})^{-\frac{1}{2}} \mb{A A\TT} (\mb{B B\TT})^{-\frac{1}{2}}}_{\text{Signal-to-noise ratio}} + \imat| \\
\end{split}
\end{align}

\begin{align}
	\mb{y} = \mb{w\TT x} \qquad \text{and} \qquad \mb{W W}\TT = \imat
\end{align}

\begin{align}
	\Rightarrow \mathrm{I}[\mb{y}:\mb{s}]
	   = \frac{1}{2} \log |\mb{W} (\mb{B B\TT})^{-\frac{1}{2}} \mb{A A\TT} (\mb{B B\TT})^{-\frac{1}{2}} \mb{W}\TT + \mb{W W}\TT|
\end{align}

\emph{Oriented PCA} seeks to maximize the mutual information between the signal s and
the \emph{reduced representation} $\mb{y} = \mb{w\TT x}$. \\

The optimum that will be found with oriented PCA is not unique because
\begin{align}
	\tilde{\mb{y}} = \mb{Q W x} \qquad \text{with} \qquad \mb{Q Q\TT} = \imat
\end{align}
has the same mutual information as
\begin{align*}
	\mathrm{I}[\mb{y}:\mb{s}] = \mathrm{I}[\tilde{\mb{y}}:\mb{s}]
\end{align*}
If one seeks to identify a transform that diagonalizes two covariance matrices at the same time,
the solution is unique -> Blind source separation \cite{Tong1991,Molgedey1994}\\

Neuroscience application of oriented PCA: Slow feature analysis.


\newcommand\hnull{\ensuremath{\text{[]}}\xspace}

\subsection{Measures: From Integers to Data Types}\label{sec:measures}

\begin{figure}
\centering
\captionsetup{justification=centering}
$$
\begin{array}{lrcl}
{\emphbf{Definition}} &
  \mathit{def} & ::=  &  \mathtt{measure} \ f :: \tau \\
              & &      &  \quad eq_1 \ldots eq_n       \\[0.05in]

{\emphbf{Equation}}   &
  \mathit{eq}  & ::=  &   f\ (D\ \overline{x}) = r    \\[0.15in]

{\emphbf{Equation to Type}} &
\quad \embed{f\ (D\ \overline{x}) = r} & \defeq & D :: \overline{\tbind{x}{\tau}} \rightarrow \tref{\mathtt{v}}{\tau}{}{f\ \mathtt{v} = r}
\end{array}
$$
\caption{Syntax of Measures.}
\label{fig:measures}
\end{figure}

So far, all our examples have used only integer and boolean expressions in refinements.
To describe properties of algebraic data types, we use \emph{measures},
introduced in prior work on Liquid Types~\cite{LiquidPLDI09}.
%
Measures are inductively defined functions that can be used in refinements and
provide an efficient way to axiomatize properties of data types.
%
For example, @emp@ determines whether a list is empty:
%
\begin{code}
  measure emp  :: [Int] -> Bool
    emp []     = true
    emp (x:xs) = false
\end{code}
The syntax for measures deliberately looks like Haskell, but it is \emph{far} more
restricted, and should really be considered as a separate language.
A measure has exactly one argument and is defined by a list of equations,
each of which has a simple pattern on the left hand side (Figure~\ref{fig:measures}).
The right-hand side of the equation is a refinement expression $r$.
Measure definitions are typechecked in the usual way; we omit the typing rules which are standard.
(Our metatheory does not support type polymorphism,
so here we simply reason about lists of integers;
however, our implementation supports polymorphism).

\paragraph{Denotational semantics}
The denotational semantics of types in \hlang in \Sref{sec:den-sem} is readily extended to
support measures.  In \hlang a refinement $r$ is an arbitrary expression and
calls to a measure are evaluated in the usual way by pattern matching.
For example, with the above definition of @emp@ it is straightforward to show that
\begin{align}
  \mathtt{[1, 2, 3]} \dcolon \tref{\mathtt{v}}{[\tint]}{}{\mathtt{not}\ (\mathtt{emp}\ \mathtt{v})} \label{type:len}
\end{align}
as the refinement @not (emp ([1, 2, 3]))@ evaluates to $\tttrue$.

\mypara{Measures as Axioms}
How can we reason about invocations of measures in the decidable logic of VCs?
A natural approach is to treat a measure like @emp@ as an uninterpreted function
and add logical axioms that capture its behaviour. This looks easy: each equation
of the measure definition corresponds to an axiom, thus:
%
\begin{align*}
\ttemp\ \hnull &= \tttrue\\
\forall \ttx, \ttxs.\, \ttemp\ (\ttx:\ttxs) &= \ttfalse
\end{align*}
%
Under these axioms the judgement~\ref{type:len} is indeed valid.
% % Measures as data constructor refinements

\mypara{Measures as Refinements in Types of Data Constructors}
Axiomatizing measures is \emph{precise}; that is,
the axioms exactly capture the meaning of measures.
Alas, axioms render SMT solvers \emph{inefficient}, and render the VC mechanism \emph{unpredictable},
as one must rely on various brittle syntactic matching and instantiation heuristics~\cite{simplifyj}.

Instead, we use a different approach that is \emph{both} precise \emph{and} efficient.
The key idea is this: \emph{instead of translating each measure equation into an axiom,
we translate each equation into a refined type for the corresponding data constructor}~\citep{LiquidPLDI09}.
This translation is given in Figure~\ref{fig:measures}.
For example, the definition of the measure @emp@ yields the following refined types for the list data constructors:
$$
\begin{array}{lcl}
\hnull  & :: & \ttreft{v}{[\tint]}{emp\ v = true}\\
{:}  & :: & \tfun{\ttx}{\tint}{\tfun{\ttxs}{[\tint]}{\ttreft{v}{[\tint]}{emp\ v = false}}}
\end{array}
$$
These types ensure that:
%
~(1) each time a list value is \emph{constructed},
its type carries the appropriate emptiness information.
Thus our system is able to statically decide that
(\ref{type:len}) is valid and
~(2) each time a list value is \emph{matched},
the appropriate emptiness information is used to
improve precision of pattern matching, as we see next.

\mypara{Using Measures}
\label{sec:pattern-match}
As an example, we use the measure @emp@ to
provide an appropriate type for the @head@ function:
%
\begin{code}
  head    :: {v:[Int] | not (emp v)} -> Int
  head xs = case xs of
              (x:_) -> x
              []    -> error "yikes"

  error   :: {v:String | false} -> a
  error   = undefined
\end{code}
%
@head@ is safe as its input type stipulates that it will only
be called with lists that are \emph{not} @[]@, and so
@error "..."@ is dead code.
%
The call to @error@ generates the subtyping query
%
\begin{align*}
   \tbind{\ttxs}{\tref{\ttxs}{[\tint]}{\trivial}{\lnot (\ttemp\ \ttxs)}}, \
   \tbind{\ttb}{\tttref{\ttb}{[\tint]}{\trivial}{(\ttemp\ \ttxs)= true}}
	 & \vdash \subtref{\tttrue}{\ttfalse}
\end{align*}
%
The match-binder $\ttb$ holds the result of the
match~\cite{SulzmannCJD07}. In the \texttt{[]} case,
we assign it the refinement of the type of \texttt{[]}
which is $(\ttemp\ \ttxs) = \tttrue$. %~\cite{LiquidPLDI09}.
%
Since the call is done inside a @case-of@ expressions
both @xs@ and @b@ are in WHNF,
thus they have \Wnf types.

The verifier \emph{accepts} the program as the above subtyping reduces to the valid VC:
\begin{align*}
\lnot (\ttemp\ \ttxs) \wedge ((\ttemp\ \ttxs)= \tttrue) \Rightarrow\ & \tttrue \Rightarrow\ \ttfalse
\end{align*}
%
Thus, our system supports idiomatic
Haskell, \eg taking the @head@ of an infinite list:
%
\begin{code}
  ex x     = head (repeat x)

  repeat   :: Int -> {v:[Int] | not (emp v)}
  repeat y = y : repeat y
\end{code}
%

\mypara{Multiple Measures}
If a type has multiple measures, we simply refine each data constructor's type
with the \emph{conjunction} of the refinements from each measure.
%
For example, consider a measure that computes the length of a list:
\begin{code}
  measure len  :: [Int] -> Int
    len ([])   = 0
    len (x:xs) = 1 + len xs
\end{code}
%
Using the translation of Figure~\ref{fig:measures},
we get the following types for list's data constructors.
%
\begin{align*}
\text{[]}  & ::  \ttreft{v}{[\tint]}{len\ v = 0}\\
{:}  & ::  \tfun{\ttx}{\tint}{\tfun{\ttxs}{[\tint]}{\ttreft{v}{[\tint]}{len\ v = 1 + (len\ xs)}}}\\
\intertext{The final types for list data are the
conjunction of the refinements from $\mathtt{len}$ and $\mathtt{emp}$:}\\
\text{[]}  & ::  \ttreft{v}{[\tint]}{emp\ v = true \land len\ v = 0}\\
{:}  & ::  \tfun{\ttx}{\tint}{\tfun{\ttxs}{[\tint]}
           {\ttreft{v}{[\tint]}{emp\ v = false \land len\ v = 1 + (len\ xs)}}}
\end{align*}