\documentclass[../main.tex]{subfiles}
\begin{document}
In this chapter, we will specifically talk math related problems. Normally, for the problems appearing in this section, they can be solved using our learned programming methodology. However, it might not inefficient (we will get LTE error on the LeetCode) due to the fact that we are ignoring their math properties which might help us boost the efficiency. Thus, learning some of the most related math knowledge can make our life easier.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% sorting
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% GCD
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Numbers}
\subsection{Prime Numbers}
A prime number is an integer greater than 1, which is only divisible by 1 and itself. First few prime numbers are : 2 3 5 7 11 13 17 19 23 ...

Some interesting facts about Prime numbers:
\begin{enumerate}
    \item 2 is the only even Prime number.
    \item  2, 3 are only two consecutive natural numbers which are prime too.
    \item \label{divide}Every prime number except 2 and 3 can represented in form of 6n+1 or 6n-1, where n is natural number.
    \item \label{godld} Goldbach Conjecture: Every even integer greater than 2 can be expressed as the sum of two primes. Every positive integer can be decomposed into a product of primes.
    \item GCD of a natural number with Prime is always one.
    \item Fermat’s Little Theorem: If n is a prime number, then for every a, $1 <= a < n $, %$a^{(n-1)} == 1 (mod n) OR a^(n-1) % n = 1$. check if it is useful in real situation
    \item Prime Number Theorem : The probability that a given, randomly chosen number n is prime is inversely proportional to its number of digits, or to the logarithm of n.
\end{enumerate}
\subsubsection{Check Single Prime Number}
Learning to check if a number is a prime number is necessary: the naive solution comes from the direct definition, for a number $n$, we try to check if it can be divided by number in range $[2, n-1]$, if it divides, then its not a prime number.
\begin{lstlisting}[language=Python]
def isPrime(n):
    # Corner case
    if (n <= 1):
        return False
    # Check from 2 to n-1
    for i in range(2, n):
        if (n % i == 0):
            return False
return True
\end{lstlisting}
There are actually a lot of space for us to optimize the algorithm. First, instead of checking till n, we can check till $\sqrt{n}$ because a larger factor of n must be a multiple of smaller factor that has been already checked. Also, because even numbers bigger than 2 are not prime, so the step we can set it to 2. The algorithm can be improved further by use feature \ref{divide} that all primes are of the form $6k \pm 1$, with the exception of 2 and 3.  Together with feature \ref{godld} which implicitly states that every non-prime integer is divisible by a prime number smaller than itself. So a more efficient method is to test if n is divisible by 2 or 3, then to check through all the numbers of form $6k \pm 1$.
\begin{lstlisting}[language=Python]
def isPrime(n):
    # corner cases
    if n <= 1:
        return False
    if n<= 3:
        return True

    if n % 2 == 0 or n % 3 == 0:
        return False

    for i in range(5, int(n**0.5)+1, 6):  # 6k+1 or 6k-1, step 6, up till sqrt(n), when i=5, check 5 and 7, (k-1, k+1)
        if n%i == 0 or n%(i+2)==0:
            return False
    return True
return True
\end{lstlisting}

\subsubsection{Generate A Range of Prime Numbers}
\paragraph{Wilson theorem} says if a number k is prime then $((k-1)! + 1) \% k$ must be 0. Below is Python implementation of the approach. Note that the solution works in Python because Python supports large integers by default therefore factorial of large numbers can be computed.
\begin{lstlisting}[language=Python]
# Wilson Theorem
def primesInRange(n):
    fact = 1
    rst = []
    for k in range(2, n):
        fact *= (k-1)
        if (fact + 1)% k == 0:
            rst.append(k)
    return rst

print(primesInRange(15))
# output
# [2, 3, 5, 7, 11, 13]
\end{lstlisting}

\paragraph{Sieve Of Eratosthenes} To  generate a list of primes. It works by recognizing \textit{Goldbach Conjecture} that all non-prime numbers are divisible by a prime number. An optimization is to only use odd number in the primes list, so that we can save half space and half time. The only difference is we need to do index mapping.
\begin{lstlisting}[language=Python]
def primesInRange(n):
    primes = [True] * n
    primes[0] = primes[1] = False
    for i in range(2, int(n ** 0.5) + 1):
        #cross off remaining multiples of prime i, start with i*i
        if primes[i]:
            for j in range(i*i,n,i):
                primes[j] = False
    rst = [] # or use sum(primes) to get the total number
    for i, p in enumerate(primes):
        if p:
            rst.append(i)
    return rst

print(primesInRange(15))
\end{lstlisting}
\subsection{Ugly Numbers}
Ugly numbers are positive numbers whose prime factors only include 2, 3, 5. We can write it as $ugly number = 2^i3^j5^k, i>=0, j>=0, k>=0$. Examples of ugly numbers: 1, 2, 3, 5, 6, 10, 15, ... The concept of ugly number is quite simple. Now let us use the LeetCode problems as example to derive the algorithms to identify ugly numbers.
\subsubsection{Check a Single Number}
263. Ugly Number (Easy)
\begin{lstlisting}
Ugly numbers are positive numbers whose prime factors only include 2, 3, 5. For example, 6, 8 are ugly while 14 is not ugly since it includes another prime factor 7.

Note:
    1 is typically treated as an ugly number.
    Input is within the 32-bit signed integer range.
\end{lstlisting}
Analysis: because the ugly number is only divisible by $2, 3, 5$, so if we keep dividing the number by these factors ($num/f$), eventually we would get $1$, if the reminder ($num\%f$) is $0$ (divisible), otherwise we stop the loop to check the number.
\begin{lstlisting}[language = Python]
def isUgly(self, num):
        """
        :type num: int
        :rtype: bool
        """
        if num ==0:
            return False
        factor = [2,3,5]
        for f in factor:
            while num%f==0:
                num/=f
        return num == 1
\end{lstlisting}
\subsubsection{Generate A Range of Number}
264. Ugly Number II (medium)
\begin{lstlisting}
Write a program to find the n-th ugly number.

Ugly numbers are positive numbers whose prime factors only include 2, 3, 5. For example, 1, 2, 3, 4, 5, 6, 8, 9, 10, 12 is the sequence of the first 10 ugly numbers.

Note that 1 is typically treated as an ugly number, and n does not exceed 1690.
\end{lstlisting}
Analysis: The first solution is we use the rules $ugly number = 2^i3^j5^k, i>=0, j>=0, k>=0$, using three for loops to generate at least 1690 ugly numbers that is in the range of $2^32$, and then sort them, the time complexity is $O(nlogn)$, with $O(n)$ in space.  However, if we need to constantly make request, it seems resasonable to save a table, and once the table is generated and saved, each time we would only need constant time to check.
\begin{lstlisting}[language = Python]
from math import log, ceil
class Solution:
    ugly = [2**i * 3**j * 5**k for i in range(32) for j in range(ceil(log(2**32, 3))) for k in range(ceil(log(2**32, 5)))]
    ugly.sort()
    def nthUglyNumber(self, n):
        """
        :type n: int
        :rtype: int
        """
        return self.ugly[n-1]
\end{lstlisting}
The second way is only generate the nth ugly number, with
\begin{lstlisting}[language=Python]
class Solution:
    n = 1690
    ugly = [1]
    i2 = i3 = i5 = 0
    for i in range(n-1):
        u2, u3, u5 = 2 * ugly[i2], 3 * ugly[i3], 5 * ugly[i5]
        umin = min(u2,u3,u5)
        ugly.append(umin)
        if umin == u2:
            i2 += 1
        if umin == u3:
            i3 += 1
        if umin == u5:
            i5 += 1

    def nthUglyNumber(self, n):
        """
        :type n: int
        :rtype: int
        """
        return self.ugly[n-1]
\end{lstlisting}
%%%%%%%%%Combinatorics%%%%
\subsection{Combinatorics}
\begin{enumerate}
    \item 611. Valid Triangle Number
\end{enumerate}
\begin{examples}[resume]
\item \textbf{Pascal's Triangle II(L119, *).} Given a non-negative index k where k <= 33, return the kth index row of the Pascal's triangle. Note that the row index starts from 0. In Pascal's triangle, each number is the sum of the two numbers directly above it.
\begin{lstlisting}[numbers=none]
Example:
Input: 3
Output: [1,3,3,1]
\end{lstlisting}
Follow up: Could you optimize your algorithm to use only O(k) extra space?
\textbf{Solution: Generate from Index 0 to K}.
\begin{lstlisting}[language=Python]
def getRow(self, rowIndex):
    if rowIndex == 0:
        return [1]
    # first, n = rowIndex+1, if n is even,
    ans = [1]
    for i in range(rowIndex):
        tmp = [1]*(i+2)
        for j in range(1, i+1):
            tmp[j] = ans[j-1]+ans[j]
        ans = tmp
    return ans
\end{lstlisting}
Triangle Counting

\end{examples}
%%%%%%%%%%%Others%%%%%%%%%%%
\subsubsection{Smallest Larger Number}
556. Next Greater Element III
\begin{lstlisting}
Given a positive 32-bit integer n, you need to find the smallest 32-bit integer which has exactly the same digits existing in the integer n and is greater in value than n. If no such positive 32-bit integer exists, you need to return -1.

Example 1:

Input: 12
Output: 21

Example 2:

Input: 21
Output: -1
\end{lstlisting}
Analysis: The first solution is to get all digits [1,2], and generate all the permutation [[1,2],[2,1]], and generate the integer again, and then sort generated integers, so that we can pick the next one that is larger. But the time complexity is O(n!).

Now, let us think about more examples to find the rule here:
\begin{lstlisting}
435798->435879
1432->2134
\end{lstlisting}
If we start from the last digit, we look to its left, find the cloest digit that has smaller value, we then switch this digit, if we cant find such digit, then we search the second last digit. If none is found, then we can not find one. Like 21. return -1. This process is we get the first larger number to the right.
\begin{lstlisting}
[5, 5, 7, 8, -1, -1]
[2, -1, -1, -1]
\end{lstlisting}
After the this we switch 8 with 7: we get
\begin{lstlisting}
4358 97
2 431
\end{lstlisting}
For the reminding digits, we do a sorting and put them back to those digit to get the smallest value
\begin{lstlisting}[language=Python]
class Solution:
    def getDigits(self, n):
        digits = []
        while n:
            digits.append(n%10) # the least important position
            n = int(n/10)
        return digits
    def getSmallestLargerElement(self, nums):
        if not nums:
            return []
        rst = [-1]*len(nums)

for i, v in enumerate(nums):
            smallestLargerNum = sys.maxsize
            index = -1
            for j in range(i+1, len(nums)):
                if nums[j]>v and smallestLargerNum > nums[j]:
                    index = j
                    smallestLargerNum = nums[j]
            if smallestLargerNum < sys.maxsize:
                rst[i] = index
        return rst

    def nextGreaterElement(self, n):
        """
        :type n: int
        :rtype: int
        """
        if n==0:
            return -1

        digits = self.getDigits(n)
        digits = digits[::-1]
        # print(digits)

        rst = self.getSmallestLargerElement(digits)
        # print(rst)
        stop_index = -1

        # switch
        for i in range(len(rst)-1, -1, -1):
            if rst[i]!=-1: #switch
                print('switch')
                stop_index = i
                digits[i], digits[rst[i]] = digits[rst[i]], digits[i]
                break
        if stop_index == -1:
            return -1

#         print(digits)

        # sort from stop_index+1 to the end
        digits[stop_index+1:] = sorted(digits[stop_index+1:])
        print(digits)

#convert the digitialized answer to integer
        nums = 0
        digit = 1
        for i in digits[::-1]:
            nums+=digit*i
            digit*=10
            if nums>2147483647:
                return -1

        return nums
\end{lstlisting}
\section{Intersection of Numbers}
In this section, intersection of numbers is to find the ``common" thing between them, for example Greatest Common Divisor and Lowest Common Multiple.
\subsection{Greatest Common Divisor}
GCD (Greatest Common Divisor) or HCF (Highest Common Factor) of two numbers $a$ and $b$ is the largest number that divides both of them. For example shown as follows:
\begin{lstlisting}
The divisors of 36 are: 1, 2, 3, 4, 6, 9, 12, 18, 36
The divisors of 60 are: 1, 2, 3, 4, 5, 6, 10, 12, 15, 30, 60
GCD = 12
\end{lstlisting}
Special case is when one number is zero, the GCD is the value of the other. $gcd(a, 0) = a$.

The basic algorithm is: we get all divisors of each number, and then find the largest common value. Now, let's see how to we advance this algorithm.  We can reformulate the last example as:
\begin{lstlisting}
36 = 2 * 2 * 3 * 3
60 = 2 * 2 * 3 * 5
GCD = 2 * 2 * 3
    = 12
\end{lstlisting}
So if we use $60-36 = 2*2*3*5 - 2*2*3*3 = (2*2*3)*(5-3) = 2*2*3*2$. So we can derive the principle that the GCD of two numbers does not change if the larger number is replaced by its difference with the smaller number.  The features of GCD:
\begin{enumerate}
    \item $gcd(a, 0) = a$
    \item $gcd(a, a) = a$,
    \item $gcd(a, b) = gcd(a-b, b)$, if $a>b$.
\end{enumerate}
Based on the above features, we can use Euclidean Algorithm to gain GCD:
\begin{lstlisting}
def euclid(a, b):
    while a != b:
         # replace larger number by its difference with the smaller number
         if a > b:
             a = a - b
         else:
             b = b - a
    return a

print(euclid(36, 60))
\end{lstlisting}
The only problem with the Euclidean Algorithm is that it can take several subtraction steps to find the GCD if one of the given numbers is much bigger than the other. A more efficient algorithm is to replace the subtraction with remainder operation. The algorithm would stops when reaching a zero reminder and now the algorithm never requires more steps than five times the number of digits (base 10) of the smaller integer.

The recursive version code:
\begin{lstlisting}[language = Python]
def euclidRemainder(a, b):
    if a == 0 :
        return b
    return gcd(b%a, a)
\end{lstlisting}
The iterative version code:
\begin{lstlisting}[language = Python]
def euclidRemainder(a, b):
    while a > 0:
         # replace one number with reminder between them
         a, b = b%a, a
    return b

print(euclidRemainder(36, 60))
\end{lstlisting}

\subsection{Lowest Common Multiple}
Lowest Common Multiple (LCM) is the smallest number that is a multiple of both $a$ and $b$. For example of 6 and 8:
\begin{lstlisting}
The multiplies of 6 are: 6, 12, 18, 24, 30, ...
The multiplies of 8 are: 8, 16, 24, 32, 40, ...
LCM = 24
\end{lstlisting}
Computing LCM is dependent on the GCD with the following formula:
\begin{equation}
    lcm(a, b) = \frac{a\times b}{gcd(a, b)}
\end{equation}

\section{Arithmetic Operations}
Because for the computer, it only understands the binary representation as we learned in Bit Manipulation (Chapter~\ref{chapter_bit}, the most basic arithmetic operation it supports are  binary addition and subtraction. (Of course, it can execute the bit manipulation too.)  The other common arithmetic operations such as Multiplication, division, modulus, exponent are all implemented/coded with the addition and subtraction as basis or in a dominant fashion. As a software engineer, have a sense of how we can implement the other operations from the given basis is reasonable and a good practice of the coding skills. Also, sometimes if the factor to compute on is extra large number, which is to say the computer can not represent, we can still compute the result by treating these numbers as strings.

In this section, we will explore operations include multiplication, division. There are different algorithms that we can use, we learn a standard one called long multiplication and long division. I am assuming you know the algorithms and focusing on the implementation of the code instead.

\paragraph{Long Multiplication}

\paragraph{Long Division} We treat the dividend as a string, e.g. dividend = 3456, and the divisor = 12. We start with 34, which has the digits as of divisor. 34/12 = 2, 10, where 2 is the integer part and 10 is the reminder. Next step, we take the reminder and join with the next digit in the dividend, we get 105/12 = 8, 9. Smilarily, 96/12 = 8, 0. Therefore we get the results by joinging the result of each dividending operation, '288'.  To see the coding, let us code it the way required by the following LeetCode Problem. In the process we need (n-m) (n, m is the total number of digits of dividend and divisor, respectively) division operation. Each division operation will be done at most 9 steps. This makes the time complexity $O(n-m)$.
\begin{examples}[resume]
\item \textbf{29. Divide Two Integers (medium)} Given two integers dividend and divisor, divide two integers without using multiplication, division and mod operator. Return the quotient after dividing dividend by divisor. The integer division should truncate toward zero.
\begin{lstlisting}[language=Python][numbers=none]
Example 1:

Input: dividend = 10, divisor = 3
Output: 3

Example 2:

Input: dividend = 7, divisor = -3
Output: -2
\end{lstlisting}

\textbf{Analysis:} we can get the sign of the result first, and then convert the dividend and divisor into its absolute value. Also, we better handle the bound condition that the divisor is larger than the vidivend, we get 0 directly. The code is given:
\begin{lstlisting}[language=Python]
def divide(self, dividend, divisor):
    def divide(dd): # the last position that divisor* val <  dd
        s, r = 0, 0
        for i in range(9):
            tmp = s + divisor
            if tmp <= dd:
                s = tmp
            else:
                return str(i), str(dd-s)
        return str(9), str(dd-s)

    if dividend == 0:
        return 0
    sign = -1
    if (dividend >0 and divisor >0 ) or (dividend < 0 and divisor < 0):
        sign = 1
    dividend = abs(dividend)
    divisor = abs(divisor)
    if divisor > dividend:
        return 0
    ans, did, dr = [], str(dividend), str(divisor)
    n = len(dr)
    pre = did[:n-1]
    for i in range(n-1, len(did)):
        dd = pre+did[i]
        dd = int(dd)
        v, pre = divide(dd)
        ans.append(v)

    ans = int(''.join(ans))*sign

    if ans > (1<<31)-1:
        ans = (1<<31)-1
    return ans
\end{lstlisting}
\end{examples}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Probability Theory
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Probability Theory}
In programming tasks, such problems are either solvable with some closed-form formula or one has no choice than to enumerate the complete search space.
\section{Linear Algebra}
\textit{Gaussian Elimination} is one of the several ways to find the solution for a system of linear euqations.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% geometry
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Geometry}
In this section, we will discuss coordinate related problems.

939. Minimum Area Rectangle(Medium)

Given a set of points in the xy-plane, determine the minimum area of a rectangle formed from these points, with sides parallel to the x and y axes.

If there isn't any rectangle, return 0.
\begin{lstlisting}
Example 1:

Input: [[1,1],[1,3],[3,1],[3,3],[2,2]]
Output: 4

Example 2:

Input: [[1,1],[1,3],[3,1],[3,3],[4,1],[4,3]]
Output: 2
\end{lstlisting}
\textbf{Combination}. This at first it is a combination problem, we pick four points and check if it is a rectangle and then what is the size. However the time complexity can be $C_n^k$, which will be $O(n^4)$. The following code implements the best combination we get, however, we receive LTE:
\begin{lstlisting}[language=Python]
def minAreaRect(self, points):
    def combine(points, idx,  curr, ans): # h and w at first is -1
        if len(curr) >= 2:
            lx, rx = min([x for x, _ in curr]), max([x for x, _ in curr])
            ly, hy = min([y for _, y in curr]), max([y for _, y in curr])
            size = (rx-lx)*(hy-ly)
            if size >= ans[0]:
                return
            xs = [lx, rx]
            ys = [ly, hy]
            for x, y in curr:
                if x not in xs or y not in ys:
                    return

            if len(curr) == 4:
                ans[0] = min(ans[0], size)
                return

        for i in range(idx, len(points)):
            if len(curr) <= 3:
                combine(points, i+1,  curr+[points[i]], ans)
        return

    ans=[sys.maxsize]
    combine(points, 0, [], ans)
    return ans[0] if ans[0] != sys.maxsize else 0
\end{lstlisting}
\textbf{Math: Diagonal decides a rectangle}. We use the fact that if we know the two diagonal points, say (1, 2), (3, 4). Then we need (1, 4), (3, 2) to make it a rectangle.  If we save the points in a hashmap, then the time complexity can be decreased to $O(n^2)$. The condition that two points are diagonal is: x1 != x2, y1 != y2. If one of them is equal, then they form a vertical or horizontal line. If both equal, then its the same points.
\begin{lstlisting}[language = Python]
class Solution(object):
    def minAreaRect(self, points):
        S = set(map(tuple, points))
        ans = float('inf')
        for j, p2 in enumerate(points): # decide the second point
            for i in range(j): # decide the firs point
                p1 = points[i]
                if (p1[0] != p2[0] and p1[1] != p2[1] and # avoid
                        (p1[0], p2[1]) in S and (p2[0], p1[1]) in S):
                    ans = min(ans, abs(p2[0] - p1[0]) * abs(p2[1] - p1[1]))
        return ans if ans < float('inf') else 0
\end{lstlisting}
\textbf{Math: Sort by column}. Group the points by x coordinates, so that we have columns of points. Then, for every pair of points in a column (with coordinates (x,y1) and (x,y2)), check for the smallest rectangle with this pair of points as the rightmost edge. We can do this by keeping memory of what pairs of points we've seen before.
\begin{lstlisting}[language=Python]
def minAreaRect(self, points):
    columns = collections.defaultdict(list)
    for x, y in points:
        columns[x].append(y)
    lastx = {}  # one-pass hash
    ans = float('inf')

    for x in sorted(columns): # sort by the keys
        column = columns[x]
        column.sort() # sort column
        for j, y2 in enumerate(column): # right most edge, up point
            for i in xrange(j):         # right most edge, lower point
                y1 = column[i]
                if (y1, y2) in lastx:  # 1: [1, 3], will be saved, when we were at 3: [1, 3], we can get the answer
                    ans = min(ans, (x - lastx[y1,y2]) * (y2 - y1))
                lastx[y1, y2] = x # y1, y2 form a tuple
    return ans if ans < float('inf') else 0
\end{lstlisting}

\section{Miscellaneous Categories}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%% rabbit and turtle to find circle or repeat number
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Floyd’s Cycle-Finding Algorithm}

Without this we detect cycle with the following code:
\begin{lstlisting}[language = Python]
def detectCycle(self, A):
        visited=set()
        head=point=A
        while point:
            if point.val in visited:
                return point
            visited.add(point)
            point=point.next
        return None
\end{lstlisting}

Traverse linked list using two pointers. Move one pointer by one and other pointer by two. If these pointers meet at some node then there is a loop. If pointers do not meet then linked list doesn’t have loop. Once you detect a cycle, think about finding the starting point.
\begin{figure}[h!]
    \centering
    \includegraphics[width = 0.98\columnwidth]{fig/floyd.png}
    \caption{Example of floyd’s cycle finding}
    \label{fig:floyd}
\end{figure}

\begin{lstlisting}[language = Python]
def detectCycle(self, A):
        #find the "intersection"
        p_f=p_s=A
        while (p_f and p_s and p_f.next):
            p_f = p_f.next.next
            p_s = p_s.next
            if p_f==p_s:
                break
        #Find the "entrance" to the cycle.
        ptr1 = A
        ptr2 = p_s;
        while ptr1 and ptr2:
            if ptr1!=ptr2:
                ptr1 = ptr1.next
                ptr2 = ptr2.next
            else:
                return ptr1
        return None
\end{lstlisting}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% exercise
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Exercise}
\subsection{Number}
313. Super Ugly Number
\begin{lstlisting}
Super ugly numbers are positive numbers whose all prime factors are in the given prime list primes of size k. For example, [1, 2, 4, 7, 8, 13, 14, 16, 19, 26, 28, 32] is the sequence of the first 12 super ugly numbers given primes = [2, 7, 13, 19] of size 4.

Note:
 (1) 1 is a super ugly number for any given primes.
 (2) The given numbers in primes are in ascending order.
 (3) 0 < k <= 100, 0 < n <= 106, 0 < primes[i] < 1000.
 (4) The nth super ugly number is guaranteed to fit in a 32-bit signed integer.
\end{lstlisting}
\begin{lstlisting}[language=Python]
def nthSuperUglyNumber(self, n, primes):
        """
        :type n: int
        :type primes: List[int]
        :rtype: int
        """
        nums=[1]
        idexs=[0]*len(primes) #first is the current idex
        for i in range(n-1):
            min_v = maxsize
            min_j = []
            for j, idex in enumerate(idexs):
                v = nums[idex]*primes[j]
                if v<min_v:
                    min_v = v
                    min_j=[j]
                elif v==min_v:
                    min_j.append(j) #we can get mutiple j if there is a tie
            nums.append(min_v)
            for j in min_j:
                idexs[j]+=1
        return nums[-1]
\end{lstlisting}
\end{document}


\section{Polynomial: Polynomials}
We will use the $[a_0,a_1,a_2 \dots]$ format for the array
representation of polynomials.  That allows the array index
to be the x exponent.  However, it makes the bookkeeping for
div and deriv a bit tricky, since they have to work
backwards.

\subsection*{eval}
NR92 recommends using the nested form:
\begin{equation}
    a_0 + a_1 x^1 + a_2 x^2 + a_3 x^3
      = a_0+ x(a_1+x(a_2+x(a_3)))
\end{equation}

This can be done in a loop, working down from a[nn]:
\begin{tt} \begin{verbatim}
     tmp:=p[nn];
     FOR i:=nn-1 TO 1 BY -1 DO
       tmp:=a[i]+x*tmp;
     END;
     tmp:=a[0]+tmp;
\end{verbatim}  \end{tt}

\subsection*{add,sub}
Trivial.

\subsection*{mul}
This is not as simple as add and sub, but it it is still
mainly bookkeeping.  No special insights needed.

\subsection*{div}
The classic form is:
\begin{equation}
   \frac{A=a_n x^n+a_{n-1} x^{n-1}+\dots+a_0}
     {B=b_m x^m+b_{m-1} x^{m-1}+\dots+b_0}
\end{equation}

We set it up for longhand as:
\begin{tt} \begin{verbatim}
          Q=quotient
          ------------------
     B    | A
           -B*q[qi]
            --------------
            remainder
\end{verbatim} \end{tt}

We repeat a pattern, using $b[m]=bmax$ as the divider, getting
q's in decreasing order and taking the remainder of the
subtraction as the start point for the next division by
bmax.  We may as well start by copying A into the remainder
and using it directly:
\begin{verbatim}
     q[qi]:=r[ri]/bmax
     for all the r's, subtract the appropriate q[qi]*b[bi]
\end{verbatim}

The problem is really one of bookkeeping, since we are
moving backwards in b,r,and q, and the start points keep
changing.  Points to remember include:
\begin{enumerate}
     \item Once the order (i.e., max exponent) of the remainder
       is less than bmax, we can quit.
     \item Q's order is A's order - B's order.
\end{enumerate}

Using $A=u, B=v$:
\begin{tt} \begin{verbatim}
     (*u/v = q, remainder r*)
     un:=NUMBER(u^); unn:=LAST(u^);
     vn:=NUMBER(v^); vnn:=LAST(v^); vmax:=v[vnn];
     qn:=un-vn+1; q:=NEW(Poly,qn);
     r^:=u^;

     FOR ri:=unn TO (unn-qnn) BY -1 DO
         qtmp:=r[ri]/vmax;
         q[ri-vnn]:=qtmp;
         ri2:=ri;
         FOR vi:=vnn TO v0 BY -1 DO
           r[ri2]:=r[ri2]-qtmp*v[vi];
           DEC(ri2);
         END;
       END;
\end{verbatim} \end{tt}

That is, we work systematrixically down through the remainders.
For each we use vmax to find q[qi], where qi started at un-
vn and decreases with the r's (which mirror the u's).  Once
q is found, we use it (in the form qtmp) to calc the
subtraction term, which starts with the biggest v and works
down to the smallest.  For each v[vi], the applicable r term
starts with the currently biggest ri (given by ri2), then
decreasing ri2 for every vi decrease.

Testing via MathCAD
\begin{equation}
   \frac{1+2x+3x^2+4x^3+5x^4}{.1+.2x+.3x^2+.4x^3}
     \rightarrow q=.6+12.5x, r=.9+.6x+.3x^2
\end{equation}

Multiplying back and adding the remainder I got
\begin{equation}
     u= .96+1.97x+2.98x^2+3.99x^3+5.0x^4
\end{equation}
Looks like the algorithm works, though the lower coefs are
pretty bad.

\subsection*{deflate}
See Swok90, pg 124.  Given polynomial a, and wanting $a/(x-c)
= b$, where a and b are in the form $a_0+a_1*x+a_2*x^2$, we set up
the longhand form:
\begin{verbatim}
            a[n]n*x^(n-1)
          ------------------------------
     x-c  | a[n]*x^n +  a[n-1]*x^(n-1) ... a0
            a[n]*x^n + -c*a[n]*x^(n-1)
            --------------------
                     (a[n-1]+c*a[n])
\end{verbatim}

Notice the $c*a[n]$ term for the next iteration.  Each step
has this addition of c times the previous solution coef.
The result is a very regular progression backwards from $x^n$
to $x^0$.  At that point we cannot divide any further, and just
have the remainder (if any).
\begin{tt} \begin{verbatim}
     m:=LAST(a);
     b:=NEW(Poly,m);  (*b is 0..m-1*)
     b[m-1]:=a[m];
     FOR i:=m-2 TO 0 BY -1 DO
       b[i]:=a[i+1]+c*b[i+1];
     END;
     rem:=a[0]+c*b[0];
\end{verbatim} \end{tt}

However, deflation is often used where it doesn't make sense
to generate a brandom new "b" polynomial every time.  So we
need an in-place approach.  Since a[] is being used for both
the old input data and the new output data, we have to save
the coef for current term:
\begin{tt} \begin{verbatim}
     (*ann = LAST(a) *)
     b:=a[ann]; asave:=a[ann-1]; a[ann-1]:=b;
     FOR j:=ann-2 TO 1 BY -1 DO
       b:=asave+c*b;
       asave:=a[j]; a[j]:=b;
     END;
     rem:=a[0]+c*a[1];
\end{verbatim} \end{tt}

Deflation is typically used where $c$ is a root of $a$, and
therefore $rem=0$.  So the last line may not be relevant.

\subsection*{deriv}
Given a polynomial with coefs a[]:
\begin{verbatim}
     p =   a0    +a1*x       +a2*x^2     +a3*x^3     +a4*x^4
     p'=   a1    +2*a2*x     +3*a3*x^2   +4*a4x^3    +0
     p"=   2*a2  +2*3*a3*x   +3*4*a4*x^2 +0          +0
\end{verbatim}

By lining them up this way, the polynomial sense of the
vector is maintained.  That is, p[2] is the coef of $x^2$ for p
and p'[2] is the coef of $x^2$ for p'.

Let's start with evaluating p(x).  The NR92 recommended way
to evaluate polynomials is to nest the multiplies.  For p(x)
we have:
\begin{equation}
     p=a_0+x*(a_1+x*(a_2+x*(a_3+x*(a_4))));
\end{equation}
In a loop, this works out as:
\begin{verbatim}
     p:=a[pnn];
     FOR i:=ann-1 TO 0 BY -1 DO
       p:=a[i]+x*p;
     END;
\end{verbatim}

Next, we need to handle derivatives of p.  We may as well
use NR92's notation of pd[], where the index is the
derivative count:
\begin{verbatim}
     p=pd[0]; p'=pd[1]; p'=pd[2]; ...pd[nd];
\end{verbatim}

In nested form:
\begin{verbatim}
     p :=a0+     x*(a1+     x*(a2+     x*(a3+     x*(a4))));
     p':=a1+     x*(2*a2+   x*(3*a3+   x*(4*a4+   x*(0))))
     p":=2*a2+   x*(2*3*a3+ x*(3*4*a4+ x*(0+      x*(0))))
\end{verbatim}

Notice the factors multiplying the a terms.  For the dth
derivative we need:
\begin{equation}
     (n)(n-1)(n-2) \cdots (n-d+1) a_n \mbox{\ (d$<$n)}
\end{equation}

We could do all the derivatives just like p, keeping track
of the offsets and inserting the factors:
\begin{verbatim}
     pd[0]:=a[pnn];
     FOR i:=1 TO nd DO pd[i]:=0.0; END;
     FOR i:=ann-1 TO 0 BY -1 DO
         pd[0]:=a[i]+x*p[0];
       FOR j:=1 TO nd DO
         factor:=1.0;
         FOR k:=i TO i-j BY -1 DO
           factor:=factor*FLOAT(k,REAL32);END;
         pd[j]:=factor*a[i-j]+x*pd[j];
       END;
     END;
\end{verbatim}

But notice something in the nested form.  Starting at the
left, a4 keeps moving to the right as another derivative is
taken.  In fact, the full (factor,a,x ) term is very similar
to the previous line's term one column to the left.  Can we
exploit that to reduce the inner loops in the above
algorithm?

We need to catch the one-lower-derivative before it gets
updated in the current cycle (i.e., the current column).  If
we worked backwards from the largest derivative, we would be
getting both the right a[] term and the right power of x.
Ignoring the factors:
\begin{verbatim}
     p[0]:=a[ann];
     FOR i:=ann-1 TO 0 BY -1 DO
       FOR j:=nd TO 1 BY -1 DO
         pd[j]:=pd[j-1]+x*pd[j];
       END;
       pd[0]:=a[i]+x*pd[0];
     END;
\end{verbatim}

Now, what's happening to the factors?
\begin{verbatim}
   init:
     p"=0; p':=0; p=a4
   iter3:
     p"=p'+x*p"=0+x*0=0
     p'=p +x*p'=a4+x*0=a4
     p =a3+x*p =a3+x*a4
   iter2:
     p"=p'+x*p"=a4+x*(0)=a4
     p'=p +x*p'=a3+x*a4+x*(a4)=a3+2*a4*x
     p =a2+x*p =a2+x*(a3+x*a4)=a2+a3*x+a4*x^2
   iter1:
     p"=p'+x*p"=a3+2*a4*x+x*(a4)=a3+3*a4*x
     p'=p
     +x*p'=a2+a3*x+a4*x^2+x*(a3+2*a4*x)=a2+2*a3*x+3*a4*x^2
     p =a1+x*p =a1+x*(a2+a3*x+a4*x^2)=a1+a2*x+a3*x^2+a4x^3
   iter0:
     p"=p'+x*p"=a2+2*a3*x+3*a4*x^2+x*(a3+3*a4*x)=a2+3*a3*x+4
     *a4*x^2
     p'=p +x*p'=a1+a2*x+a3*x^2+a4x^3+x*(a2+2*a3*x+3*a4*x^2)
          =a1+2*a2*x+3*a3*x^2+4*a4*x^3
     p =a0+x*p =a0+x*(a1+a2*x+a3*x^2+a4x^3)
          =a0+a1*x+a2*x^2+a3*x^3+a4*x^4
\end{verbatim}

Note that p comes out ok (as expected).  Note that p' comes
out ok too (definitely not expected).  p" (and higher
derivatives) are in the right shape, but the factorial terms
are incomplete.  What is the pattern?  Let's set up the
program and try it out.  Use $a[i]=1$ for all i, so we can see
the factorial terms more easily.  Experiments on
$x=1,2,3,4,5$ (comparing to MathCAD) show a pattern:  pd[0]
and pd[1] are ok.  pd[1] is off by $2=2!$ and pd[3] is off by
$6=3!$.  In other words all of them are off by a factor:
\begin{verbatim}
     pd[j]:=factorial(j)*pd[j]
\end{verbatim}

It just happens that $0!=1$ and $1!=1$, so pd[0] and pd[1] look
ok.  It appears we can clean this up by putting a factorial
onto pd after computation.  NR92 does this with a home-made
factorial.  Bu this has a hidden integer$\rightarrow$float conversion.
We may as well use the factorial function, which does a
table lookup (at the cost of a funcall).  And let's do it
for all the pd's since it is low cost to catch j=0,1, and
certainly clarifies the analysis.


\section{Introduction Outline}

We study the problem of segmenting DNA methylation data from brain cells. The data are represented by sequences - at each spatial location, two numbers, i.e. coverage and methylation counts, are observed.
Our goal is to segment the sequences, where each segment represent certain state, indicating certain functional regions. One way to represent the states is their methylation probabilities, that is, probabilities of methylation given the states.

There has been growing interests in segmentation with aligned sequences of multiple cell types (citation needed). In these datasets, regions where different cell types have drastically different methylation patterns are of key interest. These regions are called {\em differential methylation region} in computational biology literature. Specifically, if given a state, the methylation probabilities are different for different cell types, we call it a differential methylation state.

In this paper, we use a learning-based framework to study these data. We use a probablistic model - binomial hidden Markov model to model these sequences. In particular, for the single sequence model, given each state $h$ and methylation coverage $c$, the methylation count $m$ is drawn from a binomial distribution $\bin(c,p_h)$, where $p_h \in [0,1]$ is the methylation probability over state $h$. The multiple sequences model is a generalization over the single sequence model: the methylation counts over the cells $(m^1, \ldots, m^r)$ are conditionally independent given $h$, and each of them a drawn from a binomial distribution: $m^i | h \sim \bin(c^i,p_h)$.

There are several key challenges for learning on this data using the binomial HMM model:
\begin{enumerate}
\item For biological applications, we would like to use the full dataset for accurate estimation. Traditional methods such as EM takes a long time to process the data, and thus are unsuitable for this application.
\item Since we are given a large observation space (over the product space of coverage and methylation counts), a naive application of the tensor decomposition algorithm (e.g the tensor power method in~\cite{AGHKT12}) would need high time and space complexities.
\item A line of work on kernel-based tensor decomposition methods for latent variable models have been proposed~\cite{SADX14}, however the time and space complexity of running such algorithm will be at least quadratic in the size of the training data, which is prohibitive.
\end{enumerate}

In this paper, we propose a feature map based framework to address all the challenges. In addition, we develop a novel feature map, i.e. Beta-Bernoulli feature map. After recovering the expected feature map, we apply a novel recovery procedure, estimating the expected methylation probability robustly.

In the experiments, we observe model mismatches, thus making the recovered value of transition matrix and initial probability unstable. To address this issue, we introduce a novel stablization procedure.

We test the performance of our algorithm on both synthetic and real datasets. For synthetic datasets, our improvement over EM are both computational and statistical; we observed better estimation accurary and faster running time. For real datasets, we observe comparable recovery results to EM. However, our algorithm has a significant improvement in running time. %Moreover, our recovered states has a better interpretability compared to EM.


\documentclass[10pt]{article}

% Manage page layout
\usepackage[margin=2.5cm, includefoot, footskip=30pt]{geometry}
\pagestyle{plain}
\setlength{\parindent}{0em}
\setlength{\parskip}{1em}
\renewcommand{\baselinestretch}{1}

\usepackage{blkarray}
\usepackage{multirow}
\usepackage{amsmath}
\usepackage{enumerate}

\title{\textbf{Week 3.} Static games with complete information II: Elimination of dominated strategies, Nash equilibrium}
\date{}

\begin{document}
\maketitle
\vspace{-1cm}

\subsection*{Exercise 1: Iterated elimination of dominated strategies I}

\textbf{Construct a 2-player game such that:}

\begin{enumerate}
    \item Both players have 3 actions.
    \item The game cannot be solved by elimination of dominated strategies.
    \item The game can be solved by iterated elimination of dominated strategies.
\end{enumerate}

\subsection*{Exercise 2: Iterated elimination of dominated strategies II}

Consider the following 3-players game. Here the first player chooses a row,
the second player chooses a column, and the third player chooses a matrix.

\textbf{Can you solve this game using iterated elimination of strategies?}

\begin{equation*}
\begin{blockarray}{ccc}
    & \BAmulticolumn{2}{c}{\text{Matrix } 1} \\ [1em]
    & \text{Col } 1 & \text{Col } 2 \\
    \begin{block}{c(cc)}
        \text{Row } 1 & (2, 1, 6) & (3, 2, 3) \\
        \text{Row } 2 & (0, 4, 0) & (1, 0, 0) \\
    \end{block}
\end{blockarray}\qquad
%
\begin{blockarray}{ccc}
    & \BAmulticolumn{2}{c}{\text{Matrix } 2} \\ [1em]
    & \text{Col } 1 & \text{Col } 2 \\
    \begin{block}{c(cc)}
        \text{Row } 1 & (1, -1, 4) & (2, 1, 4) \\
        \text{Row } 2 & (-1, 4, 0) & (0, 0, 3) \\
    \end{block}
\end{blockarray}
\end{equation*}

\subsection*{Exercise 3: Nash equilibrium vs dominance solvability}

\textbf{Prove the following statements:}

\begin{enumerate}[(i)]
    \item If a pure strategy \(s^{(i)}_{j}\) is dominated by a pure strategy \(s^{(i)}_{k}\) and
    \(\sigma = (\sigma^{(1)}, \dots, \sigma^{(n)})\) is a Nash equilibrium, then
    \(\sigma^{(i)}_{j}=0\).
    \item If the game is dominance solvable such that the unique outcome of
    iterated elimination of dominated strategies is some pure strategy
    \(s=(s^{(1)}, \dots, s^{(n)})\), then \(s\) is a Nash equilibrium.
\end{enumerate}

[Suggestion: One could use contradiction to prove the above statements. For example, for (i)
assume that these was a Nash equilibrium with \(\sigma^{(i)}_{j}>0\), and show
that this would yield some contradiction.]

\subsection*{Exercise 4: Best responses}

Consider the stag hunt game:

\begin{equation*}
    \begin{blockarray}{cccc}
       & & \BAmulticolumn{2}{c}{\underline{\text{player 2}}} \\ [1em]
       & & \text{Stag} & \text{Hare} \\
        \begin{block}{cc(cc)}
\underline{\text{player 1}} & \text{Stag} & (10, 10) & (0, 6) \\
                            & \text{Hare} & (6, 0) & (6, 6) \\
        \end{block}
    \end{blockarray}
\end{equation*}

Suppose player 1 uses the mixed strategy \((x, 1- x)\), where \(x\) is player 1's
probability to Stag. Similarly, player 2's strategy is \((y, 1 - y)\).

\begin{enumerate}[(i)]
    \item For given \(x, y\) compute the players' payoffs \(\pi^{(1)}(x, y),
    \pi^{(2)}(x, y)\) (see Remarks 2.6, 2.7).
    \item For a given \(y\) compute player 1's best response (BR(\(y\))). In
    particular, show that there is some \(y^{*}\) such that all \(x \in [0,
    1]\) are a best response.
    \item Draw the two best response correspondences BR(\(x\)), BR(\(y\)) into a
    \(x-y\) plane. How often do they intersect? What does it mean if they
    intersect?
\end{enumerate}

\end{document}


\subsubsection{Linear Factors}

This is the the most basic case where the degree of the numerator is less than the degree of the denominator and the denominator factors into all linear factors (ie no complex roots) with no repeated roots.
In this case we can write
\begin{equation*}
	\frac{P(x)}{Q(x)} = \frac{A_1}{(x-a_1)} + \ldots + \frac{A_n}{(x-a_n)}
\end{equation*}
Multiplying each side by $Q(x)$,
\begin{equation*}
	P(x) = A_1(x-a_2) \ldots (x-a_n) + \ldots + A_n(x-a_1) \ldots (x-a_{n-1})
\end{equation*}
We can then find each $A_i$ by evaluating both sides at $x=a_i$, since every term except the ith one has an $(x-a_i)$ factor, all terms except the ith one will go to 0.
So,
\begin{equation*}
	A_i = \frac{P(a_i)}{(x-a_i) \ldots (x-a_{i-1})(x-a_{i+1}) \ldots (x-a_n)}
\end{equation*}

\input{./backgroundReview/algebraPreCalc/linearFactors_example.tex}


\section{Stable Matching}

\frame{
{Part 4: Stable Matching}

\tableofcontents[currentsection,hideallsubsections, firstsection=1, sections={1-4}]
}

\begin{frame}
  \frametitle{The Stable Marriage Problem}
  \begin{center}
    \includegraphics[width=.5\textwidth]{../img/marriage1}\\
    Which boy should marry with which girl?
  \end{center}

\end{frame}

\begin{frame}{The Stable Marriage Problem}{Each boy and girl has a preference list}

  \begin{center}
    \includegraphics[width=.4\textwidth]{../img/marriage1}
    \includegraphics[width=.45\textwidth]{../img/marriage2}
  \end{center}

  Which algorithm do you use to match them?
\end{frame}

\begin{frame}{The "Boy-greedy" algorithm}
  \structure{Boy-greedy algorithm}: Each boy, in order, marries to favorite girl:
  \begin{center}
    \includegraphics[width=.7\textwidth]{../img/marriage3}
  \end{center}
\end{frame}

\begin{frame}{The "Boy-greedy" Algorithm}

  \structure{Boy-greedy algorithm}: Each boy, in order, marries to favorite girl:
  \begin{center}
    \includegraphics[width=.7\textwidth]{../img/marriage4}
  \end{center}
\end{frame}

\begin{frame}{The "Boy-greedy" Algorithm}{Final Pairings}

  \begin{center}
    \includegraphics[width=.7\textwidth]{../img/marriage5}
  \end{center}
\end{frame}

\begin{frame}{The "Boy-greedy" Algorithm}{Rogue Couples}
  \begin{center}
    \includegraphics[width=.41\textwidth]{../img/marriage7}
    \includegraphics[width=.45\textwidth]{../img/marriage2}
  \end{center}

  \alert{Girl C} likes \structure{Boy 4} better than \structure{Boy 1}. \structure{Boy 4} likes \alert{Girl C} better than \alert{Girl B}.

  Can we find a pairing without rogue couples?
\end{frame}

\begin{frame}{A stable matching}{Using a Girl Greedy algorithm}
  \begin{center}
    \includegraphics[width=.45\textwidth]{../img/marriage8}
    \includegraphics[width=.45\textwidth]{../img/marriage2}
  \end{center}
\end{frame}

\begin{frame}
  \frametitle{Why is the Stable Marriage Problem Important?}

  {\larger
    \begin{itemize}
    \item School Admissions in the US
    \begin{itemize}
      \item Matching school preference and student preference
    \end{itemize}\bigskip

    \item Server/Client Request Matching
    \begin{itemize}
      \item In large webpages, multiple HTTP servers serve the same page for multiple clients;
      \item Servers are matched to clients by geolocation, etc;
    \end{itemize}\bigskip

    \item Etc...
    \end{itemize}
  }
\end{frame}

\subsection{``Mating Ritual'' Algorithm}

\begin{frame}{The ``Mating Ritual'' Algorithm}

    Let us describe an algorithm to {\bf always} find a
    stable matching:\bigskip

    \begin{itemize}
      \item {States}:
      \begin{itemize}
        \item Each boy is proposing to some girl.
        \item Each girl has a list of proposers.
      \end{itemize}
      \item {\bf Start State}: Every boy is proposing to their favorite girl.
    \end{itemize}

    Algorithm:
    \begin{enumerate}
      \item If all girls have $\leq 1$ proposers in their list, they are paired and the algorithm ends;
      \item Any girl with $> 1$ proposers in their list reject all except their favorite proposer;
      \item If a boy is rejected, they propose to the next girl in their list;
      \item Return to (1).
    \end{enumerate}
\end{frame}

\begin{frame}{The Mating Ritual Algorithm}{Example}

  \begin{columns}
    \column{0.45\textwidth}
    \includegraphics[width=1\textwidth]{../img/marriage2}
    \column{0.55\textwidth}
    {\large
      \begin{itemize}
      \item {\bf iter 1}: No rejections. Proposals:
        \begin{itemize}
        \item A: 2, 4, 5
        \item B:
        \item C: 1
        \item D: 3
        \item E:
        \end{itemize}

      \item {\bf iter 2}: A rejects 2 and 4. Proposals:
        \begin{itemize}
        \item A: 5
        \item B: 2
        \item C: 1, 4
        \item D: 3
        \item E:
        \end{itemize}
      \end{itemize}

    }
  \end{columns}
\end{frame}

\begin{frame}{The Mating Ritual Algorithm}{Example}

  \begin{columns}
    \column{0.45\textwidth}
    \includegraphics[width=1\textwidth]{../img/marriage2}
    \column{0.55\textwidth}
    {\large
      \begin{itemize}
      \item {\bf iter 3}: C rejects 1. Proposals:
        \begin{itemize}
        \item A: 5
        \item B: 1, 2
        \item C: 4
        \item D: 3
        \item E:
        \end{itemize}

      \item {\bf iter 4}: B rejects 1. Proposals:
        \begin{itemize}
        \item A: 5
        \item B: 2
        \item C: 4
        \item D: 3
        \item E: 1
        \end{itemize}
      \end{itemize}
    }
  \end{columns}
\end{frame}

\begin{frame}{The "Mating Ritual" Algorithm}{Proof of Correctness}

  {\larger
  To proof the correctness of an algorithm, requires that you demonstrate two facts:

    \begin{itemize}
    \item The algorithm stops at some point after the start state;

      \bigskip

    \item The algorithm is correct when it stops;
    \end{itemize}
  }
\end{frame}

\begin{frame}{Proof of Correctness}{The algorithm stops}

  {\larger
    Every day, the \structure{total number} of girls in
    the boy's lists is reduced.

    \bigskip

    \begin{itemize}
      \item Every day, {\bf At least one boy} is rejected by {\bf at least one girl}
      \begin{itemize}
        \item If no boy is rejected, it means that all girls have $\leq 1$ boys in their list
        \item If all girls have $\leq 1$ boys in their list, the algorithm stops;
      \end{itemize}
      \item At some point, every boy's list will have {\bf no girls}:
      \begin{itemize}
        \item A boy with no girls in their list will propose to no one.
        \item If no boys propose, then all girl's lists are empty.
        \item Then the algorithm stops.
      \end{itemize}
    \end{itemize}

    \bigskip

    The total size of "Boy's Lists" is \structure{strictly decreasing}, so the algorithm is guaranteed to stop.
  }
\end{frame}

\begin{frame}
  \frametitle{Proof of Correctness: No rogue couples}

  {\larger

    \begin{itemize}
    \item {\bf Lemma 1}: The rank of a girl's favorite is {\bf weakly
      increasing}\\ Every iteration, the girl rejects a favorite {\bf
      iff} she finds a \structure{better one}.

      \bigskip

    \item {\bf Lemma 2}: The rank of a boy's favorite is {\bf weakly
      decreating}\\ Every iteration, the boy stays with current
      favorite, or is rejected and goes to the next lower one.

    \end{itemize}
  }
\end{frame}

\begin{frame}
  \frametitle{Proof of Correctness: No rogue couples}

  {\larger {\bf Invariant:} If $G_i$ is not on $B_j$ list, she has a
    better curent favorite.

    \bigskip

    \begin{itemize}
      \item At the beginning of the algorithm, $G_i$ is on $B_j$ list;
      \item $G_i$ will reject a boy proposing to her only if a better favorite is also proposing to her;
      \item This implies that a girl's favorite never get worse (lemma 1)
    \end{itemize}
  }
\end{frame}

\begin{frame}
  \frametitle{Proof of Correctness: No rogue couples}

  {\larger

    {\bf Lemma:} When boy $B_i$ is paired, he cannot form a rogue
    couple.

    \bigskip

    {\bf Proof by Cases:}
    \begin{itemize}
    \item {\bf Case 1:} $B_i$ tries to form a rogue couple with
      someone not on his list. However, by \alert{Invariant}, any girl
      not on his list has a better favorite, and no rogue couple is
      possible.

    \item {\bf Case 2:} $B_i$ tries to form a rogue couple with
      someone on his list. However, by \alert{Lemma 2}, $B_i$ always
      propose to the best girl in his list, and no rogue couple is
      possible.
    \end{itemize}

    \bigskip

    {\bf Therefore}, no rogue couple is possible.

  }
\end{frame}

% \begin{frame}
%   \frametitle{Extra Topics}
%
%   {\larger
%
%     Check the class materials for ``Hall's Graphs'', for more
%     information on matching.
%
%   }
%
% \end{frame}


% Convolution appendix.

\subsection{Introduction}

fMRI data presents a distinct challenge for relating neural stimuli
to BOLD (blood-oxygen-level dependent) response. fMRI scans record changes
in oxygenation levels of hemoglobin in the brain. However, there is a delay
between the neural stimulus and the change in blood oxygen levels in a given
area. In our case, the neural stimulation comes from a event-related
experimental design. A commonly assumed hemodynamic response to a neurological
stimulus is the double gamma function that can be seen in Figure
\ref{fig:hrf}. The complete hemodynamic response function needs to be modeled
in order to better relate stimulation and the BOLD response from the fMRI scan.
It should be noted that the BOLD response is highly noisy and we are really
trying to capture the blood oxygenation level change to the stimulation.

\subsection{Mathematics}
\subsubsection{Convolution Theory and Mathematics}

To relate stimuli to BOLD response, we convolved the time courses of discrete
stimuli with the assumed response to a single stimulus. At a basic
level, convolution is a distinct combination of two functions (say $f$ and
$g$). This combination is just the ``integral that expresses the amount of
overlap of $f$ as it is shifted over another function $g$''
\cite{weissten2015convolution}.
There are many examples of this, but the following is basic idea that we will
expand off of later.

Let us define the function $f$ as a sum of two gamma functions and $g$ as a
``continuous'' specialized step function (we will examine why these functions
are valuable later). Graphically, we can see their plots in Figure \ref{fig:f}
and \ref{fig:g}, and mathematically we will define them as in the following
equations \ref{eq:gamma2} and \ref{eq:step}, respectively.

\begin{equation} \label{eq:gamma2}
f(t)=\frac{.6}{.17}\cdot  \big[G_1(6,t)-.35 \cdot G_1(12,t) \big]
\end{equation}

where $G_1(k,t) =\frac{1}{\Gamma(k)} t^{k-1} e^{-t}$ (the gamma pdf with
$\theta =1$)

\begin{equation} \label{eq:step}
 g(t)=\Big \{ \begin{tabular}{l c}
 		0  & if  5.85 $\leq$ t $\leq$ 6.15\\
 		.6  & otherwise \\
 		\end{tabular} \end{equation}

\begin{figure}[ht]
\centering
\begin{minipage}[b]{0.45\linewidth}
	\centering
	\includegraphics[width=.8\linewidth]{../images/hrf_pattern.png}
	\caption{$f$ (``Stabilized Function'').}
	\label{fig:f}
\end{minipage}
\quad
\begin{minipage}[b]{0.45\linewidth}
	\centering
		\includegraphics[width=.8\linewidth]{../images/play.png}
	\caption{$g$ (``Moving function'').}
	\label{fig:g}
\end{minipage}
\end{figure}

As mentioned in the earlier definition, if we move $g$ across $f$ from
left to right, we will see something similar to Figure \ref{fig:math} for
discrete time intervals. If we plot these values (the integration of the
differences), we will get a plot very similar to that of $f$ when $f$ is
starting at a certain point. (The plot actually ``cheats'' when $f$ is
negative, and we would have to alter definitions a little bit). If we had
multiple peaks in our $g$ function (i.e. multiple distinct``zero'' places), we
would expect to get multiple non-zero differences between the functions at
each time capture.

\begin{figure}[ht]
	\centering
	\includegraphics[width=.5\linewidth]{../images/math_convolved.png}
	\caption{Convolution of $f$ and $g$.}
	\label{fig:math}
\end{figure}

\subsubsection{Convolution Applied to Stimulus}

The ``continuous'' nature of the step function ``g'' does not extend well into
the discrete time series that we have. However, one approach for fMRI analysis
is to approach the convolution as something slightly different: mathematical
sums. For example, in the previous section, we can treat $f$ as the same, and
$g$ as $g'$ defined in equation \ref{eq:g_prime}.

\begin{equation}\label{eq:g_prime}
 g'(t)=\Big \{ \begin{tabular}{l c}
 		1  & if t=6\\
 		0  & otherwise \\
 		\end{tabular} \end{equation}

We could then find the value of the convolution of $g'$ and $f$ for discrete
integers as in equation \ref{eq:math_discrete}.

\begin{equation}  \label{eq:math_discrete}
r(t)=  f(t-6)
\end{equation}

If we allow for multiple non-zero periods in $g'$, we can get a more general
model in equation \ref{eq:math_discrete_extend}, where each $t_i$ is a value
when $g'(t_i) \neq 0$:

\begin{equation}  \label{eq:math_discrete_extend}
r(t)= \sum_{i=1}^n f(t-t_i)
\end{equation}

This equation gives a good glimpse into what the hemodynamic response would be
after stimulus at time $t_i$ for $i \in {1,...,n}$. Moreover, one could extend
the idea to include a ``strength'' value of the stimulus by changing the
$g'(t_i)$ to values other than 1. If that was the case, we would change
the response equation to Equation \ref{eq:math_discrete_final} to allow us to
include all discrete $t$ into the equation where $g'(t_i)$ is now expected to
be zero (so $n$ becomes much larger).

\begin{equation}  \label{eq:math_discrete_final}
r(t)= \sum_{i=1}^n g'(t_i) f(t-t_i)
\end{equation}

With this new equation, we can consider function $f$ and $g'$ displayed
graphically [Figure \ref{fig:hrf}, \ref{fig:on_off}, respectively] and
their ``convolved'' output [Figure \ref{fig:convolve1}].

\begin{figure}[ht]
\centering
\begin{minipage}[b]{0.45\linewidth}
	\centering
	\includegraphics[width=.8\linewidth]{../images/hrf_pattern.png}
	\caption{$f$ (``Stabilized Function'').}
	\label{fig:hrf}
\end{minipage}
\quad
\begin{minipage}[b]{0.45\linewidth}
	\centering
	\includegraphics[width=.8\linewidth]{../images/on_off_pattern.png}
	\caption{$g'$ (``Moving function'').}
	\label{fig:on_off}
\end{minipage}
\end{figure}

\begin{figure}[ht]
	\centering
	\includegraphics[width=.5\linewidth]{../images/initial_convolved.png}
	\caption{Convolution of $f$ and $g'$.}
	\label{fig:convolve1}
\end{figure}

\subsection{Approach to our Specific Problem}

\subsubsection{Returning to Our fMRI Data}

We can now apply this discrete approach to convolution between $f$ and $g'$
to our data. The $f$ is actually a common representation of the hemodynamic
response, and the $g'$ is a good representation of the stimuli from an
event-related trial \cite{brett2015course}.

\subsubsection{Naive Approach (using \texttt{np.convolve})}

A \texttt{numpy} function \texttt{np.convolve}, which takes advantage of
fast Fourier transforms for efficiency (it boils down to fewer computations
using roots of unity) is commonly used to do discrete convolution. A naive
approach for convolving two functions would use this function directly, but
\texttt{np.convolve} assumes that the intervals between stimuli mirror the
desired intervals between prediction intervals. As such, we should not naively
apply \texttt{np.convolve} to our data (though we do for a base model).
Even so, exploring this naive approach for convolving the hemodynamic response
gives an idea about what not to do, and sets a higher bar for accuracy.

\subsubsection{Needed Improvements: Moving beyond naive \texttt{np.convolve}}

Our data fails to meet the assumption that the intervals are equidistant,
which is required to naively apply \texttt{np.convolve}. Especially in our
case, there was not an simple fix, such as performing some basic rounding in
order to then correctly utilize \texttt{np.convolve}. All the following
approaches improve on the basic \texttt{np.convolve} approach's accuracy, but
ultimately circle back to incorporating \texttt{np.convolve} to improve the
speed of the convolution.

Our condition file (\texttt{cond1}) lists stimulus times for when the
individual pumped the balloon but did not pop it. For subject 001, the
first 10 data points are as follows [Figure \ref{table:cond1}]:

\vspace{5mm}

\begin{figure}[ht]
\begin{center}
\begin{tabular}{|cccccccccc|}
  \hline
0.0671 &
2.1251 &
3.7681 &
5.6601 &
7.8673 &
9.3443 &
19.7831 &
22.0402 &
23.5837 &
25.1434 \\
 \hline

  \end{tabular}
   \caption{First 10 values for Sub 001, condition 1.}
  \label{table:cond1}
\end{center}
\end{figure}

Clearly, this short time series does not align with idealized scans that
start at $t=0$ and occur every two seconds apart. As such, we had to go back
to the drawing board to try to reproduce our expected hemodynamic response for
the entire time course.

\subsection{Summary of Approaches}
Our first approach attempts to correctly match the theory underlying our data.
Our second approach tries to utilize \texttt{np.convolve} by expanding the
grid of desired results (thanks to advice from  Matthew Brett, Jean-Baptiste
Poline, and Jarrod Millman).

\subsubsection{Initial Correction to Represent Theoretical Idea}
To account for our data's lack of any easily identifiable grid structure
between when a stimulus was recorded and when our scans occurred (on the
order of every 2 seconds), we went back to the theory of convolution and
implemented code to recreate equation \ref{eq:math_discrete_final} directly.
To do so, we also had to create a function that works with all discrete points
of $f$, the stimulus response as potential starts of the hemodynamic response,
multiplied by the actual value of $f$, as seen in equation
\ref{eq:code_convolve}:

\begin{equation} \label{eq:code_convolve}
r(t)= \sum_{i=1}^n g'_{i} f(t-t_i)
\end{equation}

where $g'_{i}$ is the value of $g'$ at $t_i$ (allowing for zeros and varying
non-zero values of $g'$).

\subsubsection{Matrix Multiplication}
Equation \ref{eq:code_convolve}, reproduced below

$$r(t)= \sum_{i=1}^n g'_{i} f(t-t_i)$$

can be rewritten as a matrix multiplication problem and can be seen below:

\begin{equation} \label{eq:matrix_code_convolve}
r(t)=  g^*(t)^T f^*(t)
\end{equation}

where $g^*$ is a vectorized function of $g'$ of $t$ as a scalar output and
$f^*$ is the vector of $f$ values (irrespective of location, as the $t^*$
takes that into account). This is a useful representation, since matrix
multiplication is faster for Python's numpy arrays.

\subsubsection{Using FFT with \texttt{np.convolve}}
The ``theoretical'' solution lacked computation efficiency (despite
considerable speed improvements from matrix multiplication), so we also
approached the problem by creating a denser grid between each scan (two seconds
apart). Then we rounded the actual times of the stimulus to meet this more
finely scaled grid. This allowed us to utilize \texttt{np.convolve} with its
faster, FFT-based algorithms, before reducing back down to our two-second
grid.

\subsection{Example}

Now that we have discussed the theory and possible implementations
for convolving event-related stimulation, we will look at a basic example from
our data. In doing so, we examine the trade-offs between theoretical accuracy
and computational efficiency. We will consider just subject 001's condition
files.

\begin{figure}[ht]
\centering
	\begin{minipage}[b]{0.45\linewidth}
		\centering
		\includegraphics[width=.8\linewidth]{../images/convolution_vs_neural_stimulus}
		% needs to be from the event_related_HRF_script2.py
		\caption{\scriptsize{Different convolution functions vs. the Neural stimulus}}
		\label{fig:convolution}

	\end{minipage}
\quad
	\begin{minipage}[b]{0.45\linewidth}
		\centering
		\begin{tabular}{|l | c|}
		\hline
		name in graph       & Speed per loop \\
		\hline
		np naive approach & 14.4 $\mu$s  \\
		user 2     		    & 972 ms  \\
		user 3     		    & 1.15 s    \\
		user 4 (15 cuts)      & 98.3 ms \\
		user 4 (30 cuts)      & 185 ms  \\
		user 5     	 	    & 110 ms   \\
		\hline
		\end{tabular}
		\vspace{5mm}
		\caption{\scriptsize{Speed to create HRF predictions for Subject 001,
		all conditions}}
		\label{table:convolution}
	\end{minipage}
\end{figure}

The first method in the table ``np naive approach'' method blindly plugs
our data into the \texttt{np.convolve} function and is provided to showcase
potential speed. The ``user 2'' method  was the first approach to match the
theory, though it matches the stimulation times instead of the scan times.
The ``user 3'' method is the most theoretically sound model (and is our
standard for accuracy). The ``user 5'' model  is our matrix version of
``user 3'' and has the same accuracy, but is observably faster. The
``user 4'' methods involve the grid cut usage of \texttt{np.convolve}
with notations for the number of slices between each scan. We concluded that
"user 4 (15 cuts)" was the best approach since it gives us speed and very
close accuracy to the golden standard --- "user 3".

%While this final method does lose some accuracy compared to the more
%theoretically rigorous approach, the trade-off brings considerable gains in
%computational efficiency.

%Needed references:
%Brett, Matthew and Poline, J-B  (2013). Convolution. Retrieved from
% http://practical-neuroimaging.github.io/on_convolution.html

% Weissten, Eric W. ``Convolution''. From MathWorld - A Wolfram Web  Resource.
% Retrieved from htt://mathworld.wolfram.com/Convolution.html


In this cahpter, we describe the implementation of a sample
application---a solver for the maximum cut problem. This application
is a prototypical example of branch and cut, i.e., BCP with a fixed
set of variables. No column generation is used in this implementation.
This simplifies many of the basic tasks.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{The Max Cut Problem}
\label{MCP}

Given an undirected graph $G=(N,E)$ with edge weight function $\omega:
E \rightarrow {\rm \bf R}$, the Maximum Cut Problem (MCP) is that of
partitioning the nodes into two subsets in such a way that the total
weight of the edges in the cut separating the two sets is maximized.
This is a well-known problem---several branch and cut algorithms for
dense graphs have been presented in
\cite{A:barahona-junger-reinelt,A:desimone-rinaldi}. In the following
description, we consider complete graphs only. For a complete graph
undirected graph with $n$ nodes, a linear relaxation of the integer
programming problem is givem by
\begin{eqnarray}
\min \;\; & c^Tx & \nonumber \\
{\rm s.t.} \;\;\; && \nonumber \\
& x_{ij}+x_{jk}+x_{ik} \le  2 & \;\;\;\forall (i,j,k) \in N^3
\label{tri-0} \\
& x_{ij}-x_{jk}-x_{ik} \le  0 & \;\;\;\forall (i,j,k) \in N^3
\label{tri-1} \\
& \hskip .51in 0 \le x_{ij} \le 1 & \;\;\;\forall (i,j) \in E \\
\end{eqnarray}
Here $x_{ij}$ takes value 1 if the edge $(i,j)$ appears in the cut, and 0
otherwise. Constraints (\ref{tri-0}) - (\ref{tri-1}) are called the
{\it triangle inequalities} and they define facets of the cut polytope
(see \cite{A:barahona-mahjoub:cut-polytope}).

Another set of inequalities, which is a superset of (\ref{tri-0}) -
(\ref{tri-1}), is the following. Let $C$ be a cycle and $F \subseteq
C$ with $|F|=2k+1$. Then
\begin{equation}
\sum_{e \in F} x_e - \sum_{e \in C\backslash F} x_e \le |F|-1 \label{c}
\end{equation}
is a valid inequality. This follows from the fact that the
intersection of a cycle and a cut has even cardinality. Note, that
although this set of inequalities include those in (\ref{tri-0}) -
(\ref{tri-1}), the polytope defined by these is the same, i.e., these
inequalities can be derived from those in (\ref{tri-0}) -
(\ref{tri-1}) (see \cite{A:barahona-mahjoub:cut-polytope}).

A polynomial time separation algorithm for this class of inequalities
has been given in \cite{A:barahona-mahjoub:cut-polytope}.
However we use a faster heuristic as
follows. Let $\bar{x}$ be the fractional solution we want to separate
and define weights
\begin{equation}
w_e = c_e \cdot \max(\bar{x}_e, 1-\bar{x}_e).
\end{equation}
Then find a maximum weighted spanning tree $T$ with weights $w$. For
an edge $e \in T$, if $\bar{x}_e \ge 1-\bar{x}_e$ then the end-nodes
of $e$ should be on opposite sides of the cut---we give a label ``A''
to this edge. Otherwise, if $ \bar{x}_e < 1-\bar{x}_e$ then the
end-nodes should be on the same side of the cut, and we give the label
``B'' to this edge. Once every edge in $T$ has been labeled we have a
heuristic cut $K$. For each edge $e \notin T$, we add it to $T$ and
look at the cycle $C$ that is created. If $e \in K$, we test the
violation of an inequality (\ref{c}), where the set $F$ is given by
the A-edges. If $e \notin K$ the set $F$ is given by the A-edges and
the edge $e$. Although, as we noted above, inequalities (\ref{c}) are
implied by (\ref{tri-0}) - (\ref{tri-1}), we use (\ref{c}) because our
simple separation heuristic is faster than enumerating triangles.

\section{Implementation}

Because the size of the problems we can currently solve is small, we
can easily include all the edge variables explicitly. Hence, we do not
need to consider dynamiuc column generation. Hence, we do not need to
concern ourselves with the {\tt BCP\_vg\_user} class or the {\tt
BCP\_var\_algo} class. To simplify things further, we decided not to
use a separate cut generator either. This is usually a good approach
when cut generation is relatively inexpensive. It is also a good idea
during initial development since it makes debugging much easier.
Because we are not using a separate cut generator, we do not need to
consider the {\tt BCP\_cg\_user} class either.

As with virtually any BCP implementation, we will need to consider the
{\tt BCP\_tm\_user} and {\tt BCP\_lp\_user} classes. Also, because we
will be dynamically generating algorithmic cuts, we will need to
derive a new class to represent the cycle cuts (\ref{c}) from the
class {\tt BCP\_cut\_algo}. Finally, we will need to derive a new
class for describing the feasible solutions from {\tt BCP\_solution}.
In the remainder of the section, we provide a high-level description
of each of these classes. The reader is encouraged to look at the
source code and the HTML documentation for more detail. See Chapter
\ref{getting-started} for information on getting and examining the
source code and documentation.

\subsection{\tt MC\_tm}
\label{MC-tm}

This is the class derived from the {\tt BCP\_tm\_user} class. This
class is derived for the purpose of overriding a variety of functions
that we need to customize. These consist mainly of routiines that pass
data between the processes during parallel execution and the routines
for describing the problem core and root node. Below, we list each
function and describe how it was re-implemented.

\begin{itemize}

        \item {\tt unpack\_feasible\_solution()}: This subroutine
        exists to unpack the user-defined solution class described in
        Section \ref{MC-solution}. The corresponding {\tt
        pack\_feasible\_solution()} routine will be described in
        Section \ref{MC-lp}. Also, see Section \ref{MC-solution} for a
        description of how the feasible solutions are represented.

        \item {\tt pack\_module\_data()}: Here, we are packing the
        data that needs to be sent to the LP process. This consists of
        the number of nodes and a list of the edges in the graph. The
        corresponding {\tt unpack\_module\_data()} routines is
        described in Section \ref{MC-lp}.

        \item {\tt pack\_cut\_algo()}: Here, we pack the cycle cuts to
        be sent to the LP solver. The corresponding {\tt
        unpack\_cut\_algo()} routines is described in Section
        \ref{MC-lp}.

        \item {\tt unpack\_cut\_algo()}: Here, we unpack the cycle
        cuts that are received from the LP solver. The corresponding
        {\tt pack\_cut\_algo()} routines is described in Section
        \ref{MC-lp}.

        \item {\tt initialize\_core()}: Essentially for convenience
        and ease of implementation, we place all the variables in the
        core. This is possible since we are not using column
        generation, but may not be the most efficient method. None of
        the cuts are placed in the core since we don't have an
        inherently important subset that we know should never be
        removed from the problem.

        \item {\tt create\_root()}: To initialize the root node, we
        use some heuristics to generate an initial set of cycle cuts.
        However, as noted before, these are ``extra'' cuts and do not
        get put into the core. They may be removed later in the
        calculation.

        \item {\tt display\_feasible\_solution()}: This routine is
        used essentially to display the solutions in a more
        ``user-friendly'' way, instead of simply as a list of variable
        indices and values. See Section \ref{MC-solution} for a
        description of how the feasible solutions are represented.

\end{itemize}

\subsection{\tt MC\_lp}
\label{MC-lp}

This is the class derived from the {\tt BCP\_lp\_user} class. Again, this
class is derived for the purpose of overriding a variety of functions
that we need to customize. These consist not only of routiines that pass
data between the processes, as before, but also routines for
generating cuts and performing strong branching. Below, we list each
function and describe how it was re-implemented.

\begin{itemize}

        \item {\tt unpack\_module\_data()}: Here, we unpack the data
        sent from the TM. This data is stored in a user-defined class
        called {\tt MC\_problem}.

        \item {\tt pack\_cut\_algo()}: Here, we pack the cycle cuts to
        be sent to the TM. The cuts are represented as a list of
        edges---first the edges in the set $F$, then the edges not in
        $F$. To contruct the corresponding valid inequality, we need
        only determine which edges variables are present in the
        relaxation. See the description of {\tt cuts\_to\_rows()} below.

        \item {\tt unpack\_cut\_algo()}: Here, we unpack the cycle
        cuts that are received from the TM along with the description
        of a subproblem.

        \item {\tt modify\_lp\_parameters()}: Here, we modify the LP
        parameters before solution of the relaxation commences.

        \item {\tt test\_feasibility()}: Because integrality of the
        solution is not enough to imply feasibility, we needed to
        override this method. If it is found that the solution is
        integral but not feasible, then cuts proving the infeasibility
        are easy to derive and are added to the LP relaxation,
        allowing the solution process to continue.

        \item {\tt pack\_feasible\_solution()}: Here, any feasible
        solutions that are found are packed and sent to the TM for
        storage. See Section \ref{MC-solution} for a description of
        how the feasible solutions are represented.

        \item {\tt cuts\_to\_rows()}: This subroutine generates the
        rows of the current LP relaxation corresponding to the cuts to
        be added. For cycle cuts, this consists simply of determining
        which of the edge variables that have a positive coefficient in
        the cycle cut, i.e., the variables corresponding to the edges
        of the corresponding cycle, are active in the current
        subproblem. For each variable corresponding to an edge that is
        in the cycle cut and also active in the subproblem, the
        corresponding matrix coefficient must be added to the
        row description.

        \item {\tt compare\_cuts()}: This routine simply compares two
        cuts and determines if they are the same cut. In the case of
        cycle cuts, this is straightforward.

        \item {\tt generate\_cut\_in\_lp()}: This is the subroutine
        that generates the cuts to be added to the relaxation. A
        description of the algorithm for generating the cycle cuts was
        given in Section \ref{MCP}.

        \item {\tt select\_branching\_candidates}: Here, we select the
        edges to be branched on. We branch basically on edges that are
        have high cost and are close to value .5.

\end{itemize}

\subsection{\tt MC\_solution}
\label{MC-solution}

Although feasible solutions to this problem consist of a set of edges,
they can be more compactly represented as simply a list of the nodes
contained in either shore of the cut. This user-defined class is used
for representing the solutions in this more compact, intuitive
fashion.

\subsection{\tt MC\_cycle\_cut}

This class was derived from {\tt BCP\_cut\_algo} to contain the
representation of the cycle cuts (\ref{c}). They are stored simply as
a list of edges in the cycle. The list is in two parts---first, the
edges in the set $F$ are listed and then those not in the set $F$. Of
course, the cardinality of the set $F$ has to be stored as well.

\subsection{Other Classes}

There are a number of other classes that we have defined to hold
data used during the solution process. Please see the HTML
documentation and the source code itself for a list of these.


\subsection{Graph}
\begin{center}
\begin{longtable}{|p{4cm}|p{4cm}|p{7cm}|}
\hline
\textbf{Name} & \textbf{Worst Complexity} & \textbf{Description} \\
\hline
Depth First Search & O(\textbar E\textbar + \textbar V\textbar )
           & \begin{enumerate}
                \item Push first vertex onto a stack
                \item Pop vertex off the stack \& process
                \item Loop through vertex successors, if not already seen
                      add it to the stack
                \item Repeat until stack is empty
              \end{enumerate}\\
\hline

Breadth First Search & O(\textbar E\textbar + \textbar V\textbar )
           & \begin{enumerate}
                \item Push first vertex onto a queue
                \item Pop vertex off the queue \& process
                \item Loop through vertex successors, if not already seen
                      add it to the stack
                \item Repeat until stack is empty
              \end{enumerate}\\
\hline

BST pre-order & O(n)
              & {\begin{lstlisting}[language=Java]
Deque<TreeNode> stack = new ArrayDeque<>();
stack.push(root);
while(!stack.isEmpty()) {
  TreeNode node = stack.pop();
  // Process node here
  if (node.right != null) {
    stack.push(node.right);
  }
  if (node.left != null) {
    stack.push(node.left);
  }
}
                  \end{lstlisting}

                } \\
\hline

BST in-order & O(n)
              & {\begin{lstlisting}[language=Java]
Deque<TreeNode> stack = new ArrayDeque<>();
TreeNode node = root;
while (!stack.isEmpty() || node != null) {
  if (node != null) {
    stack.push(node);
    node = node.left;
  } else {
    node = stack.pop();
    // Process node here
    node = node.right;
  }
}
                  \end{lstlisting}

                } \\
\hline

BST post-order & O(n)
              & {\begin{lstlisting}[language=Java]
Deque<TreeNode> stack = new ArrayDeque<>();
stack.push(root);
TreeNode prev = null;
while(!stack.isEmpty()) {
  TreeNode current = stack.peek();
  if (prev == null || prev.left.equals(current) || prev.right.equals(current)) {
    if (current.left != null) {
      stack.push(current.left);
    }
    else if (current.right != null) {
      stack.push(current.right);
    }
  } else if (current.left.equals(prev)) {
    if (current.right != null) {
      stack.push(current.right);
    }
  } else {
    //Process current here
    stack.pop();
  }
  prev = current;
}
                  \end{lstlisting}

                } \\
\hline

Prims MST priority queue/min heap & O(\textbar V\textbar log\textbar V\textbar + \textbar E\textbar log\textbar V\textbar ) $\implies$ O(\textbar E\textbar log\textbar V\textbar)
            & \begin{enumerate}
                \item Push first vertex's edges onto a priority queue
                \item While the priority queue is not empty do:
                \item Poll the smallest edge and add it to the mst, if its
                      connecting vertex has other edges to it in the
                      priority queue, remove these because this edge was smaller
                      and they don't need looking at.
              \end{enumerate}\\

\hline

Kruskals MST & O(\textbar E\textbar log\textbar V\textbar)
         & \begin{enumerate}
                \item Push all edges onto a priority queue
                \item Pop first V edges off the queue, for each edge:
                \item if egde is not already in a connected tree of the answer
                      add it to the mst. This is tested via a union find data
                      structure

              \end{enumerate}\\

\hline

Dijkstra & O((\textbar V \textbar + \textbar E \textbar) log \textbar V \textbar
         & \begin{enumerate}
                \item Assign distance to the initial node 0 and the others
                      $\infty$
                 \item Add initial node to a priority queue and then whilst the
                       queue is not empty do:
                 \item Pop closest vertex off queue, for all its edges if any
                       edges make the distance closer than the current recorded distance update
                       the vertex in the priority queue
                 \item Final path can be used via back tracking
              \end{enumerate}\\

\hline

A* &
   & \begin{enumerate}
        \item A* uses a best first search, keeping a sorted priority queue
              of alternative path segments along the way
        \item It uses a heuristic $f(x)$ made up of $g(x)$ which is the
              past path cost, and $h(x)$ which is the future path cost estimation
        \item Starting with the initial node it maintains a priority queue of nodes
              (known as the fringe). The lower $f(x)$ the higher the priority
      \end{enumerate}\\
\hline

\end{longtable}
\end{center}


%% LyX 2.3.3 created this file.  For more info, see http://www.lyx.org/.
%% Do not edit unless you really know what you are doing.
\documentclass[english]{article}
\renewcommand{\sfdefault}{cmss}
\usepackage[T1]{fontenc}
\usepackage[latin9]{inputenc}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{babel}
\begin{document}

\section*{Transport equation}

The convection-diffusion equation for the transport of temperature
\textit{T} is

\begin{equation}
\frac{\partial T}{\partial t}=-U\frac{\partial T}{\partial x}+k\frac{\partial}{\partial x}(\frac{\partial T}{\partial x})+S
\end{equation}
where \textit{U} is velocity and \textit{S} a source term. For a non
existing convection case, Equation (1) becomes the diffusion Equation

\begin{equation}
\frac{\partial T}{\partial t}=k\frac{\partial}{\partial x}(\frac{\partial T}{\partial x})+S
\end{equation}
that, for a steady-state, is

\begin{equation}
0=k\frac{\partial}{\partial x}(\frac{\partial T}{\partial x})+S
\end{equation}
These equations are solved by a Finite Volume Method (FVM) and by
a Finite Difference Method (FDM).

\section*{Finite Volume Method }

In the Finite Volume Method, values for the above diferential equations
are calculated at discrete places on a grid of volumes, shown in Figure
1,
\begin{center}
\includegraphics{figure_1}
\par\end{center}

\begin{center}
\textbf{Figure 1}: Temperature at center of cell, and on left and
right cell borders.
\par\end{center}

\noindent where temperature at the center of a cell is \textit{T}\textit{\scriptsize{}P},
\textit{T}\textit{\scriptsize{}r} e \textit{T}\textit{\scriptsize{}l}
is temperature at the border between cells, right and left, \textit{T}\textit{\scriptsize{}R}
e \textit{T}\textit{\scriptsize{}L} is temperature at the right and
left cell centers. Integrating Equation 3 over a cell volume, and
considering the rate of accumulation over the volume \textit{V} equal
to the flow across the surfaces of the control volume, Equation (6),
where \textit{n} is the unit normal vector pointing out of the control
volume and \textit{A} is the cross sectional area of the volume.

\begin{equation}
\int\left[k\frac{\partial}{\partial x}(\frac{\partial T}{\partial x})+S\right]dV=0
\end{equation}

\begin{equation}
\int\left[\frac{\partial}{\partial x}(k\frac{\partial T}{\partial x})dV\right]+SV=0
\end{equation}

\begin{equation}
\int(k\frac{\partial T}{\partial x}n)dA+SV=0
\end{equation}

\noindent Equation (6) is written in terms of the flux leaving the
right face \textit{r} minus the flux entering the left face \textit{l}
in Figure (1)

\begin{equation}
(kA\frac{\partial T}{\partial x})_{r}-(kA\frac{\partial T}{\partial x})_{l}+SV=0
\end{equation}

\noindent Equation (7) for the temperature gradient at the boundaries
is discretized in terms of the temperature at the cell centers, where
$\Delta x$ is the distance between cell centers

\begin{equation}
kA\frac{T_{R}-T_{P}}{\Delta x}-kA\frac{T_{P}-T_{L}}{\Delta x}+SV=0
\end{equation}

\noindent Solving Equation (8) for \textit{T}$_{P}$ and making an
individual set of equations for each cell results in a system of equations
of the form

\begin{equation}
-\frac{kA}{\Delta x}T_{P-1}+2\frac{kA}{\Delta x}T_{P}-\frac{kA}{\Delta x}T_{P+1}=SV
\end{equation}

\noindent for each grid position except the first and last volumes
in the grid. For the first and last volumes in the grid, the temperature
T$_{A}$ and T$_{B}$ at the borders is also considered, as well as
a half distance to the border. In matrix form, where the first and
last row are the two special border cases:

\begin{equation}
\left[\begin{array}{ccccc}
\frac{kA}{\Delta x}+\frac{kA}{0.5\Delta x} & -\frac{kA}{\Delta x} & 0 & 0 & 0\\
-\frac{kA}{\Delta x} & 2\frac{kA}{\Delta x} & -\frac{kA}{\Delta x} & 0 & 0\\
0 & -\frac{kA}{\Delta x} & 2\frac{kA}{\Delta x} & -\frac{kA}{\Delta x} & 0\\
0 & 0 & -\frac{kA}{\Delta x} & 2\frac{kA}{\Delta x} & -\frac{kA}{\Delta x}\\
0 & 0 & 0 & -\frac{kA}{\Delta x} & \frac{kA}{\Delta x}+\frac{kA}{0.5\Delta x}
\end{array}\right]\left[\begin{array}{c}
T_{1}\\
T_{P-1}\\
T_{P}\\
T_{P+1}\\
T_{N}
\end{array}\right]=\left[\begin{array}{c}
SV+T_{A}\frac{kA}{0.5\Delta x}\\
SV\\
SV\\
SV\\
SV+T_{B}\frac{kA}{0.5\Delta x}
\end{array}\right]
\end{equation}

\noindent This matrix is solved for the vector of temperatures \textit{T}.

\section*{Finite Difference Method}

The diffusion equation (2)

\begin{equation}
\frac{\partial T}{\partial t}=k\frac{\partial^{2}T}{\partial x^{2}}+S
\end{equation}
can be approximated by finite differences with a forward difference
in time and a central difference in space scheme.

\begin{equation}
\frac{T_{i}^{n+1}-T_{i}^{n}}{\varDelta t}=k\frac{T_{i-1}^{n}-2T_{i}^{n}+T_{i+1}^{n}}{\varDelta x^{2}}+S_{i}^{n}
\end{equation}

\noindent that can be solved for the temperature at the next time
step \textit{T}$^{n+1}$

\begin{equation}
T_{i}^{n+1}=T_{i}^{n}+k\frac{\varDelta t}{\varDelta x^{2}}(T_{i-1}^{n}-2T_{i}^{n}+T_{i+1}^{n})+S_{i}^{n}
\end{equation}

\section*{References}
\begin{enumerate}
\item Computational Fluid Dynamics Fundamentals Course. A. Wimshurst. 2019.
\item An Introduction to Computational Fluid Dynamics: The Finite Volume
Method. H. Versteeg, W. Malalasekera. 2007.
\item Finite Difference Computing with PDEs. A Modern Software Approach.
H. Langtangen, S. Linge. 2016.
\end{enumerate}

\end{document}


\subsection{Interpretations}

An interpretation assigns meaning to propositional variables in a formula.

For example an interpretation of the formula \(\theta \lor \gamma \) assigns values to each of \(\theta \) and \(\gamma \).

\subsection{Satifisable}

A formula is satisfisable if there is some interpretation where it is true.

For example \(\theta \) is satisfisable but \(\theta \land \neg \theta \) is not.

\subsection{Tautology}

A formula is a tautology if it is true in all interpretations.

Examples of tautologies include:
\begin{itemize}
\item \(\theta \lor \neg \theta \)
\end{itemize}


\section{Marciani Normal Form}
\label{sec:marciani-normal-form}

We introduce the notion of context-free grammars in Marciani Normal Form (MNF).
We prove that every MNF grammar generates a regular language.

\begin{definition}
	\label{dfn:mnf}
	A context-free grammar $G=(V,V_{N},S,P)$ is said to be in Marciani Normal
	Form (MNF) iff $G$ enjoys the looking forward property and there exists a
	pseudo-regular partition $\Upsilon_{A}$ for all symbol $A\in V_{T}$.
\end{definition}

\begin{theorem}
	\label{thm:mnf}
	A context-free grammar $G=(V_{T},V_{N},S,P)$ in Marciani Normal Form always
	generates a regular language.

	\begin{proof}
		If $G$ is in Marciani Normal Form, then for each non-terminal $A$ there
		exists a pseudo-regular partition $\Upsilon_{A}$.
		Notice that to every pseudo-regular partition
		$\Upsilon_{A}=\{\{p|p\in P_{A},A\rightarrow\alpha A\},
		\{p|p\in P_{A},A\rightarrow A\beta\},\{p|p\in P_{A},A\rightarrow\gamma\}\}$
		can be associated a both-linear language equation $A=\alpha A+A\beta+\gamma$.
		By the application of the Marciani's Rule, we know that the least
		solution of the previous equation is $\alpha^{*}\gamma\beta^{*}$,
		so $L(A)=L(\alpha^{*}\gamma\beta^{*})$.

		As a consequence of the previous results and the looking forward property,
		there exists a regular expression $e$ such that $L(S)=L(e)$, then $L(S)$
		is a regular language, that is $G$ is regular.
	\end{proof}
\end{theorem}


\section*{Common angles}

\begin{table}[!ht]
\setlength{\tabcolsep}{1em} % Increase the horizontal table cell margin
\centering
  \tabulinesep=1.5mm
  \begin{tabu}{|c|c|c|c|c|c|c|c|}
    \hline
    \textbf{Degrees}
      & $\SI{0}{\degree}$
      & $\SI{30}{\degree}$
      & $\SI{45}{\degree}$
      & $\SI{60}{\degree}$
      & $\SI{90}{\degree}$\\
    \hline
    \textbf{Radians}
      & $\displaystyle 0$
      & $\displaystyle \frac{\pi}{6}$
      & $\displaystyle \frac{\pi}{4}$
      & $\displaystyle \frac{\pi}{3}$
      & $\displaystyle \frac{\pi}{2}$\\
    \hline
    \textbf{\bm{$\sin \theta$}}
      & $\displaystyle 0$
      & $\displaystyle \frac{1}{2}$
      & $\displaystyle \frac{\sqrt 2}{2}$
      & $\displaystyle \frac{\sqrt 3}{2}$
      & $\displaystyle 1$\\
    \hline
    \textbf{\bm{$\cos \theta$}}
      & $\displaystyle 1$
      & $\displaystyle \frac{\sqrt 3}{2}$
      & $\displaystyle \frac{\sqrt 2}{2}$
      & $\displaystyle \frac{1}{2}$
      & $\displaystyle 0$\\
    \hline
    \textbf{\bm{$\tan \theta$}}
      & $\displaystyle 0$
      & $\displaystyle \frac{\sqrt 3}{3}$
      & $\displaystyle 1$
      & $\displaystyle \sqrt 3$
      &\\
    \hline
  \end{tabu}
\end{table}