\newpage\section{Binomial Identities}

\theo{https://en.wikipedia.org/wiki/Vandermonde's_identity}{Vandermonde's identity}{
    For every positive integer $m, n, k$
    \begin{center}
        \tcboxmath[arc=0mm, colback=black!5!white, boxrule=0mm]{
            \binom{m+n}{k} = \sum_{i=0}^{k} \binom{m}{i}\binom{n}{k-i}
        }
    \end{center}
    \begin{center}
        \tcboxmath[arc=0mm, colback=black!5!white, boxrule=0mm]{
            {2n \choose n} = {n \choose 0}^2+ {n \choose 1}^2 \dots + {n \choose n}^2
        }
    \end{center}
}

\prob{https://artofproblemsolving.com/community/c6h1061914p4600670}{Somewhere}{M}{
    Let $n$ and $m$ be positive integers with $n<m$. Prove that
    \begin{center}
        \tcboxmath[arc=0mm, colback=black!5!white, boxrule=0mm]{
        \sum_{k=0}^n (-1)^{n-k}\frac{1}{m-k}\binom{n}{k}=\frac{1}{(n+1)\binom{m}{n+1}}
        }
    \end{center}
}

\solu{[MellowMelon]
    The left side is the $n$th finite difference of the unique $n$th degree
    polynomial satisfying $P(x) = \frac{1}{m-x}$ for $x = 0,1, \ldots, n$.
    This means that if $a_n$ is the $x^n$ coefficient of $P$, then the left
    side is equal to $n! \cdot a_n$. Setting this equal to the right side, it
    suffices to show that
    \[ a_n = \frac{1}{(n+1)! \binom{m}{n+1}} = \frac{1}{m(m-1)\cdots (m-n)}. \]

    Let $Q(x) = (x-m)P(x) + 1$. Then $Q$ is degree $n+1$, has roots
    $0,1,\ldots,n$, and has leading coefficient $a_n$. So
    \[ Q(x) = a_nx(x-1)(x-2)\cdots (x-n). \]
    We also know $Q(m) = 1$. Plugging $x = m$ into the above equation and
    isolating $a_n$ gives us exactly what we need to show.
}

\rem{
    The idea to express the left hand side with finite differences is pretty
    simple. The crux idea is to finding a $Q$ that works. Since we have to
    show $a_n\left(m\right)\left(m-1\right) \dots \left(m-n\right) =1$, which
    looks like a polynomial with leading coefficient $a_n$ with roots $0, 1, 2,
    \dots n$ and $Q(m) = 1$. How can we get that?

    The takeaway is, if the problem requires you to find a polynomial that
    almost looks like the one given in the problem, try to modify the given
    polynomial in some way first.

    In this particular case was to modify $P$ bu using the fact $(x-m)P(x) =
    -1$.
}

\solu{[Hydroxide]
    Let $P$ be the unique polynomial of degree at most $n$ such that
    $P(0)=P(1)=\cdots=P(n)=1$. Obviously $P(x)=1$, but we can also find $P$
    from Lagrange interpolation. This gives us the equality of polynomials

    \[ \sum_{k=0}^n \frac{(x-0)\cdots (x-(k-1))(x-(k+1))\cdots
    (x-n)}{(k-0)\cdots (k-(k-1))(k-(k+1))\cdots (k-n)}=1. \]

    Plugging in $x=m$ and doing a ton of rearranging gives the desired result.
}

\rem{
    Both solutions can easily be extended to prove the more general result
    that if $0\le p<n<m$ are integers, then
    \begin{center}
        \tcboxmath[arc=0mm, colback=black!5!white, boxrule=0mm]{
    \sum_{k=0}^n (-1)^{n-k}\frac{1}{m-k} \binom{n}{k}k^p=\frac{m^p}{(n+1)\binom{m}{n+1}}
        }
    \end{center}
}

\prob{https://artofproblemsolving.com/community/c6h358774}{USA TST 2010
    P8}{M, 10/10}{
    Let $m,n$ be positive integers with $m \geq n$, and let $S$ be the
    set of all $n$-term sequences of positive integers $(a_1, a_2, \ldots
    a_n)$ such that $a_1 + a_2 + \cdots + a_n = m$. Show that
    \begin{center}
        \tcboxmath[arc=0mm, colback=black!5!white, boxrule=0mm]{
            \sum_S 1^{a_1} 2^{a_2} \cdots n^{a_n} = {n \choose n} n^m -
            {n \choose n-1} (n-1)^m + \cdots + (-1)^{n-2} {n \choose 2} 2^m +
            (-1)^{n-1} {n \choose 1}
        }
    \end{center}
}

\solu{[Combinatorial, MellowMelon]
    Look at the right side, try to translate it to
    combinatorial model. If we had used the inclusion-exclusion method, the
    right side would be the number of ways to color $ m $ balls with
    $ n $ colors, with each color appearing at least once. Now our remaining
    job is to prove the same for the left side.\\

    Trying to interpret the left hand side, we first notice that we need to
    partition the balls in $n$ parts with sizes $a_1, a_2, \dots  a_n$ where
    the first partition will have $1$ color to choose from, the second one
    will have $2$ and so on. We need to refine this idea for using it.
}

\solu{[Generating Function, MellowMelon]
    The first step of a gf solution is to decide on a gf solution. Now we can
    think of the left hand side as a function of $m$, namely $b_m$. So we can
    make a generating function with it.\\

    We let $F_n(x) = b_0 + b_1 x + b_2 x^2 + \cdots$ and so we get:
    \begin{align*}
        F_n(x) &= (x+x^2+\cdots)(2x+4x^2+\cdots)\cdots (nx+n^2x^2+\cdots), \\
               &= \frac{x}{1-x} \frac{2x}{1-2x} \cdots \frac{nx}{1-nx}, \\
               &= \frac{n! x^n}{(1-x)(1-2x) \cdots (1-nx)}. \\
    \end{align*}

    Now we need a generating function for the right hand side as well. The
    right hand side looks the $n$th finite difference of the function
    $_mg_0(x) = x^m$, that is
    \[_mg_n(0) = {n \choose n} n^m - {n \choose n-1} (n-1)^m + \cdots +
    (-1)^{n-2} {n \choose 2} 2^m + (-1)^{n-1} {n \choose1}\]

    Now we can make a generating function with this:
    \[G(y) =\ _0g_n(0)y^0 +\ _1g_n(0)y^1 \dots \]
    But this isn't very useful as we are getting the $n$th finite differences
    on the right side, where we want to work with the original polynomials.\\

    So what we do is, we unravel the $_mg_n(0)$ as original polynomials and
    get:
    \[G(x, y) = x^0y^0 + x^1y^1 + x^2+y^2\dots \]
    Where our desired generating function for $n$ is:
    \[G_n(0, y) =\ _0g_n(0)y^0 +\ _1g_n(0)y^1 +\dots \]

    And we want to show that
    \[G_n(0, y) = \frac{n! x^n}{(1-x)(1-2x) \cdots (1-nx)} \]

    Now we notice that we can actually use induction on $n$ since we definded
    \[G_{n+1}(x, y) = G_n(x+1, y) - G_n(x, y)\]

    And so we are done.
}

\lem{}{
    \[G(x)=\sum_{n=0}^{\infty} \binom{n+k}{k} x^n=\frac{1}{(1-x)^{k+1}}\]
}

\begin{prooof}
    We have the above relation because ${n+k \choose k}$ can be interpret as
    the number of ways write $n$ as the sum of $k+1$ numbers. And the
    generating function for that is:
    \[G(x) = (1+x+x^2\dots )^{k+1}\]
\end{prooof}

\prob{https://artofproblemsolving.com/community/c6h1076934p5964577}{Binom1}{}{
    Solve for positive integer $n$
    \begin{center}
        \tcboxmath[arc=0mm, colback=black!5!white, boxrule=0mm]{
            {2n \choose 0}-{2n-1 \choose 1}+{2n-2 \choose 2}- \dots  +(-1)^n
            {n \choose n}=?
        }
    \end{center}
}

\solu{[Generating funtion, Tintarn]
    Denote this sum $a_n$. Denote $F(x)=\sum_{n=0}^{\infty} a_nx^{2n}$.
    \begin{align*}
        F(x)&=\sum_{k=0}^{\infty} (-1)^k \sum_{n=0}^{\infty} \binom{2n-k}{k}
        x^{2n}\\
        &=\sum_{k=0}^{\infty} (-x^2)^k \sum_{n=0}^{\infty} \binom{2n+k}{k}
        x^{2n}
    .\end{align*}
    Now, using the well-known power series
    \[\boxed{G(x)=\sum_{n=0}^{\infty} \binom{n+k}{k}
    x^n=\frac{1}{(1-x)^{k+1}}}\]
    we find
    \[\sum_{n=0}^{\infty} \binom{2n+k}{k} x^{2n}=\frac{G(x)+G(-x)}{2}\]
    and hence
    \begin{align*}
        F(x)&=\frac{1}{2(1-x)} \sum_{k=0}^{\infty} \left( \frac{x^2}{x-1} \right)^k
        +\frac{1}{2(1+x)} \sum_{k=0}^{\infty} \left( \frac{-x^2}{x+1} \right)^k\\[.5em]
            &=\frac{1}{2(x^2-x+1)}+\frac{1}{2(x^2+x+1)}\\[.5em]
            &=\frac{x^2+1}{x^4+x^2+1}\\[.5em]
            &=\frac{1-x^4}{1-x^6}\\
            &=(1-x^4)(1+x^6+x^{12}+x^{18}+\dotsc)\\
            &=1-x^4+x^6-x^{10}+x^{12}-x^{16}+\dots
    .\end{align*}

    \vspace{1em}
    and hence $\boxed{a_{3k}=1, a_{3k+1}=0, a_{3k+2}=-1}$.
}

\solu{[Combinaorial Model, MellowMelon]
    Consider the number of ways to tile a $1 \times 2n$ rectangle with squares
    and dominoes. Let $E$ be the set of ways to do it with an even number of
    dominoes, and let $O$ be the same for odd. We want to find $|E| - |O|$.\\

    Trying some basic cases, we see that the only values we get are $-1, 0,
    1$. So we want to find a pairing between the two sets that is almost a
    bijection, but doesn't work for only one element. And that bijection is:\\

    \textit{Consider the first consecutive pair of squares or domino which
    either comes at the very start or comes right after another domino. Swap
    the pair of squares to a domino or vice versa.}
}

\prob{https://artofproblemsolving.com/community/c6h1076934p5964577}{Binom 2}{}{
    \[{n \choose 0}^2-{n \choose 1}^2+{n \choose 2}^2-\dots +(-1)^n{n \choose n}^2=?\]
}

\solu{[Generating function, TinTarn]
    Denote \[f(m,n)=\sum_{k=0}^{\infty} (-1)^k \binom{n}{k} \binom{n}{m-k}\]
    We are looking for $f(n,n)$.  We have
    \begin{align*}
        \sum_{m=0}^{\infty} f(m,n) x^m &=\sum_{k=0}^{\infty} (-1)^k \binom{n}{k}
        \sum_{m=0}^{\infty} \binom{n}{m-k} x^m\\
        &=\left[\sum_{k=0}^{\infty} (-x)^k \binom{n}{k} \right]
        \left[\sum_{m=0}^{\infty} x^m \binom{n}{m} \right]\\
        &=(1-x)^n(1+x)^n\\
        &=(1-x^2)^n\\
        &=\sum_{t=0}^{\infty} (-1)^t \binom{n}{t} x^{2t}
    .\end{align*}

    Hence we see that for odd $m$ we have $f(m,n)=0$ and hence for odd $n$ we have
    $\boxed{f(n,n)=0}$.\\

    Now, for $n=2k$ even, we find $\boxed{f(2k,2k)=(-1)^k \binom{2k}{k}}$.
}

\solu{[Combinatorial, MellowMelon]
    Consider the number of ways to paint the squares of a $2 \times n$
    rectangle red and blue such that both rows have the same number of red
    squares. Let $E$ be the set of ways with an even number of red squares in each
    row, and let $O$ be the same for odd. We want to find $|E| - |O|$.\\

    \textit{Pair the selections as follows: take the first column with both
        squares the same color, and flip both colors. This maps elements in
        $E$ to $O$ and vice versa and is invertible, so except cases where the
    pairing is undefined, $E$ and $O$ have the same number of elements.}
}


\documentclass{ximera}
\input{../preamble}
\title{Exercises: Integration by Parts}
%%%%%\author{Philip T. Gressman}

\begin{document}
\begin{abstract}
Various exercises relating to integration by parts.
\end{abstract}
\maketitle

Compute the indefinite integrals below. Since there are many possible answers (which differ by constant values), use the given instructions if needed to choose which possible answer to use.

\begin{exercise}%[APEX0602IBP05]
\[  \int x\sin x\ dx = \answer{\sin x - x\cos x} +C\]
(Add a constant to your answer if needed so that it equals $0$ at $x = 0$.)
%
%
\end{exercise}

\begin{exercise}%[APEX0602IBP06]
\[ \int xe^{-x}\ dx = \answer{-e^{-x}-xe^{-x}}+C\]
(Add a constant to your answer if needed so that it equals $-1$ at $x = 0$.)
%
%
\end{exercise}

\begin{exercise}%[APEX0602IBP10]
\[  \int x^3e^{x}\ dx\ = \answer{x^3e^x-3x^2e^x+6xe^x-6e^x}+C\]
(Write your answer so that it has no constant term.)
%
%
\end{exercise}

\begin{exercise}%[APEX0602IBP25]
\[ \int x^2\ln |x| \ dx = \answer{\frac{1}{3} x^3 \ln |x|-\frac{x^3}{9}}+C\]
Don't forget absolute values in your logarithm.
(Add a constant to your answer as necessary so that it equals $-1/9$ at $x = 1$.)
%
%
\end{exercise}

\begin{exercise}%[APEX0602IBP12]
\[ \int e^x\sin x\ dx = \answer{\frac{1}{2} e^x(\sin x-\cos x)}+C\]
(Add a constant to your answer if needed so that it equals $-1/2$ at $x = 0$.)
\begin{hint}
This is a case in which you need to treat the integration by parts as an equation and \textit{solve} for the answer.
\end{hint}
%
%
\end{exercise}

\begin{exercise}%[APEX0602IBP17]
\[ \int \arcsin x\ dx = \answer{\sqrt{1-x^2}+x \arcsin(x)}+C \]
(Add a constant to your answer if needed so that it equals $1$ at $x = 0$.)
\begin{hint}
Write $\arcsin x = 1 \cdot \arcsin x$.
\end{hint}
%
%
\end{exercise}

\section*{Sample Quiz Questions}
\begin{question}%%%%%[IBPibp:explin1413]

Compute the definite integral
\[\int_1^4 e^{3x} (x + 1) ~ dx.\]
(Hints won't be revealed until after you choose a response.)
\begin{multiplechoice}
\choice[correct]{\(\displaystyle \frac{14}{9} e^{12} - \frac{5}{9} e^{3}\)}
\choice{\(\displaystyle \frac{17}{9} e^{12} - \frac{5}{9} e^{3}\)}
\choice{\(\displaystyle \frac{17}{9} e^{12} - \frac{8}{9} e^{3}\)}
\choice{\(\displaystyle \frac{20}{9} e^{12} - \frac{8}{9} e^{3}\)}
\choice{\(\displaystyle \frac{20}{9} e^{12} - \frac{11}{9} e^{3}\)}
\choice{\(\displaystyle \frac{23}{9} e^{12} - \frac{11}{9} e^{3}\)}
\end{multiplechoice}
\begin{feedback}
Integrate by parts, integrating the exponential and differentiating polynomials.
\begin{hint}
\[\begin{aligned}
    \int_1^4 e^{3x} &  (x + 1) ~ dx \\
    & = \left. \frac{e^{3x}}{3} (x+1) \right|_1^4 - \int_1^4 \frac{e^{3x}}{3} dx \\
    & = \frac{5}{3} e^{12} - \frac{2}{3} e^{3} - \left. \frac{e^{3x}}{9} \right|_1^4 \\
    & = \frac{14}{9} e^{12} - \frac{5}{9} e^{3}
\end{aligned}\]
\end{hint}
\end{feedback}

\end{question}
\begin{question}%%%%%[IBPibp:fourier2alt4]

Compute the definite integral
\[\int_{\pi}^{2 \pi} x \sin 4 x ~ dx.\]
(Hints won't be revealed until after you choose a response.)
\begin{multiplechoice}
\choice{\(\displaystyle 0\)}
\choice{\(\displaystyle \frac{\pi}{4}\)}
\choice[correct]{\(\displaystyle -\frac{\pi}{4}\)}
\choice{\(\displaystyle \frac{3 \pi}{4}\)}
\choice{\(\displaystyle -\frac{3 \pi}{4}\)}
\choice{\(\displaystyle \frac{7 \pi}{4}\)}
\end{multiplechoice}
\begin{feedback}
Integrate by parts, integrating the trig functions and differentiating polynomials.
\begin{hint}
\[ \begin{aligned}
    \int_{\pi}^{2 \pi} x & \sin 4 x ~ dx \\
    & = \left. - \frac{\cos 4x}{4} x \right|_{\pi}^{2 \pi} + \int_{\pi}^{2 \pi} \frac{\cos 4x}{4} dx \\
    & = - \frac{2 \pi }{4} + \frac{(-1)^{4} \pi }{4} - \left. \frac{\sin 4x}{16} \right|_{\pi}^{2 \pi} = -\frac{\pi}{4}
\end{aligned}\]
\end{hint}
\end{feedback}

\end{question}
\begin{question}%%%%%[IBPibp:miscarctan5]

Compute the indefinite integral
\[\int \arctan 5x ~ dx.\]
(Hints won't be revealed until after you choose a response.)
\begin{multiplechoice}
\choice[correct]{\(\displaystyle x \arctan 5x - \frac{1}{10} \ln | 1 + 25x^2|  + C\)}
\choice{\(\displaystyle x \arctan 5x - \frac{1}{12} \ln | 1 + 25x^2|  + C\)}
\choice{\(\displaystyle x \arctan 5x - \frac{1}{14} \ln | 1 + 25x^2|  + C\)}
\choice{\(\displaystyle x \arctan 5x + \frac{1}{10} \ln | 1 + 25x^2|  + C\)}
\choice{\(\displaystyle x \arctan 5x + \frac{1}{12} \ln | 1 + 25x^2|  + C\)}
\choice{\(\displaystyle x \arctan 5x + \frac{1}{14} \ln | 1 + 25x^2|  + C\)}
\end{multiplechoice}
\begin{feedback}
Integrate by parts, integrating the coefficient 1 and differentiating arctangent.
\begin{hint}
\[\begin{aligned}
    \int & \arctan 5x ~ dx \\
    & = x \arctan 5x - \int \frac{5x}{1+25x^2} ~ dx \\
    & = x \arctan 5x - \frac{1}{10} \int \frac{50x}{1+25x^2} ~ dx \\
    & = x \arctan 5x - \frac{1}{10} \ln | 1 + 25x^2|  + C
\end{aligned}\]
\end{hint}
\end{feedback}

\end{question}
\begin{question}%%%%%[IBPibp:expcos35]

Compute the indefinite integral
\[\int e^{3x}  \cos 5x ~ dx.\]
(Hints won't be revealed until after you choose a response.)
\begin{multiplechoice}
\choice{\(\displaystyle \frac{e^{3x} (2 \cos 5x + 5 \sin 5x)}{29} + C\)}
\choice[correct]{\(\displaystyle \frac{e^{3x} (3 \cos 5x + 5 \sin 5x)}{34} + C\)}
\choice{\(\displaystyle \frac{e^{3x} ( \cos 5x + 3 \sin 5x)}{20} + C\)}
\choice{\(\displaystyle \frac{e^{3x} ( \cos 5x + 2 \sin 5x)}{15} + C\)}
\choice{\(\displaystyle \frac{e^{3x} (2 \cos 5x + 7 \sin 5x)}{53} + C\)}
\choice{\(\displaystyle \frac{e^{3x} (3 \cos 5x + 7 \sin 5x)}{58} + C\)}
\end{multiplechoice}
\begin{feedback}
Integrate by parts, integrating the exponential and differentiating cosine (or vice-versa), then solve for the antiderivative.
\begin{hint}
\[ \begin{aligned}
    \int e^{3x} & \cos 5x ~ dx \\
    & = \frac{e^{3x}}{3} \cos 5x - \int \frac{e^{3x}}{3} (-5 \sin 5x) ~ dx \\
    & = \frac{e^{3x} \cos 5x}{3} + \frac{5}{3} \int e^{3x} \sin 5x ~ dx \\
    & = \frac{e^{3x} \cos 5x}{3} + \frac{5}{3} \frac{e^{3x}}{3} \sin 5x - \frac{5}{3} \int \frac{e^{3x}}{3} (5 \cos 5x) ~ dx \\
    & = \frac{e^{3x} (3 \cos 5x + 5 \sin 5x)}{9} - \frac{25}{9} \int e^{3x} \cos 5x ~ dx  \\
    \Rightarrow   & \qquad \frac{34}{9}  \int e^{3x}  \cos 5x ~ dx  = \frac{e^{3x} (3 \cos 5x + 5 \sin 5x)}{9} \\
    \Rightarrow & \qquad  \int  e^{3x}  \cos 5x ~ dx = \frac{e^{3x} (3 \cos 5x + 5 \sin 5x)}{34} + C
\end{aligned}\]
\end{hint}
\end{feedback}

\end{question}

\section*{Sample Exam Questions}

\begin{question}%%%%%[2015C.11]

Compute the integral below.
\[ \int_{\frac{1}{2}}^\infty \frac{ \ln (2x)}{x^2} dx \]
\begin{multiplechoice}
\choice{\(1 - \ln 2\)}
\choice[correct]{\(2\)}
\choice{\(\displaystyle \ln 2 - \frac{1}{2}\)}
\choice{\(\displaystyle \frac{1}{2}\)}
\choice{\(2 - 2 \ln 2\)}
\choice{the integral diverges}
\end{multiplechoice}

\end{question}

\begin{question}%%%%%[2016C.05]

Compute the indefinite integral indicated below. [Hint: Write \(\displaystyle \frac{1}{\cos^2 \theta} = \sec^2 \theta\) and integrate by parts.]
\[ \int \left( 1 + \frac{\ln | \sin \theta|}{\cos^2 \theta} \right) d \theta \]
\begin{multiplechoice}
\choice{\(\displaystyle (\sin \theta) \ln |\sin \theta| + C\)}
\choice{\(\displaystyle (\cos \theta) \ln |\sin \theta| + C\)}
\choice[correct]{\(\displaystyle (\tan \theta) \ln |\sin \theta| + C\)}
\choice{\(\displaystyle (\csc \theta) \ln |\sin \theta| + C\)}
\choice{\(\displaystyle (\sec \theta) \ln |\sin \theta| + C\)}
\choice{\(\displaystyle (\cot \theta) \ln |\sin \theta| + C\)}
\end{multiplechoice}

\end{question}

\end{document}


\chapter{Second Order Boundary Value Problems}

\section{2nd order two-point boundary value problems}

\begin{intro}
  We have already seen, that boundary value problems have very
  different stability properties than initial value
  problems. Here, we will discuss a special class of boundary value
  problems of the form
  \begin{gather}
    \label{eq:bvp-second:1}
    -u''(x) + \beta(x) u'(x) + \gamma(x)u(x) = f(x),
    \qquad u(a) = u_a,
    \quad u(b) = u_b.
  \end{gather}

  In order to make this problem more amenable to mathematical
  investigation, we introduce the set
  \begin{gather*}
    \mathcal B =
    \Bigl\{ u\in C^2(a,b) \cap C[a,b]
    \;\Big|\; u(a) = u_a\;\wedge\; u(b) = u_b
    \Bigr\}.
  \end{gather*}

  Then, we can see the left hand side of the differential equation as
  a differential operator applied to $u$ and thus mapping $\mathcal B$
  to the set of continuous functions. Namely, we define
  \begin{gather}
    \label{eq:fd:2}
    \begin{split}
      L: \mathcal B &\to C[a,b] \\
      u & \mapsto -u'' + \beta u' + \gamma u.
    \end{split}
  \end{gather}
  In addition, we would like to simplify our life and get rid of the
  inhomogeneous boundary values $u_a$ and $u_b$. To this end, let
  \begin{gather*}
    u_B(x) = u_a \frac{b-x}{b-a} + u_b \frac{x-a}{b-a},
  \end{gather*}
  and introduce the new function $u_0 = u+u_B$. Then, $u_0$ solves
  the boudnary value problem
  \begin{multline*}
    -u_0''(x) + \beta(x) u'(x) + \gamma(x)u(x) = f(x) -
    \beta(x)\frac{u_b-u_a}{b-a} - \gamma(x)u_B(x),
    \\ u(a) =  u(b) = 0.
  \end{multline*}
  Thus, it is sufficient to consider the boundary value problem
\end{intro}

\input{definitions/bvp-second}

\begin{remark}
  This definition exhibits a major change in paradigm. Before, we
  considered a differential equation as an equation which determines
  the derivative of a function in a point. Now, we are looking at a
  linear system of equations, albeit one, which is not of finite
  dimension. This paradigm change will be essential when we consider
  partial differential equations in future semesters.

  On the other hand, the equality in equation~\eqref{eq:bvp-second:2}
  is understood point-wise, such that in fact nothing but our point of
  view has changed.
\end{remark}

\begin{intro}
  Again, we subdivide the interval $I= [a,b]$ into subintervals, but
  the subdivision does not involve IVP solvers on subintervals, but
  much more like in the original subdivision in
  Definition~\ref{Definition:partitioning}, the the solution will only
  be defined at the partitioning points $t_k$, $k=0,\dots,n$.

  Thus, like with one-step and multistep methods we will have values
  $y_0, y_1,\dots, y_n$, but the sequence has a defined end at $y_n$
  due to the right boundary of the interval.

  While one-step methods directly discretize the Volterra integral
  equation in order to compute a solution at every new step,
  \textbf{finite difference methods} discretize the differential
  equation on the whole interval at once and then solve the resulting
  discrete (finite-dimensional) system of equations.

  We have accompished the first step and decided that instead of
  function values in every point of the interval $I$, we only
  approximate $u(t_k)$ in the points of the partition. What is left is
  the definition of the discrete operator representing the equation.
\end{intro}

\input{definitions/difference-operators}

\begin{remark}
  The 3-point stencil is the product of forward and backward
  difference operators.
  \begin{gather*}
    D^2_h u(x) = D^+_hu(x)D^-_hu(x) = D^-_hu(x)D^+_hu(x).
  \end{gather*}
  For simplicity, we only present finite differences of uniform
  subdivisions. Nevertheless, the definition of the operators can be
  extended easily to $h$ changing between intervals.
\end{remark}

\input{definitions/fd-consistency}
\input{theorems/fd-consistency}

\begin{proof}
  We begin to show consistency of the first two operators by Taylor
  expansion: for some $\xi\in(x,x+h)$, there holds
  \begin{align*}
    u'(x) - D^+_h u(x) &= u'(x) - \frac{u(x+h) - u(x)}h \\
    &= u'(x) - \frac{u(x)+h u'(x) + \tfrac{h^2}{2} u''(\xi) - u(x)}h
    \\
    &= \tfrac h2 u''(\xi).
  \end{align*}
  The same computation can be applied to $D^-_h u(x)$. It is clear
  that we need an additional symmetry argument for te other two,
  otherwise their consistency order would be lower. Therefore, we
  follow the line of argument that we introduced in
  Lemma~\ref{Lemma:lmm-bramble-hilbert}, and which here reads: a
  difference operator $D^\alpha_h$ approximating a derivative of order
  $\alpha$ is consistent of order $p$, if and only if it is exact for all
  polynomials of degree $p+\alpha-1$. We realize this by computing the
  Taylor polynomial $p(h)$ of degree $p+\alpha-1$ and the remainder term
  involving $u^{(p+\alpha)}(\xi)$. Then,
  \begin{gather*}
    D^\alpha_h u(x) = \frac{p(h)+\frac{h^{\alpha+p}}{(\alpha+p)!}u^{(p+\alpha)}(\xi)}{h^\alpha}.
  \end{gather*}
  Now we employ that the formula is exact for $p(h)$ and thus
  \begin{gather*}
    u^{(\alpha)}(x) - D^\alpha_h u(x)
    = \frac{h^p}{(\alpha+p)!}u^{(p+\alpha)}(\xi).
  \end{gather*}
  We now write
  \begin{gather*}
    p(\xi) = a_0 + a_1 (\xi-x) + a_2 (\xi-x)^2 + a_3 (\xi-x)^3 + \cdots
  \end{gather*}
  The central difference $D^c_h$ is exact for linear polynomials,
  since ti evaluates to zero for a constant and $D^c_h (\xi-x) =
  1$. But additionally, we observe
  \begin{gather*}
    \frac{d}{d\xi} (\xi-x)^2 \Bigr|_{\xi=x}
    = D^c_h (\xi-x)^2 \Bigr|_{\xi=x}= 0.
  \end{gather*}
  Thus, the central difference is exact for polynomials of degree 2
  and consistent of second order.

  For the 3-point stencil, we observe that $D^2_h u(x)=0$ for any
  function $u$ such that $u(x+h) - u(x-h) = u(x)$, in particular any
  odd polynomial in $\xi-x$. Furthermore,
  \begin{gather*}
    D^2_h  (\xi-x)^2 \Bigr|_{\xi=x}= \frac{h^2-0+h^2}{h^2}
    = 2 = \frac{d^2}{d\xi^2} (\xi-x)^2
  \end{gather*}
\end{proof}

\begin{remark}
  When applied to the equation $u'=f(t,u)$ the solutions obtained by
  forward and backward differences correspond to the explicit and
  implicit Euler methods, respectively.
\end{remark}

\input{definitions/fd}

\input{definitions/fd-example}

\begin{remark}
  Like our view to the continuous boundary value problem has changed,
  the discrete one is now a fully coupled linear system which has to
  be solved by methods of linear algebra, not by time stepping
  anymore. In fact, we have $n+1$ variables $y_0,\dots, y_n$ and $n+1$
  equations, such that here existence and uniqueness of solutions are
  equivalent.
\end{remark}

\input{definitions/fd-matrix}

\begin{remark}
  The first and last row of the matrix $L_h$ are redundant, since they
  simply say $y_0 = y_n = 0$. They can be eliminated, such that we
  obtain the reduced system
  \begin{gather}
    \label{eq:fd:8}
    L_h y =
    \begin{pmatrix}
      \lambda_1 & \nu_1\\
      \mu_2 & \lambda_2 & \ddots \\
      & \ddots & \ddots & \nu_{n-2} \\
      &&\mu_{n-1}& \lambda_{n-1}
    \end{pmatrix}
    \begin{pmatrix}
      y_1\\\vdots\\y_{n-1}
    \end{pmatrix}
    =
    \begin{pmatrix}
      f_1 \\\vdots\\f_{n-1}
    \end{pmatrix} = f_h.
  \end{gather}
  In this form, the operator $L_h$ is consistent with $L$ in the sense
  that it only describes the differential operator, not the boundary
  values. Therefore, we will use this form in our further analysis.

  When it comes to implementation, both versions have their
  merits. Obviously, the new operator involves less unknowns. On the
  other hand, the discretization with boundary unknowns is more
  straight-forward.
\end{remark}

\section{Existence, stability, and convergence}

\begin{intro}
  Since the solution of the discretized boundary value problem is a
  problem in linear algebra, we have to study properties of the matrix
  $L_h$. The shortest and most elegant way to prove stability is
  through the properties of M-matrices, which we present here very
  shortly. We are not dwelling on this approach too long, since it is
  sufficient for stability, but by far not necessary and constrained
  to low order methods.

  The fact that $L_h$ is an M-matrix requires some knowledge of
  irreducible weakly diagonal dominant matrices, which the author
  considers as outdated as the whole concept of m-matrices. We will
  just quote this result without proof.
\end{intro}

\input{definitions/m-matrix}

\begin{Lemma}{m-matrix-fd1}
  The matrix $L_h$ defined above is an M-matrix provided that
  \begin{gather}
    \label{eq:fd:3}
    \gamma_k \ge 0, % > -\frac2{h^2},
    \qquad
    \abs{\beta_k} < \frac2h.
  \end{gather}
\end{Lemma}

\begin{proof}
  It is clear that these two conditions are sufficient for the first
  M-matrix property. The proof of positivity of the inverse is based
  on irreducible diagonal dominance, which is too long and too
  specialized for these notes.
\end{proof}

\begin{remark}
  The finite element method provides much more powerful to deduce
  solvability and stability of the discrete problem.
\end{remark}

\input{theorems/m-matrix-inverse}

\begin{proof}
  Let $x\in \R^n$ and $y = A^{-1} x$. Then,
  \begin{align*}
    \abs{y_i} & = \abs{\sum c_{ij} x_j} \\
    & \le \sum c_{ij} \abs{x_j} \\
    & \le \norm{x}_\infty \sum c_{ij} v_j.
  \end{align*}
  Thus,
  \begin{gather*}
    \abs{y_i} \le  \norm{x}_\infty \bigl(A^{-1} v\bigr)_i
    = \norm{x}_\infty \bigl(A^{-1} Aw\bigr)_i \le \norm{x}_\infty \abs{w_i}.
  \end{gather*}
  Taking the maximum over all $i$, we obtain
  \begin{gather*}
    \norm{A^{-1}}_\infty = \sup_{u\in
      \R^n}\frac{\norm{A^{-1}u}_\infty}{\norm{u}_\infty}
    \le \norm{w}_\infty.
  \end{gather*}
\end{proof}

\input{theorems/fd-stability}

\begin{proof}
  Take the function
  \begin{gather*}
    p(x) = (x-a)(b-x) = -x^2+(a+b)x-ab,
  \end{gather*}
  with derivatives $p'(x) = a+b-2x$ and $p''(x) = -2$, and a maximum
  of $(b-a)^2/4$ at $x=(a+b)/2$. Choose the values $p_k = p(x_k)$. By
  consistency, we have and $k=1,\dots,n-1$
  \begin{gather*}
    (L_h p)_k \ge 2 - \beta_k\abs{b-a} + \gamma_k \abs{b-a}^2
    \ge 2-\delta.
  \end{gather*}
  Thus, the vector with entries $w_k = p_k/(2-\delta)$ can be used to
  bound the inverse of $L_h$ by Lemma~\ref{Lemma:m-matrix-inverse}.
\end{proof}

\begin{remark}
  The assumptions of the previous theorem involve two sets of
  conditions on the parameters $\beta_k$ and
  $\gamma_k$. Condition~\eqref{eq:fd:1} is actually a condition on the
  continuous problem. The condition on $\gamma_k$ is indeed necessary,
  as will be seen when we study partial differential equations. The
  condition on $\beta_k$ is not necessary in this form, but a better
  estimate again requires far advanced analysis.

  The other set of conditions relates the coefficients to the mesh
  size. Again, the condition on $\beta_k$ can be avoided as seen in
  the next example. The condition on $\gamma_k$ is already implied by
  $-\gamma_k \le (b-a)^2$, which is a small restriction compared
  to~\eqref{eq:fd:1}, as soon as the partition has 3 interior
  points. Thus, it is not crucial.
\end{remark}

\input{definitions/upwind}
\input{theorems/fd-convergence}

\begin{proof}
  We apply the difference operator $L_h$ to $u$ (as the vector of
  function values in the points $t_k$) and $y$ to obtain
  \begin{gather*}
    L_h(u-y) = (L_h - L) u + Lu - L_h y = \tau + f - f = \tau,
  \end{gather*}
  where $\tau = (\tau_1,\dots,\tau_{n-1})^T$ is the vector, which
  measures the consistency error  $(L_h-L) u$ in each $t_k$. The
  entries $\tau_k$ are bounded by $c h^p$ by the consistency estimate.
  For the error, there holds
  \begin{gather*}
    u-y = L_h^{-1} L_h(u-y) =  L_h^{-1} \tau.
  \end{gather*}
  Using the stability assumption, we obtain
  \begin{gather*}
    \norm{u-y}_\infty \le \norm{L_h^{-1}}_\infty \norm{\tau}_\infty
    \le c h^p,
  \end{gather*}
  where the constant $c$ depends on $\norm{L_h^{-1}}_\infty$ and the
  constant in the consistency estimate, but not on $h$.
\end{proof}

\begin{remark}
  Finite differences can be generalized to higher order by extending
  the stencils by more than one point to the left and right of the
  current point. Whenever we add two points to the symmetric
  difference formulas, we can gain two orders of consistency.
  \begin{gather*}
    \underbrace{\bullet\quad
      \underbrace{\bullet\quad
        \underbrace{\bullet\quad\phantom{\bullet}\quad\bullet}_{u'+\mathcal O(h^2)}
        \quad\bullet}_{u'+\mathcal O(h^4)}
      \quad\bullet}_{u'+\mathcal O(h^6)}
    \qquad\qquad
    \underbrace{\bullet\quad
      \underbrace{\bullet\quad
        \underbrace{\bullet\quad\bullet\quad\bullet}_{u''+\mathcal O(h^2)}
        \quad\bullet}_{u''+\mathcal O(h^4)}
      \quad\bullet}_{u''+\mathcal O(h^6)}
  \end{gather*}
  Similarly, we can define one-sided difference formulas, which get us
  close to multistep methods. The matrices generated by these formulas
  are not M-matrices anymore, although you can show for the 4th order
  formula for the second derivative that it yields a product of two
  M-matrices. While this rescues the theory in a particular instance,
  M-matrices do not provide a theoretical framework for general high
  order finite differences anymore.

  Very much like the starting procedures for high order multistep
  methods, high order finite differences cause problems at the
  boundaries. Here, the formulas must be truncated and for instance be
  replaced by one-sided formulas of equal order.

  All these issues motivate the study of different discretization
  methods in the next course.
\end{remark}

\section{The Laplacian and harmonic functions}

\begin{intro}
  The two-point boundary value problem has a natural extension to
  higher dimensions. There, we deal with partial derivatives
  $\frac{\partial}{\partial x}$, $\frac{\partial}{\partial y}$, and
  $\frac{\partial}{\partial z}$. As an outlook towards topics
  discussed in classes on partial differential equations and their
  numerical analysis, we close these notes by a short introduction at
  hand of examples.
\end{intro}

\begin{Definition}{laplace-equation}
  the \define{Laplacian} in two (three) space dimensions is the sum of
  the second partial derivatives
  \begin{gather}
    \label{eq:fd:10}
    \Delta u = \frac{\partial^2}{\partial x^2}u
    + \frac{\partial^2}{\partial y^2}u
    \left(+ \frac{\partial^2}{\partial z^2}u\right)
    = \operatorname{div}(\nabla u)
  \end{gather}
  The \define{Laplace equation} is the partial differential equation
  \begin{gather}
    \label{eq:fd:11}
    -\Delta u = 0.
  \end{gather}
  The \define{Poisson equation} is the partial differential equation
  \begin{gather}
    \label{eq:fd:9}
    -\Delta u = f.
  \end{gather}
  Solutions to the Laplace equations are called \define{harmonic functions}.
\end{Definition}

\subsection{Properties of harmonic functions}

\begin{Theorem*}{mean-value}{Mean-value formula for harmonic functions}
  Let $u\in C^2(\Omega)$ be a solution to the Laplace equation. Then,
  $u$ has the mean value property
  \begin{gather}
    \label{eq:mean-value:1}
    u(\mathbf x) = \frac1{r^{d-1}\omega(d)}
    \int_{\partial B_r(\mathbf x)} u(\mathbf y) \,ds,
  \end{gather}
  where $\partial B_r(\mathbf x) \subset \Omega$ is the sphere of
  radius $r$ around $\mathbf x$ and $\omega(d)$ is the volume of the
  unit sphere in $\R^d$.
\end{Theorem*}

\begin{proof}
  First, we rescale the problem to
  \begin{gather*}
    \Phi(r) = \frac1{r^{d-1}\omega(d)}
    \int_{\partial B_r(\mathbf x)} u(\mathbf y) \,ds
    = \frac1{\omega(d)} \int_{\partial B_1(0)} u(\mathbf x+r\mathbf z) \,ds.
  \end{gather*}
  Then, we notice that
  \begin{align*}
    \Phi'(r)
    &= \frac1{\omega(d)} \int_{\partial B_1(0)}
      \nabla u(\mathbf x+r\mathbf z)\cdot \mathbf z \,ds_z\\
    &= \frac1{r^{d-1}\omega(d)} \int_{\partial B_r(\mathbf x)}
      \nabla u(\mathbf y)\cdot\frac{\mathbf y-\mathbf x}{r} \,ds_y\\
    &= \frac1{r^{d-1}\omega(d)} \int_{\partial B_r(\mathbf x)}
    \frac{\partial}{\partial \mathbf n} u(\mathbf y) \,ds_y\\
    &= \frac1{r^{d-1}\omega(d)} \int_{B_r(\mathbf x)}
    \Delta u(\mathbf y)\,d\mathbf y = 0.
  \end{align*}
  Between the last two lines, we used the Gauß theorem for the vector
  valued function $\nabla u$. Therefore, $\Phi(r)$ is
  constant. Because of continuity, we have
  \begin{gather*}
    \lim_{r\to 0} \Phi(r) =
    \lim_{r\to 0}\frac1{r^{d-1}\omega(d)} \int_{\partial B_r(\mathbf x)} u(\mathbf y)
    \,ds
    = u(\mathbf x),
  \end{gather*}
  which proves our theorem.
\end{proof}

\input{theorems/maximum}

\begin{proof}
  Let $\mathbf x_0$ be such a maximum and let $r>0$ such that
  $B_r(\mathbf x_0) \subset \Omega$. Assume that there is a point
  $\mathbf x$ on $\partial B_r(\mathbf x_0)$, such that $u(\mathbf x)
  < u(\mathbf x_0)$. Then, this holds for points $\mathbf y$ in a
  neighborhood of $\mathbf x$. Thus, in order that the mean value
  property holds, there must be a subset of $\partial B_r(\mathbf
  x_0)$ where $u(\mathbf y) > u(\mathbf x_0)$, contradicting that
  $\mathbf x_0$ is a maximum. Thus, $u(\mathbf x) = u(\mathbf x_0)$
  for all $\mathbf x \in B_r(\mathbf x_0)$ for all $r$ such that
  $B_r(\mathbf x_0) \subset \Omega$.

  Let now $\mathbf x\in \Omega$ be arbitrary. Then, there is a
  (compact) path from $\mathbf x_0$ to $\mathbf x$ in $\Omega$. Thus,
  the path can be covered by a finite set of overlapping balls inside
  $\Omega$, and the argument above can be used iteratively to conclude
  $u(\mathbf x) = u(\mathbf x_0)$.
\end{proof}

\begin{corollary}
  Let $u \in C^2(\Omega)$ be a solution to the Laplace equation. Then,
  its maximum and its minimum lie on the boundary, that is, there are
  points
  $\underline{\mathbf x}, \overline{\mathbf x} \in\partial\Omega$,
  such that
  \begin{gather*}
    u(\underline{\mathbf x}) \le u(\mathbf x)
    \le u(\overline{\mathbf x})
    \quad\forall \mathbf x\in\Omega.
  \end{gather*}
\end{corollary}

\begin{proof}
  If the maximum of $u$ is attained in an interior point, the maximum
  principle yields a constant solution and the theorem holds
  trivially. On the other hand, the maximum principle does not make
  any prediction on points at the boundary, which therefore can be
  maxima. The same holds for the minimum, since $-u$ is a solution to
  the Laplace equation as well.
\end{proof}

\begin{corollary}
  Solutions to the Poisson equation with homogeneous boundary
  conditions are unique.
\end{corollary}

\begin{proof}
  Assume there are two functions $u,v\in C^2(\Omega)$ with $u=v=0$ on
  $\partial\Omega$ such that
  \begin{gather*}
    -\Delta u = -\Delta v = f.
  \end{gather*}
  Then, $w=u-v$ solves the Laplace equation with $w=0$ on
  $\partial\Omega$. Due to the maximum principle, $w \equiv 0$ and
  $u=v$.
\end{proof}

\section{Finite differences}
\input{definitions/square-domain}
\input{definitions/dirichlet}
\input{definitions/cartesian-grid}
\input{definitions/lexicographic}
\input{definitions/5-point-stencil}

\begin{remark}
  From now on, we will call the discrete solution $u_h$ in order to
  avoid confusion with the coordinate direction $y$.
\end{remark}

\input{theorems/fd-matrix}

\begin{Theorem}{5-point-m-matrix}
  The matrix generated by the 5-point stencil is an M-matrix and the
  discrete problem
  \begin{gather*}
    L_h u_h = f
  \end{gather*}
  is stable in the sense that there is a constant $c$ independent of
  the grid spacing $h$ such that
  \begin{gather*}
    \norm{L_h^{-1}} \le c.
  \end{gather*}
\end{Theorem}

\begin{proof}
  The proof of M-matrix property is identical to the proof for 2-point
  boundary value problems, which was omitted there.  The same way as
  there, the function
  \begin{gather*}
    w(x,y) = (x-a)(b-x)(y-a)(b-y)
  \end{gather*}
  can be employed to show boundedness of $\norm{L_h^{-1}}$.
\end{proof}

\input{theorems/fd2-consistency}

\begin{proof}
  We apply the analysis of Lemma~\ref{Lemma:fd-consistency} in $x$-
  and $y$-directions separately, obtaining
  \begin{align*}
    \abs{\frac{\partial^2}{\partial x^2} u(x,y)
      - \frac{u(x+h,y) -2u(x,y) + u(x-h,y)}{h^2}} & \le c h^2 \\
    \abs{\frac{\partial^2}{\partial y^2} u(x,y)
      - \frac{u(x,y+h) -2u(x,y) + u(x,y-h)}{h^2}} & \le c h^2,
  \end{align*}
  an conclude consistency of the sum.
\end{proof}

\input{theorems/fd2-maximum}

\begin{proof}
  From equation~\eqref{eq:5-point-stencil:1}, it is clear that a
  discrete mean value property holds, that is, $y_k$ is the mean value
  of its four neighbors. Therefore, if $y_k \ge y_j$, for all
  neighboring indices $j$ of $k$, we have $y_j = y_k$. We conclude by
  following a path through the grid points.
\end{proof}

\section{Evolution equations}

After an excursion to second order differential equations depending on
a spatial variable, we are now returning to problems depending on
time. But this time, on time \emph{and} space. As for the
nomenclature, we have encountered ordinary differential equations as
equations or systems depending on a time variable only, then partial
differential equations (PDE) with several, typically a spatial
independent variables. While the problems considered here are covered
by the definition of PDE, time and space are fundamentally different
as long as we stay away from black holes, such that a distinction is
reasonable. Therefore, we introduce the concept of

\begin{Definition}{evolution-equation}
  An equation of the form
  \begin{gather}
    \partial_t u(t,x) + F(t,u(t,x)) = 0,
  \end{gather}
  where $u(t,.)$ is in a function space $V$ on a domain $\Omega$, and
  $F$ is a differential operator with respect to the spatial variables
  $x$ only, is an \define{evolution equation} of first order (in
  time).

  An \define{initial boundary value problem} (\define{IBVP}) for this
  evolution equation completes the differential equation by conditions
  \begin{xalignat}2
    u(0,x) &= u_0 & x&\in\Omega \\
    u(t,x) &= g & x&\in\partial\Omega,\; t>0.
  \end{xalignat}
\end{Definition}

\section{Fundamental solutions}

\begin{Definition}
  The function
  \begin{gather}
    \label{eq:fd:fundamental}
    \Phi(x) =
    \begin{cases}
      -\tfrac{1}{2\pi} \log \abs{x} & d=2\\
      \tfrac1{d(d-2)\omega_d} \frac1{\abs{x}^{d-2}} & d\ge 3,
    \end{cases}
  \end{gather}
  for $x\in \R^d$ is the \define{fundamental solution} to the Laplace
  equation. Here, $\omega_d$ is the volume of the unit ball in $\R^d$.
\end{Definition}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "notes"
%%% End:


\documentclass[10pt]{article}

\title{Alignment with dynamic programming}
\author{Heng Li}

\begin{document}

\maketitle

\section{General notations}

Suppose we have two sequences: a \emph{target} sequence and a \emph{query}
sequence. The length of the target sequence is $\ell_t$ with each residue
indexed by $i$.  The length of query is $\ell_q$ with each residue indexed by
$j$. Gaps on the target sequence are \emph{deletions} and gaps on the query are
\emph{insertions}. Function $S(i,j)$ gives the score between two residues on
the target and the query, respectively. $q>0$ is the gap open/initiation
penalty and $e>0$ the gap extension penalty. A gap of length $k$ costs
$q+k\cdot e$.

\section{Global alignment with affine-gap penalties}

\subsection{Durbin's formulation}

The original Durbin's formulation is:
\begin{eqnarray*}
M_{ij}&=&\max\{M_{i-1,j-1}, E_{i-1,j-1}, F_{i-1,j-1}\} + S(i,j)\\
E_{ij}&=&\max\{M_{i-1,j}-q, E_{i-1,j}\} - e\\
F_{ij}&=&\max\{M_{i,j-1}-q, F_{i,j-1}\} - e
\end{eqnarray*}
This formulation disallows a deletion immediately followed an insertion, or
vice versa. A more general form is:
\begin{eqnarray*}
M_{ij}&=&\max\{M_{i-1,j-1}, E_{i-1,j-1}, F_{i-1,j-1}\} + S(i,j)\\
E_{ij}&=&\max\{M_{i-1,j}-q, E_{i-1,j}, F_{i-1,j}-q\} - e\\
F_{ij}&=&\max\{M_{i,j-1}-q, E_{i,j-1}-q, F_{i,j-1}\} - e
\end{eqnarray*}

\subsection{Green's formulation}

If we define:
\[H_{ij}=\max\{M_{ij},E_{ij},F_{ij}\}\]
the Durbin's formulation can be transformed to
\begin{eqnarray*}
E_{ij} &=& \max\{H_{i-1,j}-q, E_{i-1,j}\} - e \\
F_{ij} &=& \max\{H_{i,j-1}-q, F_{i,j-1}\} - e \\
H_{ij} &=& \max\{H_{i-1,j-1}+S(i,j), E_{ij}, F_{ij}\}
\end{eqnarray*}
I first saw this formulation in Phrap developed by Phil Green, though it may
have been used earlier. If we further introduce
\begin{eqnarray*}
E'_{ij}&=&E_{i+1,j}\\
F'_{ij}&=&F_{i,j+1}
\end{eqnarray*}
we have
\begin{eqnarray*}
H_{ij} &=& \max\{H_{i-1,j-1}+S(i,j),E'_{i-1,j},F'_{i,j-1}\}\\
E'_{ij}&=& \max\{H_{ij}-q,E'_{i-1,j}\}-e\\
F'_{ij}&=& \max\{H_{ij}-q,F'_{i,j-1}\}-e
\end{eqnarray*}
In fact, we more often use this set of equations in practical implementations.
The initial conditions are
\begin{eqnarray*}
H_{-1,j}&=&
  \left\{\begin{array}{ll}
    0 & (j=-1)\\
	-q-(j+1)\cdot e & (0\le j<\ell_q)
  \end{array}\right.\\
H_{i,-1}&=&
  \left\{\begin{array}{ll}
    0 & (i=-1)\\
	-q-(i+1)\cdot e & (0\le i<\ell_t)
  \end{array}\right.\\
E'_{-1,j}&=&E_{0,j}=H_{-1,j}-q-e=-2q-(j+2)\cdot e\\
F'_{i,-1}&=&F_{i,0}=-2q-(i+2)\cdot e
\end{eqnarray*}

\subsection{Suzuki's formulation}

\subsubsection{Standard coordinate}
Now let
\begin{eqnarray*}
u'_{ij}&=&H_{ij}-H_{i-1,j}\\
v'_{ij}&=&H_{ij}-H_{i,j-1}\\
x'_{ij}&=&E'_{ij}-H_{ij}\\
y'_{ij}&=&F'_{ij}-H_{ij}
\end{eqnarray*}
We have
\begin{eqnarray}\label{eq:x}
x'_{ij}&=&\max\{-q,E'_{i-1,j}-H_{i-1,j}+H_{i-1,j}-H_{ij}\}-e\\\nonumber
&=&\max\{-q,x'_{i-1,j}-u'_{ij}\}-e
\end{eqnarray}
Similarly
\begin{equation}
y'_{ij}=\max\{-q,y'_{i,j-1}-v'_{ij}\}-e
\end{equation}
To derive the equation to compute $u'(i,j)$ and $v'(i,j)$, we note that
\begin{eqnarray*}
H_{ij}-H_{i-1,j-1}
&=&\max\{S(i,j),E'_{i-1,j}-H_{i-1,j-1},F'_{i,j-1}-H_{i-1,j-1}\}\\
&=&\max\{S(i,j),x'_{i-1,j}+v'_{i-1,j},y'_{i,j-1}+u'_{i,j-1}\}
\end{eqnarray*}
and
\[H_{ij}-H_{i-1,j-1}=u'_{ij}+v'_{i-1,j}=v'_{ij}+u'_{i,j-1}\]
We can derive the recursive equation for $u'_{ij}$ and $v'_{ij}$:
\begin{eqnarray*}
z'_{ij}&=&\max\{S(i,j),x'_{i-1,j}+v'_{i-1,j},y'_{i,j-1}+u'_{i,j-1}\}\\
u'_{ij}&=&z'_{ij}-v'_{i-1,j}\\
v'_{ij}&=&z'_{ij}-u'_{i,j-1}\\
x'_{ij}&=&\max\{0,x'_{i-1,j}+v'_{i-1,j}-z'_{ij}+q\}-q-e\\
y'_{ij}&=&\max\{0,y'_{i,j-1}+u'_{i,j-1}-z'_{ij}+q\}-q-e
\end{eqnarray*}

From eq.~(\ref{eq:x}) we can infer that $x'_{ij}\ge-q-e$ and similarly
$y'_{ij}\ge-q-e$. We further have:
\[
u'_{ij}=H_{ij}-H_{i-1,j-1}-v'_{i-1,j}\ge x'_{i-1,j}\ge-q-e
\]
Therefore, we have a lower bound $-q-e$ for $u'$, $v'$, $x'$ and $y'$.
This motivates us to redefine the four variables as:
\begin{eqnarray*}
u''_{ij}&=&H_{ij}-H_{i-1,j}+q+e\\
v''_{ij}&=&H_{ij}-H_{i,j-1}+q+e\\
x''_{ij}&=&E'_{ij}-H_{ij}+q+e\\
y''_{ij}&=&F'_{ij}-H_{ij}+q+e
\end{eqnarray*}
The recursion becomes
\begin{eqnarray*}
z''_{ij}&=&\max\{S(i,j)+2q+2e,x''_{i-1,j}+v''_{i-1,j},y''_{i,j-1}+u''_{i,j-1}\}\\
u''_{ij}&=&z''_{ij}-v''_{i-1,j}\\
v''_{ij}&=&z''_{ij}-u''_{i,j-1}\\
x''_{ij}&=&\max\{0,x''_{i-1,j}-u''_{ij}+q\}=\max\{0,x''_{i-1,j}+v''_{i-1,j}-z''_{ij}+q\}\\
y''_{ij}&=&\max\{0,y''_{i,j-1}-v''_{ij}+q\}=\max\{0,y''_{i,j-1}+u''_{i,j-1}-z''_{ij}+q\}
\end{eqnarray*}
Here $z_{ij}$ is a temporary variable. $u''$, $v''$, $x''$ and $y''$ are all
non-negtive.

\subsubsection{Rotated coordinate}

We let
\begin{eqnarray*}
r&=&i+j\\
t&=&i
\end{eqnarray*}
We have
\begin{eqnarray*}
z_{rt}&=&\max\{S(t,r-t)+2q+2e,x_{r-1,t-1}+v_{r-1,t-1},y_{r-1,t}+u_{r-1,t}\}\\
u_{rt}&=&z_{rt}-v_{r-1,t-1}\\
v_{rt}&=&z_{rt}-u_{r-1,t}\\
x_{rt}&=&\max\{0,x_{r-1,t-1}+v_{r-1,t-1}-z_{rt}+q\}\\
y_{rt}&=&\max\{0,y_{r-1,t}+u_{r-1,t}-z_{rt}+q\}
\end{eqnarray*}
Due to the definition of $r$ and $t$, the following inequation must stand:
\[0\le r-t \le\ell_q-1\]
\[0\le t \le\ell_t-1\]
where $\ell_t$ is the length of the sequence indexed by $i$ and $\ell_q$ the
length indexed by $j$. In case of banded alignment with a fixed diagonal band
of size $w$,
\[-w\le j-i\le w\]
In the $(r,t)$ coordinate, it is:
\[\frac{r-w}{2}\le t\le \frac{r+w}{2}\]
Putting these together:
\[0\le r\le \ell_q+\ell_t-2\]
\[\max\left\{0,r-\ell_q+1,\frac{r-w}{2}\right\}\le t\le\min\left\{\ell_t-1,r,\frac{r+w}{2}\right\}\]

\subsubsection{Initial conditions}
\[x_{r-1,-1}=x''_{-1,r}=E'_{-1,r}-H_{-1,r}+q+e=0\]
\[y_{r-1,r}=y''_{r,-1}=0\]
\[v_{r-1,-1}=v''_{-1,r}=H_{-1,r}-H_{-1,r-1}+q+e=\left\{\begin{array}{ll}
  q & (r>0) \\
  0 & (r=0)
\end{array}\right.\]
\[u_{r-1,r}=u''_{r,-1}=H_{r,-1}-H_{r-1,-1}+q+e=\left\{\begin{array}{ll}
  q & (r>0) \\
  0 & (r=0)
\end{array}\right.\]

\section{Alignment with dual affine-gap penalties}

\subsection{Green's formulation}

\begin{eqnarray*}
H_{ij} &=& \max\{H_{i-1,j-1}+S(i,j),E'_{i-1,j},F'_{i,j-1},\tilde{E}'_{i-1,j},\tilde{F}'_{i,j-1}\}\\
E'_{ij}&=& \max\{H_{ij}-q,E'_{i-1,j}\}-e\\
F'_{ij}&=& \max\{H_{ij}-q,F'_{i,j-1}\}-e\\
\tilde{E}'_{ij}&=& \max\{H_{ij}-\tilde{q},\tilde{E}'_{i-1,j}\}-\tilde{e}\\
\tilde{F}'_{ij}&=& \max\{H_{ij}-\tilde{q},\tilde{F}'_{i,j-1}\}-\tilde{e}
\end{eqnarray*}
The initial conditions are:
\begin{eqnarray*}
H_{-1,j}&=&
  \left\{\begin{array}{ll}
    0 & (j=-1)\\
	\max\{-q-(j+1)\cdot e,-\tilde{q}-(j+1)\cdot\tilde{e}\} & (0\le j<\ell_q)
  \end{array}\right.\\
H_{i,-1}&=&
  \left\{\begin{array}{ll}
    0 & (i=-1)\\
	\max\{-q-(i+1)\cdot e,-\tilde{q}-(i+1)\cdot\tilde{e}\} & (0\le i<\ell_t)
  \end{array}\right.\\
E'_{-1,j}&=&E_{0,j}=H_{-1,j}-q-e\\
F'_{i,-1}&=&F_{i,0}=H_{i,-1}-q-e\\
\tilde{E}'_{-1,j}&=&\tilde{E}_{0,j}=H_{-1,j}-\tilde{q}-\tilde{e}\\
\tilde{F}'_{i,-1}&=&\tilde{F}_{i,0}=H_{i,-1}-\tilde{q}-\tilde{e}
\end{eqnarray*}

\subsection{Suzuki's formulation}

\begin{eqnarray*}
z'_{ij}&=&\max\{S(i,j),x'_{i-1,j}+v'_{i-1,j},y'_{i,j-1}+u'_{i,j-1},\\
&&\tilde{x}'_{i-1,j}+v'_{i-1,j},\tilde{y}'_{i,j-1}+u'_{i,j-1}\}\\
u'_{ij}&=&z'_{ij}-v'_{i-1,j}\\
v'_{ij}&=&z'_{ij}-u'_{i,j-1}\\
x'_{ij}&=&\max\{0,x'_{i-1,j}+v'_{i-1,j}-z'_{ij}+q\}-q-e\\
y'_{ij}&=&\max\{0,y'_{i,j-1}+u'_{i,j-1}-z'_{ij}+q\}-q-e\\
\tilde{x}'_{ij}&=&\max\{0,\tilde{x}'_{i-1,j}+v'_{i-1,j}-z'_{ij}+\tilde{q}\}-\tilde{q}-\tilde{e}\\
\tilde{y}'_{ij}&=&\max\{0,\tilde{y}'_{i,j-1}+u'_{i,j-1}-z'_{ij}+\tilde{q}\}-\tilde{q}-\tilde{e}
\end{eqnarray*}
In the rotated coordinate:
\begin{eqnarray*}
z_{rt}&=&\max\{S(t,r-t),x_{r-1,t-1}+v_{r-1,t-1},y_{r-1,t}+u_{r-1i,t},\\
&&\tilde{x}_{r-1,t-1}+v_{r-1,t-1},\tilde{y}_{r-1,t}+u_{r-1,t}\}\\
u_{rt}&=&z_{rt}-v_{r-1,t-1}\\
v_{rt}&=&z_{rt}-u_{r-1,t}\\
x_{rt}&=&\max\{0,x_{r-1,t-1}+v_{r-1,t-1}-z_{rt}+q\}-q-e\\
y_{rt}&=&\max\{0,y_{r-1,t}+u_{r-1,t}-z_{rt}+q\}-q-e\\
\tilde{x}_{rt}&=&\max\{0,\tilde{x}_{r-1,t-1}+v_{r-1,t-1}-z_{rt}+\tilde{q}\}-\tilde{q}-\tilde{e}\\
\tilde{y}_{rt}&=&\max\{0,\tilde{y}_{r-1,t}+u_{r-1,t}-z_{rt}+\tilde{q}\}-\tilde{q}-\tilde{e}
\end{eqnarray*}
By definition, it is easy to see the initial conditions except $u$ and $v$:
\[x_{r-1,-1}=x'_{-1,r}=E'_{-1,r}-H_{-1,r}=-q-e\]
\[y_{r-1,r}=y'_{r,-1}=F'_{r,-1}-H_{r,-1}=-q-e\]
\[\tilde{x}_{r-1,-1}=-\tilde{q}-\tilde{e}\]
\[\tilde{y}_{r-1,-1}=-\tilde{q}-\tilde{e}\]
\[v_{r-1,-1}=H_{-1,r}-H_{-1,r-1}=\left\{\begin{array}{ll}
\max\{-q-e,-\tilde{q}-\tilde{e}\} & (r=0)\\
-e & (r<\lceil\frac{\tilde{q}-q}{e-\tilde{e}}-1\rceil)\\
r(e-\tilde{e})-(\tilde{q}-q)-\tilde{e} & (r=\lceil\frac{\tilde{q}-q}{e-\tilde{e}}-1\rceil)\\
-\tilde{e} & (r>\lceil\frac{\tilde{q}-q}{e-\tilde{e}}-1\rceil)
\end{array}\right.\]

\end{document}


\section{The Fundamental Theorem of Calculus}\label{sec:FTC}
Let's recast the first example from the previous section. Suppose that
the speed of the object is $3t$ at time $t$. How far does the object
travel between time $t=a$ and time $t=b$? We are no longer assuming
that we know where the object is at time $t=0$ or at any other
time. It is certainly true that it is {\it somewhere,} so let's
suppose that at $t=0$ the position is $k$. Then just as in the
example, we know that the position of the object at any time is
$\ds 3t^2/2+k$. This means that at time $t=a$ the position is
$\ds 3a^2/2+k$ and at time $t=b$ the position is $\ds 3b^2/2+k$. Therefore the
change in position is $\ds 3b^2/2+k-(3a^2/2+k)=3b^2/2-3a^2/2$. Notice that
the $k$ drops out; this means that it doesn't matter that we don't
know $k$, it doesn't even matter if we use the wrong $k$, we get the
correct answer.

What about the second approach to this problem, in the new form? We
now want to approximate the change in position between time $a$ and
time $b$. We take the interval of time between $a$ and $b$, divide it
into $n$ subintervals, and approximate the distance traveled during
each. The starting time of subinterval number $i$ is now
$a+(i-1)(b-a)/n$, which we abbreviate as $\ds t_{i-1}$, so that
$\ds t_0=a$, $\ds t_1=a+(b-a)/n$, and so on. The speed of the object is
$f(t)=3t$, and each subinterval is $(b-a)/n=\Delta t$ seconds long.
The distance traveled during subinterval number
$i$ is approximately $\ds f(t_{i-1})\Delta t$, and the total change in
distance is approximately
$$
  f(t_0)\Delta t+f(t_1)\Delta t+\cdots+f(t_{n-1})\Delta t.
$$
The exact change in position is the limit of this sum as $n$ goes to
infinity. We abbreviate this sum using \dfont{sigma notation}:
$$
  \sum_{i=0}^{n-1} f(t_i)\Delta t =
f(t_0)\Delta t+f(t_1)\Delta t+\cdots+f(t_{n-1})\Delta t.
$$
The notation on the left side of the equal sign uses a large capital
sigma, a Greek letter, and the left side is an abbreviation for the
right side. The answer we seek is
$$
  \lim_{n\to\infty}\sum_{i=0}^{n-1} f(t_i)\Delta t.
$$
Since this must be the same as the answer we have already obtained, we
know that
$$
  \lim_{n\to\infty}\sum_{i=0}^{n-1} f(t_i)\Delta t={3b^2\over
  2}-{3a^2\over 2}.
$$
The significance of $\ds 3t^2/2$, into which we substitute $t=b$ and
$t=a$, is of course that it is a function whose derivative is $f(t)$.
As we have discussed, by the time we know that we want to compute
$$
  \lim_{n\to\infty}\sum_{i=0}^{n-1} f(t_i)\Delta t,
$$
it no longer matters what $f(t)$ stands for---it could be a speed, or
the height of a curve, or something else entirely. We know that
the limit can be computed by finding any function with derivative
$f(t)$, substituting $a$ and $b$, and subtracting. We summarize this
in a theorem. First, we introduce some new notation and terms.

We write
$$
  \int_a^b f(t)\,dt = \lim_{n\to\infty}\sum_{i=0}^{n-1} f(t_i)\Delta t
$$
if the limit exists. That is, the left hand side means, or is an
abbreviation for, the right hand side. The symbol $\int$ is called an
\dfont{integral sign}, and the whole
expression is read as ``the integral of $f(t)$ from $a$ to $b$.'' What
we have learned is that this integral can be computed by finding a
function, say $F(t)$, with the property that $F'(t)=f(t)$, and then
computing $F(b)-F(a)$. The function $F(t)$ is called an \dfont{antiderivative} of $f(t)$.
Now the theorem:

\begin{theorem}{Fundamental Theorem of Calculus}{fundamental_theorem_I}
Suppose that $f(x)$ is
continuous on the interval $[a,b]$. If $F(x)$ is any antiderivative of
$f(x)$, then
$$
  \int_a^b f(x)\,dx = F(b)-F(a).
$$
\end{theorem}

Let's rewrite this slightly:
$$
  \int_a^x f(t)\,dt = F(x)-F(a).
$$
We've replaced the variable $x$ by $t$ and $b$ by $x$. These are just
different names for quantities, so the substitution doesn't change the
meaning. It does make it easier to think of the two sides of the
equation as functions. The expression
$$
  \int_a^x f(t)\,dt
$$
is a function: plug in a value for $x$, get out some other value. The
expression $F(x)-F(a)$ is of course also a function, and it has a nice
property:
$$
  {d\over dx} (F(x)-F(a)) = F'(x) = f(x),
$$
since $F(a)$ is a constant and has derivative zero. In other words, by
shifting our point of view slightly, we see that the odd looking
function
$$
  G(x)=\int_a^x f(t)\,dt
$$
has a derivative, and that in fact $G'(x)=f(x)$. This is really just a
restatement of the Fundamental Theorem of Calculus, and indeed is
often called the Fundamental Theorem of Calculus. To avoid confusion,
some people call the two versions of the theorem ``The Fundamental
Theorem of Calculus, part I'' and ``The Fundamental
Theorem of Calculus, part II'', although unfortunately there is no
universal agreement as to which is part I and which part II. Since it
really is the same theorem, differently stated, some people simply
call them both ``The Fundamental
Theorem of Calculus.''

\begin{theorem}{Fundamental Theorem of Calculus}{fundamental_theorem_II}
Suppose that $f(x)$ is
continuous on the interval $[a,b]$ and let
$$
  G(x)=\int_a^x f(t)\,dt.
$$
Then $G'(x)=f(x)$.
\end{theorem}

We have not really proved the Fundamental Theorem. In a nutshell, we
gave the following argument to justify it: Suppose we want to know the
value of
$$
  \int_a^b f(t)\,dt = \lim_{n\to\infty}\sum_{i=0}^{n-1} f(t_i)\Delta t.
$$
We can interpret the right hand side as the distance traveled by an
object whose speed is given by $f(t)$. We know another way to compute
the answer to such a problem: find the position of the object by
finding an antiderivative of $f(t)$, then substitute $t=a$ and $t=b$
and subtract to find the distance traveled. This must be the answer to
the original problem as well, even if $f(t)$ does not represent a
speed.

What's wrong with this? In some sense, nothing. As a practical matter
it is a very convincing argument, because our understanding of the
relationship between speed and distance seems to be quite solid. From
the point of view of mathematics, however, it is unsatisfactory to
justify a purely mathematical relationship by appealing to our
understanding of the physical universe, which could, however unlikely
it is in this case, be wrong.

A complete proof is a bit too involved to include here, but we will
indicate how it goes. First, if we can prove the second version of the
Fundamental Theorem, Theorem~\ref{thm:fundamental_theorem_II}, then
we can prove the first version from that:

\begin{proof} Proof of Theorem~\ref{thm:fundamental_theorem_I}.

We know from Theorem~\ref{thm:fundamental_theorem_II} that
$$
  G(x)=\int_a^x f(t)\,dt
$$
is an antiderivative of $f(x)$, and therefore any antiderivative
$F(x)$ of $f(x)$ is of the form $F(x)=G(x)+k$. Then
\begin{eqnarray*}
  F(b)-F(a)=G(b)+k-(G(a)+k) &=& G(b)-G(a)\cr
  &=&\int_a^b f(t)\,dt-\int_a^a f(t)\,dt.\cr
\end{eqnarray*}
It is not hard to see that $\ds \int_a^a f(t)\,dt=0$, so this means that
$$
  F(b)-F(a)=\int_a^b f(t)\,dt,
$$
which is exactly what Theorem~\ref{thm:fundamental_theorem_I} says.
\end{proof}

So the real job is to prove
Theorem~\ref{thm:fundamental_theorem_II}. We will sketch the proof,
using some facts that we do not prove. First, the following identity
is true of integrals:
$$
  \int_a^b f(t)\,dt = \int_a^c f(t)\,dt + \int_c^b f(t)\,dt.
$$
This can be proved directly from the definition of the integral, that
is, using the limits of sums. It is quite easy to see that it must be
true by thinking of either of the two applications of integrals that
we have seen. It turns out that the identity is true no matter what
$c$ is, but it is easiest to think about the meaning when
$a\le c\le b$.

First, if $f(t)$ represents a speed, then we know that the three
integrals represent the distance traveled between time $a$ and time $b$;
the distance traveled between time $a$ and time $c$; and
the distance traveled between time $c$ and time $b$. Clearly the sum of
the latter two is equal to the first of these.

Second, if $f(t)$ represents the height of a curve, the three
integrals represent the area under the curve between $a$ and $b$;
the area under the curve between $a$ and $c$;
and the area under the curve between $c$ and $b$. Again it is clear
from the geometry that the first is equal to the sum of the second and
third.

\begin{proof} Proof of Theorem~\ref{thm:fundamental_theorem_II}.

We want to compute $G'(x)$, so we start with the definition of the
derivative in terms of a limit:
\begin{eqnarray*}
  G'(x)&=&\lim_{\Delta x\to0}{G(x+\Delta x)-G(x)\over\Delta x}\cr
  &=&\lim_{\Delta x\to0}{1\over \Delta x}\left(
  \int_a^{x+\Delta x} f(t)\,dt - \int_a^x f(t)\,dt\right)\cr
  &=&\lim_{\Delta x\to0}{1\over \Delta x}\left(
  \int_a^{x} f(t)\,dt + \int_x^{x+\Delta x} f(t)\,dt -
  \int_a^x f(t)\,dt\right)\cr
  &=&\lim_{\Delta x\to0}{1\over \Delta x}\int_x^{x+\Delta x} f(t)\,dt.\cr
\end{eqnarray*}
Now we need to know something about
$$
  \int_x^{x+\Delta x} f(t)\,dt
$$
when $\Delta x$ is small; in fact, it is very close to
$\Delta x f(x)$, but we will not prove this. Once again, it is easy to
believe this is true by thinking of our two applications:
The integral
$$
  \int_x^{x+\Delta x} f(t)\,dt
$$
can be interpreted as the distance traveled by an object over a very
short interval of time. Over a sufficiently short period of time, the
speed of the object will not change very much, so the distance
traveled will be approximately the length of time multiplied by the
speed at the beginning of the interval, namely, $\Delta x
f(x)$. Alternately, the integral may be interpreted as the area under
the curve between $x$ and $x+\Delta x$. When $\Delta x$ is very small,
this will be very close to the area of the rectangle with base $\Delta
x$ and height $f(x)$; again this is $\Delta x
f(x)$. If we accept this, we may proceed:
$$
  \lim_{\Delta x\to0}{1\over \Delta x}\int_x^{x+\Delta x} f(t)\,dt
  =\lim_{\Delta x\to0}{\Delta x f(x)\over \Delta x}=f(x),
$$
which is what we wanted to show.
\end{proof}

It is still true that we are depending on an interpretation of the
integral to justify the argument, but we have isolated this part of
the argument into two facts that are not too hard to prove. Once the
last reference to interpretation has been removed from the proofs of
these facts, we will have a real proof of the Fundamental Theorem.

Now we know that to solve certain kinds of problems, those that lead
to a sum of a certain form, we ``merely'' find an antiderivative and
substitute two values and subtract. Unfortunately, finding
antiderivatives can be quite difficult. While there are a small number
of rules that allow us to compute the derivative of any common
function, there are no such rules for antiderivatives. There are some
techniques that frequently prove useful, but we will never be able to
reduce the problem to a completely mechanical process.

Due to the close relationship between an integral and an
antiderivative, the integral sign is also used to mean
``antiderivative''. You can tell which is intended by whether the
limits of integration are included:
$$
  \int_1^2 x^2\,dx
$$
is an ordinary integral, also called a
\dfont{definite integral},
because it has a definite value, namely
$$
  \int_1^2 x^2\,dx={2^3\over3}-{1^3\over3}={7\over3}.
$$
We use
$$
  \int x^2\,dx
$$
to denote the antiderivative of $\ds x^2$, also called an
\dfont{indefinite integral}.
So this is evaluated as
$$
  \int x^2\,dx = {x^3\over 3}+C.
$$
It is customary to include the constant $C$ to indicate that there are
really an infinite number of antiderivatives. We do not need this $C$
to compute definite integrals, but in other circumstances we will need
to remember that the $C$ is there, so it is best to get into the habit
of writing the $C$.
When we compute a definite integral, we first find an antiderivative
and then substitute. It is convenient to first display the
antiderivative and then do the substitution; we need a notation
indicating that the substitution is yet to be done. A typical solution
would look like this:
$$
  \int_1^2 x^2\,dx=\left.{x^3\over 3}\right|_1^2 =
  {2^3\over3}-{1^3\over3}={7\over3}.
$$
The vertical line with subscript and superscript is used to indicate
the operation ``substitute and subtract'' that is needed to finish the
evaluation.

We seem to have found a pattern. When attempting to solve a previous question, we found the antiderivative of $x^2$ to be $x^3/3+c$ (as it was when solving the indefinite integral). Likewise, when we first began, we were trying to determine a position based on velocity, and $3t$ gave rise to $3t^2/2+k$.

As will be formalized later, we see that in these cases, the power is increased to $n+1$, but we also divide through by this factor, $n+1$. So $x$ becomes $x^2/2$, $x^2$ becomes $x^3/3$, and $x^3$ will become $x^4/4$.

Now we will also try with negative and fraction values in the following example.

\begin{example}{Fundamental Theorem of Calculus}{FundamentalTheoremCalculus}
Evaluate $\ds\int_1^4 x^3+\sqrt{x}+\frac{1}{x^2}\,dx $.
\end{example}

\begin{solution}
\[ \begin{array}{lcl}
\ds\int_1^4 x^3+\sqrt{x}+\frac{1}{x^2}\,dx
	& = & \ds\left.\frac{x^4}{4}+\frac{2x^{3/2}}{3}-x^{-1}\right|_1^4\\
\\
	& = & \ds\left(\frac{(4)^4}{4}+\frac{2(4)^{3/2}}{3}-4^{-1}\right) \\
\\
	& & \ds-\left(\frac{(1)^4}{4}+\frac{2(1)^{3/2}}{3}-1^{-1}\right)\\
\\
	& = & \ds\frac{415}{6}
\end{array}\]\
\end{solution}

\begin{formulabox}[Properties of Definite Integrals]
Some properties are as follows:
$$\mbox{Order of limits matters:}\qquad\int_a^b f(x)\,dx=-\int_b^a f(x)\,dx$$
$$\mbox{If interval is empty, integral is zero:}\qquad\int_a^a f(x)\,dx=0$$
$$\mbox{Constant Multiple Rule:}\qquad\int_a^b cf(x)\,dx=c\int_a^b f(x)\,dx$$
$$\mbox{Sum/Difference Rule:}\qquad\int_a^b f(x)\pm g(x)\,dx=\int_a^b f(x)\,dx\pm\int_a^b g(x)\,dx$$
$$\mbox{Can split up interval $[a,b]=[a,c]\cup[c,b]$:}\qquad\int_a^b f(x)\,dx=\int_a^c f(x)\,dx+\int_c^b f(x)\,dx$$
$$\mbox{The variable does not matter!:}\qquad\int_a^b f(x)\,dx=\int_a^b f(t)\,dt$$
\end{formulabox}

The reason for the last property is that a definite integral is a \ifont{number}, not a function, so the variable is just a placeholder that won't appear in the final answer.

Some additional properties are \ifont{comparison} types of properties.

\begin{formulabox}[Comparison Properties of Definite Integrals]
$$\mbox{If $f(x)\geq 0$ for $x\in[a,b]$, then:}\qquad\int_a^b f(x)\,dx\geq 0.$$
$$\mbox{If $f(x)\geq g(x)$ for $x\in[a,b]$, then:}\qquad\int_a^b f(x)\,dx\geq \int_a^b g(x)\,dx.$$
$$\mbox{If $m\leq f(x)\leq M$ for $x\in[a,b]$, then:}\qquad m(b-a)\leq \int_a^b f(x)\,dx\leq M(b-a).$$
\end{formulabox}

\begin{example}{Properties of Definite Integrals}{PropertiesDefiniteIntegrals}
Suppose $\ds{\int_a^b f(x)~dx=7}$ and $\ds{\int_a^b g(x)~dx=3}$. Find:
\begin{multicols}{2}
\begin{enumerate}
	\item	$\ds\int_a^b 2f(x)-3g(x)\,dx$.
	\item	$\ds\int_{b}^{a} 2g(x)\,dx$.
	\item	$\ds\int_a^a f(x)\cdot g(x)\,dx$.
	\item	$\ds\int_a^c f(x)~dx+\int_c^b f(x)\,dx$.
\end{enumerate}
\end{multicols}
\vspace{5mm}
\end{example}
\begin{solution}
\begin{enumerate}
	\item	$\ds\int_a^b 2f(x)-3g(x)\,dx=\ds 2\int_a^b f(x)\,dx-3\int_a^b g(x)\,dx=2(7)-3(3)=5$.
	\item	$\ds\int_{b}^{a} 2g(x)\,dx=\ds -2\int_{a}^{b} g(x)\,dx=-2(3)=-6$.
	\item	$\ds\int_a^a f(x)\cdot g(x)\,dx=0$.
	\item	$\ds\int_a^c f(x)\,dx+\int_c^b f(x)\,dx=\ds\int_a^b f(x)\,dx=7$.
\end{enumerate}
\end{solution}

We next evaluate a definite integral using three different techniques.

\begin{example}{Three Different Techniques}{ThreeDifferentTechniques}
Evaluate $\ds{\int_0^2 x+1~dx}$ by
\begin{enumerate}
\item Using FTC II (the shortcut)
\item Using the definition of a definite integral (the limit sum definition)
\item Interpreting the problem in terms of areas (graphically)
\end{enumerate}
\end{example}

\begin{solution}
1. The shortcut (FTC II) is the method of choice as it is the fastest.
Integrating and using the `\ifont{top minus bottom}' rule we have:
\begin{eqnarray*}
\int_0^2 x+1~dx&=&\left.\frac{x^2}{2}+x\right|_0^2\\
&=&\left[\frac{2^2}{2}+2\right]-\left[\frac{0^2}{2}+0\right]=4.
\end{eqnarray*}

2. We now use the definition of a definite integral.
We divide the interval $[0,2]$ into $n$ subintervals of equal width $\Delta x$, and from each interval choose a point $x_i^*$.
Using the formulas
$$\Delta x = \frac{b-a}{n}\qquad\mbox{and}\qquad x_i=a+i\Delta x,$$
we have
$$\Delta x = \frac{2}{n}\qquad\mbox{and}\qquad x_i=0+i\Delta x=\frac{2i}{n}.$$
Then taking $x_i^*$'s as right endpoints for convenience (so that $x_i^*=x_i$), we have:
\begin{eqnarray*}
\int_0^2 x+1~dx & = & \ds\lim_{n\to\infty}\sum_{i=1}^n f(x_i^*)\Delta x\\
\\
& = & \ds\lim_{n\to\infty}\sum_{i=1}^n f\left(\frac{2i}{n}\right) \frac{2}{n}\\
\\
& = & \ds\lim_{n\to\infty}\sum_{i=1}^n \left(\frac{2i}{n}+1\right) \frac{2}{n}\\
\\
& = & \ds\lim_{n\to\infty}\sum_{i=1}^n \left(\frac{4i}{n^2}+\frac{2}{n}\right)\\
\\
& = & \ds\lim_{n\to\infty}\left(\sum_{i=1}^n \frac{4i}{n^2}+\sum_{i=1}^n \frac{2}{n} \right)\\
\\
& = & \ds\lim_{n\to\infty}\left(\frac{4}{n^2}\sum_{i=1}^n i+\frac{2}{n}\sum_{i=1}^n 1 \right)\\
\\
& = & \ds\lim_{n\to\infty}\left(\frac{4}{n^2}\frac{n(n+1)}{2}+\frac{2}{n}n \right)\\
\\
& = & \ds\lim_{n\to\infty}\left(2+\frac{2}{n}+2 \right)\\
\\
& = & 4.
\end{eqnarray*}

3. Finally, let's evaluate the net area under $x+1$ from $0$ to $2$.
$$\includegraphics[width=5in]{images/int-ex}$$
Thus, the area is the sum of the areas of a rectangle and a triangle.
Hence,
\begin{eqnarray*}
\int_0^2 x+1~dx&=&\mbox{Net Area}\\
&=&\mbox{Area of rectangle} + \mbox{Area of triangle}\\
&=&(2)(1)+\frac{1}{2}(2)(2)\\
&=&4.
\end{eqnarray*}
\end{solution}

We next apply FTC to differentiate a function.

\begin{example}{Using FTC}{UsingFTC}
\exfont{Differentiate} the following function:
$$g(x)=\int_{-2}^x \cos(1+5t)\sin t\,dt.$$
\end{example}

\begin{solution}
We simply apply the Fundamental Theorem of Calculus directly to get:
$$g'(x)=\cos(1+5x)\sin x.$$
\end{solution}

Using the Chain Rule we can derive a formula for some more complicated problems.
We have:
$$\frac{d}{dx}\int_a^{v(x)}f(t)\,dt=f(v(x))\cdot v'(x).$$

Now what if the upper limit is constant and the lower limit is a function of $x$?
Then we interchange the limits and add a minus sign to get:
$$\frac{d}{dx}\int_{u(x)}^af(t)\,dt=-\frac{d}{dx}\int_a^{u(x)} f(t)\,dt=-f(u(x))\cdot u'(x).$$

Combining these two we can get a formula where both limits are a function of $x$.
We break up the integral as follows:
$$\int_{u(x)}^{v(x)} f(t)\,dt=\int_{u(x)}^a f(t)\,dt+\int_a^{v(x)}f(t)\,dt.$$
We just need to make sure $f(a)$ exists after we break up the integral.
Then differentiating and using the above two formulas gives:

\begin{formulabox}[FTC I + Chain Rule:]
$$\frac{d}{dx}\int_{{u(x)}}^{{v(x)}} f(t)\,dt=f({v(x)}){v'(x)}-f({u(x)}){u'(x)}$$
\end{formulabox}

Many textbooks do not show this formula and instead to solve these types of problems will use FTC I along with the tricks we used to derive the formula above.
Either method is perfectly fine.

\begin{example}{FTC I + Chain Rule}{FTCIChainRule}
Differentiate the following integral:
$$\int_{10x}^{x^2} t^3\sin(1+t) \,dt.$$
\end{example}

\begin{solution}
We will use the formula above.
We have $f(t)=t^3\sin(1+t)$, $u(x)=10x$ and $v(x)=x^2$.
Then $u'(x)=10$ and $v'(x)=2x$.
Thus,
\begin{eqnarray*}
\frac{d}{dx}\int_{10x}^{x^2} t^3\sin(1+t) \,dt&=&(x^2)^3\sin(1+(x^2))(2x)-(10x)^3\sin(1+(10x))(10)\\
\\
&=&2x^7\sin(1+x^2)-10000x^3\sin(1+10x)
\end{eqnarray*}
\end{solution}

\begin{example}{FTC I + Chain Rule}{FTCIChainRule2}
Differentiate the following integral with respect to $x$:
$$\int_{x^3}^{2x} 1+\cos t\,dt$$
\end{example}

\begin{solution}
Using the formula we have:
$$\frac{d}{dx}\int_{x^3}^{2x} 1+\cos t\,dt=(1+\cos(2x))(2)-(1+\cos(x^3))(3x^2).$$
\end{solution}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Opensolutionfile{solutions}[ex]
\section*{Exercises for Section \ref{sec:FTC}}

\begin{enumialphparenastyle}

%%%%%%%%%%
\begin{ex}
Evaluate $\ds \int_1^4 t^2+3t\,dt$
\begin{sol}
 $87/2$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Evaluate $\ds \int_0^\pi \sin t\,dt$
\begin{sol}
 $2$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Evaluate $\ds \int_1^{10} {1\over x}\,dx$
\begin{sol}
 $\ln(10)$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Evaluate $\ds \int_0^5 e^x\,dx$
\begin{sol}
 $\ds e^5-1$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Evaluate $\ds \int_0^3 x^3\,dx$
\begin{sol}
 $\ds 3^4/4$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Evaluate $\ds \int_1^2 x^5\,dx$
\begin{sol}
 $\ds 2^6/6 -1/6$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 Find the derivative of $\ds G(x)=\int_1^x t^2-3t\,dt$
\begin{sol}
 $\ds x^2-3x$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 Find the derivative of $\ds G(x)=\int_1^{x^2} t^2-3t\,dt$
\begin{sol}
 $\ds 2x(x^4-3x^2)$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 Find the derivative of $\ds G(x)=\int_1^x e^{t^2}\,dt$
\begin{sol}
 $\ds e^{x^2}$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 Find the derivative of $\ds G(x)=\int_1^{x^2} e^{t^2}\,dt$
\begin{sol}
 $\ds 2xe^{x^4}$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 Find the derivative of $\ds G(x)=\int_1^x \tan(t^2)\,dt$
\begin{sol}
 $\ds \tan(x^2)$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
 Find the derivative of $\ds G(x)=\int_{10x}^{x^2} \tan(t^2)\,dt$
\begin{sol}
 $\ds 2x\tan(x^4)-10\tan(100x^2)$
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Suppose $\int_{1}^{4}f(x)\,dx=2$ and $\int_{1}^{4}g(x)\,dx=7$. Find $\int_{1}^{4}(5f(x)+3g(x))\,dx$ and $\int_{1}^{4}(6-2f(x))\,dx$.
\begin{sol}
	31, 14
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Suppose $\int_{-2}^{5}f(x)\,dx=3$ and $\int_{1}^{5}f(x)\,dx=-2$. Find $\int_{-2}^{1}f(x)\,dx$.
\begin{sol}
	5
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
If $f$ is continuous on $[a,b]$, we define the average of $f(x)$ on $[a,b]$ to be
\begin{equation*}
\text{avg}_{[a,b]}(f)=\frac{1}{b-a}\int_{a}^{b}f(x)\,dx.
\end{equation*}
\begin{enumerate}
	\item	What is the average of $\sqrt{x}$ on the interval $[0,1]?$
	\item	If the average of $f(x)$ on $[0,2]$ and on $[2,5]$ are 6 and 4 respectively, then what is its average on $[0,5]$?
\end{enumerate}
\begin{sol}
\begin{enumerate}
	\item	2/3
	\item	24/5
\end{enumerate}
\end{sol}
\end{ex}

\end{enumialphparenastyle}


In this work, we present a data-driven IRL-based social navigation pipeline.\\
An overview of the pipeline:
\\ \textbf{An image showing the block diagram of the pipeline including the environment, the feature extractor and the other components}
We will describe each component of the pipeline in greater detail.
\section*{The IRL block:}
Inverse reinforcement learning(IRL) or inverse optimal control(IOC) has been in vogue in recent years when it comes to training robots to perform real-world tasks. This is understandable as assigning rewards to individual states to illicit out desired behaviors is challenging. IRL provides a better alternative of obtaining the underlying reward function as well as a trained agent on the obtained reward function using demonstrations from the expert.

To keep the chapter self-contained, we will briefly go over the definition of a Markov decision process, which is at the base of reinforcement and inverse reinforcement learning.
A Markov decision process or MDP can be defined as a tuple ($\mathcal{S}$,$\mathcal{A}$,T,$\gamma$, $\mathcal{R}$)  where,
\begin{itemize}
	\item $\mathcal{S}$ is the set of all possible states.
	\item $\mathcal{A}$ is the set of all possible actions.
	\item T is the state transition dynamics, i.e. the probability of moving to a state given its previous state and action, $P(s^{'}|s,a)$ .
	\item $\gamma$ is the discounting factor.
	\item $\mathcal{R}$ is the set of rewards $R:  \mathcal{S} \mapsto \mathbb{R} $ is the reward function. In practice, instead of using the raw states, hand engineered features are extracted from the states with are then used to calculate the reward of a particular state. This alleviates a lot of complexity when dealing with large continuous state-spaces.
	\end{itemize}
The goal of the IRL is to infer a reward function that best explains the behavior of the expert. The expert behavior is represented in terms of expert demonstrations or trajectories, $D = \{ \tau_1, \tau_2, \tau_3, \dots,
\tau_{M} \}$ in the context of navigation. Each of these trajectories, in turn, can be further broken down into a collection of states $\tau_{i} = \{ s_{0}, s_{1}, s_{2}, \dots, s_{T} \}$ as visited by the expert in the trajectory.

We base our work on Wulfmeier's paper, which in turn is a neural network adaptation of the work done by Ziebart 2008.
According to the max entropy formulation, the probability of the occurrence of a trajectory is directly proportional to the reward it receives.
\begin{align}
Entropy equation - Ziebart equation 2
\end{align}
and is equal to:
\begin{align}
equation with z
\end{align}
Given a set of expert demonstrations, an optimal reward structure should maximize the probability of the occurrence of the expert demonstrations and their associated states. Mathematically, this is given by
\begin{align}
the equation for the loglikelihood-Ziebart equation 6
\end{align}
The original paper by Ziebart used a linear combination of weights as reward functions. But linear representations have limited capabilities when it comes to expressing complex reward functions. This problem is addressed by Wulfmeier 2015 where they restructure the maxent IRL formulation using neural networks. Neural networks are universal function approximators, and this vastly improves on the amount of complexity the reward functions can encapsulate.

\begin{align}
derive the gradient expression in equation 3 of iros paper
\end{align}

The original formulation of MEDIRL was in the context of a model-based setting and the state transition matrix was used to calculate the agent SVF. While this produces an exact value of the agent SVF, assuming the availability of the state transition matrix is fairly optimistic for most real-world tasks including navigation. In an attempt to make things less constrained we take the model-free approach and focus on calculating the SVF using sampling-based methods.
The SVF calculation:
The main challenge going model free is the calculation of the Z value, which previously could be calculated using dynamic programming [citation of the paper].
Under the assumption of a model-free but deterministic environment, the SVF of a deterministic policy can be reasonably computed by taking trajectory samples of the policy from the context of all the existing pedestrians in the scene.
\begin{align}
equation 4 from iros2020
\end{align}
where the $\mathcal{P}$ represents state transitions obtained from sampling and not the state transition dynamics. \textbf{We argue that this assumption is reasonable in a navigation setting because the task is not inherently uncertain, and most transition dynamic uncertainty can be attributed to sensory noise and control error. We summarize our approach in algorithm 1. (Taken word-to-word from iros manuscript)}
\begin{algorithm}
	algorithm 1 from iros2020
\end{algorithm}

For solving the MDP, we employ actor-critic methods, which we will describe in detail in the next section.

%\subsection*{Overview of the algorithm used}
%The algorithm trains for two networks, the reward network that, given the features of a state returns the reward associated with it,\\
%\textbf{equation}\\ stating this.
%and the policy network, which given the same, returns the best possible action.\\
%\textbf{equation}\\
%The method starts with randomly initializing the weights of the reward network. This reward network is then used in the  RL block to train an agent which is optimal for the current reward structure. Once, an optimal policy is obtained, the policy is then sampled from, in the environment to obtain roll outs or trajectories in this case. A trajectory is given by the sequence of states visited by the agent {s1, s2, ... sn}.
%Once the trajectories are obtained, they are used to calculate the state visitation frequency. The difference between the expert and the agent SVF is used to calculate the loss
%\textbf{equation}
%This loss is then back propagated through the reward network to update the weights.
%Once the weights are updated, the new network is again fed into the RL block. This iterative process continues until completion.
%Explanation of the L1 regularization over l2 regularization

\section*{The RL block:}

Actor-critic methods are a class of reinforcement learning algorithms that are built upon policy gradient methods.
In policy-gradient methods, the goal is to iteratively improve the performance of a given policy which is achieved by maximizing the expected return of the policy. Mathematically, the objective of a policy gradient method can be expressed as:
\begin{align}
maximize \;\; J( \theta )  &\; = \; \mathbb{E} [ R | \pi_{\theta} ] \\
					   & \; = \; \mathbb{E}[ \sum^{T-1}_{t=0} r_{t+1}| \pi_{\theta}]
\end{align}
where, $r_{t}$ is the reward obtained at time $t$ and $\pi_{\theta}$ is the policy with parameters $\theta$.\\
This leads to an update function:
\begin{align}
\delta J ( \theta ) =
\end{align}
Vanilla policy gradient methods suffer from high biases because they do not have a normalizing factor for the second term. This is addressed by introducing a
baseline.
\begin{align}
Equation with the baseline to the vanilla policy gradient.
\end{align}
The baseline can be calculated in various ways. One of the resulting algorithms is the A2C or the Advantage actor-critic algorithm, which we use to solve our MDP.
The A2C method:
Two symbiotic agents at play here. The actor and the critic.
The critic estimates the value function of a given state.
The actor uses this information to update its policy distribution.
\begin{algorithm}
	The actor-critic algorithm.
\end{algorithm}

\section*{The feature extractor}
The feature extractor is a vital component in the navigation pipeline. It acts as a medium that enables the learning algorithm to interact with the environment. The performance of a learning agent is highly contingent on the design of the feature extractor. \cite{vasquez_et_al}\\

The feature vector generated by the feature extractor can be broken down into 2 broad components: local and global components.
\begin{itemize}
	\item The \textbf{local component}, as the name suggests, comprises of the information from nearby surroundings of the agent captured in the form of a binary feature vector. This provides an approximate idea of the obstacles in the vicinity.
	\item The \textbf{global component}, on the other hand, provides a rough direction of where the goal is with respect to the agent. This provides the agent with a purpose of navigation, preventing it from just rambling around the map.
\end{itemize}
 The environment, by design, publishes a detailed description of obstacles in the map, including their location, orientation, and velocity, and the goal. Having access to such a detailed picture of the surroundings on a mobile robot navigating any given scene in the real world is highly unlikely and difficult to obtain. This is additionally addressed by the feature extractor, which also acts as an information moderator, receiving raw information from the environment and packaging it in a feature vector that can be readily constructed by a mobile robot on the go using off-the-shelf sensors.\\
 Both the local and the global components along with their subcomponents are described in greater detail below.
Talk about the relative orientation calculation

\subsection*{The global component}
The global information is further comprised of 3 elements:

\subsubsection*{Relative goal orientation}
This acts as a compass, providing a rough estimate of the location of the goal based on the current position and orientation of the agent. This is denoted by a $9 \times 1$ one-hot vector, where the presence of the goal in any one of the bins is marked by a $1$ keeping the rest to $0$. The $360 \degree$ around the agent are divided into $8$ equal divisions forming the first 8 bins of $45 \degree$ each. The $9^{th}$ bin denotes the contact of the agent with the goal. The structuring of the bins is shown below.
\begin{figure}[!htbp]
	Figure showing the relative goal orientation
\end{figure}

\subsubsection*{Change in orientation}
Represented by a $5 \times 1$ one-hot vector, the change in orientation captures the magnitude of the change in the orientation of the agent in consecutive steps. The value ranges between $0 \degree$ -  $ 180 \degree$. This value is binned in one of 5 asymmetric bins. The rationale behind the uneven distribution is that empirically we have observed that human motion is smooth. So, the bins are constructed in a way to have a finer resolution in the lower range. This helps capture the nuances in the human motion in greater detail leading to better encapsulation of the essence of the navigational pattern. The division of the range is shown below.

\begin{table}[htbp]
	\caption{Bin thresholds orientation change features.}
	\label{orientation-change-bins}
	\begin{center}
		\renewcommand{\arraystretch}{1.3}
		\begin{tabular}{|c|c|}
			\hline
			Feature & Threshold \\
			\hline
			$\phi_{O1}$ & $\alpha_{OC} \in \left[ 0 , \frac{\pi}{9} \right)$ \\

			$\phi_{O2}$ & $\alpha_{OC} \in \left[ \frac{\pi}{9} , \frac{2\pi}{9} \right)$ \\

			$\phi_{O3}$ & $\alpha_{OC} \in \left[ \frac{2\pi}{9} , \frac{3\pi}{9} \right)$ \\

			$\phi_{O5}$ & $\alpha_{OC} \in \left[ \frac{3\pi}{9} , \frac{4\pi}{9} \right)$ \\

			$\phi_{O6}$ & $\alpha_{OC} \in \left[ \frac{4\pi}{9} , \pi \right)$ \\
			\hline
		\end{tabular}
	\end{center}
\end{table}
\subsection*{Deviation from the goal}
Represented by a $4 \times 1$ one-hot vector, the deviation from goal captures the magnitude of the angle between the vector to the goal from the current position of the agent and the current orientation vector of the agent. The value ranges from $0 \degree$ - $ 180 \degree$ which is again asymmetrically divided into 4 bins, with greater emphasis is laid on the lower degrees due to reasons mentioned earlier.

\begin{table}[htbp]
	\caption{Bin thresholds for deviation from the goal.}
	\label{deviation-from-goal-bins}
	\begin{center}
		\renewcommand{\arraystretch}{1.3}
		\begin{tabular}{|c|c|}
			\hline
			Feature & Threshold \\
			\hline
			$\phi_{GA1}$ & $\alpha_{GA} \in \left[ 0 , \frac{\pi}{8} \right)$ \\

			$\phi_{GA2}$ & $\alpha_{GA} \in \left[ \frac{\pi}{8} , \frac{\pi}{4} \right)$ \\

			$\phi_{GA3}$ & $\alpha_{GA} \in \left[ \frac{\pi}{4} , \frac{3\pi}{4} \right)$ \\

			$\phi_{GA4}$ & $\alpha_{GA} \in \left[ \frac{3\pi}{4} , \pi \right]$ \\
			\hline
		\end{tabular}
	\end{center}
\end{table}

\subsubsection*{Speed}
The entire range of all possible speed values are divided into 6 divisions and the current speed of the agent is represented by a $ 6 \times 1$ one-hot feature vector. The entire range of possible speeds are divided into 6 divisions with each division corresponding to an element of the one-hot vector.

%\begin{table}[htbp]
%	\caption{Thresholds for the qantization of the speed of the agent.}
%	\label{deviation-from-goal-bins}
%	\begin{center}
%		\renewcommand{\arraystretch}{1.3}
%		\begin{tabular}{|c|c|}
%			\hline
%			Raw speed & Speed bin \\
%			\hline
%			0 - 0.2 & 0 \\
%			0.2 - 0.4 & 1 \\
%			0.4 - 0.6 & 2 \\
%			\hline
%		\end{tabular}
%	\end{center}
%	\end{table}

\subsection*{The local information}
Taking inspiration from previous works in this field \cite{fahad-et-al} \cite{vasquez-et-al}, we use spatial bins to effectively break the region surrounding the agent into discrete segments and calculate a 'risk' metric for each of these bins.

\subsubsection*{Creation of the bins}
There are 16 spatial bins surrounding the agent. They are arranged in two concentric circles around the agent. The region between the agent and the inner circle is then broken into 8 equal divisions. These form bins from 1 - 8. Similarly, the region between the first and the second bin is again divided into 8 equal divisions which form bins 9 - 16.
\subsubsection*{Calculation of the risk}
'Risk' is a term we label the 'threat' the agent faces at a given time from any of these bins. Here 'threat' can loosely be seen as a measure of the possibility of hitting an obstacle. The higher the threat, the greater the chance that if the agent and the obstacles continue on their current course it will end in a collision.
The 'risk factor' is divided in 3 levels: high, low and something in between.
\begin{itemize}
	\item High risk:
When the relative motion of an obstacle is towards the agent and can lead to a collision if not intervened.
	\item Low risk:
When the relative motion of an obstacle is away from the agent.
	\item Med risk:
Anything in between
\end{itemize}
The calculation of the risk values are based on the following entities:
\begin{align}
	\vec{o}_{rel} = & \;\; \vec{o}_{obs} - \vec{o}_{agent}  \\
	\vec{d}_{rel} =  &\;\; \vec{d}_{agent} - \vec{d}_{obs} \\
	\theta_{risk} =  & \;\; \angle (\vec{o}_{rel}, \vec{o}_{rel}) \\
	\mathbf{s}_{obs} = & \;\; \tan(\theta_{risk}) \times |\vec{d_{rel}}| \\
	\mathbf{T} = & \;\; \text{agent witdh} + \text{obstacle witdh}
\end{align}
where $\vec{o}_{rel}$ is the relative orientation of the obstacle w.r.t the agent, $\vec{d}_{rel}$ is the relative position of the agent w.r.t to the obstacle, $\theta_{risk}$ is the angle between the vectors, $\vec{o}_{rel}$ and $\vec{d}_{rel}$,  $\mathbf{s}_{obs}$ is the estimated \textbf{safety margin} between the agent and the obstacle and $\mathbf{T}$ is a predefined value which if maintained between the agent and an obstacle should guarantee a collision-free trajectory.\\
An obstacle is marked as 'high risk' when $\theta$ is less than $90\degree$ and the safety margin is less than $\mathbf{T}$. If $\theta$ < $90 \degree$, this indicates that the obstacle is moving away from the agent and hence chances of collision are less and hence low risk. Anything that does not fall in the above two categories are considered as medium risk. The risk calculation conditions and values are summarized in Table \ref{risk-categorization-table}

\begin{table}[htbp]
	\caption{Categorization of the risk.}
	\label{risk-categorization-table}
	\begin{center}
		\renewcommand{\arraystretch}{1.3}
		\begin{tabular}{|c|c|}
			\hline
			Risk value & Risk condition \\
			\hline
			High & $\theta < 90\degree$ \&  $\mathbf{s}_{obs}$ < $\mathbf{T}$   \\

			Low & $\theta > 90\degree$\\

			Medium & otherwise \\
			\hline
		\end{tabular}
	\end{center}
\end{table}
The risk value of each bin is represented using a $3 \times 1$ one-hot vector. One thing to note is the risk is calculated for individual obstacles present in the bin separately. And it is not uncommon to have more than one obstacle falling in different risk divisions from the same spatial bin. In that case, the risk value assigned to that bin is the highest risk posed among all the obstacles that fall under that spatial bin.
\begin{figure}[!htbp]
	\text{Graphics that shows each of the risk conditions}
\end{figure}

\section*{The SVF calculation}
One of the main challenges of IRL is calculating the expected state distribution or the state visitation frequency (svf) of a given agent. (equation x). In model-based environments, this can be calculated using the state transition matrix and dynamic programming \cite{wulfmeier-deepirl}. Having access to the state dynamics of the environment might be difficult and not always readily available, especially in real-world applications like navigation. Instead, we relax this assumption for a model-free but deterministic environment. \textbf{Explanation as to why this is a reasonable assumption} This is a reasonable assumption in the context of a navigation problem because the movements of pedestrians in general are inherently deterministic. %and any observed uncertainty by the agent can be attributed to the error in measurement by the onboard sensors.\\

Due to the lack of state dynamics we use sampling techniques to get a \textbf{good} estimate of the svf. While this can be time consuming and computationally expensive, the problem is drastically simplified when using greedy policies. (greedy soft actor critic).\textbf{Why?}
Svf calculation for an agent following a greedy policy in a deterministic environment can be calculated by taking a single sample trajectory for each pedestrian by replacing them with the agent and letting it run till completion. The calculation of the expected state visitation frequency of the agent is shown in equation \ref{agent-approx}
\begin{align}
\label{agent-approx}
\text{Equation to calculate the state visitation frequency}
\end{align}
Additionally, we also introduce a \textbf{smoothing technique} for the calculated svf. \\
\textbf{What is smoothing?}\\
In the traditional way of calculating the SVF, the observation of a given state contributes to the increment of the visitation frequency of that state by 1.
Instead for a single observation, we opt for the increment in the visitation frequency of a set of neighboring states based on the closeness of the neighboring states to the state observed. Here smoothing is defined as the distribution of the visitation weight of a state over its set of neighboring states based on their spatial similarity.\\
\textbf{Why smoothing?}
Having a 1 to 1 mapping between the observation and the increment of the SVF misses out on the fact that not all states \textit{are equally different.} \textbf{differ from each other in equal magnitude} \\
\textbf{For example:} consider 3 different states, which differ only in their goal location component. Now, if two of the states indicated the goal to be in the $2^{nd}$ spatial bin and $3^{rd}$ spatial bin, then the difference between these two states are smaller than a $3^{rd}$ state where the goal is in the $6^{th}$ bin.
This inequality in the differences among different states are accounted for in the smoothing, by increasing the weighted increment of a set of neighboring states of the observed state (based on their similarity) rather than increasing the value of the observed state only. \\
\textbf{How smoothing?}
The state vector comprises of different components, and the 'smoothed' state is obtained by convolving a smoothing kernel to each of them separately. The values used in the kernel, and the type of convolution applied depends on the nature of the spatial division the feature represents and is summarised in the Table \ref{conv-table}.

\begin{table}
	\caption{Table showing the details of the convolution used for smoothing the state feature vector.}
	\label{conv-table}
	\begin{center}
		\renewcommand{\arraystretch}{1.3}
		\begin{tabular}{|c|c|c|}
			\hline
			Feature component & Convolution Kernel & Convolution type\\
			\hline
			Relative goal orientation & $ [0.1, \;0.8, \; 0.1 ]$ & Wrap  \\

			Change in orientation & $[ 0.1, \; 0.8; \;0.1 ]$ & Same \\

			Deviation from goal & $[0.9,\; 0.1]$, $[0.1,\; 0.9]$,
									      $[0.05, \; 0.9, \; 0.05]$, $[0.1,\; 0.9]$ & Same \\
			Local spatial bins & $[ 0.1, \; 0.8,\;0.1 ]$ & Wrap \\
			Speed info & $[ 0.1, \; 0.8,\;0.1 ]$  & Same \\
			\hline
		\end{tabular}
	\end{center}
\end{table}