\documentclass[a4paper,12pt]{article}

% Font
\usepackage[T1]{fontenc}
\usepackage{gentium}

% Math packages
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{bm}

% Define symbol shortcuts
\newcommand{\cc}{\mathcal{C}}
\newcommand{\dd}{\mathcal{D}}
\newcommand{\hh}{\mathcal{H}}
\newcommand{\xx}{{\bm x}}
\newcommand{\yy}{{\bm y}}

% Math environment
\newtheorem*{thm}{Theorem}

% Better list management:
% - vertical spacing in lists
% - items in lists start with dash not bullet point.
\usepackage{enumitem}
\setlist{label=\textemdash,
  itemsep=0pt, topsep=3pt, partopsep=0pt}

% Include graphics
\usepackage{graphicx}
\usepackage{subcaption}

% Page format
\usepackage[top=2cm,left=2cm,right=2cm,bottom=2cm]{geometry}

\begin{document}
%%% HEADER
\raisebox{0.6in}[0in]{\makebox[\textwidth][r]{\it Unproofed version }}
\vspace{-0.7in}

\begin{center}
\bf\large MA2823: Foundations of Machine Learning \\
Chapter 8: Tree-based Methods
\end{center}

\noindent
Lecturer: Chlo\'e-Agathe Azencott
\hfill
Scribe: LIN Laurent \\
\null \hfill PHAM Olivier \\
\null \hfill SCHNITZLER Pierre-Louis

\noindent
\rule{\textwidth}{1pt}

\medskip

In this chapter we will :
\begin{itemize}
\item see how to build decision trees (and more precisely how to grow a tree and when to stop growing a tree);
\item explain why they are examples of non-metric learning and hierarchical learning;
\item combine decision trees (or other weak learners) to make more powerful classifiers.
\end{itemize}

\section{Decision trees}

\subsection{Hierarchical learning}

As single classifiers assign a class to an object x using a single operation and use a single set of features for all classes, they can face difficulties in case of non-metric learning, so when classes have multi-modal distributions or when features are nominal. We are talking about nominal data when the attributes are discrete without any natural notion of similarity/ordering (such as \{color, shape, texture, size\} for a fruit classification). In order to avoid those problems, decision trees are based on hierarchical classifiers, which make multiple successive tests. \\

\begin{figure}[h]
  \centering
  \includegraphics[width=0.5\textwidth]{figures/Structure.jpg}
  \caption{Example of a decision tree}
  \label{fig:ensemble learning}
\end{figure}

\subsection{Multiclass classification}

There are strategies for reducing a problem of multiclass classification to multiple binary classification problems. It can be categorized into One-versus-one (Build K (K-1) / 2 classifiers and make them vote) and One-versus-all (Build K classifiers and make them vote). To solve those problems, we have to use an algorithm that naturally handles multiple classes. It is the case of tree algorithms and Neural networks (that we will study in Chap. 10).\\

\subsection{Vocabulary Reminder}
Here is a short reminder of tree vocabulary:\\
\begin{figure}[h]
  \centering
  \includegraphics[width=0.5\textwidth]{figures/s1.jpg}
  \caption{Tree vocabulary}
  \label{fig:ensemble learning}
\end{figure}
\subsection{Partition of the feature space}

Classification and regression trees (CART) is the generic name for a recursive procedure to split a training set and organize it into a tree. The models are obtained by recursively partitioning
the training set into smaller and smaller subsets and fitting a simple prediction model within each partition. As a result, the partitioning can be represented graphically as a decision tree. During the CART procedure we will have to do several choices:
\begin{itemize}
\item Binary or multi-way splits?
\item Which feature(s) to use at each node? i.e. how to split?
\item When to stop growing a tree?

\end{itemize}

\section{Binary-tree versus non binary-tree}

A binary tree is a tree data structure in which each node has at most two children, which are referred to as the left child and the right child. A tree with arbitrary branching factor can always be equivalently represented by a binary tree.  \\
The following binary tree is equivalent to the non-binary tree of Figure 1.

\begin{figure}[h]
  \centering
  \includegraphics[width=0.5\textwidth]{figures/s2.jpg}
  \caption{Binary tree equivalent to Figure. 1}
  \label{fig:ensemble learning}
\end{figure}

\newpage
\section{How to grow a tree}

\subsection{Introduction}

We are only considering monothetic trees, that is to say trees with only one feature per node. In this conditions, the decision boundary is orthogonal to the axes. The splitting variable (j) and the splitting point (s) define 2 regions: \\
\begin{figure}[h]
  \centering
  \includegraphics[width=0.5\textwidth]{figures/s3.jpg}
  \label{fig:ensemble learning}
\end{figure}

For a regression tree, we choose j and s to minimize SE (standard error):\\
\begin{figure}[h]
  \centering
  \includegraphics[width=0.5\textwidth]{figures/s4.jpg}
  \label{fig:ensemble learning}
\end{figure}

For a classification tree, we choose j and s to minimize impurity : \\
\begin{figure}[h]
  \centering
  \includegraphics[width=0.5\textwidth]{figures/s5.jpg}
  \label{fig:ensemble learning}
\end{figure}

\subsection{Impurity}

Decision trees are grown by posing a series of questions about the feature of the items. In order to do so, we add question nodes incrementally, using labeled training examples to guide the choice of questions. Ideally, a single, simple question would perfectly split the training examples into their classes. If no question exists that gives such a perfect separation, we choose a question that separates the examples as cleanly as possible. \\

Several measures have been designed to evaluate the degree of inhomogeneity, or impurity, in a set of items. For decision trees, the most common measures are entropy, the Gini index and the classification error. Given a measure of impurity, we choose a question that minimizes the weighted average of the impurity of the resulting children nodes.

\subsection{Classification error}

The classification error corresponds to the minimum probability that a training point will be misclassied at node (s,j) and is defined by the following formula :
\[
\textrm{Imp}(R_m)= 1 - \textrm{max}_{  k}(\hat{p}_{mk})
\]
where \( \hat{p}_{mk} \) is the proportion of training instances from class k in \(R_m\). \\

The value of the classification error is always between 0 and 1. If all examples from one class belong to \(R_m\) then \( \textrm{Imp}(R_m)= 1 - \textrm{max}{(1)} = 0 \). If we have 2 balanced classes and instances are randomly split at (s, j), then \( \textrm{Imp}(R_m)= 1 - \textrm{max}{(
0.5, 0.5)} = 0.5 \)

\subsection{Entropy}

The concept of entropy comes from the information theory. Entropy is a measurement of information (or rather lack of information). We calculate the information gain by making a split. This measures how you reduce the uncertainty about the label. It is defined by the following formula:
\[
\textrm{Imp}(R_m)= -  \sum_{k}\hat{p}_{mk} \log_{2}(\hat{p}_{mk})
\]
where \( \hat{p}_{mk} \) is the proportion of training instances from class k in \(R_m\). \\

If all examples from one class belong to \(R_m\) then \( \textrm{Imp}(R_m) = 0 \). If we have 2 balanced classes and instances are randomly split at (s, j), then \( \textrm{Imp}(R_m)= - (\frac{1}{2}\log_{2}(\frac{1}{2})+ \frac{1}{2}\log_{2}(\frac{1}{2}))= 1 \)

\subsection{Gini Impurity}

Used by the CART (classification and regression tree) algorithm, Gini impurity is a measure of how often a randomly chosen element from the set would be incorrectly labeled if it was randomly labeled according to the distribution of labels in the subset.

It is defined by the following formula:
\[
\textrm{Imp}(R_m)=  \sum_{k=1}^{K}\hat{p}_{mk}(1-\hat{p}_{mk})
\]
where \( \hat{p}_{mk} \) is the proportion of training instances from class k in \(R_m\). \\

If all examples from one class belong to \(R_m\) then \( \textrm{Imp}(R_m) = 0 \). If we have 2 balanced classes and instances are randomly split at (s, j), then \( \textrm{Imp}(R_m)= \frac{1}{2}\frac{1}{2}+ \frac{1}{2}\frac{1}{2}= \frac{1}{2} \) . \\

If we have two regions \( R_l \) and \( R_r \) then the impurity is defined by:
\[
\textrm{GI}(j,s)= \frac{\left | R_l(j,s) \right |}{n_{tot}}  \textrm{Imp}(R_l(j,s)) + \frac{\left | R_r(j,s) \right |}{n_{tot}}  \textrm{Imp}(R_r(j,s))
\]
Assuming that the split respects the overall distribution \( \forall k ,p_{mk}=\frac{\left | C_k \right |}{N} \) , then all regions are identically distributed and have Gini impurity:
\[
\textrm{GI}(R)= \sum_{k=1}^{K} \frac{\left | C_k \right |}{N}  (1-\frac{\left | C_k \right |}{N} ) = 1 - \sum_{k=1}^{K} (\frac{\left | C_k \right |}{N})^{2} \]

For a K-way split  \(\textrm{GI}(j,s)= \sum_{l=1}^{K} \frac{1}{K} \textrm{GI}(R) = \textrm{GI}(R)\) and in particular if \(\left | C_k \right | = \frac{N}{K} \) then \(\textrm{GI}= 1 - \frac{1}{K} \). If the split is perfect then all regions have a proportion of 1 of one class and of 0 of
the other, and hence have Gini impurity \(\textrm{GI}(R)=0 \).

\subsection{When to stop growing a tree?}

If a tree is too large it might overfit while a small tree might underfit. Therefore we have to establish a strategy in order to decide when to stop growing a tree.
There are two main strategies:
\begin{itemize}
\item we can either grow the tree until a minimum node size (number of training points in the region) is reached
\item or we can prune the tree using cost-complexity pruning
\end{itemize}

\subsection*{Cost-complexity pruning}

In a similar way to what we did with regularisation we define the cost complexity with the following formula which includes a complexity penalty :
\[
C_\alpha (Tree)=  \sum_{m=1}^{\# Tree} N_m Q_m(Tree) + \alpha (\# Tree)
\]
where:
\begin{itemize}
\item \#Tree is the number of regions in our pruned tree
\item \(N_m\) is the number of training instances in \(R_m\)
\item \(Q_m\) is the error on \(R_m\)
\item \(\alpha \) is a coefficient which defines the balance between model complexity and goodness of fit
\end{itemize}

The goal is to find a subtree which will minimise the cost-complexity.
The more leaf nodes that the tree contains the higher complexity of the tree because we have more flexibility in partitioning the space into smaller pieces, and therefore more possibilities for fitting the training data. There is also the issue of how much importance to put on the size of the tree. The complexity parameter \(\alpha \) adjusts that.

\section{Advantages and drawbacks of trees}

\begin{tabular}{|c|c|}
  \hline
  Advantages & Drawbacks  \\
  \hline
  Easy to explain & Bad predictive accuracy in general  \\
  Mirror human-decision making & \\
  Displayed graphically and easily interpreted & \\
  Can easily handle quatitative variables & \\
  Naturally handle multiclass problems & \\
  \hline

\end{tabular}\\

In order to improve the predictive accuracy we will use forests.

\newpage
\section{Forest}

\paragraph{Ensemble learning.}The idea is that aggregating many weak learners can substantially increase their performance, because trees have weak predictive power in general. This method is called ensemble learning and uses the principle of wisdom of crowds, that by combining multiple individual classifiers, we can average out their uncorrelated errors. This is what we can observe with the example below, by combining many 'staircase' boundaries, we finally obtain a diagonal separation. \\

\begin{figure}[h]
  \centering
  \includegraphics[width=0.5\textwidth]{figures/Staircase.png}
  \caption{Principle of ensemble learning}
  \label{fig:ensemble learning}
\end{figure}

\subsection{Building ensembles}

To build ensembles, we have to:

\begin{itemize}
\item \textbf{subsample the training data} : by either bagging the data, picking multiple times the same point, or boosting which is resampling based on performance.

\item Use \textbf{different features} (with multiple input representations, we will also learn how to select features in the chapter 11).

\item Use different \textbf{parameters} of the learning algorithm.\\

\end{itemize}

When we combine weak learners :

\begin{itemize}
\item if it is a non-trainable combination: it should be done by \textbf{voting} (classification) or \textbf{averaging} (regressing).

\item if it is a trainable combination: we use \textbf{weighted averaging} based on performance on a validation set or \textbf{meta-learner} with which the outputs of the individual learners are features for another learning algorithm.

\end{itemize}

\subsection{Bagging trees}

The principle is to bootstrap the training data (take repeated samples) and build one predictor from each of these samples. The final prediction is the \textbf{average} of the samples when considering a \textbf{regression} problem and a \textbf{majority vote} for a \textbf{classification} problem.

\subsection{Random forest}

The idea in random forests is to improve the variance reduction of bagging by reducing the correlation between the trees, without increasing the variance too much. This is achieved in the tree-growing process through random selection of the input variables: before each split, select q (out of p) input variables at random as candidates for splitting. We typically choose q = $\sqrt[]{p}$ .
This method has very good predictive power in practice!

\section{Summary}

Decision trees are easy to interpret.\\
Decisions trees elegantly deal with :
\begin{itemize}
\item Quantitative variables
\item Multiple classes
\item Multimodal distributions
\end{itemize}
Decision trees have limited predictive power, but this can be addressed thanks to ensemble methods:
\begin{itemize}
\item Bagging
\item Random forests
 \end{itemize}
\end{document}


% -------------------------------------------------------- %
% SkipQueue
% by: Isai Barajas Cicourel

% -------------------------------------------------------- %
% Document Start

\section{\textbf{Skiplist-Based Unbounded Priority Queue}}

% -------------------------------------------------------- %
% Particular Caes

\subsection{Particular Case}
\par
In this experiment we implement a skip-list priority queue which in contrast to the \textit{FineGrainedHeap} priority queue algorithm, that the underlying heap structure requires complex, coordinated rebalancing. We requires no rebalancing.
\par

% -------------------------------------------------------- %
% Solution Information

\subsection{Solution}
\par
According to the theory the \textit{PrioritySkipList} class sorts items by priority instead of by hash value, ensuring that high-priority items (the ones we want to remove first) appear at the front of the list. Removing the item with highest priority is done lazily. A node is logically removed by marking it as removed, and is later physically removed by unlinking it from the list.
\par
\begin{lstlisting}[frame=single,breaklines=true]
  public T removeMin() {
      Node<T> node = skiplist.findAndMarkMin();

      if (node != null) {
        skiplist.remove(node);
        return node.item;
      } else{
        return null;
      }
  }
\end{lstlisting}
\par
The \textit{add()} and \textit{remove()} calls take skiplist nodes instead of items as arguments and results. These methods are straightforward adaptations of the corresponding \textit{LockFreeSkipList} methods. This class nodes differ from \textit{LockFreeSkipList} nodes in two fields, an integer score field, and an \textit{AtomicBoolean} marked field used for logical deletion from the priority queue (not from the skiplist).
\begin{lstlisting}[frame=single,breaklines=true]
  boolean add(Node node) {
    int bottomLevel = 0;
    Node<T>[] preds = (Node<T>[]) new Node[MAX_LEVEL + 1];
    Node<T>[] succs = (Node<T>[]) new Node[MAX_LEVEL + 1];
    while (true) {
      boolean found = find(node, preds, succs);
      if (found) { // if found it's not marked
        return false;
      } else {
        for (int level = bottomLevel; level <= node.topLevel; level++) {
          Node<T> succ = succs[level];
          node.next[level].set(succ, false);
        }
        // try to splice in new node in bottomLevel going up
        Node<T> pred = preds[bottomLevel];
        Node<T> succ = succs[bottomLevel];
        node.next[bottomLevel].set(succ, false);
        if (!pred.next[bottomLevel].compareAndSet(succ, node, false, false)) {// lin point
          continue; // retry from start
        }
        // splice in remaining levels going up
        for (int level = bottomLevel+1; level <= node.topLevel; level++) {
          while (true) {
            pred = preds[level];
            succ = succs[level];
            if (pred.next[level].compareAndSet(succ, node, false, false))
              break;
            find(node, preds, succs); // find new preds and succs
          }
        }
        return true;
      }
    }
  }

  boolean remove(Node<T> node) {
    int bottomLevel = 0;
    Node<T>[] preds = (Node<T>[]) new Node[MAX_LEVEL + 1];
    Node<T>[] succs = (Node<T>[]) new Node[MAX_LEVEL + 1];
    Node<T> succ;
    while (true) {
      boolean found = find(node, preds, succs);
      if (!found) {
        return false;
      } else {
        // proceed to mark all levels
        // some levels could stil be unthreaded by concurrent add() while being marked
        // other find()s could be modifying node's pointers concurrently
        for (int level = node.topLevel; level >= bottomLevel+1; level--) {
          boolean[] marked = {false};
          succ = node.next[level].get(marked);
          while (!marked[0]) { // until I succeed in marking
            node.next[level].attemptMark(succ, true);
            succ = node.next[level].get(marked);
          }
        }
        // proceed to remove from bottom level
        boolean[] marked = {false};
        succ = node.next[bottomLevel].get(marked);
        while (true) { // until someone succeeded in marking
          boolean iMarkedIt = node.next[bottomLevel].compareAndSet(succ, succ, false, true);
          succ = succs[bottomLevel].next[bottomLevel].get(marked);
          if (iMarkedIt) {
            // run find to remove links of the logically removed node
            find(node, preds, succs);
            return true;
          } else if (marked[0]) return false; // someone else removed node
          // else only succ changed so repeat
        }
      }
    }
  }
\end{lstlisting}

% -------------------------------------------------------- %
% Experiment

\subsection{Experiment Description}
\par
The test creates $8$ threads, eight that need to be coordinate in order to \textit{add()} and \textit{removeMin()} an array of values. All threads have to cooperate to add and remove elements from the queue and a validation of the priorities is done to validate non-ascending priorities.
If that is not the case, a fail will be raised.
\par

% -------------------------------------------------------- %
% Results

\subsection{Observations and Interpretations}

\par
The test needs some changes in order to execute and avoid starvation.
\begin{lstlisting}[frame=single,breaklines=true]
------------- Standard Output ---------------
add
OK.
testParallelAdd
OK.
testParallelBoth

\end{lstlisting}

% -------------------------------------------------------- %
% Results

\subsection{Proposed changes to fix the problem}

\par
The \textit{findAndMarkMin()} method must choose the coresponding element that will be removed in order to mark it and later on the \textit{remove()} method we verify the marked node and remove it, but the atomicity of this operations are not guarantee, thus when a node is marked it can be marked by other thread causing the remove process to hand, since the internal logic tries to remove the node and has no escape sequence when the mark is affected.
\par
\begin{lstlisting}[frame=single,breaklines=true]
  public T removeMin() {
    synchronized(this) {
      Node<T> node = skiplist.findAndMarkMin();

      if (node != null) {
        skiplist.remove(node);
        return node.item;
      } else{
        return null;
      }
    }
  }
\end{lstlisting}
\par
By doing so, the execution finishes as expected and no errors are found.
\begin{lstlisting}[frame=single,breaklines=true]
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.358 sec

------------- Standard Output ---------------
add
OK.
testParallelAdd
OK.
testParallelBoth
OK.
------------- ---------------- ---------------
test-single:
BUILD SUCCESSFUL (total time: 0 seconds)
\end{lstlisting}


\section{Adaptive Segmentation of MRI Data}

\subsection{Introduction}

\begin{frame}
  \frametitle{Introduction}

  \structure{Magnetic Resonance Imaging (MRI)} is an important acquisition technique. \\[.5cm]

  It features:

  \begin{itemize}
    \item high spatial resolution
    \item good soft tissue contrast
    \item does not incorporate ionizing radiation (as computed tomography) \\[.75cm]
  \end{itemize}
  \pause

  Several applications require the \structure{segmentation} (classification) of the acquired images into \structure{tissue types}.
\end{frame}

\begin{frame}
  \frametitle{Introduction \cont}

  Difficulties arise from:

  \begin{itemize}
    \item missing intensity normalization (like Hounsfield units in CT) \pause
    \item \structure{intensity inhomogeneities}, also known as bias field \\
      (RF coils, acquisition sequences)
  \end{itemize}
  \pspread

  \begin{figure}
    \centering
    \subfloat[with bias field]{
      \includegraphics[width=.23\linewidth]{\pngdir/MRI_bias.\png}
    }
    \qquad
    \subfloat[corrected]{
      \includegraphics[width=.23\linewidth]{\pngdir/MRI_nobias.\png}
    }
    \caption{MRI intensity inhomogeneity (Courtesy of F. J{\"a}ger)}
  \end{figure}
\end{frame}

\begin{frame}
  \frametitle{Introduction \cont}

  Effect of the bias field on ML segmentation:

  \begin{figure}
    \centering
    \subfloat[]{
      \label{syn-a}
      \includegraphics[width=.22\linewidth]{\pngdir/SyntheticBiased.\png}
    }
    \onslide<2->
    \subfloat[]{
      \label{syn-b}
      \includegraphics[width=.22\linewidth]{\pngdir/SyntheticBiasField.\png}
    }
    \onslide<3->
    \subfloat[]{
      \label{syn-c}
      \includegraphics[width=.22\linewidth]{\pngdir/SyntheticBadSegmentation.\png}
    }
    \onslide<4>
    \subfloat[]{
      \label{syn-d}
      \includegraphics[width=.22\linewidth]{\pngdir/SyntheticGoodSegmentation.\png}
    }
    \onslide<1->
    \caption{Synthetic image \subref{syn-a} overlaid with artificial bias field \subref{syn-b}, result of ML segmentation \subref{syn-c}, result after modeling bias field within segmentation \subref{syn-d} (Courtesy of W. Wells).}
  \end{figure}
\end{frame}

\begin{frame}
  \frametitle{Introduction \cont}

  W.\,M.\,Wells et al. presented an approach to improve MR brain segmentation (1996):

  \begin{itemize}
    \item statistical approach to intensity-based segmentation of MRI
    \item statistical modeling of bias field (smoothness constraint)
    \item usage of EM algorithm for simultaneous computation of tissue classification and intensity inhomogeneity correction
  \end{itemize}
  \pspread

  \structure{Typical EM problem:}

  \begin{itemize}
    \item The missing data is the tissue class assignment for each pixel. \pause
    \item If the tissue was classified, the bias field could easily be computed. \pause
    \item If the bias field was known, the tissue classification would be much easier.
  \end{itemize}
\end{frame}

\subsection{Bias Field Model}

\begin{frame}
  \frametitle{Bias Field Model}

  \begin{itemize}
    \item Let $\tilde{X}_i$ be the (unknown) intensity of the $i$-th voxel of the MRI data and \\
      $B_i$ the corresponding bias field. \pause
    \item The bias field is assumed to be multiplicative:
      \begin{displaymath}
        X_i = \tilde{X}_i \cdot B_i
      \end{displaymath}
      \pause
    \item Using a log-transform on the intensities yields an additive bias field model:
      \begin{displaymath}
        Y_i = \log X_i = \log \tilde{X}_i + \beta_i ~,~\mbox{with}~ \beta_i = \log B_i
      \end{displaymath}
      \pause
    \item The bias field is then:
      \begin{displaymath}
        \vec \beta = \left(\beta_0, \beta_1, \ldots, \beta_{n-1}\right)^T
      \end{displaymath}
      with $n$ being the number of voxels.
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Bias Field Model \cont}

  \begin{itemize}
    \item The bias field is assumed to change smoothly over the entire image domain.
    \item It is modeled by an $n$-dimensional zero mean Gaussian prior:
      \begin{displaymath}
        p(\vec \beta) = {\mathcal N}(\vec{\beta} ; \vec{0}, \mat{\Psi}_{\vec{\beta}})
      \end{displaymath}
  \end{itemize}
  \pspread

  \structure{Notes:}

  \begin{itemize}
    \item $\mat{\Psi}_{\vec{\beta}}$ is a $n \times n$-dimensional covariance matrix \pause
    \item $\mat{\Psi}_{\vec{\beta}}$ is too large to compute directly in practice \pause
    \item Instead of the full covariance matrix, a banded estimate is chosen in practice.
  \end{itemize}
\end{frame}

\subsection{Bayesian Approach}

\begin{frame}
  \frametitle{Bayesian Approach}

  \structure{Variables:}

  \begin{eqnarray*}
    Y_i         &\phantom{=}& \mbox{log-transformed observed intensity at $i$-th voxel} \\
    \Gamma_i    &\phantom{=}& \mbox{tissue class of the $i$-th voxel} \\
    \mu_\Gamma  &\phantom{=}& \mbox{mean intensity for tissue class $\Gamma$} \\
    \psi_\Gamma &\phantom{=}& \mbox{variance of tissue class $\Gamma$}
  \end{eqnarray*}

  The intensities are assumed to be scalar values, therefore: $\mu_\Gamma, \psi_\Gamma \in \real$
\end{frame}

\begin{frame}
  \frametitle{Bayesian Approach \cont}

  Assuming \structure{statistical independence} of the intensities, the probability density for the entire image $\vec Y = \left(Y_0, Y_1, \ldots, Y_{n-1} \right)^T$ is: \pause

  \begin{displaymath}
    p(\vec Y | \vec \beta) = \prod_i p(Y_i | \beta_i)
  \end{displaymath}
  \pause

  The probability of the observations is modeled as a \structure{Gaussian mixture} over the tissue classes:
  \begin{displaymath}
    p(Y_i | \beta_i) = \pause \sum_{\Gamma} p(Y_i, \Gamma | \beta_i) = \pause
    \sum_{\Gamma} p(\Gamma) \, p(Y_i | \Gamma, \beta_i)
  \end{displaymath}
  \pause

  with
  \begin{displaymath}
    p(Y_i | \Gamma, \beta_i) = {\mathcal N}(Y_i ; \mu_\Gamma + \beta_i, \psi_{\Gamma})
  \end{displaymath}
\end{frame}

\begin{frame}
  \frametitle{Bayesian Approach \cont}

  \structure{Observations so far:}

  \begin{itemize}
    \item Each tissue class is modeled with a normal distribution. \\[.25cm] \pause
    \item The modeling of the observed intensity distribution yields a Gaussian mixture model. \\[.25cm] \pause
    \item $p(\Gamma)$ is a stationary prior probability for the tissue class. \\[.25cm] \pause
    \item The estimators for the GMM are non-linear!
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Bayesian Approach \cont}

  Using \structure{Bayes rule} to derive an objective function for the bias field:
  \begin{displaymath}
    p(\vec \beta | \vec Y) = \frac{p(\vec Y | \vec \beta) \, p(\vec \beta)} {p(\vec Y)}
  \end{displaymath}
  \pspread

  Applying the \structure{MAP principle} yields an estimator for the bias field:

  \begin{eqnarray*}
    \hat{\vec \beta} &=& \argmax_{\vec \beta} p(\vec \beta | \vec Y) \\ \pause
                     &=& \argmax_{\vec \beta} \log p(\vec \beta | \vec Y) \\ \pause
                     &=& \argmax_{\vec \beta} \big( \log p(\vec Y | \vec \beta) + \log p(\vec \beta) \big)
  \end{eqnarray*}
\end{frame}

\begin{frame}
  \frametitle{Gradient Computation}

  At the optimum, the gradient w.\,r.\,t.\ $\vec \beta$ has to be zero:

  \begin{eqnarray*}
    \frac{\partial}{\partial \beta_i} \log p(\vec \beta | \vec Y)
    &=& \frac{\partial}{\partial \beta_i} \left( \log p(\vec Y | \vec \beta) + \log p(\vec \beta) \right) \\ \pause
    &=& \frac{\partial}{\partial \beta_i} \left( \sum_j \log p(Y_j | \beta_j) + \log p(\vec \beta) \right) \\ \pause
    &=& \frac{\frac{\partial}{\partial \beta_i} p(Y_i | \beta_i)}{p(Y_i | \beta_i)} + \frac{\frac{\partial}{\partial \beta_i} p(\vec \beta)}{p(\vec \beta)} \\
    &\stackrel{!}{=}& 0.
  \end{eqnarray*}
\end{frame}

\begin{frame}
  \frametitle{Gradient Computation \cont}

  {\footnotesize
  \begin{eqnarray*}
    \frac{\frac{\partial}{\partial \beta_i} p(Y_i | \beta_i)}{p(Y_i | \beta_i)}
    &=& \underbrace{
          \frac{\sum_{\Gamma} p(\Gamma) \, \frac{\partial}{\partial \beta_i} {\mathcal N}(Y_i ; \mu_\Gamma + \beta_i, \psi_{\Gamma}) }
               {\sum_{\Gamma} p(\Gamma) \, {\mathcal N}(Y_i ; \mu_\Gamma + \beta_i, \psi_\Gamma)}
        }_{\mbox{\small substitute GMM}} \\[.25cm] \pause
    &=& \frac{\sum_{\Gamma} p(\Gamma) \, {\mathcal N}(Y_i ; \mu_\Gamma + \beta_i, \psi_\Gamma) \, \psi_\Gamma^{-1} \, (Y_i - \mu_\Gamma - \beta_i)}
             {\sum_\Gamma p(\Gamma) \, {\mathcal N}(Y_i ; \mu_\Gamma + \beta_i, \psi_\Gamma) } \\[.25cm] \pause
    &=& \sum_\Gamma W_{i\Gamma} \left( \psi_\Gamma^{-1} (Y_i - \mu_\Gamma - \beta_i) \right)
  \end{eqnarray*}
  }

  Weight for the $i$-th voxel and tissue class $\Gamma$:
  {\footnotesize
  \begin{eqnarray*}
    W_{i\Gamma} :=
      \frac{p(\Gamma) \cdot \mathcal{N}(Y_i ; \mu_\Gamma + \beta_i, \psi_\Gamma) }
           {\sum_{\Gamma} p(\Gamma) \cdot \mathcal{N}(Y_i ; \mu_\Gamma + \beta_i, \psi_\Gamma)}
  \end{eqnarray*}
  }
\end{frame}

\begin{frame}
  \frametitle{Gradient Computation \cont}

  Rewriting the equation:

  {\footnotesize
  \begin{eqnarray*}
    \frac{\frac{\partial}{\partial \beta_i} p(Y_i | \beta_i)}
         {p(Y_i | \beta_i)}
    &=& \sum_\Gamma W_{i\Gamma} \left( \psi_\Gamma^{-1} (Y_i - \mu_\Gamma - \beta_i) \right) \\ \pause
    &=& \sum_\Gamma W_{i\Gamma} \, \psi_j^{-1} \, (Y_i - \mu_\Gamma) - \sum_\Gamma W_{i\Gamma} \, \psi_\Gamma^{-1} \, \beta_i \\ \pause
    &=& \overline{R_i} - \overline{\psi^{-1}}_i ~\beta_i
  \end{eqnarray*}
  }

  Mean residual:
  {\footnotesize
  \begin{displaymath}
    \overline{R_i} := \sum_\Gamma W_{i\Gamma} \, \psi_\Gamma^{-1} \, (Y_i - \mu_\Gamma)
  \end{displaymath}
  }

  Mean inverse variance:
  {\footnotesize
  \begin{displaymath}
    \overline{\psi^{-1}}_i := \sum_\Gamma W_{i\Gamma} \, \psi_\Gamma^{-1}
  \end{displaymath}
  }
\end{frame}

\begin{frame}
  \frametitle{Gradient Computation \cont}

  Finishing gradient computation:

  {\small
  \begin{eqnarray*}
    \nabla_{\vec \beta} \log p(\vec \beta | \vec Y)
      &=& \overline{\vec R} - \overline{\mat{\Psi}^{-1}} \vec \beta + \frac{\nabla_{\vec \beta} p(\vec \beta)}{p(\vec \beta)} \\
      &=& \overline{\vec R} - \overline{\mat{\Psi}^{-1}} \vec \beta - \mat{\Psi}_{\vec{\beta}}^{-1} \vec \beta \\
      &\stackrel{!}{=}& \vec{0}
  \end{eqnarray*}
  }
  \pause

  It follows that:
  {\small
  \begin{displaymath}
    \hat{\vec \beta} = \mat{H} \overline{\vec R} \quad \mbox{with}~
    \mat{H} \equiv \left[\overline{\mat{\Psi}^{-1}} + \mat{\Psi}_{\vec{\beta}}^{-1} \right]^{-1}
  \end{displaymath}
  }
  \pause

  $\mat{H}$ is a linear operator that is applied to the mean residual field. \\
  In fact, $\hat{\vec{\beta}}$ can be obtained by low pass filtering of $\overline{\vec R}$ and $\overline{\mat{\Psi}^{-1}}$.
\end{frame}

\subsection{EM-Algorithm}

\begin{frame}
  \frametitle{EM-Algorithm}

  \structure{EM-Algorithm for the adaptive segmentation problem:}

  \begin{eqnarray}
    W_{i\Gamma}
    &\leftarrow& \frac{p(\Gamma) \cdot \mathcal{N}(Y_i | \mu_\Gamma + \beta_i, \psi_\Gamma)}
                      {\sum_\Gamma p(\Gamma) \cdot \mathcal{N} (Y_i | \mu_\Gamma + \beta_i, \psi_\Gamma)} \label{eq:E} \\[.3cm]
    \hat{\vec \beta}
    &\leftarrow& \mat{H} \overline{\vec R} \label{eq:M}
  \end{eqnarray}
  \pause

  \begin{itemize}
    \item \structure{E-step:} equation \eqref{eq:E} yields the posterior tissue class probabilities for a known bias field \pause
    \item \structure{M-step:} equation \eqref{eq:M} yields the new bias field for the current estimates for the tissue probabilities \pause
    \item \structure{Result:} iterating 5-10 times between the E- and the M-step is usually sufficient
  \end{itemize}
\end{frame}

\subsection{Results}

\begin{frame}
  \frametitle{Results}

  \begin{figure}
    \centering
    \subfloat[]{
      \label{brain-a}
      \includegraphics[width=.22\linewidth]{\pngdir/BrainInput.\png}
    }
    \subfloat[]{
      \label{brain-b}
      \includegraphics[width=.22\linewidth]{\pngdir/BrainBadSegment.\png}
    }
    \subfloat[]{
      \label{brain-c}
      \includegraphics[width=.22\linewidth]{\pngdir/BrainBiasField.\png}
    }
    \subfloat[]{
      \label{brain-d}
      \includegraphics[width=.22\linewidth]{\pngdir/BrainAdaptiveSegment.\png}
    }
    \caption{\footnotesize Results of conventional segmentation \subref{brain-b} compared to adaptive segmentation \subref{brain-d} with computed bias field \subref{brain-c} on brain image \subref{brain-a}
(Courtesy of W. Wells).}
  \end{figure}
\end{frame}

\begin{frame}
  \frametitle{Results \cont}

  \begin{figure}
    \centering
    \subfloat[]{
      \label{surf-a}
      \includegraphics[width=.3\linewidth]{\pngdir/BrainGrayMatter.\png}
    }
    \subfloat[]{
      \label{surf-b}
      \includegraphics[width=.3\linewidth]{\pngdir/BrainBadWhiteMatter.\png}
    }
    \subfloat[]{
      \label{surf-c}
      \includegraphics[width=.3\linewidth]{\pngdir/BrainAdaptiveWhiteMatter.\png}
    }
    \caption{\footnotesize Gray matter surface \subref{surf-a} for the previous image example, white matter surface of the conventional algorithm \subref{surf-b} and for the adaptive segmentation  \subref{surf-c} (Courtesy of W. Wells).}
  \end{figure}
\end{frame}

\begin{frame}
  \frametitle{Results \cont}

  \begin{figure}
    \centering
    \subfloat[]{
      \label{bias-a}
      \includegraphics[width=.3\linewidth]{\pngdir/MRIBrainBias.\png}
    }
    \subfloat[]{
      \label{bias-b}
      \includegraphics[width=.3\linewidth]{\pngdir/MRIBrainBiasField.\png}
    }
    \subfloat[]{
      \label{bias-c}
      \includegraphics[width=.3\linewidth]{\pngdir/MRIBiasCorrected.\png}
    }
    \caption{\footnotesize MRI image with bias field \subref{bias-a}, computed bias field \subref{bias-b} and image corrected at the brain region \subref{bias-c} (Courtesy of W. Wells).}
  \end{figure}
\end{frame}

\subsection{Model Extensions}

\begin{frame}
  \frametitle{Model Extensions}

  \structure{Drawbacks} of initial adaptive segmentation algorithm:

  \begin{itemize}
    \item brains should be extracted from entire data \pause
    \item algorithm does not incorporate neighborhood of pixels \pause
    \item purely intensity-based model
  \end{itemize}
  \pspread

  \structure{Extensions} of the algorithm:

  \begin{itemize}
    \item incorporation of atlases for spatial probability maps of tissue classes \pause
    \item definition of vector space for probabilistic atlases to get shape models \pause
    \item voxel neighborhood relations modeled by Markov random fields \pause
    \item incorporation into Bayesian model that is solved by EM approach
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Model Extensions \cont}

  Result using an extended model:

  \begin{figure}
    \includegraphics[width=.8\linewidth]{\pngdir/MRIBrainAtlasSegmentation.\png}
    \caption{\footnotesize MRI segmentation of the thalamus and caudate using an atlas-based EM segmentation algorithm (Courtesy of K. Pohl).}
  \end{figure}
\end{frame}

\subsection{Lessons Learned}

\begin{frame}
  \frametitle{Lessons Learned}

  \begin{itemize}
    \item Bayesian approach for MRI data segmentation \\[.25cm]
    \item incorporation of bias field estimation \\[.25cm]
    \item nonlinear problem is solved iteratively using EM algorithm \\[.25cm]
    \item improvement of results by incorporating atlases
  \end{itemize}
\end{frame}

\input{nextTime.tex}

\subsection{Further Readings}

\begin{frame}
  \frametitle{Further Readings}

  \begin{itemize}
    \item Original paper on adaptive MRI segmentation: \\[.15cm]
      W.\,M.\ Wells, R.\,Kikinis, W.\,E.\,L.\ Grimson, F.\ Jolesz: \\
      \structure{Adaptive segmentation of MRI data}, \\
      IEEE Transactions on Medical Imaging, 15:429-442, 1996. \\[.3cm]
    \item F.\ J{\"a}ger, J.\ Hornegger: \\
      \structure{Nonrigid registration of joint histograms for intensity standardization \\ in magnetic resonance imaging}, \\
      IEEE Transactions on Medical Imaging, 28(1):137-150, 2009.
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Further Readings \cont}

  Extensions of the model with shape models, atlas registration and MRFs: \\[.15cm]

  \begin{itemize}
    \item K.\,M.\ Pohl, J.\ Fisher, J.\,J.\ Levitt, M.\,E.\ Shenton, R.\ Kikinis, W.\,E.\,L.\ Grimson, W.\,M.\ Wells: \\
      \structure{A Unifying Approach to Registration, Segmentation, and \\ Intensity Correction}, \\
      Proc. MICCAI, pp.\ 310-318, 2005. \\[.3cm]
      % In Proc. MICCAI 2005: Eighth International Conference on Medical Image Computing and Computer Assisted Intervention, Palm Springs, CA, U.\,S.\,A.,
      %Springer-Verlag, Part I, vol. 3749 of Lecture Notes in Computer Science,
    \item K.\,M.\ Pohl, J.\ Fisher, S.\ Bouix, M.\,E.\ Shenton, R.\,W.\ McCarley, W.\,E.\,L.\ Grimson, R.\ Kikinis, W.\,M.\ Wells: \\
      \structure{Using the logarithm of odds to define a vector space on \\ probabilistic atlases}, \\
      Medical Image Analysis, 11(6), pp.\ 465-477, 2007.
  \end{itemize}
\end{frame}

\subsection{Comprehensive Questions}

\begin{frame}
  \frametitle{Comprehensive Questions}

  \begin{itemize}
    \item What is the idea of combined MR segmentation and \\
      bias field correction? \\[.5cm]
    \item What is the E-step in this context? \\[.5cm]
    \item What is the M-step? \\[.5cm]
    \item How can the update formulas be derived?
  \end{itemize}
\end{frame}


% !TEX encoding = UTF-8 Unicode
% !TEX spellcheck = en_US
% !TEX root = ../../../ICMA2020.tex

\subsection{Process-Based Excitation}

One of the main advantages of the process-based approach is that both data collection and processing can be automated. Additionally, no special excitation trajectory has to be generated.
The parameter estimation is done as described in algorithm\,\ref{alg:ModelReductionAlgorithm} using the WLS method. The estimated parameters for process 1 and 2 are given in Tab.\,\ref{tab:ParamThetaProcess}. Here $\hat{\boldsymbol{\theta}}_{0}$ denotes the estimated parameters for the full model and $\hat{\boldsymbol{\theta}}_{\text{r}}$ denotes the estimated parameters for the reduced model. For both processes an error-tolerance $e_{\mathrm{tol}}$ of \SI{5}{\%} was chosen.

\begin{table}[tb]
	\caption{Estimated parameters for process 1 and 2.}\label{tab:ParamThetaProcess}
	\vspace{-0.2cm}
	\centering
	\begin{tabular}[h]{|r||c|c||c|c|}\hline
		\input{Chapters/Experimental_Results/Process_Based_Excitation/Table_ParametersProcess}
		\hline
	\end{tabular}
\end{table}

For process 1 $15$ parameters were removed during the model reduction. The condition number of the full model for the excitation of process~1 is $61.7$. As a result of the model reduction, the condition number for the reduced design matrix was decreased to $8.46$. Because of the significantly lower condition number the parameter estimation becomes more robust. For process~1 the friction parameters can not be neglected since all joints perform a sufficient motion. During the model reduction of process~2 $16$ dynamics parameters and $3$ friction parameters were removed. A peculiarity of process~2 is that there is little to no movement in the joints 4 and 6. This results in a significantly higher condition number of $1632$ for the full model compared to process~1. This was reduced to $2.73$ during the model reduction. Since some joints perform little to no movement the respective friction parameters can be neglected and are removed by the model reduction algorithm. The model prediction errors for the process-models applied to the respective process are given in Tab.\,\ref{tab:errorProcessOnProcess}. Both models A and B yield similar model prediction errors for for process 1 and 2. Therefore only the model prediction errors for model A are displayed in Tab.\,\ref{tab:errorModelAOnProcess} to have a comparison to the model which was derived from the best possible excitation.% This is due to the fact that the models A and B are of similar quality when compared by their model errors as shown in section \ref{subsec:OptimalExcitation_Result}. The model prediction errors for model A and B on both processes are given in Tab. \ref{tab:errorModelAOnProcess} and \ref{tab:errorModelBOnProcess} respectively.

\begin{table}[tb]
	\caption{Prediction error for the process based models on their respective process.}\label{tab:errorProcessOnProcess}
	\vspace{-0.2cm}
	\centering
	\begin{tabular}[h]{|r||c|c||c|c|}\hline
		\input{Chapters/Experimental_Results/Process_Based_Excitation/Table_errorProcessOnProcess}
		\hline
	\end{tabular}
\end{table}

\begin{table}[tb]
	\caption{Prediction error for model A on process 1 and 2.}\label{tab:errorModelAOnProcess}
	\centering
	\vspace{-0.2cm}
	\begin{tabular}[h]{|r||c|c||c|c|}\hline
		\input{Chapters/Experimental_Results/Process_Based_Excitation/Table_errorModelAonProcess}
		\hline
	\end{tabular}
\end{table}

%\begin{table}[tb]
%	\caption{Prediction error for model B on process 1 and 2.}\label{tab:errorModelBOnProcess}
%	\centering
%	\begin{tabular}[h]{|r||c|c||c|c|}\hline
%		\input{Chapters/Experimental_Results/Process_Based_Excitation/Table_errorModelBonProcess}
%		\hline
%	\end{tabular}
%\end{table}

The removed model parameters are negligible for the observed process since without them the model error did not increase by more than \SI{5}{\%}. The parameters $5$, $12$, $19$ and $22$ had relatively low changes in their values throughout the model reduction and were estimated with similar values for all models. Removing them from the model also yields a significantly higher prediction error when compared to the other parameters.

It is only possible to evaluate the physical consistency (see e.g. \cite{Gautier.2013}) of the parameters 8, 10, 13, 15, 16, 20, 22 and 23 since not all of the model parameters are independently identifiable. The mentioned parameters are inertia values and therefore must be positive. For the reduced models of process 1 and 2 this is the case for all mentioned parameters which were kept in the model.

For the validation trajectories of both processes the design matrix was calculated using the desired values of the joint angular positions, accelerations and velocities. This was done because the models are expected to be used in a model-based feedforward control scheme which also uses the desired values. As already explained in Sec.\,\ref{subsec:OptimalExcitation_Result}, the greater deviations between the model predictions and the measured torques result from the noise on the measured torques and the dynamic behavior of the controlled system. This effect is the strongest in joints 4 and 5 for process~2 where nearly no motion is performed which results in a high signal-to-noise ratio.

It should be noted that the model reduction only keeps parameters which are needed to describe the joint torques of the observed process. Therefore the resulting models may not be generally applicable. The process based identification has to be repeated for every new process the robot has to perform, and the resulting model can only be applied to that process. The parameter estimation is done through the robot application, which has to be implemented on the controller regardless. Therefore no additional expenditure arises.

Despite the restraints applied on the workspace, the parameter estimation with an optimized trajectory could still be performed. However, restricting the robot workspace hampers the excitation of all parameters, as can be seen in Tab.\,\ref{tab:ParamTheta}. Applying the model reduction algorithm to models A and B for their respective excitation trajectory showed that some parameters in model B had significantly less influence on the model output when compared to model A. Compared by the model errors, both process-based models can predict the measured torque data with a similar accuracy as the models A and B. Depending on the restrictions placed on the robot workspace optimal excitation of all parameters may not always be possible. Therefore the process-based parameter estimation can be used as an alternative if the optimal excitation of all parameters is not possible.

%The order in which the parameters are removed from the model is based on their sensitivity. Whether a parameter can be removed is determined by the accumulated model error during the iteration steps. Therefore the removed parameters vary, depending on the observed trajectory.


\documentclass[19pt]{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}

% partial derivative as a fraction
\newcommand{\fracpd}[2]{
  \ensuremath{\frac{\partial #1}{\partial #2}}
}

% fraction with parenthesis around it
\newcommand{\pfrac}[2]{
  \ensuremath{ \left( \frac{#1}{#2} \right)}
}

\begin{document}

% \section{Model}
%
% \textit{About notation} Without loss of generality when I write $x_t$ I really mean $x(s^t)$. When the history dependence is potentially unclear, I will it out in longer form.
%
% The Lagrangian is
%
% \begin{alignat}{2} \label{eq:lagrangian}
%   \mathcal{L} &= \mu_{1,0} V_{1,0}\left(c_{1,0}, V_{1,1}\right) &+& \mu_{2,0} V_{2,0}\left(c_{2,0}, V_{2,1} \right) \notag \\
%   & & \sum_{t=0}^{\infty} \sum_{s^t}  \biggm \{
%       &\lambda_{1}(s^t) \left( Y_1\left(\chi_{1,t}/g_t, 1\right) - a_{1,t} - a_{2,t} - i_{1,a,t} - i_{2,a,t} \right) + \notag \\
%   & & &\lambda_{2}(s^t) \left( Y_2\left(\chi_{2,t}/g_t, \xi_t\right) - b_{1,t} - b_{2,t} - i_{1,b,t} - i_{2,b,t} \right) + \notag \\
%   & & &\lambda_{3}(s^t) \left( (1-\delta_1)\frac{\chi_{1,t-1}}{g_{t-1}} + H_1 \left(i_{1,a,t-1}, i_{1,b,t-1} \right) - \chi_{1,t} \right) + \notag \\
%   & & &\lambda_{4}(s^t) \left( (1-\delta_2)\frac{\chi_{2,t-1}}{g_{t-1}} + H_2 \left(i_{2,a,t-1}, i_{2,b,t-1} \right) - \chi_{2,t} \right)  \biggm\},
% \end{alignat}
%
% where $c_{i,t} = G_i(a_{i,t}, b_{i,t})$ and $\chi_{i,t} = g(s^t) k_i(s^t)$ for $i=1,2$ and $t=0, 1, \dots, \infty$.
%
% \section{FOC}
%
% The control variables are: $\left\{a_{n}(s^t), b_{n}(s^t), i_{n,a}(s^t), i_{n,b}(s^t), \chi_{n}(s^t) \right\}$ for $t=0,1,\dots, \infty$, $n \in {1,2}$, and all histories of states $s^t$. The FONC are
%
% \begin{alignat}{2}
%   a_{1}(s^t)   : \quad & \mu_{1,0} \fracpd{V_{1,0}}{V_{1,t}}\fracpd{V_{1,t}}{c_{1,t}} \fracpd{c_{1,t}}{a_{t,1}} &= \lambda_{1}(s^t) \label{eq:foc_a1} \\
%   b_{1}(s^t)   : \quad & \mu_{1,0} \fracpd{V_{1,0}}{V_{1,t}}\fracpd{V_{1,t}}{c_{1,t}} \fracpd{c_{1,t}}{b_{t,1}} &= \lambda_{2}(s^t) \label{eq:foc_b1} \\
%   i_{1,a}(s^t) : \quad & \sum_{s^{t+1}} \lambda_3(s^{t+1}) \fracpd{H_1(i_{1,a}(s^t),i_{1,b}(s^t)))}{i_{1,a}(s^t)} &= \lambda_1(s^t) \label{eq:foc_i1a} \\
%   i_{1,b}(s^t) : \quad & \sum_{s^{t+1}} \lambda_3(s^{t+1}) \fracpd{H_1(i_{1,a}(s^t),i_{1,b}(s^t)))}{i_{1,b}(s^t)} &= \lambda_2(s^t)\label{eq:foc_i1b} \\
%   \chi_{1}(s^t): \quad & \lambda_1(s^t) \fracpd{Y_1(\chi_{1,t}/g_t, 1)}{(\chi_{1,t}/g_t)} \frac{1}{g_t} + \sum_{s^{t+1}} \frac{(1 - \delta_1)}{g(s^{t+1}|s^t)}\lambda_3(s^{t+1}) &= \lambda_3(s^t) \label{eq:foc_chi1} \\
%   a_{2}(s^t)   : \quad & \mu_{2,0} \fracpd{V_{2,0}}{V_{2,t}}\fracpd{V_{2,t}}{c_{2,t}} \fracpd{c_{2,t}}{a_{t,2}} &= \lambda_{1}(s^t)\label{eq:foc_a2} \\
%   a_{2}(s^t)   : \quad & \mu_{2,0} \fracpd{V_{2,0}}{V_{2,t}}\fracpd{V_{2,t}}{c_{2,t}} \fracpd{c_{2,t}}{b_{t,2}} &= \lambda_{2}(s^t)\label{eq:foc_b2} \\
%   i_{2,a}(s^t) : \quad & \sum_{s^{t+1}} \lambda_4(s^{t+1}) \fracpd{H_2(i_{2,a}(s^t),i_{2,b}(s^t)))}{i_{4,a}(s^t)} &= \lambda_1(s^t) \label{eq:foc_i2a} \\
%   i_{2,b}(s^t) : \quad & \sum_{s^{t+1}} \lambda_4(s^{t+1}) \fracpd{H_2(i_{2,a}(s^t),i_{2,b}(s^t)))}{i_{4,b}(s^t)} &= \lambda_2(s^t)\label{eq:foc_i2b} \\
%   \chi_{2}(s^t): \quad & \lambda_2(s^t) \fracpd{Y_2(\chi_{2,t}/g_t, 1)}{(\chi_{2,t}/g_t)} \frac{1}{g_t} + \sum_{s^{t+1}} \frac{(1 - \delta_1)}{g(s^{t+1}|s^t)}\lambda_4(s^{t+1}) &= \lambda_4(s^t) \label{eq:foc_chi2}
% \end{alignat}
%
% \section{Algebra} \label{sec:algebra}
%
%   \subsection{Definitions} \label{sub:definitions}
%
%   Before proceeding, we will gather some tools we will use while analyzing the above conditions. First, notice that by the chain rule we have
%
%   \begin{align} \label{eq:chain_derivs}
%     \fracpd{V_{i,0}}{V_{i,t}} = \prod_{\tau=1}^{t} \fracpd{V_{i,\tau-1}}{V_{i,\tau}}.
%   \end{align}
%
%   The notice that we can write the SDF for agent $i$ as
%
%   \begin{align*}
%     M_{i,t+1} &= \fracpd{V_{i,t}}{V_{i,t+1}} \frac{\partial V_{i,t+1}/ \partial c_{i,t+1}}{\partial V_{i,t}/ \partial c_{i,t}} \\
%     &= \beta \left( \frac{C_{t+1}}{C_t} \right)^{\rho - 1} \left( \frac{V_{i,t+1}}{E \left[ V_{i,t+1}^{\alpha} \right]^{1/\alpha}} \right)^{\alpha-\rho}.
%   \end{align*}
%
%   We will also write stochastic discount factor in $a$ and $b$ units as follows:
%
%
%
%   We will make use of the time $t$ Pareto weight, which we define as
%
%   \begin{align} \label{eq:defn_pareto_weight}
%     \mu_{i,t} &= \mu_{i,0} \fracpd{V_{i,0}}{V_{i,t}} \fracpd{V_{i,t}}{c_{i,t}} \notag \\
%     &= \mu_{i,0} \prod_{\tau=1}^{t} \fracpd{V_{i,\tau-1}}{V_{i,\tau}} \fracpd{V_{i,t}}{c_{i,t}} \notag  \\
%     &= \mu_{i,t-1} \fracpd{V_{i,t}}{V_{i,t+1}} \frac{\partial V_{i,t+1}/ \partial c_{i,t+1}}{\partial V_{i,t}/ \partial c_{i,t}} \notag \\
%     &= \mu_{i, t-1} M_{i,t}
%   \end{align}
%
%   We will also work with the variable, $S_t = \frac{\mu_{1,t}}{\mu_{2,t}}$, which according to \eqref{eq:defn_pareto_weight} can be written recursively as
%
%   \begin{align*}
%     S_t &= S_{t-1} \frac{M_{1,t-1}}{M_{2,t-1}}
%   \end{align*}
%
%   \subsection{Working with FOC} \label{sub:Working_with_foc}
%
%
%   We can take \eqref{eq:foc_i1a} and turn it into
%
%   \begin{align*}
%     \frac{1}{\fracpd{H_1(i_{1, a}(s^t), i_{1, b}(s^t)}{i_{1, a}(s^t)}} &= E \left[ M_{1, t+1} P_{k, t+1} \right]
%   \end{align*}
%
%   We can also transform \eqref{eq:foc_chi1} to
%
%   \begin{align*}
%     P_{k, t} &= \fracpd{Y(\chi_{1, t} / g(s^t))}{\chi_{1, t} / g(s^t)} \frac{1}{g(s^t)} + E \left[ \frac{(1 - \delta)}{g(s^{t+1})} M_{1, t+1} P_{k, t+1} \right]
%   \end{align*}
%
%   where $P_{k, t} = \frac{\lambda_{3, t}}{\lambda_{1, t}}$ and can be interpreted as the price of capital in terms of good $a$.
%
%   Using \eqref{eq:defn_pareto_weight}, \eqref{eq:foc_a1}, and \eqref{eq:foc_a2} we can derive
%
%   \begin{align} \label{eq:eqn_for_a}
%     \mu_{1,t} \fracpd{c_{1,t}}{a_{1,t}} &= \mu_{2,t} \fracpd{c_{2,t}}{a_{2,t}} \notag \\
%     S_{t} \fracpd{c_{1,t}}{a_{1,t}} &= \fracpd{c_{2,t}}{a_{2,t}}
%   \end{align}
%
%   Similarly using \eqref{eq:defn_pareto_weight}, \eqref{eq:foc_b1}, and \eqref{eq:foc_b2} we can derive
%
%   \begin{align} \label{eq:eqn_for_b}
%     \mu_{1,t} \fracpd{c_{1,t}}{b_{1,t}} &= \mu_{2,t} \fracpd{c_{2,t}}{b_{2,t}} \notag \\
%     S_{t} \fracpd{c_{1,t}}{b_{1,t}} &= \fracpd{c_{2,t}}{b_{2,t}}
%   \end{align}

\clearpage
\section{Starting from Dave's Exchange notes} \label{sec:StartingfromDavesExchangenotes}

Start from Dave's exchange economy notes, but add back heterogeneity in agent parameters ($\rho, \alpha, \beta, \omega, \sigma$). Combine the FOC in $a_1$ and $a_2$ to obtain

\begin{align} \label{eq:FOC_a1_a2}
  \frac{w_1}{w_2} &= \frac{(1 - \beta_2) c_2^{\rho_2 - \sigma_2} \omega_2 a_2^{\sigma_2 - 1}}{(1 - \beta_1) c_1^{\rho_1 - \sigma_1} (1-\omega_1) a_1^{\sigma_1 - 1}}
\end{align}

Also combine FOC in $b_1$ and $b_2$:

\begin{align} \label{eq:FOC_b1_b2}
  \frac{w_1}{w_2} &= \frac{(1 - \beta_2) c_2^{\rho_2 - \sigma_2} (1-\omega_2) b_2^{\sigma_2 - 1}}{(1 - \beta_1) c_1^{\rho_1 - \sigma_1} \omega_1 b_1^{\sigma_1 - 1}}
\end{align}

Combine these two to obtain the equation he has in the middle of the page:

\begin{align*}
  \frac{\omega_2 a_2^{\sigma_2 - 1}}{(1 - \omega_1) a_1^{\sigma_1 - 1}} &= \frac{(1-\omega_2) b_2^{\sigma_2 - 1}}{\omega_1 b_1^{\sigma_1 - 1}}
\end{align*}

For now let's assume that $\sigma_1 = \sigma_2 = \sigma$ and define $s_a := a_1/y_1$ and $s_b = b_1/y_2$. We can now derive

\begin{align*}
  \left( \frac{s_b}{1 - s_b} \right)^{\sigma - 1} &= \frac{(1-\omega_1)(1-\omega_2)}{\omega_1 \omega_2} \left( \frac{s_a}{1 - s_a} \right)^{\sigma-1},
\end{align*}

which can be solved in closed form for

\begin{align} \label{eq:soln_sb}
  s_b = \frac{\left( \frac{(1-\omega_1)(1-\omega_2)}{\omega_1 \omega_2} \right)^{\frac{1}{\sigma - 1}} \frac{s_a}{1 - s_a}}{1 + \left( \frac{(1-\omega_1)(1-\omega_2)}{\omega_1 \omega_2} \right)^{\frac{1}{\sigma - 1}} \frac{s_a}{1 - s_a}}.
\end{align}

Then, for a given $w_1/w_2 := \spadesuit, y_1=1$ and $y_2=z_2/z_1$ we can use \eqref{eq:FOC_a1_a2} as a residual to search over $s_a$ and we can use \eqref{eq:FOC_b1_b2} as the needed residual to form the RHS of the regression that updates $\spadesuit_t$ to $\spadesuit_{t+1}$.

\section{Algorithm} \label{sec:Algorithm}

Now the computation algorithm will be presented. As an initialization phase, do the following:

\begin{itemize}
  \item Choose a large simulation length $T$
  \item Choose an initial state $\spadesuit_0$ and $z_{2,0}/z_{1,0}$.
  \item Simulate the exogenous process forward to obtain $\left\{z_{2,t}/z_{1,t}  \right\}_{t=1}^T$
  \item Choose an initial coefficient vector $b = \begin{bmatrix}b_0 & b_1 & b_2\end{bmatrix}$ in the equation
  \begin{align*}
    \spadesuit_{t+1} = b_0 + b_1 \spadesuit_t + b_2 (\log z_{2,t} - \log z_{1,t})
  \end{align*}
\end{itemize}

Then, one iteration of the simulation algorithm is carried out as follows:

\begin{enumerate}
  \item Use the coefficients $b$, simulated history for $z_2/z_1$, and the initial condition $\spadesuit_0 = 1$ to simulate $\left\{ \spadesuit_t \right\}_{t=1}^{T}$
  \item Using this history, use equations \eqref{eq:FOC_a1_a2} and \eqref{eq:soln_sb} so solve for $\left\{ s_{a,t}, s_{b,t} \right\}_{t=1}^{T}$
  \item Given $s_{a,t}, s_{b,t}$, and $z_{2,t}/z_{1,t}$ we know $y_{1,t}=1$, $y_{2,t} = z_{2,t}/z_{1,t}$, $a_{1,t} = s_{a,t} y_{1,t}$, $a_{2,t} = (1-s_{a,t}) y_{1,t}$, $b_{1,t} = s_{b,t} y_{2,t}$, $b_{2,t} = (1-s_{b,t}) y_{2,t}$, $c_{1,t} = h_1(a_{1,t}, b_{1,t})$, and $c_{2,t} = h_2(a_{2,t}, b_{2,t})$. We can use \eqref{eq:FOC_b1_b2} to form an updated guess for what $\spadesuit_t$ should be.
  \item Given this updated guess we can now run the following regression:

    \begin{align*}
      \hat{\spadesuit}_{t+1} = \hat{b}_0 + \hat{b}_1 \tilde{\spadesuit}_t + \hat{b}_2 (\log z_{2,t} - \log z_{1,t})
    \end{align*}

    where $\hat{\spadesuit}_{t+1}$ is obtained in step 1 from using the old coefficient vector and $\tilde{spadesuit}_t$ is the lefthand side of \eqref{eq:FOC_b1_b2} as computed in step 3.
  \item Then say that the new coefficient vector is

    \begin{align*}
      b = (1 - \xi) \hat{b} + \xi b,
    \end{align*}

    where $\hat{b}$ is the new coefficient vector from the previous step and $b$ is the coefficient vector in hand when you simulated forward in step 1.

  \item Finally, check convergence of the time series of $\{\spadesuit_t\}$ computed on successive iterations
\end{enumerate}

\end{document}


%% Submissions for peer-review must enable line-numbering
%% using the lineno option in the \documentclass command.
%%
%% Preprints and camera-ready submissions do not need
%% line numbers, and should have this option removed.
%%
%% Please note that the line numbering option requires
%% version 1.1 or newer of the wlpeerj.cls file.

\documentclass[fleqn,10pt,numbers]{wlpeerj} % for journal submissions
% \documentclass[fleqn,10pt]{wlpeerj} % for preprint submissions

\usepackage{lmodern}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[scaled=0.8]{DejaVuSansMono}

\usepackage{graphicx}
\usepackage[all]{xy}
\usepackage{amsmath}
\usepackage{caption}
\graphicspath{ {images/} }

% Makes quote characters in monospace font not be curly
\usepackage{upquote}

\usepackage{amsmath}
\usepackage{url}

% Order matters here
\usepackage{xr-hyper}
\usepackage{hyperref}
\externaldocument[S-]{supplement}

\hypersetup{
  colorlinks=true,
  urlcolor=blue,
  citecolor=black,
  filecolor=black % cross-file links, which do not actually work unless the
                  % supplement is named supplement.pdf
}

% for nice units
\usepackage{siunitx}

% for images: png, pdf, etc
\usepackage{graphicx}

% for nice table formatting, i.e., /toprule, /midrule, etc
\usepackage{booktabs}

% to allow for \verb++ declarations in captions.
\usepackage{cprotect}

% to allow usage of \mathbb symbols
\usepackage{amssymb}

\usepackage{longtable}

\usepackage{listings}

\newcommand\email[1]{\href{mailto:#1}{#1}}
\title{SymPy: symbolic computing in Python}

\input{authors}

% \keywords{Python, computer algebra system, symbolics}

\begin{abstract}
  SymPy is an open source computer algebra system written in pure Python. It
  is built with a focus on extensibility and ease of use, through both
  interactive and programmatic applications. These characteristics have led
  SymPy to become a popular symbolic library for the scientific Python
  ecosystem. This paper presents the architecture of SymPy, a description of
  its features, and a discussion of select submodules. The
  supplementary material provide additional examples and further outline
  details of the architecture and features of SymPy.
\end{abstract}

\begin{document}

\flushbottom
\maketitle
\thispagestyle{empty}

\section{Introduction}

%% What sympy is, where to download etc.
%%
%% List other major CAS's.
%%
%% Why SymPy.

\input{introduction.tex}

\section{Overview of Capabilities}
\label{sec:features}

%% List of Features and how to use
%%
%% Quick overview of the main modules, what it can do and so on. It should probably provide examples how to use sympy.
%%
%% See also the supplement (below)

\input{features}

\section{Numerics}
\label{sec:numerics}

%% Description of some algorithms (example: integration with Risch, Meijer G, Gruntz, polys)
%%
%% Description of numerics/mpmath (Fredrik)

\input{numerics}

\section{Physics Submodule}
\label{sec:domain_specific}

\input{domain_specific}

\section{Architecture}
\label{sec:architecture}

\input{architecture}

\section{Projects that Depend on SymPy}
\label{sec:other-proj}
\input{projects_that_depend_on_sympy}

\section{Conclusion and Future Work}
\label{sec:conclusion}

\input{conclusion_and_future_work}

\phantomsection
\bibliography{paper}

\end{document}