\hypertarget{exploring-distributions}{%
\chapter{Exploring Distributions}\label{exploring-distributions}}

\hypertarget{introduction}{%
\section{Introduction}\label{introduction}}

This is the second in a series of notebooks that make up a
\href{https://allendowney.github.io/PoliticalAlignmentCaseStudy/}{case
study in exploratory data analysis}. This case study is part of the
\href{https://allendowney.github.io/ElementsOfDataScience/}{\emph{Elements
of Data Science}} curriculum.

In this notebook, we:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
  Look at responses to the variable \passthrough{\lstinline!polviews!},
  which represent political alignment on a 7-point scale from liberal to
  conservative.
\item
  Compare the distribution of responses in 1974 and 1990.
\item
  Plot the mean and standard deviation of responses over time as a way
  of quantifying changes in political alignment and polarization.
\item
  Use local regression to plot a smooth line through noisy data.
\item
  Use cross tabulation to compute the fraction of respondents in each
  category over time.
\item
  Plot the results using a custom color palette.
\end{enumerate}

As an exercise, you will look at changes in political party affiliation
over the same period.

The following cell installs the \passthrough{\lstinline!empiricaldist!}
library if necessary.

\begin{lstlisting}[language=Python,style=source]
try:
    import empiricaldist
except ImportError:
    !pip install empiricaldist
\end{lstlisting}

If everything we need is installed, the following cell should run
without error.

\begin{lstlisting}[language=Python,style=source]
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

from empiricaldist import Pmf
\end{lstlisting}

The following cell defines a function I use to decorate the axes in
plots.

\begin{lstlisting}[language=Python,style=source]
def decorate(**options):
    """Decorate the current axes.

    Call decorate with keyword arguments like
    decorate(title='Title',
             xlabel='x',
             ylabel='y')

    The keyword arguments can be any of the axis properties
    https://matplotlib.org/api/axes_api.html
    """
    ax = plt.gca()
    ax.set(**options)

    handles, labels = ax.get_legend_handles_labels()
    if handles:
        ax.legend(handles, labels)

    plt.tight_layout()
\end{lstlisting}

\hypertarget{loading-the-data}{%
\section{Loading the data}\label{loading-the-data}}

In the previous notebook, we downloaded GSS data, loaded and cleaned it,
resampled it to correct for stratified sampling, and then saved the data
in an HDF5 file, which is much faster to load. In this and the following
notebooks, we'll download the HDF5 file and load it.

The following cell downloads the file if necessary.

\begin{lstlisting}[language=Python,style=source]
from os.path import basename, exists

def download(url):
    filename = basename(url)
    if not exists(filename):
        from urllib.request import urlretrieve
        local, _ = urlretrieve(url, filename)
        print('Downloaded ' + local)

download('https://github.com/AllenDowney/PoliticalAlignmentCaseStudy/' +
         'raw/master/gss_eda.3.hdf5')
\end{lstlisting}

This file contains three DataFrames containing resamples of the GSS
data. We'll work with the first resampling,
\passthrough{\lstinline!gss0!}, to get started; at the end of this
notebook, we'll see the other two as well.

\begin{lstlisting}[language=Python,style=source]
datafile = 'gss_eda.3.hdf5'
gss = pd.read_hdf(datafile, 'gss0')
gss.shape
\end{lstlisting}

\begin{lstlisting}[style=output]
(64814, 169)
\end{lstlisting}

\hypertarget{political-alignment}{%
\section{Political alignment}\label{political-alignment}}

The people surveyed as part of the GSS were asked about their
``political alignment'', which is where they place themselves on a
spectrum from liberal to conservative.

The variable \passthrough{\lstinline!polviews!} contains responses to
the
\href{https://gssdataexplorer.norc.org/projects/52787/variables/178/vshow}{following
question}:

\begin{quote}
We hear a lot of talk these days about liberals and conservatives. I'm
going to show you a seven-point scale on which the political views that
people might hold are arranged from extremely liberal--point 1--to
extremely conservative--point 7. Where would you place yourself on this
scale?
\end{quote}

Here are the valid responses:

\begin{lstlisting}[style=output]
1   Extremely liberal
2   Liberal
3   Slightly liberal
4   Moderate
5   Slightly conservative
6   Conservative
7   Extremely conservative
\end{lstlisting}

To see how the responses have changed over time, we'll inspect them at
the beginning and end of the observation period.

First I'll select the column.

\begin{lstlisting}[language=Python,style=source]
polviews = gss['polviews']
\end{lstlisting}

Then compute a Boolean Series that's \passthrough{\lstinline!True!} for
responses from 1974.

\begin{lstlisting}[language=Python,style=source]
year74 = (gss['year'] == 1974)
\end{lstlisting}

Now we can select the responses from 1974.

\begin{lstlisting}[language=Python,style=source]
polviews74 = polviews[year74]
\end{lstlisting}

As in the previous notebook, we'll use \passthrough{\lstinline!values!}
to plot the values in the series and their frequencies.

\begin{lstlisting}[language=Python,style=source]
def values(series):
    """Count the values and sort.

    series: pd.Series

    returns: series mapping from values to frequencies
    """
    return series.value_counts().sort_index()
\end{lstlisting}

Here are the responses from 1974.

\begin{lstlisting}[language=Python,style=source]
values(polviews74)
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &  polviews \\
\midrule
1.0 &        31 \\
2.0 &       201 \\
3.0 &       211 \\
4.0 &       538 \\
5.0 &       223 \\
6.0 &       181 \\
7.0 &        30 \\
\bottomrule
\end{tabular}

And here are the responses from 2018.

\begin{lstlisting}[language=Python,style=source]
year18 = (gss['year'] == 2018)
polviews18 = polviews[year18]
values(polviews18)
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &  polviews \\
\midrule
1.0 &        89 \\
2.0 &       269 \\
3.0 &       265 \\
4.0 &       891 \\
5.0 &       310 \\
6.0 &       342 \\
7.0 &        92 \\
\bottomrule
\end{tabular}

\hypertarget{pmfs}{%
\section{PMFs}\label{pmfs}}

To visualize these distributions, we'll use the Probability Mass
Function (PMF), which is similar to a histogram. The difference is that
the PMF is ``normalized'', which means that it shows the percentage of
people who gave each response, rather than the number.

I'll use the \passthrough{\lstinline!Pmf!} class from
\passthrough{\lstinline!empiricaldist!} to compute them.

\begin{lstlisting}[language=Python,style=source]
from empiricaldist import Pmf
\end{lstlisting}

Here's the distribution from 1974:

\begin{lstlisting}[language=Python,style=source]
pmf74 = Pmf.from_seq(polviews74)
pmf74.bar(label='1974', color='C0', alpha=0.7)

decorate(xlabel='Political view on a 7-point scale',
         ylabel='Fraction of population',
         title='Distribution of political views')
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{02_polviews_files/02_polviews_30_0.pdf}
\end{center}

And from 2018:

\begin{lstlisting}[language=Python,style=source]
pmf18 = Pmf.from_seq(polviews18)
pmf18.bar(label='2018', color='C1', alpha=0.7)

decorate(xlabel='Political view on a 7-point scale',
         ylabel='Fraction of population',
         title='Distribution of political views')
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{02_polviews_files/02_polviews_32_0.pdf}
\end{center}

In both cases, the most common response is \passthrough{\lstinline!4!},
which is the code for ``moderate''. And few respondents describe
themselves as ``extremely'' liberal or conservative.

So maybe we're not so polarized after all.

To make it easier to compare the distributions, I'll plot them side by
side.

\begin{lstlisting}[language=Python,style=source]
pmf74.bar(label='1974', width=-0.45, align='edge', alpha=0.7)

pmf18.bar(label='2018', width=0.45, align='edge', alpha=0.7)

decorate(xlabel='Political view on a 7-point scale',
         ylabel='Fraction of population',
         title='Distribution of political views')
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{02_polviews_files/02_polviews_34_0.pdf}
\end{center}

Now we can see the changes in the distribution more clearly. It looks
like the number of people at the extremes (1 and 7) has increased, and
the fraction of liberal (2) and slightly liberal (3) has decreased.

\textbf{Exercise:} To summarize these changes, we can compare the mean
and standard deviation of \passthrough{\lstinline!polviews!} in 1974 and
2018.

The mean of the responses measures the balance of people in the
population with liberal or conservative leanings. If the mean increases
over time, that might indicate a shift in the population toward
conservatism.

The standard deviation measures the dispersion of views in the
population; if it increases over time, that might indicate an increase
in polarization.

Compute the mean and standard deviation of
\passthrough{\lstinline!polviews74!} and
\passthrough{\lstinline!polviews18!}.

What do they indicate about changes over this interval?

\hypertarget{time-series}{%
\section{Time series}\label{time-series}}

At this point we have looked at the endpoints, 1974 and 2018, but we
don't know what happened in between.

To see how the distribution changes over time, we can group by year and
compute the mean of \passthrough{\lstinline!polviews!} during each year.

First I'll use \passthrough{\lstinline!groupby!} to group the
respondents by year.

\begin{lstlisting}[language=Python,style=source]
gss_by_year = gss.groupby('year')
gss_by_year
\end{lstlisting}

\begin{lstlisting}[style=output]
<pandas.core.groupby.generic.DataFrameGroupBy object at 0x7f9b95ac7250>
\end{lstlisting}

The result is a \passthrough{\lstinline!DataFrameGroupBy!} object that
represents a collection of groups. We can loop through the groups and
display the number of respondents in each:

\begin{lstlisting}[language=Python,style=source]
for year, group in gss_by_year:
    print(year, len(group))
\end{lstlisting}

\begin{lstlisting}[style=output]
1972 1613
1973 1504
1974 1484
1975 1490
1976 1499
1977 1530
1978 1532
1980 1468
1982 1860
1983 1599
1984 1473
1985 1534
1986 1470
1987 1819
1988 1481
1989 1537
1990 1372
1991 1517
1993 1606
1994 2992
1996 2904
1998 2832
2000 2817
2002 2765
2004 2812
2006 4510
2008 2023
2010 2044
2012 1974
2014 2538
2016 2867
2018 2348
\end{lstlisting}

In many ways the \passthrough{\lstinline!DataFrameGroupBy!} behaves like
a \passthrough{\lstinline!DataFrame!}. We can use the bracket operator
to select a column:

\begin{lstlisting}[language=Python,style=source]
polviews_by_year = gss_by_year['polviews']
polviews_by_year
\end{lstlisting}

\begin{lstlisting}[style=output]
<pandas.core.groupby.generic.SeriesGroupBy object at 0x7f9b95a1e710>
\end{lstlisting}

A column from a \passthrough{\lstinline!DataFrameGroupBy!} is a
\passthrough{\lstinline!SeriesGroupBy!}. If we invoke
\passthrough{\lstinline!mean!} on it, the results is a series that
contains the mean of \passthrough{\lstinline!polviews!} for each year of
the survey.

\begin{lstlisting}[language=Python,style=source]
mean_series = polviews_by_year.mean()
\end{lstlisting}

And here's what it looks like.

\begin{lstlisting}[language=Python,style=source]
mean_series.plot(color='C2', label='polviews')
decorate(xlabel='Year',
         ylabel='Mean (7 point scale)',
         title='Mean of polviews')
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{02_polviews_files/02_polviews_46_0.pdf}
\end{center}

\textbf{Exercise:} The standard deviation quantifies the spread of the
distribution, which is one way to measure polarization.

Plot standard deviation of \passthrough{\lstinline!polviews!} for each
year of the survey from 1972 to 2018.

Does it show evidence of increasing polarization?

\hypertarget{local-regression}{%
\subsection{Local regression}\label{local-regression}}

In the previous section we plotted mean and standard deviation of
\passthrough{\lstinline!polviews!} over time. Both plots are quite
noisy.

We can use \href{https://en.wikipedia.org/wiki/Local_regression}{local
regression} to compute a smooth line through these data points.

The following function takes a Pandas Series and uses and algorithm
called LOWESS to compute a smooth line. LOWESS stands for ``locally
weighted scatterplot smoothing''.

\begin{lstlisting}[language=Python,style=source]
from statsmodels.nonparametric.smoothers_lowess import lowess

def make_lowess(series):
    """Use LOWESS to compute a smooth line.

    series: pd.Series

    returns: pd.Series
    """
    y = series.values
    x = series.index.values

    smooth = lowess(y, x)
    index, data = np.transpose(smooth)

    return pd.Series(data, index=index)
\end{lstlisting}

We'll use the following function to plot data points and the smoothed
line.

\begin{lstlisting}[language=Python,style=source]
def plot_series_lowess(series, color):
    """Plots a series of data points and a smooth line.

    series: pd.Series
    color: string or tuple
    """
    series.plot(linewidth=0, marker='o', color=color, alpha=0.5)
    smooth = make_lowess(series)
    smooth.plot(label='', color=color)
\end{lstlisting}

The following figure shows the mean of
\passthrough{\lstinline!polviews!} and a smooth line.

\begin{lstlisting}[language=Python,style=source]
mean_series = gss_by_year['polviews'].mean()
plot_series_lowess(mean_series, 'C2')
decorate(ylabel='Mean (7 point scale)',
         title='Mean of polviews',
         xlabel='Year',
         xlim=[1972, 2020])
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{02_polviews_files/02_polviews_53_0.pdf}
\end{center}

One reason the PMFs for 1974 and 2018 did not look very different is
that the mean seems to have gone up (more conservative) and then down
again (more liberal).

Generally, it looks like the U.S. has been trending toward liberal for
the last 20 years, or more, at least in the sense of how people describe
themselves.

\textbf{Exercise:} Use \passthrough{\lstinline!plot\_series\_lowess!} to
plot the standard deviation of \passthrough{\lstinline!polviews!} with a
smooth line.

\hypertarget{cross-tabulation}{%
\section{Cross tabulation}\label{cross-tabulation}}

In the previous sections, we treated \passthrough{\lstinline!polviews!}
as a numerical quantity, so we were able to compute means and standard
deviations.

But the responses are really categorical, which means that each value
represents a discrete category, like ``liberal'' or ``conservative''.

In this section, we'll treat \passthrough{\lstinline!polviews!} as a
categorical variable. Specifically, we'll compute the number of
respondents in each category for each year, and plot changes over time.

Pandas provides a function called \passthrough{\lstinline!crosstab!}
that computes a
\href{https://en.wikipedia.org/wiki/Contingency_table}{cross
tabulation}.

It takes two Series as arguments and returns a DataFrame.

\begin{lstlisting}[language=Python,style=source]
year = gss['year']
column = gss['polviews']

xtab = pd.crosstab(year, column)
\end{lstlisting}

Here are the first few lines from the result.

\begin{lstlisting}[language=Python,style=source]
xtab.head()
\end{lstlisting}

\begin{tabular}{lrrrrrrr}
\toprule
polviews &  1.0 &  2.0 &  3.0 &  4.0 &  5.0 &  6.0 &  7.0 \\
year &      &      &      &      &      &      &      \\
\midrule
1974 &   31 &  201 &  211 &  538 &  223 &  181 &   30 \\
1975 &   56 &  184 &  207 &  540 &  204 &  162 &   45 \\
1976 &   31 &  198 &  175 &  564 &  209 &  206 &   34 \\
1977 &   37 &  181 &  214 &  594 &  243 &  164 &   42 \\
1978 &   21 &  140 &  255 &  559 &  265 &  187 &   25 \\
\bottomrule
\end{tabular}

It contains one row for each value of \passthrough{\lstinline!year!} and
one column for each value of \passthrough{\lstinline!polviews!}. Reading
the first row, we see that in 1974, 31 people gave response 1,
``extremely liberal'', 201 people gave response 2, ``liberal'', and so
on.

The number of respondents varies from year to year, so we need to
``normalize'' the results, which means computing for each year the
\emph{fraction} of respondents in each category, rather than the count.

\passthrough{\lstinline!crosstab!} takes an optional argument that
normalizes each row.

\begin{lstlisting}[language=Python,style=source]
xtab_norm = pd.crosstab(year, column, normalize='index')
\end{lstlisting}

Here's what that looks like for the 7-point scale.

\begin{lstlisting}[language=Python,style=source]
xtab_norm.head()
\end{lstlisting}

\begin{tabular}{lrrrrrrr}
\toprule
polviews &       1.0 &       2.0 &       3.0 &       4.0 &       5.0 &       6.0 &       7.0 \\
year &           &           &           &           &           &           &           \\
\midrule
1974 &  0.021908 &  0.142049 &  0.149117 &  0.380212 &  0.157597 &  0.127915 &  0.021201 \\
1975 &  0.040057 &  0.131617 &  0.148069 &  0.386266 &  0.145923 &  0.115880 &  0.032189 \\
1976 &  0.021877 &  0.139732 &  0.123500 &  0.398024 &  0.147495 &  0.145378 &  0.023994 \\
1977 &  0.025085 &  0.122712 &  0.145085 &  0.402712 &  0.164746 &  0.111186 &  0.028475 \\
1978 &  0.014463 &  0.096419 &  0.175620 &  0.384986 &  0.182507 &  0.128788 &  0.017218 \\
\bottomrule
\end{tabular}

To make the results easier to interpret, I'm going to replace the
numeric codes 1-7 with strings. First I'll make a dictionary that maps
from numbers to strings:

\begin{lstlisting}[language=Python,style=source]
# recode the 7 point scale with words
d7 = {1: 'Extremely liberal',
      2: 'Liberal',
      3: 'Slightly liberal',
      4: 'Moderate',
      5: 'Slightly conservative',
      6: 'Conservative',
      7: 'Extremely conservative'}
\end{lstlisting}

Then we can use the \passthrough{\lstinline!replace!} function like
this:

\begin{lstlisting}[language=Python,style=source]
polviews7 = gss['polviews'].replace(d7)
\end{lstlisting}

We can use \passthrough{\lstinline!values!} to confirm that the values
in \passthrough{\lstinline!polviews7!} are strings.

\begin{lstlisting}[language=Python,style=source]
values(polviews7)
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &  polviews \\
\midrule
Conservative           &      8495 \\
Extremely conservative &      1770 \\
Extremely liberal      &      1699 \\
Liberal                &      6299 \\
Moderate               &     21444 \\
Slightly conservative  &      8864 \\
Slightly liberal       &      6981 \\
\bottomrule
\end{tabular}

If we make the cross tabulation again, we can see that the column names
are strings.

\begin{lstlisting}[language=Python,style=source]
xtab_norm = pd.crosstab(year, polviews7, normalize='index')
xtab_norm.head()
\end{lstlisting}

\begin{tabular}{lrrrrrrr}
\toprule
polviews &  Conservative &  Extremely conservative &  Extremely liberal &   Liberal &  Moderate &  Slightly conservative &  Slightly liberal \\
year &               &                         &                    &           &           &                        &                   \\
\midrule
1974 &      0.127915 &                0.021201 &           0.021908 &  0.142049 &  0.380212 &               0.157597 &          0.149117 \\
1975 &      0.115880 &                0.032189 &           0.040057 &  0.131617 &  0.386266 &               0.145923 &          0.148069 \\
1976 &      0.145378 &                0.023994 &           0.021877 &  0.139732 &  0.398024 &               0.147495 &          0.123500 \\
1977 &      0.111186 &                0.028475 &           0.025085 &  0.122712 &  0.402712 &               0.164746 &          0.145085 \\
1978 &      0.128788 &                0.017218 &           0.014463 &  0.096419 &  0.384986 &               0.182507 &          0.175620 \\
\bottomrule
\end{tabular}

We are almost ready to plot the results, but first we need some colors.

\hypertarget{color-palettes}{%
\section{Color palettes}\label{color-palettes}}

Seaborn provides a variety of color palettes,
\href{https://seaborn.pydata.org/tutorial/color_palettes.html}{which you
can read about here}.

To represent political views, I'll use a diverging palette from blue to
red.

\begin{lstlisting}[language=Python,style=source]
palette = sns.color_palette('RdBu_r', 7)
sns.palplot(palette)
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{02_polviews_files/02_polviews_74_0.pdf}
\end{center}

The middle color is white, which won't work when we plot it, so I will
replace it with a purple color from another palette.

\begin{lstlisting}[language=Python,style=source]
muted = sns.color_palette('muted', 7)
purple = muted[4]
sns.palplot(muted)
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{02_polviews_files/02_polviews_76_0.pdf}
\end{center}

Here's the modified diverging palette with purple in the middle.

\begin{lstlisting}[language=Python,style=source]
palette[3] = purple
sns.palplot(palette)
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{02_polviews_files/02_polviews_78_0.pdf}
\end{center}

A feature of this color map is that the colors are meaningful, at least
in countries that use blue, purple, and red for these points on the
political spectrum. A drawback of this color map is that some some of
the colors are indistinguishable to people who are
\href{https://davidmathlogic.com/colorblind}{color blind}.

Now I'll make a dictionary that maps from the responses to the
corresponding colors.

\begin{lstlisting}[language=Python,style=source]
columns = ['Extremely liberal',
           'Liberal',
           'Slightly liberal',
           'Moderate',
           'Slightly conservative',
           'Conservative',
           'Extremely conservative']
\end{lstlisting}

\begin{lstlisting}[language=Python,style=source]
color_map = dict(zip(columns, palette))

for key, value in color_map.items():
    print(key, value)
\end{lstlisting}

\begin{lstlisting}[style=output]
Extremely liberal (0.16339869281045757, 0.44498269896193776, 0.6975009611687812)
Liberal (0.4206843521722416, 0.6764321414840447, 0.8186851211072664)
Slightly liberal (0.7614763552479817, 0.8685121107266438, 0.924567474048443)
Moderate (0.5843137254901961, 0.4235294117647059, 0.7058823529411765)
Slightly conservative (0.9824682814302191, 0.8006920415224913, 0.7061130334486736)
Conservative (0.8945790080738177, 0.5038062283737024, 0.39976931949250283)
Extremely conservative (0.7284890426758939, 0.15501730103806227, 0.1973856209150327)
\end{lstlisting}

\hypertarget{plotting}{%
\section{Plotting}\label{plotting}}

To plot the results, I use the following function, which takes a
\passthrough{\lstinline!DataFrame!} and plots each column using
\passthrough{\lstinline!plot\_series\_lowess!}.

\begin{lstlisting}[language=Python,style=source]
def plot_columns_lowess(table, columns, colors):
    """Plot the columns in a DataFrame.

    table: DataFrame with a cross tabulation
    columns: list of column names, in the desired order
    colors: mapping from column names to colors
    """
    for col in columns:
        series = table[col]
        plot_series_lowess(series, colors[col])
\end{lstlisting}

The following function sets the position of the figure legend.

\begin{lstlisting}[language=Python,style=source]
def anchor_legend(x, y):
    """Place the upper left corner of the legend box.

    x: x coordinate
    y: y coordinate
    """
    plt.legend(bbox_to_anchor=(x, y), loc='upper left', ncol=1)
\end{lstlisting}

Here are the 7 categories plotted as a function of time.

\begin{lstlisting}[language=Python,style=source]
plot_columns_lowess(xtab_norm, columns, color_map)
decorate(xlabel='Year',
         ylabel='Proportion',
         title='Fraction of people with each political view',
         xlim=[1972, 2020])

anchor_legend(1.02, 1.02)
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{02_polviews_files/02_polviews_87_0.pdf}
\end{center}

This way of looking at the results suggests that changes in political
alignment during this period have generally been slow and small.

The fraction of self-described moderates has not changed substantially.

The fraction of conservatives increased, but seems to be decreasing now;
the number of liberals seems to be increasing.

The fraction of people at the extremes has increased, but it is hard to
see clearly in this figure.

We can get a better view by plotting just the extremes.

\begin{lstlisting}[language=Python,style=source]
columns2 = ['Extremely liberal', 'Extremely conservative']

plot_columns_lowess(xtab_norm, columns2, color_map)
decorate(xlabel='Year',
         ylabel='Proportion',
         title='Fraction of people with extreme political views',
         xlim=[1970, 2020])

anchor_legend(1.02, 1.02)
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{02_polviews_files/02_polviews_89_0.pdf}
\end{center}

This figure shows that the fraction of people who describe themselves as
``extreme'' has increased from about 2.5\% to about 4\%.

In relative terms, that's a big increase. But in absolute terms these
tails of the distribution are still small.

\textbf{Exercise:} Let's do a similar analysis with
\passthrough{\lstinline!partyid!}, which encodes responses to the
question:

\begin{quote}
Generally speaking, do you usually think of yourself as a Republican,
Democrat, Independent, or what?
\end{quote}

The valid responses are:

\begin{lstlisting}[style=output]
0   Strong democrat
1   Not str democrat
2   Ind,near dem
3   Independent
4   Ind,near rep
5   Not str republican
6   Strong republican
7   Other party
\end{lstlisting}

You can
\href{https://gssdataexplorer.norc.org/projects/52787/variables/141/vshow}{read
the codebook for \passthrough{\lstinline!partyid!} here}.

Here are the steps I suggest:

\begin{enumerate}
\def\labelenumi{\arabic{enumi})}
\item
  If you have not already saved this notebook, you should do that first.
  If you are running on Colab, select ``Save a copy in Drive'' from the
  File menu.
\item
  Now, before you modify this notebook, make \emph{another} copy and
  give it an appropriate name.
\item
  Search and replace \passthrough{\lstinline!polviews!} with
  \passthrough{\lstinline!partyid!} (use ``Edit-\textgreater Find and
  replace'').
\item
  Run the notebook from the beginning and see what other changes you
  have to make.
\end{enumerate}

You will have to make changes in \passthrough{\lstinline!d7!} and
\passthrough{\lstinline!columns!}. Otherwise you might get a message
like

\passthrough{\lstinline!TypeError: '<' not supported between instances of 'float' and 'str'!}

Also, you might have to drop ``Other party'' or change the color
palette.

And you should change the titles of the figures.

What changes in party affiliation do you see over the last 50 years? Are
things going in the directions you expected?

Write a headline (or a couple) that describe the most substantial
changes you see.


\documentclass[../PHYS306Notes.tex]{subfiles}

\begin{document}
\section{Lecture 33}
\subsection{Lecture Notes - Lyapunov Exponents, Bifurcation Diagrams, State-Space orbits, and Poincare Sections}
\subsubsection{Period doubling cascade - "Route to Chaos"}
Period doubles each time the driving strength of the driven damped pendulum is increased past $\gamma_n$:
\[\delta = \lim_{n\rightarrow\infty}\frac{\gamma_{n-1} - \gamma_{n-2}}{\gamma_n - \gamma_{n-1}} = 4.6692016\ldots\]
This is the universal "Feigenbaum number".
\subsubsection{The driven damped pendulum revisisted}
We return to the driven damped pendlum from last day. Giving two pendulums driving strengths of $\gamma = 1.503$, and one with initial phase of $\phi_0 = 0$ and the other with $\phi_0 = 0.005$, we can see that after time, the trajectories begin to diverge significantly.
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img1.png}
\end{center}
With a driving strength of 1.077, and an initial phase shift of $-28$, we see that there is definitely a divergence in the two trajectories, but it looks periodic.
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img2.png}
\end{center}
With $\Delta \phi = -27$, the difference is different from $\Delta \phi = -28$ but still periodic. With $\Delta \phi = -29$, we see that we actually reach a constant difference:
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img3.png}
\end{center}
With a slightly different driving strength, we see:
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img4.png}
\end{center}
We recognize that we are slightly further into the chaotic regime; looking at the difference between the two pendulums, we see that there is no repeated pattern (there are slight differneces between cycles, so there doesn't appear to be periodic/perfect repetition).

\subsubsection{Sensitivity to Initial Conditions \& Lyapunov Exponents}
$\Phi(t) = \Phi_2(t) - \Phi_1(t)$ os the difference between two solutions with slightly different initial conditions. For linear oscillations,
\[\Delta \Phi(t) = D\exp(-\beta t)\cos(\omega t - \delta)\]
In general:
\[\Delta \Phi(t) \sim K\exp(\lambda t)\]
$\lambda$ is the Lyapunov exponent, with periodic motion when it is negative and chaotic motion when it is positive. It is often best to plot $\log\abs{\Delta \Phi(t)} \sim \lambda t + \text{Const.}$ to see what happens with time.Doing so, we should either see a line with a positive or negative slope, depending on whether the trajectory is divergent (chaotic) or convergent (periodic) respectively. Plotting this for our two driven damped pendulums, we see:
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img5.png}
\end{center}
The peaks follow the linear trend, and the dips correspond to when $\Delta \Phi$ becomes negative. For driving of 1.07 (higher), we se that we still have convergence, but the Lyapunov exponent is less negative; it takes longer for the difference between the two osicllators to vanish.
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img6.png}
\end{center}

Ramping it up to a driving strength of 1.105, we get into the Chaotic regime, where the overall slope is positive:
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img7.png}
\end{center}
But bringing it up to 1.13, we actually go back to the non-chaotic regime:
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img8.png}
\end{center}

\subsubsection{Bifurcation Diagrams}
It gets quite confusing as to when the motion is chaotic, or not! A nice way of visualizing this is with a bifurcation diagram. We plot the driving strength on the $x$ axis and $\phi(t)$ on the y axis:
\begin{center}
    \includegraphics[scale=0.6]{Lecture-33/l33-img9.png}
\end{center}
Here, we only plot the phase at specific times. If the motion is periodic and we take pictures at specific intervals, we always end up at the same point (e.g. before $\gamma_1$). However, past $\gamma_1$, we get period doubling and hence taking snapshots of the pendulum at the original period, we will now see two periods. Past $\gamma_2$, we see 4 different values, and so on. Past $\gamma = 1.0845$ we get into the chaotic regime. There is one more subtlety; once we increase the driving strength past a certain point, the pendulum can roll over the top, so the angle can go to infinity; this is a bit inconvenient! Although we could make the phase $2\pi$ periodic, another fix is to just plot the velocity as  a function of the driving strength:
\begin{center}
    \includegraphics[scale=0.6]{Lecture-33/l33-img10.png}
\end{center}
We can see distint regions of periodicity and chaos; the circled part of $a$ was the period doubling we were studying earlier, $b$ is chaotic, then $c$ goes back to regular/periodic motion, then it gets chaotic for a while $d$, then we have regular motion for a while $e$ and so on. These diagrams can be quite useful to see these regions of chaos and regularity.

\subsubsection{State Space Orbits}
Very similar to phase space diagrams we did with Hamiltonian mechanics, but now we plot $\dot{\phi}$ versus $\phi$. Looking at this plot letting it run for a little while, we can see that there are four different cycles. (4 different trajectories); we are in a period 4 scenario.
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img11.png}
\end{center}
Going to 1.0826, we see that after the initial transience, looking very carefully we have a period 8 scenario:
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img12.png}
\end{center}
And for 1.087 we get period 16 and so on.
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img13.png}
\end{center}
As we increase the driving strength even further, the motion becomes truly aperiodic.But, increasing it back to 1.1, we get back to a periodic region (where the pendulum starts to roll over):
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img14.png}
\end{center}
And increasing it further, the motion gets chaotic again:
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img15.png}
\end{center}
\subsubsection{Poincare Sections}
We can plot a point per cycle to get a Poincare section:
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img16.png}
    \includegraphics[scale=0.7]{Lecture-33/l33-img17.png}
\end{center}
So for the 4-period case, we expect to see 4 dots, which is indeed the case (though two are close together in the below plot):
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img18.png}
\end{center}
Pictured below is a strange attractor:
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img19.png}
\end{center}
This is actually a fractal; fractals have scale invariance/self-similarity.

\subsubsection{The Logistic map}
The logistic map is defined as $x_{t+1} \mapsto rx_t(1 - x_t)$, describing the reproduction and starvation of a population. Looking a the bifurcation diagram as we vary $r$, we see a similar development of chaotic behavior as the pendulum, due to the nonlinearity:
\begin{center}
    \includegraphics[scale=0.7]{Lecture-33/l33-img20.png}
\end{center}

\end{document}


\documentclass[thesis.tex]{subfiles}

\begin{document}
\chapter{Background and Related Work}
The field of neuroinformatics, despite its recent emergence, has burgeoned rather rapidly, has produced and adopted a wide variety of techniques that can be used for performing white matter integrity and structure research. Since my work has greatly relied on the ideas and methods used in CoBundleMAP and within the FBA, in this chapter an overview of these approaches will be given, together with my reflections on this particular topic.

%=======================================================================================================
\section{Diffusion weighted MRI}
%------------------------------------------------------------------------------------
\subsection{General information}
Diffusion weighted imaging (DWI) is a well-established, noninvasive method of acquiring information about the structure of biological tissues. It is based on the fact that in unrestricted mediums water movement is absolutely random (Brownian movement), while restricted areas force molecules to diffuse in specific directions. These two possibilities are called isotropic diffusion and anisotropic diffusion respectively. Diffusion weighted MRI is capable of measuring the amount of such water motion along any chosen axis. To define the direction of the diffusion in three-dimensional space, observations are made by applying diffusion gradient pulses along several non-collinear orientations \cite{ChanraudDTI}.
Anisotropic water diffusion inside of biological tissue is explained as the result of molecular movement being restricted by specific barriers such as cell membranes or axon fibres \cite{diffusion1990Moseley}. Specifically in the brain white matter fiber tracts, that consist of axon bundles, the presence of myelination forces water to diffuse preferentially along axonal fiber directions \cite{WMdiffusion} \cite{Mori1999DiffusionMR}. Existence of such dominant direction is the basis for a wide variety of techniques. Based on diffusion tensor analysis, white matter tractography and calculation of various characteristic metrics can be executed \cite{Basser1995InferringMF}. Among such measures are Fractional Anisotropy, Mean Diffusivity, Axial Diffusivity and Radial Diffusivity, all of which are used in existing implementation of CoBundleMap pipeline, and will be discussed further in more details.

DWI allows us to perform a noninvasive analysis of the micro-structure of tissue, which is extremely useful in cases of white matter research, where possibility of in vivo investigation is of significant value. DWI has significantly contributed to our understanding of the human brain. The possibility to examine the white matter bundle structures and, when present, find pathological abnormalities has helped us comprehend various neurological diseases and disorders \cite{dwiDiseases, ChanraudDTI, dtiGeneralGood}, as well as effects of aging on human brain \cite{moseleyAging} and mechanisms of neuroplasticity \cite{Tournier2004FODdeconv, dwiDiseases}.

%------------------------------------------------------------------------------------
\subsection{Diffusion tensor imaging}
Diffusion Tensor Imaging (DTI) is a quantitative imaging method that provides the possibility to infer brain micro-structure properties based on data acquired through DWI \cite{BasserHow}. Common uses of the DTI include: white matter fiber tractography, brain connectivity analysis, and assessment of parameters that describe integrity and general state of both white and grey matter.

% \begin{figure}
% \centering
% \includegraphics[width=14cm,height=14cm,keepaspectratio]{thesis_radomskyi/images/dti-scheme-cite-Basser1995InferringMF.png}
% \caption{\textbf{DTI Scheme.} This scheme depicts the basic steps involved in diffusion tensor imaging approach. The output of DTI for each processed voxel is a 3x3 diffusion tensor and a $T_2$-weighted scalar A(0) \cite{Basser1995InferringMF}.}
% \label{fig:dti-scheme}
% \end{figure}

The main idea of this approach lies in estimating a symmetric and positive definite tensor $D_{eff}$ -- effective diffusion tensor, within each voxel. This is performed by selecting a local orthogonal coordinate system and calculating three corresponding diffusion coefficients in these directions \cite{Basser1994DTI}. The diffusion tensor, being a symmetric 3 $\times$ 3 matrix, can be described by three positive eigenvalues ($\lambda_1, \lambda_2, \lambda_3$) and three orthogonal eigenvectors ($e_1, e_2, e_3$). These eigenvalues represent the magnitude of diffusion within each given voxel, while eigenvectors reflect the corresponding directions. Such tensor may be interpreted as a mathematical representation of three-dimensional ellipsoid \cite{Basser1995InferringMF}, describing diffusion characteristics in individual voxels (Figure \ref{fig:dti-scheme}).

\begin{figure}
\centering
\includegraphics[width=14cm,height=14cm,keepaspectratio]{thesis_radomskyi/images/dti-scheme-cite-Basser1995InferringMF.png}
\caption{\textbf{DTI Scheme.}}
\label{fig:dti-scheme}
\end{figure}

From the parameters that are contained in DT, a set of scalar quantities that measure fundamental features of diffusion within tissues can be extracted  \cite{Basser1995InferringMF, Basser1996FA}.

The most basic of such metrics is eigenvalue average, or trace of DT, referred to as Mean Diffusivity (MD). It is rotationally invariant and reflects the magnitude of overall diffusion within the voxel:
\[ MD = \frac{tr(D)}{3} = \frac{\lambda_1 + \lambda_2 + \lambda_3}{3} \]
Higher values of MD are indicators of isotropic diffusion (possible axon degeneration or demyelination), while lower MD is primarily observed in regions of anisotropic diffusion (high myelination, dense axonal packing). Another rotationally invariant measure extracted from TD is FA. It describes the degree of orientational preference within a voxel, or in other words, amount of anisotropy, and is often used as a measure of white matter integrity. In relation to DT eigenvalues, it defines the extent to which one eigenvalue prevails over other two and is computed based on MD:
\[ FA = \sqrt{\frac{(\lambda_1 - MD)^2 + (\lambda_2 - MD)^2 + (\lambda_3 - MD)^2}{2(\lambda_{1}^2 + \lambda_{2}^2 + \lambda_{3}^2)}} \]

Existing implementation of CoBundleMAP also performs computation of two more DT-derived metrics related to analysis of white matter pathologies \cite{dtiGeneralGood} -- Axial Diffusivity (AD) and Radial Diffusivity (RD). AD reflects axonal integrity and \cite{RDmyelination} equals to the value of the largest eigenvalue:
\[AD = \lambda_1\]
while RD is defined by secondary eigenvalues and reflect myelin integrity \cite{RDmyelination}:
\[RD = \frac{\lambda_2 + \lambda_3}{2}\]
Though it is worth mentioning that interpretation of AD and RD should be performed with care and preferably in combination with other DTI-derived features \cite{rd-ad-crit}.
Nevertheless, analysis of diffusion measures and their correlations is a reliable method of gaining valuable insights about the condition of white matter tissue. For example, simultaneous rise in FA combined with lower RD is a sign of dense axonal packing or high myelination, while the mirrored case -- low FA and high RD are signals of either axonal degeneration or demyelination \cite{fa-fd-correlation}.
%------------------------------------------------------------------------------------
\subsection{Voxel based analysis}

Voxel-based analysis (VBA) is a technique used for voxel-wise analysis of diffusion images across a given population of subjects, capable of identifying local differences and correlations in the structure of brain tissue. It is an important and well-established approach, permitting comparison of the voxels between the same areas of different brains, which in case of significant differences can reveal the presence of a degenerative disease. The essential principle on which VBA is based, is retrieval of anatomical correspondences between the subjects by spatially normalizing images to s a common stereotactic space (usually a brain atlas), using image registration algorithms \cite{vbaMechelli}. Image registration commonly consists of two distinct steps, which sequentially increase similarity between the input and reference images. First one is computation of the linear transformation, including estimation of the rigid-body transformation, used for rotation and translation of the image, followed by global scaling and skewing. This allows to perform correction of differences in the brain size across the population as well as revision of possible orientation variations. Second step accounts for global nonlinear differences and involves estimation of the coefficients of the basis function, minimizing the residual sum of squared differences between them and simultaneously maximizing the smoothness of the deformation \cite{vbaAshburner}. Deformation field, derived from image registration, contains information about adjustments made for matching voxels of the input image and the template. One of the important outcomes of the image spatial normalization, is that exact voxel-wise volume fluctuations can be computed from extracted deformation fields as Jacobian determinants. Analysis of Jacobian determinants as well as the deformation fields themselves establishes what is known as tensor-based morphometry or deformation-based morphometry \cite{vbaKurth}.

\subsection{Constrained spherical deconvolution}
Spherical deconvolution is an approach used for estimations of the fiber orientation distribution (FOD) function within the voxels of diffusion-weighted images. Introduction of this approach allowed to perform identification of multiple fiber orientations within a single voxel, as shown on Figure \ref{fig:sd-is-good}. It is based on several assumptions about the nature of diffusion inside white matter tissue and implies that it is possible to approximate the measured diffusion-weighted signal as a sum of the signals emitted by each of the orientationally distinct fiber population that is present in the sample \cite{csd1TOURNIER}. Therefore, the signal measured from a single fiber population is expressed as a symmetric response function $R(\theta)$, where $\theta$ is elevation angle in spherical coordinates, while the signal originating from multiple incoherently aligned populations is defined as a sum of such functions, aligned with respect to their orientation (via azimuthal angle $\phi$ in spherical coordinates) and weighted by the share of volume ($f_{i}$), that each population occupies within the voxel \cite{csd1TOURNIER}: \[S(\theta, \phi) = \sum_{i}f_{i}A_{i}R(\theta)\]
Here, $A_{i}$ is an operator of rotation onto direction ($\theta_{i}, \phi_{i}$). The signal can be then subsequently interpreted as a convolution over spherical coordinates of the FOD function and response function:
\[S(\theta, \phi) = F(\theta, \phi) \otimes R(\theta)\]
FOD estimation can then be seen as a problem of inferring the distribution from the measured signal, given a suitable response function \cite{csdDellAcqua2019}. Constrained spherical deconvolution (CSD) is a further development of these ideas. The main improvement of CSD over simple spherical deconvolution is the introduction of a non-negativity constraint on the values in the computed FOD, leading to elimination of the noise caused by high angular frequencies \cite{dwi2fod2-csd}.

\begin{figure}
\centering
\includegraphics[width=14cm,keepaspectratio]{thesis_radomskyi/images/sd-is-good.png}
\caption{\textbf{Comparison of DTI and SD.} Diffusion tensor (left) is capable of describing the average diffusion profile in each image voxel, thus capturing only dominant fiber population. Spherical deconvolution (right), on the other hand, is capable of identifying multiple populations. \cite{csdDellAcqua2019}.}
\label{fig:sd-is-good}
\end{figure}

%=======================================================================================================
\section{Fixel based analysis}
Although VBA of diffusion MRI, namely analysis of tensor-derived FA values, is a rather common approach to researching structure of white matter in the brain, it has some restrictions. Since FA, as well as other diffusion measures, is computed as a value averaged within the whole voxel, it doesn't account for the fact that most white matter voxels (around 90\% as reported in \cite{crossingFibers2013}) contain several crossing fibres. Inside such voxels FA is affected by all fiber populations that are present, thus producing values which do not reliably represent any underlying tract. Figure \ref{fig:percent-of-non-dominant-volume-fraction} shows statistics of affected white matter voxels and degree of their contamination with non-dominant fibre orientations.
This could not only affect precision of the whole system, but also prevents us from a more robust and specific analysis of individual fibres.
A possible solution for this problem could be utilization of diffusion MRI mixture models, which are able to recognise multiple fibre populations within a single voxel. A common name for such a fibre population is \textit{fixel} \cite{aboutFixels2015Raffelt}, while commodity of methods that are using this approach is called Fixel-based analysis (FBA). In this section I will make an overview of papers introducing fixel-based approach to white matter analysis, as well new metrics for measuring nervous fibre integrity, density and thickness.

\begin{figure}
\centering
\includegraphics[width=10cm,height=10cm,keepaspectratio]{thesis_radomskyi/images/percent-of-non-dominant-volume-fraction-cite-crossingFibers2013.png}
\caption{\textbf{Non-dominant Volume Fraction of White Matter Voxels} Histogram depicting non-dominant volume fraction measured by CSD over all white matter voxels \cite{crossingFibers2013}}
\label{fig:percent-of-non-dominant-volume-fraction}
\end{figure}

\subsection{Fiber density and cross-section}
Investigation of white matter tissue is closely related to estimation of its integrity, ability to connect different regions of the brain and transfer information. The extent to which separate bundles contribute to such connectivity may be characterised by the total number of axons within the bundle, their thickness and degree of myelination. One of the proposed metrics, that is capable of measuring these properties of white matter tracts is Apparent Fiber Density (AFD) \cite{afd2012Raffelt}. This metric introduces a novel approach to computation of fiber-specific parameters, allowing to perform analysis of individual fiber populations in voxels that contain multiple tracks, by leveraging the information, obtained within image spatial normalization process. The important development of this approach is Fiber density cross-section (FDC) metric \cite{fdcAndFBA2017Raffelt}. This method further improves the ideas of AFD and defines FDC as a measure of white matter integrity, consisting of two related values -- Fiber density (FD), which is basically AFD, and Fiber cross-section (FC), which describes local volumetric differences within the examined population. Both metrics, similar to AFD, are based on information, contained within deformation fields which are produced by image registration. In addition to allowing computation of fiber-specific parameters, approach adopted within FBA also allows to derive better insights about the structure and integrity of the tissue. As can be observed from the Figure \ref{fig:difference-fd-fc-fdc}, new metrics allow to distinguish between different types of intra-axonal volume changes, providing important information for deeper analysis of underlying causes.

\begin{figure}
\centering
\includegraphics[width=10cm,height=10cm,keepaspectratio]{thesis_radomskyi/images/difference-fd-fc-fdc.jpg}
\caption{\textbf{Different types of intra-axonal volume reduction} By performing analysis of both FC and FD values we may achieve better insights about underlying causes\cite{fdcAndFBA2017Raffelt}}
\label{fig:difference-fd-fc-fdc}
\end{figure}

\section{CoBundleMAP}
Since the main objective of this work is to further extend CoBundleMAP, it is essential to understand its inner mechanisms and the principles upon which it is built. BundleMAP and CoBundleMAP are dMRI image processing tools, developed within the Department of Computer Science, University of Bonn. Results obtained through these pipelines contain, among other things, feature vectors that can be further utilized in machine learning and data processing algorithms, or used for visual analysis of specific white matter areas. Following subsections contain a short overview of steps performed and algorithms used within the scope of CoBundleMAP pipeline. Since CoBundleMAP is an improved, two-dimensional manifestation of BundleMAP, I will start with the overview of ideas implemented in the latter.

\subsection{BundleMAP overview}
BundleMAP tries to solve the problem of deriving features, which are specific for an interpretive evaluation of different degenerative white matter diseases, as well as performing research of the tissue structure. The main principle of BundleMAP is to combine joint parametrization and manifold learning for extracting bundle-specific duffusivity measures \cite{Khatami2017BundleMap}. Joint parametrization can be described as a process of finding anatomical correspondences between specific brain regions among given population of patients. If we know such unambiguous areas, we can perform localised comparisons between subjects and find areas of interest for each given application case. Features that are extracted contain such diffusion measures as FA, MD, RD and AD.

The pipeline can be logically split into four distinct steps (Figure \ref{fig:bundlemap-steps}): fiber tractography, followed by supervised and unsupervised outlier removal, manifold learning, and feature extraction. Next subsections contain more specific information and explanations about each performed step.

\begin{figure}
\centering
\includegraphics[width=14cm,height=14cm,keepaspectratio]{thesis_radomskyi/images/bundlemap_steps.jpg}
\caption{\textbf{BundleMAP Main Steps.} BundleMAP consists of four main steps: white matter fiber tractography, outlier removal, joint parametrization, and computation of spatially localised features along the BundleMAP coordinate \cite{Khatami2017BundleMap}}
\label{fig:bundlemap-steps}
\end{figure}

\subsubsection{Image preprocessing and tractography}
The first part of the BundleMAP processing pipeline consists of each subject's DWI data processing, extraction of diffusivity measures, and performing tractography for a set of predefined white matter tracts. These steps rely on principles adopted in VBA, namely analysis of subject population by performing registration towards a common template. In the case of BundleMAP, registration is performed by using Montreal Neurological Institute atlas, which represents an average, healthy human brain \cite{mniAtlasReference}. This introduces the presence of two different spaces -- one that is specific for the each of the subjects being processed (subject space), and the subject-agnostic MNI space. One of the advantages of using this approach, is the ability to predefine regions of interest in the space which is independent from the examined population, and then perform automatical mapping of these areas into the subject space.
The first data processing step within the pipeline is computation of fiber orientation distributions of the input dMRI images, using the spherical deconvolution algorithms \cite{dwi2fod2-csd, dwi2fod2-msmt-csd}, present within MRtrix \cite{mrtrixGeneral2019} and FSL \cite{fsl1, fsl2} software packages. This is followed by extraction of diffusion measures and using a registration algorithm to calculate two non-linear warp fields, which establish the spatial correspondence between the image and the MNI template. These steps produce all the data required for performing white matter tractography. Thereafter tractography is performed in the subject space using MRtrix \cite{mrtrixGeneral2019} implementation of the iFOD2 algorithm \cite{tckgen} for probabilistic streamlines tractography.

\subsubsection{Outlier removal}
Execution of voxelwise statistical analysis on a set of dMRI images is prone to multiple problems due to the imperfections of data and deficiency of registration and tracking algorithms \cite{voxelwiseImperfections}. In order to improve tracking data resulting from the first pipeline step, two strategies are implemented within the CoBundleMAP. The first one is based on the knowledge of the anatomy of human brain. Since the analysis is performed on a known set of white matter bundles, information about natural constraints of such bundles is applied to cut off any fibers that do not comply with predefined spacial thresholds. Afterwards all remaining fibers from all subjects are combined and processed using a one-class support vector machine (SVM), which identifies the smallest possible area in the input space that contains majority of the samples. Additionally, a parameter $\nu \in (0, 1]$ is specified, controlling the fraction of samples to be discarded \cite{Khatami2017BundleMap}.

\subsubsection{Manifold learning}
Computation of the joint fiber bundles parametrization across the whole examined population allows to perform a robust analysis by leveraging the inter-subject spatial correspondence of computed features within specific white matter tracts.
BundleMAP is based on the idea of representing individual white matter tracts as special cases of a generalized, abstract fiber bundle. Each fiber bundle core is thus regarded as a one-dimensional manifold, which then could be acquired through manifold learning techniques \cite{Khatami2017BundleMap}, such as ISOMAP \cite{isomapTenenbaum}. Fibers belonging to the same bundle are first warped to the MNI space using nonlinear transformations computed during image registration. In order to produce a \textit{joint} parametrization, ISOMAP algorithm needs to be applied to the accumulated data from the tracts of the whole population. Since the combined number of vertices across all subjects is too big for distance matrix calculation, BundleMAP first selects a set of representative streamlines, which will be used for manifold learning by performing \textit{k-means} clustering on tract fibers. Afterwards, vertices which belong to the selected streamlines are used for ISOMAP input. The values for the vertices that were excluded from the manifold computations, are interpolated as a weighted average of \textit{k} nearest vertices belonging to the representative fibres in MNI space \cite{Khatami2017BundleMap}. An example of the joint parametrization, produced by BundleMAP for the left corticospinal tract can be seen on Figure \ref{fig:bundlemap-parametrisation-results}.

\begin{figure}
\centering
\includegraphics[width=14cm,height=14cm,keepaspectratio]{thesis_radomskyi/images/bundlemap-parametrisation-results.jpg}
\caption{\textbf{BundleMAP Joint Parametrization Results.} Image depicting results of joint parametrization of the left corticospinal tract, performed with BundleMAP. As can be seen, corresponding anatomical locations of the tract have matching colors, indicating correct parametrization \cite{Khatami2017BundleMap}}
\label{fig:bundlemap-parametrisation-results}
\end{figure}

\subsubsection{Feature extraction}
The streamlines and respective vertex values, produced via manifold learning, are represented as a function of ISOMAP coordinate, but since white matter bundles are of different size and proportions across the subjects, the range of ISOMAP coordinate is also varying within the population. In order to achieve a universal correspondence between the tracts, originating from distinct diffusion images, ISOMAP coordinate is normalised to the $[0,1)$ range. This allows to perform the analysis of extracted metrics along the reciprocal parts of the bundles, independent from dissimilarities of their initial size and form. BundleMAP pipeline produces feature vectors by partitioning the whole coordinate range into $n$ equal bins and assigning them an average of computed metrics values belonging to the corresponding segment of the tract.

\subsection{CoBundleMAP overview}
As mentioned before, CoBundleMAP is based on the same principles as the BundleMAP, but contains several important modifications that allow to considerably improve parameterization results. The most important changes are outlined in the corresponding source \cite{Khatami2019CoBundleMap}, in this subsection a short overview is given based on original paper and personal experience with the pipeline.

\subsubsection{Hemisphere correspondence}
One of the substantial differences of between CoBundleMAP and previous approach is computation of inter-hemisphere correlations between corresponding white matter bundles of individual subjects. This allows to achieve additional coherence between left and right parts of the same tract. This allows to significantly improve joint parametrization and observe additional asymmetries, leading to deeper insights about tissue structure and affected white matter bundles. To achieve this, CoBundleMAP computes registration of the points in the left tract to the corresponding points of the right tract by treating the vertices of the streamlines as point clouds and applying an iterative closest point approach \cite{Khatami2019CoBundleMap}. Consequently, the manifold learning is performed using streamlines from both hemispheres, producing an idealistic white matter bundle representation, expressing left and right tracts together. This is different from the approach used by BundleMAP, where each produced manifold was a manifestation of either right or left tract.

\subsubsection{Bundle parameterization}
Another prominent improvement of the new pipeline is the way how bundle processing and manifold learning are performed. CoBundleMAP is able to produce two-dimensional parameterization by setting the dimensionality of ISOMAP output. This allows to investigate changes of extracted diffusion parameters along two axis, distinguishing even more specific correlations. CoBundleMAP additionally improves the
mechanism of ISOMAP coordinate normalization. Instead of directly re-scaling produced result, the ranges of produced ISOMAP coordinates are first divided into discrete sections. Within these sections, number of contributing vertices is computed and compared against a predefined threshold, thus eliminating the sections that do not have enough support. This approach essentially trims the bundle ends, which tend to contain abnormally lower averaged metric values due to low sample count.

%\label{ch:background}

\end{document}


%###############################################################
%\section{Theory of Globally Controlled Polyomino Assembly}\label{sec:Theory}
\section{Theory: Polyomino Assembly by Global Control}\label{sec:Theory}
%###############################################################

This section explains how to design factories that build arbitrary-shaped 2D polyominoes.
 We first assign species to individual tiles of the polyomino, second discover a build path, and finally build an assembly line of factory components that each add one tile to a partially assembled polyomino and pass the polyomino to the next component.

\subsection{Model}\label{subsec:model}
Assume the following rules:
%\begin{enumerate}
1.) A planar  grid \emph{workspace} $W$ is filled with a number of unit-square particles (each occupying one cell of the grid)  and some fixed unit-square blocks.  Each unit square in the workspace is either  \emph{free}, which a particle may occupy or \emph{obstacle} which a particle may not occupy.  Each square in the grid can be referenced by its Cartesian coordinates $\bm{x}=(x,y)$.
2.) All particles are commanded in unison: the valid commands are  ``Go Up" ($u$), ``Go Right" ($r$), ``Go Down" ($d$), or ``Go Left" ($l$).
3.) Particles all move until they  hit an obstacle, hit a stationary particle, or share an edge with a compatible particle.
	%\begin{enumerate}
%		\item hit an obstacle
%		\item hit a stationary particle
%		\item share an edge with a compatible particle
%	\end{enumerate}
If a particle shares an edge with a compatible particle the two particles bond and from then on move as a unit.
This letter uses \emph{cycles} of movement commands in the order $\langle r,d,l,u \rangle$. We assume the area of $W$ is finite and issue each command long enough for the particles to reach their maximum extent.
%\end{enumerate}

%###############################################################
\subsection{Arbitrary 2D shapes require two particle species}\label{subsec:RobotSpecies}
%###############################################################
Polyominoes have \emph{four-point connectivity}: a 4-connected square is a neighbor to every square that shares an edge with it.

\begin{lemma}
  Any polyomino can be constructed using just two species
  \end{lemma}
\begin{proof}
Label a grid with an alternating pattern like a checkerboard.  Any desired polyomino can be constructed on this checkerboard, and all joints are between dissimilar species.
  An example shape is shown in Fig.~\ref{fig:Grid}. Red and blue colors are used to indicate particles of different species.
  \end{proof}

   \begin{figure}
   \centering
   \vspace{0.2em}
\begin{overpic}[width =.8\columnwidth]{Grid2.pdf}
\end{overpic}
\caption{\label{fig:Grid}Any polyomino can be constructed with two compatible robot species, shown here with red and blue tiles.
}
\end{figure}

  The sufficiency of two species to construct any shape gives many options for implementation.  The two species could correspond to any gendered connection,
including ionic charge, magnetic polarity, or hook-and-loop type fasteners. Large populations of these two species can then be stored in separate hoppers and, like two-part epoxy,  only assemble when dissimilar particles come in contact.

%###############################################################
\subsection{Complexity Handled in This Letter}\label{sec:ComplexityHandled}
%###############################################################

2D part geometries vary in difficulty.  Fig.~\ref{fig:IncreasingDifficulty} shows parts with increasing  complexity.

   \begin{figure}
   \centering
\begin{overpic}[width =\columnwidth]{IncreasingDifficulty3.pdf}
\end{overpic}\vspace{-2em}
\caption{\label{fig:IncreasingDifficulty}Polyomino parts. Assembly difficulty increases from left to right.
}
\end{figure}
Label the first particle in the assembly process the \emph{seed particle}.
 Part 1 is shaped as a `\#' symbol.  Though it has an interior hole, any of the 16 particles could serve as the seed particle, and the shape could be constructed around it.  The second shape is a spiral, and must be constructed from the inside-out.  If the outer spiral was completed first, there would be no path to add particles to finish the interior because added particles would have to slide past compatible particles.  Increasing the number of species would not solve this problem, because there is a narrow passage through the spiral that forces incoming parts to slide past the edges of all the bonded particles.
The third shape contains a loop, and the interior must be finished before the loop is closed.
Shape 4 is the combination of a left-handed and a right-handed spiral.
Adding one particle at a time in 2D cannot assemble this part, because each spiral must be constructed from the inside-out.
 Instead, this part must be divided into sub-assemblies that are each constructed, and then combined.
 Shape 5 contains compound overhangs, and may be impossible to construct with additive 2D manufacturing using only two species.
 The algorithms in this letter detect if the desired shape can be constructed one particle at a time.
 If so, a build order is provided, and a factory layout is designed.

% A polyomino is said to be \emph{column convex} if each column has no holes. Similarly, a polyomino is said to be row convex if each row has no holes. A polyomino is said to be \emph{convex} if it is row and column convex.
%
%\begin{lemma}\label{lemma:convexonjectsCanbeConstructedAdditively}
%Any convex polyomino can be constructed by adding one particle at a time
%\end{lemma}
%\begin{proof}
%Select any pixel as the \emph{seed block}, or root node.  Perform a breadth-first search starting at the seed block, labelling each block in the order they are expanded.  Constructing the shape according to the ordering ensures that the polyomino is convex at every step of construction.
%\end{proof}

%The proof of \ref{lemma:convexonjectsCanbeConstructedAdditively} assumes the existence of fixtures for assembly.
%\todo{describe fixtures for adding one particle at a time}

%Some non-convex polynominos cannot be constructed one particle at a time, as illustrated in Fig. ~\ref{fig:IncreasingDifficulty}.    For instance, a polynomino consisting of a clockwise and a counterclockwise square spiral, joined at the ends with a gap of one unit between the spirals must be constructed by first assembling each spiral, and then combining the sub assemblies.

%###############################################################
\subsection{Discovering a Build Path}
%###############################################################

Given a polyomino, Alg.~\ref{alg:FindBuildPath} determines if the polyomino can be built by adding one component at a time.
 The  problem of determining a build order is difficult because there are $O(n!)$ possible build orders, and many of them may  violate the constraints given in Section \ref{subsec:model}.
 Each new tile must have a straight-line path to its goal position in the polyomino that does not collide with any other tile, does not slide past an opposite specie tile, and terminates in a mating configuration with an opposite specie tile.
However, as in many robotics problems, the inverse problem of deconstruction is easier than the forward problem of construction.

\begin{algorithm}
\newcommand\algotext[1]{\end{algorithmic}#1\begin{algorithmic}[1]}
%\begin{algorithmic}[1]
%\scriptsize
\caption{\sc {FindBuildPath}($\mathbf{P})$   \label{alg:FindBuildPath}}
$\mathbf{P}$ is the $x,y$ coordinates of a 4-connected polyomino. % that has at least a 1-tile empty border.
Returns $ \mathbf{C} $, $ \mathbf{c} $ and $\mathbf{m}$ where $ \mathbf{C} $ contains sequence of polyomino coordinates, $ \mathbf{c} $ is a vector of color labels, and $\mathbf{m}$ is a vector of directions for assembly.
\begin{algorithmic}[1]

\State\hbox{$ \mathbf{c}\leftarrow${\sc{LabelColor}}($\mathbf{P}$)}
\State $\{\mathbf{C},\mathbf{m} \}= ${\sc {Decompose}}$(\mathbf{P},\mathbf{c})$
\State \Return $\{ \mathbf{C},\mathbf{c}, \mathbf{m} \} $
\end{algorithmic}
\end{algorithm}

   \begin{figure}
   \centering
\begin{overpic}[width =\columnwidth]{DeconstructionOrderMattersSlide.pdf}
\end{overpic}\vspace{-2em}
\caption{\label{fig:DeconstructionOrderMatters} Deconstruction order matters if loops are present.  Loops occur when the 8-connected freespace has more than one connected component.  In the top row the green tile is removed first, resulting in a polyomino that cannot be decomposed. However, if the bottom right tile is removed first, deconstruction is possible.
}
\end{figure}

Alg.~\ref{alg:FindBuildPath}  first assigns each tile in the polyomino a color, then calls the recursive function {\sc {Decompose}}, which returns either a build order of polyomino coordinates and the directions to build, or an empty list if the part cannot be constructed.
{\sc {Decompose}} starts by calling the function {\sc {Erode}}.  {\sc {Erode}} first counts the number of components in the 8-connected freespace. An 8-connected square is a neighbor to every square that shares an edge or vertex with it. If there is more than one connected component, the polyomino contains loops.
 {\sc {Erode}} maintains an array of the remaining tiles in the polyomino $\mathbf{R}$.
 In the inner \textit{for loop} at line  \ref{alg:line:forloopTotryremovinEachTileERODE}, a temporary array $\mathbf{T}$ is generated that contains all but the $j$th tile in $\mathbf{R}$ sorted by the number of neighbors so a tile with one neighbor is checked before tiles with two or three.
This \textit{for loop} simply checks (1) if the $j$th tile can be removed along a straight-line path without  colliding with any other particle or sliding past an opposite specie tile in line \ref{alg:line:checkpathtileERODE},  (2) that its removal does not fragment the remaining polyomino into more than one piece in line \ref{alg:line:NumConnectedCompERODE}, and (3) that its removal does not break a loop in line \ref{alg:line:Num8ConnectedCompERODE}.
If no loops are present, this algorithm requires at most  $n/2 (1 + n)$ iterations, because there are $n$ particles to remove, and each iteration considers one less particle than the previous iteration.

Polyominoes with loops require care, because decomposing them in the wrong order can make disassembly impossible, as shown in Fig.~\ref{fig:DeconstructionOrderMatters}.
If loops exist then  {\sc {Erode}} may return only a partial decomposition, so {\sc {Decompose}} must then try every possible break point and recursively call {\sc {Decompose}} until either a solution is found, or all possible decomposition orders have been tested.  The worst-case number of function calls of  {\sc {Decompose}}  are proportional to the factorial of the number of loops, $O( |\text{\sc 8-ConnComp}(\neg\mathbf{P})| !)$. Though large, this is much less than $O(n!)$.

\begin{algorithm}
\newcommand\algotext[1]{\end{algorithmic}#1\begin{algorithmic}[1]}
%\scriptsize
\caption{\sc {Erode}($\mathbf{P},\mathbf{c})$   \label{alg:Erode}}
$\mathbf{P}$ is the $x,y$ coordinates of a 4-connected polyomino  and $ \mathbf{c} $ is a vector of color labels.
Returns $ \mathbf{R} $, $ \mathbf{C} $, $\mathbf{m}$, and $\mathbf{\ell}$ where $ \mathbf{R} $  is a list of coordinates of the remaining polyomino, $ \mathbf{C} $ contains sequence of tile coordinates that were removed,   $\mathbf{m}$ is a vector of directions for assembly, and $\mathbf{\ell}$ if loops were encountered. $\mathbf{d} \gets\{r,d,l,u\}$
\begin{algorithmic}[1]
%\State\hbox{$ \mathbf{c}\leftarrow${\sc{LabelColor}}($\mathbf{P}$)}
%\State $ \{ \mathbf{R},\mathbf{C}, \mathbf{m}, \ell \} \gets ${\sc {Erode}}$(\mathbf{P},\mathbf{c})$
\State\hbox{$\mathbf{C} \leftarrow \{\}, \mathbf{m} \leftarrow \{\}, \mathbf{\ell} \gets \textrm{\sc False},  \mathbf{R}\leftarrow \mathbf{P}$}
\State $w \gets |\text{\sc 8-ConnComp}(\neg\mathbf{R})|$

\While{$1 <  |\mathbf{R}|  $}
\State  \emph{successRemove} $\gets$ {\sc False}
\State\hbox{$ \mathbf{R}\leftarrow${\sc{Sort}}($\mathbf{R}$)} \Comment{sort by number of neighbors}
\For{$j\leftarrow 1, j \le  |\mathbf{R}| $}
\State $\mathbf{p} \gets \mathbf{R}_j,  \mathbf{T} \gets  \mathbf{R}  \backslash   \mathbf{R}_j$

\For{$ k \leftarrow 1, k \le  4$   \label{alg:line:forloopTotryremovinEachTileERODE} }
\If{{\sc CheckPathTile}($\mathbf{T},\mathbf{p}, \mathbf{d}_k, \mathbf{c}$) \label{alg:line:checkpathtileERODE} \textbf{and}
\\ \textbf{~~~~~~~~~~~~~~}
$1 = |\text{\sc 4-ConnComp}(\mathbf{T})|$  \label{alg:line:NumConnectedCompERODE}}
\If{$w = |\text{\sc 8-ConnComp}(\neg\mathbf{T})|$  \label{alg:line:Num8ConnectedCompERODE}}
\State $ \mathbf{R}\leftarrow   \mathbf{T}$, \emph{successRemove} $\gets$ {\sc True}
\State  $\mathbf{C}_{ 1+|\mathbf{R}|} \gets \mathbf{p},  \mathbf{m}_{ |\mathbf{R}|}  \gets \mathbf{d}_k$
\Else { $  \mathbf{\ell} \gets \textrm{\sc True}$}
\EndIf
\State \textbf{break}
\EndIf
\EndFor
\EndFor
\If {  \emph{successRemove} $=$ {\sc False}}
\State  \hbox{$\mathbf{C} \leftarrow \{\}, \mathbf{m} \leftarrow \{\}$}
\State \textbf{break}
\EndIf
\EndWhile
\If {$ |\mathbf{R}| = 1$}
\State  $\mathbf{C}_{ 1} \gets \mathbf{R}_1 $
\EndIf
\State \Return $\{ \mathbf{R},\mathbf{C}, \mathbf{m}, \ell \}$
\end{algorithmic}
\end{algorithm}

\vspace{10em}
\begin{algorithm}
\newcommand\algotext[1]{\end{algorithmic}#1\begin{algorithmic}[1]}
%\scriptsize
\caption{\sc {Decompose}($\mathbf{P},\mathbf{c})$   \label{alg:Decompose}}
$\mathbf{P}$ is the $x,y$ coordinates of a 4-connected polyomino and $ \mathbf{c} $ is a vector of color labels.
Returns $ \mathbf{C} $ and $\mathbf{m}$ where $ \mathbf{C} $ contains sequence of polyomino coordinates and $\mathbf{m}$ is a vector of directions for assembly. $\mathbf{d} \gets\{u,d,l,r\}$
\begin{algorithmic}[1]
\State $ \{ \mathbf{R},\mathbf{C}, \mathbf{m}, \ell \} \gets ${\sc {Erode}}$(\mathbf{P},\mathbf{c})$
\If {$|  \mathbf{R} | = 0 \textbf{ or } \neg \ell$}
\State \Return $\{ \mathbf{C},\mathbf{m} \}$
\EndIf
\For{$j\leftarrow 1, j \le  |\mathbf{R}| $}
\State $\mathbf{p} \gets \mathbf{R}_j,  \mathbf{T} \gets  \mathbf{R}  \backslash   \mathbf{R}_j$
\For{$ k \leftarrow 1, k \le  4$   \label{alg:line:forloopTotryremovinEachTileDecompose} }
\If{{ ( \sc CheckPathTile}($\mathbf{T},\mathbf{p}, \mathbf{d}_k, \mathbf{c}$) \label{alg:line:checkpathtileDecompose} \textbf{and }
\\ \textbf{~~~~~~~~~~~~ }
$1 = |\text{\sc 4-ConnComp}(\mathbf{T})|$)  \label{alg:line:NumConnectedCompDecompose}}
\State $\{\mathbf{C2},\mathbf{m2} \}\gets ${\sc {Decompose}}$(\mathbf{T},\mathbf{c})$
\If {$\mathbf{C2}  \ne \{\}$}
%\State  $\mathbf{C}_{ 1+|\mathbf{R}|} \gets \mathbf{p},  \mathbf{m}_{ |\mathbf{R}|}  \gets \mathbf{d}_k$
\State $\mathbf{C}_{1:|\mathbf{C2}|+1} \gets \{\mathbf{C2},\mathbf{p}\}$
\State $ \mathbf{m}_{1:|\mathbf{m2}|+1} \gets \{\mathbf{m2},\mathbf{d}_k\}$
\State \Return $\{ \mathbf{C}, \mathbf{m} \}$
\EndIf
\State \textbf{break}
\EndIf
%\EndIf
\EndFor
\EndFor
%\State $\mathbf{C} \gets \{\}, \mathbf{m} \gets \{\}$
\State \Return $\{ \mathbf{C}\gets \{\}, \mathbf{m}\gets \{\} \}$
\end{algorithmic}
\end{algorithm}

%###############################################################
%\subsection{Assembling Tiles}
%###############################################################

%###############################################################
\subsection{Hopper Construction}\label{subsec:HopperConstruction}
%###############################################################
Two-part adhesives react when components mix.  Placing components in separate containers prevents mixing.  Similarly, storing many particles of a single specie in separate containers allows controlled mixing.
%WIKI: harden by mixing two or more components which chemically react.

We can design \emph{part hoppers}, containers that store similarly labelled particles.  These particles will not bond with each other.  The hopper shown in Fig.~\ref{fig:HopperCW} releases one particle every cycle. Delay blocks are used to ensure the $n$th part hopper does not start releasing particles until cycle $n$. For ease of exposition, this letter has a unique hopper for each tile position. This enables precise positioning of different materials, but a particle logic system could use just two hoppers, similar to our particle logic systems in [9].

   \begin{figure}
  %  \vspace{-1em}
   \centering
\begin{overpic}[width =\columnwidth]{hopperV4.pdf}
\end{overpic}\\ \vspace{-1em}
\caption{\label{fig:HopperCW}Hopper with five delays. The hopper is filled with similarly-labelled robots that will not combine.  Every clockwise command cycle releases one robot from the hopper.  %\textcolor{red}{replace with new hopper design}
}
\end{figure}

\begin{figure}
   \centering
\begin{overpic}[width =\columnwidth]{24tilefactory.pdf}
\end{overpic}
\begin{overpic}[width =\columnwidth]{Spiraltilefactory.pdf}
\end{overpic}\\ \vspace{-1em}
\caption{\label{fig:24Tilefactory}A twenty-four tile factory, step 82 for a `\#' shape and a twenty-one tile factory, step 66 for a spiral (zoom in for details in this vector graphic).
}
\end{figure}

%###############################################################
\subsection{Part Assembly Jigs}\label{subsec:PartAssemblyJigs}
%###############################################################

Assembly is an iterative procedure.
A factory layout is generated by  {\sc{BuildFactory}}($\mathbf{P}, n_c$), described in Alg.~\ref{alg:BuildFactory}. This function takes a 2D polyomino $\mathbf{P}$ and, if $\mathbf{P}$ has a valid build path, designs an obstacle layout to generate $n_c$ copies of the polyomino. A polyomino is composed of $|\mathbf{P}| = n$ tiles.

For each tile, the function
 {\sc{FactoryAddTile}} $(n_c,\mathbf{b}, m,C, c,w)$
  described in  Alg.~\ref{alg:FactoryAddTile}
is called to generate an obstacle configuration $\mathbf{A}$.
$\mathbf{A}$  forms a hopper that releases a particle each iteration and a chamber that temporarily holds the partially-assembled polyomino $\mathbf{b}$ and guides the new particle $C$ to the correct mating position. A 24-tile factory is shown in  Fig.~\ref{fig:24Tilefactory}.

%\todo{Sheryl, add the algorithmic environment for Build Factory}
\begin{algorithm}
\newcommand\algotext[1]{\end{algorithmic}#1\begin{algorithmic}[1]}
%\scriptsize
\caption{ \sc{BuildFactory}($\mathbf{P}, n_c$)\label{alg:BuildFactory}}
$\mathbf{P}$ is the $x,y$ coordinates of a 4-connected polyomino.  $n_c$ is the number of parts desired.
Returns a two dimensional array $ \mathbf{F} $ containing the factory obstacles and filled hoppers.
\begin{algorithmic}[1]
\State$\mathbf{F} \leftarrow \{\}$ \Comment{the factory obstacle array}

\State \{$\mathbf{C},\mathbf{c}, \mathbf{m}$\} $  \leftarrow$ {\sc{FindBuildPath}}($\mathbf{P}$)
 \If{$ \{\} = \mathbf{m}$}
 \State \Return  $ \mathbf{F} $
 \EndIf
 \State$\{ \mathbf{A}, \mathbf{b} \}\leftarrow${\sc{FactoryFirstTile}}$(n_c, \mathbf{c}_i,w)$
 \For{$i\leftarrow 2, i \le  |\mathbf{c}| )$}
 \State$\{\mathbf{A},\mathbf{b}\}\leftarrow${\sc{FactoryAddTile}}$(n_c,\mathbf{b}, \mathbf{m}_{i-1},\mathbf{C}_i, \mathbf{c}_i,w)$
 \State$ \mathbf{F} \leftarrow${\sc{ConcatFactories}}$(\mathbf{F},\mathbf{A})$
\EndFor
\State \Return  $ \mathbf{F} $
%\State{\sc{DisplayFactory}}($factoryLayout$)
\end{algorithmic}
\end{algorithm}

\begin{algorithm}
\newcommand\algotext[1]{\end{algorithmic}#1\begin{algorithmic}[1]}
%\scriptsize
\caption{\sc {FactoryAddTile}$(n_c,\mathbf{b}, m,C, c,w)$ \label{alg:FactoryAddTile}}
\begin{algorithmic}[1]
\State$
\{ \mathbf{hopper}\}\leftarrow${\sc{Hopper}}$(c,n_c,w)$
\If{ $m = d \textbf{ and } \left(     C_x  \le \max \mathbf{b}_x
                         \textbf{ or }  C_y     < \min \mathbf{b}_y \right)  }$

\State$\{\mathbf{A},\mathbf{b}\}\leftarrow${\sc{downdir}}$(\mathbf{hopper},\mathbf{b},\mathbf{C})$

\ElsIf{ $m = l \textbf{ and} \left(     C_y  \le \max \mathbf{b}_y
                         \textbf{ or }  C_x     > \max \mathbf{b}_x \right)  }$

\State$\{\mathbf{A},\mathbf{b}\}\leftarrow${\sc{leftdir}}$(\mathbf{hopper},\mathbf{b},\mathbf{C})$
\ElsIf{ $m = l \textbf{ and} \left(     C_x  \ge \max \mathbf{b}_x
                         \textbf{ or }  C_y     > \max \mathbf{b}_y \right)  }$

\State$\{\mathbf{A},\mathbf{b}\}\leftarrow${\sc{updir}}$(\mathbf{hopper},\mathbf{b},\mathbf{C})$
\ElsIf{ $m = r \textbf{ and } \left(     C_y     \ge \min \mathbf{b}_y
                       \textbf{ or }  C_x  < \min \mathbf{b}_x   \right)  }$
\State$\{\mathbf{A},\mathbf{b}\}\leftarrow${\sc{rightdir}}$(\mathbf{hopper},\mathbf{b},\mathbf{C})$

\EndIf

\State \Return $\{ \mathbf{A}, \mathbf{b} \}$

\end{algorithmic}
\end{algorithm}


\section{Introduction}

OpenMath~\cite {BusCapCar:oms04} is a semantic representation format of mathematical objects and formulae.
In a nutshell, OpenMath standardizes six basic object types (symbols, variables, numbers, strings, and foreign objects), three ways of building complex objects: (function) application, binding, and management facilities like structure sharing and error reporting.
The OpenMath object model underlies Content MathML~\cite{CarlisleEd:MathML3:on}, making it well-integrated with MathML presentation.

There are several encodings of OpenMath Objects, most notably the XML and Binary encodings.
In this paper we propose another one based on JSON, a lightweight data-interchange format that used heavily in the Web Applications area.

JSON~\cite{JSON:web}, short for \textbf{J}ava\textbf{S}cript \textbf{O}bject \textbf{N}otation, is a lightweight data-interchange format.
While being a subset of JavaScript, it is defined independently.
JSON can represent both primitive types and composite types.

Primitive JSON data types are strings (e.g. \lstinline{"Hello world"}), Numbers (e.g. \lstinline{42} or \lstinline{3.14159265}), Booleans (\lstinline{true} and \lstinline{false}) and \lstinline{null}.
Composite JSON types are either (non-homogeneous) arrays (e.g. \lstinline{[1, "two", false]}) or key-value pairs called objects (e.g. \lstinline|{"foo": "bar", "answer": 42}|).

Constructs corresponding to JSON objects are found in most programming languages.
Furthermore, the syntax is very simple; hence many languages have built-in facilities for translating their existing data structures to and from JSON.
The use for an OpenMath JSON encoding is clear: It would enable easy use of OpenMath across many languages.

In the next Section we survey two existing OpenMath JSON encodings.
Section~\ref{sec:encoding} proposes a new encoding that combines the advantages and alleviates their disadvantages.
We give a thorough specification of the encoding, present a JSON schema implemented in TypeScript, and provide a web service that validates JSON-encoded OpenMath and transforms OpenMath objects between XML and JSON encodings.
Section~\ref{sec:concl} concludes the paper.

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "paper"
%%% End:

%  LocalWords:  standardizes textbf textbf cript textbf bject otation sec:concl


\section{Introduction}

Community detection is an essential task for cyberspace mining, which has been successfully employed to explore users’ resemblance for retrieval/recommendation enhancement and user behavior analysis. Taking social media and e-commerce as examples, the complex, and often heterogeneous, relations among users and other objects, e.g., products, reviews, and messages, can be encapsulated as bipartite graphs, and the topology can help to synthesize and represent users with a coarser and broader view.

While a graph is well-connected, conventional methods, e.g., modularity-based approach \cite{newman2004fast}, spectral approach \cite{nascimento2011spectral}, dynamic approach \cite{peixoto2017modelling} and deep learning approach \cite{chiang2019cluster}, are able to estimate the internal/external connectivity and generate high-quality communities directly on nodes \cite{fortunato2016community}.

For sparse graphs, as mentioned in the Chapter \ref{ch:intro}, to deal with the lack-of-connection problem, I propose  a novel research task – Cross-Graph Community Detection. The idea is based on the fact that an increasing number of small apps are utilizing the user identity information inherited from giant providers, i.e., users can easily login a large number of new apps by using Facebook and Google ID. In such ecosystem, the main large graph can provide critical information to enlighten the community detection on many small sparse graphs. Note that, in spit of the small sparse graphs can engage with a specific field, the main graph is quite comprehensive and noisy. As Figure \ref{fig:c4_example} shows, not all the connections in Amazon (shopping graph) can be equally important for the two candidate app graphs. In the example, three mutual users are selected where $u_1$ and $u_2$ mainly share similar shopping interests on cosmetics and $u_1$ and $u_3$ mainly share similar shopping interests on food products in Amazon. Then, with deliberate propagation from main graph, in the Cosmetic graph, $u_1$ and $u_2$ have a better chance to be grouped together, while $u_1$ and $u_3$ are more likely to be assigned the same community ID in the Cooking graph. Therefore, the proposed model should be able to differentiate various kinds of information from the main graph for each candidate sparse graph to enhance its local community detection performance.

As another challenge, small sparse graphs often suffer from training data insufficiency, e.g., the limited connections in these graphs can hardly tell the community residency information. In this study, I employed a novel data augmentation approach - cross-graph pairwise learning. Given a candidate user and an associated user triplet, the proposed model can detection the community closeness superiority by leveraging main graph and the sparse graph simultaneously. Moreover, the proposed pairwise learning method can cope with the main graph heterogeneity issue and reduce noisy information by taking care of graph local structure. Theoretically, I can offer at most $\mathcal{O}(N^{3})$ user triplets to learn graph community structure while conventional community detection methods by default can only be applied on $\mathcal{O}(N)$ users  ($N$ is the number of users in the sparse graph).

Therefore, I propose an innovative \textit{Pairwise Cross-graph Community Detection} (PCCD) model for enhanced sparse graph user community detection. Specifically, given user $u_i$ and its associated triplet $\langle u_{i},u_{j},u_{k}\rangle$, I aim to predict their pairwise community relationship, e.g., compared with user $u_{k}$, user $u_{j}$ should have closer, similar or farther community closeness to user $u_i$.


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{indentfirst}

\title{CS131 B1: HW 3}
\author{Duy Nguyen}
\date{2 October 2016}

\begin{document}

\maketitle

\section*{Question 1}
\subsection*{a.}
\begin{tabular}{l|l}
    The program has bugs or works correctly. & $B \lor C$ \\
    The program is bug-free or needs debugging. & $\neg B \lor N$ \\ \hline
    the program works correctly or needs debugging.&$C \lor N$.
\end{tabular}

This argument is valid since it follows the valid argument form of \textit{resolution}.
\subsection*{b.}
\begin{tabular}{l|l}
    The error is in function A or B. & $A \lor B$ \\ \hline
    I need to debug function A & $A$
\end{tabular}

We construct a truth table:

\begin{tabular}{cc|cc}
    $A$&$B$&$A \lor B$&$A$ \\ \hline
    T&T&T&T\\
    T&F&T&T\\
    \textbf{F}&\textbf{T}&\textbf{T}&\textbf{F}\\
    F&F&F&F
\end{tabular}

We note that there is one case where a true premise leads to a false conclusion. Thus this argument is invalid.

\subsection*{c.}
\begin{tabular}{l}
    All integers are rational numbers. \\
    All rational numbers can be represented as $p/q$, \\
    where
p and q are integers. \\
    There are no such p and q which satisfy $x=p/q$. \\ \hline
    x is irrational and x is not an integer.
\end{tabular}

 We have: ``all integers are rational numbers" and ``all rational numbers can be represent as $p/q$" are both true since this is the definition of rational number. So if the third premise ``there are no such p and q which satisfy $x=p/q$" is true, then x is not an rational number. And since integers are subset of rational numbers, x is not an integer. Therefore, this argument is valid.

 \subsection*{d.}
 \begin{tabular}{l}
     All the planets of the solar system orbit the Sun. \\
     Pluto orbits the Sun. \\ \hline
     Pluto is a planet of
the solar system.
 \end{tabular}

Both of the premises are true. However, the conclusion is false. Therefore, this is not a valid argument.

\section*{Question 2}
\subsection*{a. Every CS student knows Python and Java.}
$\forall x (C(x) \rightarrow (P(x) \land J(x)))$

\begin{tabular}{ll}
    Where: & \textbf{x} are all people. \\
    & \textbf{C(x)} are those who are a CS student. \\
    & \textbf{P(x)} is a person who knows Python.\\
    & \textbf{J(x)} is a person who knows Java.
\end{tabular}

\subsection*{b. Some CS students know C++ or C\#.}
$\exists x (C(x) \land (P(x) \lor S(x))$

\begin{tabular}{ll}
    Where: & \textbf{x} are all people. \\
    & \textbf{C(x)} are those who are a CS student. \\
    & \textbf{P(x)} is a person who knows C++.\\
    & \textbf{S(x)} is a person who knows C\#.
\end{tabular}

\subsection*{c. For every successful person there is someone even more successful, for every unhappy person there is someone more unhappy.}

$\forall x \exists y P(x,y)$

\begin{tabular}{ll}
    Where: & \textbf{x} is a successful/unhappy person. \\
    & \textbf{P(x,y)} is where y is more successful/unhappy than x
\end{tabular}

\subsection*{d. If y=f(x) is a function, then for every x there exists only one y.}
$ y = f(x) \rightarrow \forall x \exists y $

\section*{Question 3}
\subsection*{2a.}
$\neg \forall x (C(x) \rightarrow (P(x) \land J(x)))$

$\exists x \neg (C(x) \rightarrow (P(x) \land J(x)))$

$\exists x (C(x) \land \neg (P(x) \land J(x)))$

$\exists x (C(x) \land (\neg P(x) \lor \neg J(x)))$

There exists a person who is a CS student and does not know Python nor Java.

\subsection*{2b.}
$\neg \exists x (C(x) \land (P(x) \lor S(x))$

$\forall x \neg (C(x) \land (P(x) \lor S(x))$

$\forall x (\neg C(x) \lor \neg (P(x) \lor S(x))$

$\forall x (\neg C(x) \lor (\neg P(x) \land \neg S(x))$

For all people, there is not a CS student or there is not a person who know both C++ and C\#.

\end{document}


\subsection{SecretModp}
This is the same as the $\sint$ in the underlying SCALE virtual machine.

\subsubsection{Conversion of Data}
\func{SecretModp::from(a)}
This takes an $\sint$ integer value $a$ and loads it into a value
of type $\SecretModp$.
You can also pass in an argument of a type $\ints$, $\ClearModp$,
$\SecretBit$ and $\SecretI$.
\begin{lstlisting}
    let sa = SecretModp::from(ConstI32::<8>);

    let aa: i64 = 10;
    let mut a2=SecretModp::from(aa);

    let cint_a = ClearModp::from(ConstI32::<10>);
    let sint_a = SecretModp::from(cint_a);

    let sb = SecretBit::from(false);
    let ssb = SecretModp::from(sb);

    // See the main SCALE manual for the semantics of this conversion
    let si = SecretI64::from(ConstI32::<10>);
    let ssi = SecretModp::from(si);
\end{lstlisting}

\func{SecretModp::from_unsigned(a)}
The above conversion from $\SecretI$ to $\SecretModp$ is a signed conversion.
To get an unsigned conversion you perform.
\begin{lstlisting}
    let si = SecretI64::from(ConstI32::<-10>);
    let ssi = SecretModp::from_unsigned(si);
\end{lstlisting}

\subsubsection{IO}
The IO class functions for $\SecretModp$ datatypes can be access via the following commands:

\func{SecretModp::private_input(x)}
Loads an $\SecretModp$ from the channel $x$ and player $p$.
\begin{lstlisting}
    let a=SecretModp::private_input(Player::<p>, Channel::<10>);
\end{lstlisting}

\func{a.private_output(x)}
Writes an $\SecretModp$ to channel $x$ and player $p$.
\begin{lstlisting}
   let v = SecretModp::from(ConstI32::<1>);
   v.private_output(Player::<0>, Channel::<10>);
\end{lstlisting}

\subsubsection{Memory Access}
\func{a.store_in_mem(x)}
To store data into memory location $50$ say you need to execute,
recalling that memory is accessed by $32$ bit values.
\begin{lstlisting}
    unsafe{ sa.store_in_mem(ConstU32::<50>); }
\end{lstlisting}
To store to a variable location you use
\begin{lstlisting}
    let x: i64 = 50;
    unsafe{ sa.store_in_mem(x); }
\end{lstlisting}

\func{SecretModp::load_from_mem(x)}
To load data from memory you do
\begin{lstlisting}
    let sa=SecretModp::load_from_mem(ConstU32::<50>);
    let x: i64 = 50;
    let sb=SecretModp::load_from_mem(x);
\end{lstlisting}

\subsubsection{Testing Data}
\func{x.test()}
In the SCALE target this takes the $\SecretModp$ value $x$,
applies a reveal operation to it, and then writes the
resulting $\ClearModp$ value into memory.

In the Test target this takes a value stored in the $\ClearModp$
memory saved on the last SCALE invocation, and compares it to
$x$. If the two values are the same it prints the value, and the
line number in the rust file where this was executed.
Otherwise it aborts.

\func{x.test_value(y)}
Test whether the value held in $x$ is the same as the value held in $y$.
This is the same as \verb|x.test()| except the value is compared to
$y$ and not the value emulated in the test environment.