\hypertarget{distributions}{%
\chapter{Distributions}\label{distributions}}

In this chapter we'll see three ways to describe a set of values:

\begin{itemize}
\item
  A probability mass function (PMF), which represents a set of values
  and the number of times each one appears in a dataset.
\item
  A cumulative distribution function (CDF), which contains the same
  information as a PMF in a form that makes it easier to visualize, make
  comparisons, and perform some computations.
\item
  A kernel density estimate (KDE), which is like a smooth, continuous
  version of a histogram.
\end{itemize}

For examples, we'll use data from the General Social Survey (GSS) to
look at distributions of age and income, and to explore the relationship
between income and education.

But we'll start with one of the most important ideas in statistics, the
distribution.

\hypertarget{distributions-1}{%
\section{Distributions}\label{distributions-1}}

A distribution is a set of values and their corresponding probabilities.
For example, if you roll a six-sided die, there are six possible
outcomes, the numbers \passthrough{\lstinline!1!} through
\passthrough{\lstinline!6!}, and they all have the same probability,
\passthrough{\lstinline!1/6!}.

We can represent this distribution of outcomes with a table, like this:

\begin{longtable}[]{@{}ll@{}}
\toprule
Value & Probability\tabularnewline
\midrule
\endhead
1 & 1/6\tabularnewline
2 & 1/6\tabularnewline
3 & 1/6\tabularnewline
4 & 1/6\tabularnewline
5 & 1/6\tabularnewline
6 & 1/6\tabularnewline
\bottomrule
\end{longtable}

More generally, there can be any number of values, the values can be any
type, and the probabilities do not have to be equal.

To represent distributions in Python, we will use a library called
\passthrough{\lstinline!empiricaldist!}, for ``empirical distribution'',
which means it is based on data rather than a mathematical formula.

\passthrough{\lstinline!empiricaldist!} provides an object called
\passthrough{\lstinline!Pmf!}, which stands for ``probability mass
function''. A \passthrough{\lstinline!Pmf!} object contains a set of
possible outcomes and their probabilities.

For example, here's a \passthrough{\lstinline!Pmf!} that represents the
outcome of rolling a six-sided die:

\begin{lstlisting}[language=Python,style=source]
from empiricaldist import Pmf

outcomes = [1,2,3,4,5,6]
die = Pmf(1/6, outcomes)
\end{lstlisting}

The first argument is the probability of each outcome; the second
argument is the list of outcomes. We can display the result like this.

\begin{lstlisting}[language=Python,style=source]
die
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &     probs \\
\midrule
1 &  0.166667 \\
2 &  0.166667 \\
3 &  0.166667 \\
4 &  0.166667 \\
5 &  0.166667 \\
6 &  0.166667 \\
\bottomrule
\end{tabular}

A \passthrough{\lstinline!Pmf!} object is a specialized version of a
Pandas \passthrough{\lstinline!Series!}, so it provides all of the
attributes and methods of a \passthrough{\lstinline!Series!}, plus some
additional methods we'll see soon.

\hypertarget{the-general-social-survey}{%
\section{The General Social Survey}\label{the-general-social-survey}}

The examples in this chapter are based on a new dataset, the General
Social Survey (GSS). The GSS has run annually since 1972; it surveys a
representative sample of adult residents of the U.S. and asks questions
about demographics, personal history, and beliefs about social and
political issues.

It is widely used by politicians, policy makers, and researchers,
including me. The GSS dataset contains hundreds of columns; using an
online tool call \href{https://gssdataexplorer.norc.org/}{GSS Explorer}
I've selected just a few and created a data extract.

Like the NSFG data we used in the previous chapter, the GSS data is
stored in a fixed-width format, described by a Stata data dictionary.

\begin{lstlisting}[language=Python,style=source]
dict_file = 'GSS.dct'
data_file = 'GSS.dat.gz'
\end{lstlisting}

We will use the \passthrough{\lstinline!statadict!} library to read the
data dictionary.

\begin{lstlisting}[language=Python,style=source]
from statadict import parse_stata_dict

stata_dict = parse_stata_dict(dict_file)
\end{lstlisting}

The data file is compressed, but we can use the
\passthrough{\lstinline!gzip!} library to open it.

\begin{lstlisting}[language=Python,style=source]
import gzip

fp = gzip.open(data_file)
\end{lstlisting}

The result is an object that behaves like a file, so we can pass it as
an argument to \passthrough{\lstinline!read\_fwf!}:

\begin{lstlisting}[language=Python,style=source]
import pandas as pd

gss = pd.read_fwf(fp,
                  names=stata_dict.names,
                  colspecs=stata_dict.colspecs)
gss.shape
\end{lstlisting}

\begin{lstlisting}[style=output]
(64814, 8)
\end{lstlisting}

The result is a \passthrough{\lstinline!DataFrame!} with 64818 rows, one
for each respondent, and 6 columns, one for each variable. Here are the
first few rows.

\begin{lstlisting}[language=Python,style=source]
gss.head()
\end{lstlisting}

\begin{tabular}{lrrrrrrrr}
\toprule
{} &  YEAR &  ID\_ &  AGE &  EDUC &  SEX &  GUNLAW &  GRASS &  REALINC \\
\midrule
0 &  1972 &    1 &   23 &    16 &    2 &       1 &      0 &  18951.0 \\
1 &  1972 &    2 &   70 &    10 &    1 &       1 &      0 &  24366.0 \\
2 &  1972 &    3 &   48 &    12 &    2 &       1 &      0 &  24366.0 \\
3 &  1972 &    4 &   27 &    17 &    2 &       1 &      0 &  30458.0 \\
4 &  1972 &    5 &   61 &    12 &    2 &       1 &      0 &  50763.0 \\
\bottomrule
\end{tabular}

You can probably guess what the variables are, and I'll explain them as
we go along. But if you want more information, you can always read the
codebook at
\url{https://gssdataexplorer.norc.org/projects/52787/variables/vfilter}.

\hypertarget{distribution-of-education}{%
\section{Distribution of Education}\label{distribution-of-education}}

To get started with this dataset, let's look at the distribution of
\passthrough{\lstinline!EDUC!}, which records the number of years of
education for each respondent. First I'll select a column from the
\passthrough{\lstinline!DataFrame!} and use
\passthrough{\lstinline!value\_counts!} to see what values are in it.

\begin{lstlisting}[language=Python,style=source]
gss['EDUC'].value_counts().sort_index()
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &   EDUC \\
\midrule
0  &    165 \\
1  &     47 \\
2  &    152 \\
3  &    257 \\
4  &    319 \\
5  &    402 \\
6  &    828 \\
7  &    879 \\
8  &   2724 \\
9  &   2083 \\
10 &   2880 \\
11 &   3743 \\
12 &  19663 \\
13 &   5360 \\
14 &   7160 \\
15 &   2910 \\
16 &   8355 \\
17 &   1967 \\
18 &   2384 \\
19 &    920 \\
20 &   1439 \\
98 &     73 \\
99 &    104 \\
\bottomrule
\end{tabular}

The result from \passthrough{\lstinline!value\_counts!} is a set of
possible values and the number of times each one appears, so it is a
kind of distribution.

The values \passthrough{\lstinline!98!} and \passthrough{\lstinline!99!}
are special codes for ``Don't know'' and ``No answer''. We'll use
\passthrough{\lstinline!replace!} to replace these codes with
\passthrough{\lstinline!NaN!}.

\begin{lstlisting}[language=Python,style=source]
import numpy as np

educ = gss['EDUC'].replace([98, 99], np.nan)
\end{lstlisting}

We've already seen one way to visualize a distribution, a histogram.
Here's the histogram of education level.

\begin{lstlisting}[language=Python,style=source]
import matplotlib.pyplot as plt

educ.hist(grid=False)
plt.xlabel('Years of education')
plt.ylabel('Number of respondents')
plt.title('Histogram of education level');
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_36_0.pdf}
\end{center}

Based on the histogram, we can see the general shape of the distribution
and the central tendency -- it looks like the peak is near 12 years of
education. But a histogram is not the best way to visualize this
distribution.

An alternative is a \passthrough{\lstinline!Pmf!}.
\passthrough{\lstinline!Pmf!} provides a function called
\passthrough{\lstinline!from\_seq!} that takes any kind of sequence --
like a list, tuple, or Pandas \passthrough{\lstinline!Series!} -- and
computes the distribution of the values in the sequence.

\begin{lstlisting}[language=Python,style=source]
pmf_educ = Pmf.from_seq(educ, normalize=False)
type(pmf_educ)
\end{lstlisting}

\begin{lstlisting}[style=output]
empiricaldist.empiricaldist.Pmf
\end{lstlisting}

The keyword argument \passthrough{\lstinline!normalize=False!} indicates
that we don't want to normalize this PMF. I'll explain what that means
soon.

Here's what the first few rows look like.

\begin{lstlisting}[language=Python,style=source]
pmf_educ.head()
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &  probs \\
\midrule
0.0 &    165 \\
1.0 &     47 \\
2.0 &    152 \\
\bottomrule
\end{tabular}

In this dataset, there are \passthrough{\lstinline!165!} respondents who
report that they have had no formal education, and
\passthrough{\lstinline!47!} who have only one year. Here the last few
rows.

\begin{lstlisting}[language=Python,style=source]
pmf_educ.tail()
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &  probs \\
\midrule
18.0 &   2384 \\
19.0 &    920 \\
20.0 &   1439 \\
\bottomrule
\end{tabular}

There are \passthrough{\lstinline!1439!} respondents who report that
they have 20 or more years of formal education, which probably means
they attended college and graduate school.

You can use the bracket operator to look up a value in a Pmf and get the
corresponding count:

\begin{lstlisting}[language=Python,style=source]
pmf_educ[20]
\end{lstlisting}

\begin{lstlisting}[style=output]
1439
\end{lstlisting}

Usually when we make a PMF, we want to know the \emph{fraction} of
respondents with each value, rather than the counts. We can do that by
setting \passthrough{\lstinline!normalize=True!}; then we get a
normalized PMF, that is, a PMF where the values in the second column add
up to 1.

\begin{lstlisting}[language=Python,style=source]
pmf_educ_norm = Pmf.from_seq(educ, normalize=True)
pmf_educ_norm.head()
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &     probs \\
\midrule
0.0 &  0.002553 \\
1.0 &  0.000727 \\
2.0 &  0.002352 \\
\bottomrule
\end{tabular}

Now if we use the bracket operator, the result is a fraction. For
example, the fraction of people with 12 years of education is about
30\%:

\begin{lstlisting}[language=Python,style=source]
pmf_educ_norm[16]
\end{lstlisting}

\begin{lstlisting}[style=output]
0.12926033077030183
\end{lstlisting}

\passthrough{\lstinline!Pmf!} provides a \passthrough{\lstinline!bar!}
method that plots the values and their probabilities as a bar chart.

\begin{lstlisting}[language=Python,style=source]
pmf_educ_norm.bar(label='EDUC')

plt.xlabel('Years of education')
plt.xticks(range(0, 21, 4))
plt.ylabel('PMF')
plt.title('Distribution of years of education')
plt.legend();
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_50_0.pdf}
\end{center}

In this figure, we can see that the most common value is 12 years, but
there are also peaks at 14 and 16, which correspond to two and four
years of college.

For this data, the PMF is probably a better choice than the histogram.
The PMF shows all unique values, so we can see where the peaks are.
Because the histogram puts values into bins, it obscures some details.
With this dataset, and the default number of bins, we couldn't see the
peaks at 14 and 16 years.

But PMFs have limitations, too, as we'll see. But first, let's get some
practice with PMFs.

\textbf{Exercise:} Let's look at another column in this
\passthrough{\lstinline!DataFrame!}, \passthrough{\lstinline!YEAR!},
which represents the year each respondent was interviewed.

Make an unnormalized PMF for \passthrough{\lstinline!YEAR!} and display
the result. How many respondents were interviewed in 2018?

\hypertarget{cumulative-distribution-functions}{%
\section{Cumulative distribution
functions}\label{cumulative-distribution-functions}}

Now we'll see another way to represent a distribution, the cumulative
distribution function (CDF). \passthrough{\lstinline!empiricaldist!}
provides a \passthrough{\lstinline!Cdf!} object that represents a CDF.
We can import it like this:

\begin{lstlisting}[language=Python,style=source]
from empiricaldist import Cdf
\end{lstlisting}

As an example, suppose we have a sequence of five values:

\begin{lstlisting}[language=Python,style=source]
values = 1, 2, 2, 3, 5
\end{lstlisting}

Here's the \passthrough{\lstinline!Pmf!} of these values.

\begin{lstlisting}[language=Python,style=source]
Pmf.from_seq(values)
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &  probs \\
\midrule
1 &    0.2 \\
2 &    0.4 \\
3 &    0.2 \\
5 &    0.2 \\
\bottomrule
\end{tabular}

If you draw a random value from \passthrough{\lstinline!values!}, the
PMF tells you the chance of getting \passthrough{\lstinline!x!}, for any
value of \passthrough{\lstinline!x!}. So the probability of the value
\passthrough{\lstinline!1!} is \passthrough{\lstinline!1/5!}; the
probability of the value \passthrough{\lstinline!2!} is
\passthrough{\lstinline!2/5!}; and the probabilities for
\passthrough{\lstinline!3!} and \passthrough{\lstinline!5!} are
\passthrough{\lstinline!1/5!} each.

A CDF is similar in the sense that it contains values and their
probabilities; the difference is that the probabilities in the CDF are
the the cumulative sum of the probabilities in the PMF.

Here's the \passthrough{\lstinline!Cdf!} for the same five values.

\begin{lstlisting}[language=Python,style=source]
Cdf.from_seq(values)
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &  probs \\
\midrule
1 &    0.2 \\
2 &    0.6 \\
3 &    0.8 \\
5 &    1.0 \\
\bottomrule
\end{tabular}

If you draw a random value from \passthrough{\lstinline!values!},
\passthrough{\lstinline!Cdf!} tells you the chance of getting a value
\emph{less than or equal to} \passthrough{\lstinline!x!}, for any given
\passthrough{\lstinline!x!}.

So the \passthrough{\lstinline!Cdf!} of \passthrough{\lstinline!1!} is
\passthrough{\lstinline!1/5!} because one of the five values in the
sequence is less than or equal to 1.

The \passthrough{\lstinline!Cdf!} of 2 is \passthrough{\lstinline!3/5!}
because three of the five values are less than or equal to 2.

And the \passthrough{\lstinline!Cdf!} of 5 is
\passthrough{\lstinline!5/5!} because all of the values are less than or
equal to 5.

\hypertarget{cdf-of-age}{%
\section{CDF of Age}\label{cdf-of-age}}

Now let's look at a more substantial \passthrough{\lstinline!Cdf!}, the
distribution of ages for respondents in the General Social Survey.

The \href{https://gssdataexplorer.norc.org/variables/53/vshow}{variable
we'll use} is \passthrough{\lstinline!'AGE'!}. According to the
codebook, the range of the values is from \passthrough{\lstinline!18!}
to \passthrough{\lstinline!89!}, where \passthrough{\lstinline!89!}
means ``89 or older''. The special codes \passthrough{\lstinline!98!}
and \passthrough{\lstinline!99!} mean ``Don't know'' and ``Didn't
answer''.

I'll use \passthrough{\lstinline!replace!} to replace the special codes
with \passthrough{\lstinline!NaN!}.

\begin{lstlisting}[language=Python,style=source]
age = gss['AGE'].replace([98, 99], np.nan)
\end{lstlisting}

We can compute the \passthrough{\lstinline!Cdf!} of these values like
this:

\begin{lstlisting}[language=Python,style=source]
cdf_age = Cdf.from_seq(age)
\end{lstlisting}

\passthrough{\lstinline!Cdf!} provides a method called
\passthrough{\lstinline!plot!} that plots the CDF as a line. Here's what
it looks like.

\begin{lstlisting}[language=Python,style=source]
cdf_age.plot()

plt.xlabel('Age (years)')
plt.ylabel('CDF')
plt.title('Distribution of age');
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_68_0.pdf}
\end{center}

The \(x\)-axis is the ages, from 18 to 89. The \(y\)-axis is the
cumulative probabilities, from 0 to 1.

\passthrough{\lstinline!cdf\_age!} can be used as a function, so if you
give it an age, it returns the corresponding probability (in a NumPy
array).

\begin{lstlisting}[language=Python,style=source]
q = 51
p = cdf_age(q)
p
\end{lstlisting}

\begin{lstlisting}[style=output]
array(0.63318676)
\end{lstlisting}

\passthrough{\lstinline!q!} stands for ``quantity'', which is what we
are looking up. \passthrough{\lstinline!p!} stands for probability,
which is the result. In this example, the quantity is age 51, and the
corresponding probability is about \passthrough{\lstinline!0.63!}. That
means that about 63\% of the respondents are 51 years old or younger.

The arrow in the following figure shows how you could read this value
from the CDF, at least approximately.

\begin{lstlisting}[language=Python,style=source]
cdf_age.plot()

x = 17
draw_line(p, q, x)
draw_arrow_left(p, q, x)

plt.xlabel('Age (years)')
plt.xlim(x-1, 91)
plt.ylabel('CDF')
plt.title('Distribution of age');
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_73_0.pdf}
\end{center}

The CDF is an invertible function, which means that if you have a
probability, \passthrough{\lstinline!p!}, you can look up the
corresponding quantity, \passthrough{\lstinline!q!}.
\passthrough{\lstinline!Cdf!} provides a method called
\passthrough{\lstinline!inverse!} that computes the inverse of the
cumulative distribution function.

\begin{lstlisting}[language=Python,style=source]
p1 = 0.25
q1 = cdf_age.inverse(p1)
q1
\end{lstlisting}

\begin{lstlisting}[style=output]
array(31.)
\end{lstlisting}

In this example, we look up the probability
\passthrough{\lstinline!0.25!} and the result is
\passthrough{\lstinline!31!}.\\
That means that 25\% of the respondents are age 31 or less. Another way
to say the same thing is ``age 31 is the 25th percentile of this
distribution''.

If we look up probability \passthrough{\lstinline!0.75!}, it returns
\passthrough{\lstinline!59!}, so 75\% of the respondents are 59 or
younger.

\begin{lstlisting}[language=Python,style=source]
p2 = 0.75
q2 = cdf_age.inverse(p2)
q2
\end{lstlisting}

\begin{lstlisting}[style=output]
array(59.)
\end{lstlisting}

In the following figure, the arrows show how you could read these values
from the CDF.

\begin{lstlisting}[language=Python,style=source]
cdf_age.plot()

x = 17
draw_line(p1, q1, x)
draw_arrow_down(p1, q1, 0)

draw_line(p2, q2, x)
draw_arrow_down(p2, q2, 0)

plt.xlabel('Age (years)')
plt.xlim(x-1, 91)
plt.ylabel('CDF')
plt.title('Distribution of age');
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_79_0.pdf}
\end{center}

The distance from the 25th to the 75th percentile is called the
\textbf{interquartile range}, or IQR. It measures the spread of the
distribution, so it is similar to standard deviation or variance.

Because it is based on percentiles, it doesn't get thrown off by extreme
values or outliers, the way standard deviation does. So IQR is more
\textbf{robust} than variance, which means it works well even if there
are errors in the data or extreme values.

\textbf{Exercise:} Using \passthrough{\lstinline!cdf\_age!}, compute the
fraction of the respondents in the GSS dataset that are \emph{older}
than 65.

\textbf{Exercise:} The distribution of income in almost every country is
long-tailed, which means there are a small number of people with very
high incomes. In the GSS dataset, the column
\passthrough{\lstinline!REALINC!} represents total household income,
converted to 1986 dollars. We can get a sense of the shape of this
distribution by plotting the CDF.

Select \passthrough{\lstinline!REALINC!} from the
\passthrough{\lstinline!gss!} dataset, make a
\passthrough{\lstinline!Cdf!} called
\passthrough{\lstinline!cdf\_income!}, and plot it. Remember to label
the axes!

\hypertarget{comparing-distributions}{%
\section{Comparing distributions}\label{comparing-distributions}}

So far we've seen two ways to represent distributions, PMFs and CDFs.
Now we'll use PMFs and CDFs to compare distributions, and we'll see the
pros and cons of each.

One way to compare distributions is to plot multiple PMFs on the same
axes. For example, suppose we want to compare the distribution of age
for male and female respondents.

First I'll create a Boolean Series that's true for male respondents.

\begin{lstlisting}[language=Python,style=source]
male = (gss['SEX'] == 1)
\end{lstlisting}

And another that's true for female respondents.

\begin{lstlisting}[language=Python,style=source]
female = (gss['SEX'] == 2)
\end{lstlisting}

Now I can select ages for the male and female respondents.

\begin{lstlisting}[language=Python,style=source]
male_age = age[male]
female_age = age[female]
\end{lstlisting}

And plot a Pmf for each.

\begin{lstlisting}[language=Python,style=source]
pmf_male_age = Pmf.from_seq(male_age)
pmf_male_age.plot(label='Male')

pmf_female_age = Pmf.from_seq(female_age)
pmf_female_age.plot(label='Female')

plt.xlabel('Age (years)')
plt.ylabel('PMF')
plt.title('Distribution of age by sex')
plt.legend();
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_90_0.pdf}
\end{center}

The plot is pretty noisy. In the range from 40 to 50, it looks like the
PMF is higher for men. And from 70 to 80, it is higher for women. But
both of those differences might be due to random variation.

Now let's do the same thing with CDFs; everything is the same except we
replace \passthrough{\lstinline!Pmf!} with
\passthrough{\lstinline!Cdf!}.

\begin{lstlisting}[language=Python,style=source]
cdf_male_age = Cdf.from_seq(male_age)
cdf_male_age.plot(label='Male')

cdf_female_age = Cdf.from_seq(female_age)
cdf_female_age.plot(label='Female')

plt.xlabel('Age (years)')
plt.ylabel('CDF')
plt.title('Distribution of age by sex')
plt.legend();
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_92_0.pdf}
\end{center}

In general, CDFs are smoother than PMFs. Because they smooth out
randomness, we can often get a better view of real differences between
distributions. In this case, the lines are close together until age 40;
after that, the CDF is higher for men than women. So what does that
mean?

One way to interpret the difference is that the fraction of men below a
given age is generally more than the fraction of women below the same
age. For example, about 79\% of men are 60 or less, compared to 76\% of
women.

\begin{lstlisting}[language=Python,style=source]
cdf_male_age(60), cdf_female_age(60)
\end{lstlisting}

\begin{lstlisting}[style=output]
(array(0.78599958), array(0.75529908))
\end{lstlisting}

Going the other way, we could also compare percentiles. For example, the
median age woman is older than the median age man, by about one year.

\begin{lstlisting}[language=Python,style=source]
cdf_male_age.inverse(0.5), cdf_female_age.inverse(0.5)
\end{lstlisting}

\begin{lstlisting}[style=output]
(array(43.), array(44.))
\end{lstlisting}

\textbf{Exercise:} What fraction of men are over 80? What fraction of
women?

\begin{lstlisting}[language=Python,style=source]
1-cdf_male_age(80), 1-cdf_female_age(80)
\end{lstlisting}

\begin{lstlisting}[style=output]
(0.0258566323313012, 0.03806458772611254)
\end{lstlisting}

\hypertarget{income}{%
\section{Income}\label{income}}

As another example, let's look at household income and compare the
distribution before and after 1995 (I chose 1995 because it's roughly
the midpoint of the survey). The variable
\passthrough{\lstinline!REALINC!} represents household income in 1986
dollars.

I'll make a Boolean \passthrough{\lstinline!Series!} to select
respondents interviewed before and after 1995.

\begin{lstlisting}[language=Python,style=source]
pre95 = (gss['YEAR'] < 1995)
post95 = (gss['YEAR'] >= 1995)
\end{lstlisting}

Now we can plot the PMFs.

\begin{lstlisting}[language=Python,style=source]
income = gss['REALINC'].replace(0, np.nan)

Pmf.from_seq(income[pre95]).plot(label='Before 1995')
Pmf.from_seq(income[post95]).plot(label='After 1995')

plt.xlabel('Income (1986 USD)')
plt.ylabel('PMF')
plt.title('Distribution of income')
plt.legend();
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_102_0.pdf}
\end{center}

There are a lot of unique values in this distribution, and none of them
appear very often. As a result, the PMF is so noisy and we can't really
see the shape of the distribution.

It's also hard to compare the distributions. It looks like there are
more people with high incomes after 1995, but it's hard to tell. We can
get a clearer picture with a CDF.

\begin{lstlisting}[language=Python,style=source]
Cdf.from_seq(income[pre95]).plot(label='Before 1995')
Cdf.from_seq(income[post95]).plot(label='After 1995')

plt.xlabel('Income (1986 USD)')
plt.ylabel('CDF')
plt.title('Distribution of income')
plt.legend();
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_104_0.pdf}
\end{center}

Below \$30,000 the CDFs are almost identical; above that, we can see
that the post-1995 distribution is shifted to the right. In other words,
the fraction of people with high incomes is about the same, but the
income of high earners has increased.

In general, I recommend CDFs for exploratory analysis. They give you a
clear view of the distribution, without too much noise, and they are
good for comparing distributions, especially if you have more than two.

\textbf{Exercise:} In the previous figure, the dollar amounts are big
enough that the labels on the \passthrough{\lstinline!x!} axis are
crowded. Improve the figure by expressing income in 1000s of dollars
(and update the \passthrough{\lstinline!x!} label accordingly).

\textbf{Exercise:} Let's compare incomes for different levels of
education in the GSS dataset

To do that we'll create Boolean Series to identify respondents with
different levels of education.

\begin{itemize}
\item
  In the U.S, 12 years of education usually means the respondent has
  completed high school (secondary education).
\item
  A respondent with 14 years of education has probably completed an
  associate degree (two years of college)
\item
  Someone with 16 years has probably completed a bachelor's degree (four
  years of college or university).
\end{itemize}

Define Boolean \passthrough{\lstinline!Series!} named
\passthrough{\lstinline!high!}, \passthrough{\lstinline!assc!}, and
\passthrough{\lstinline!bach!} that are true for respondents with

\begin{itemize}
\item
  12 or fewer years of education,
\item
  13, 14, or 15 years, and
\item
  16 or more.
\end{itemize}

Compute and plot the distribution of income for each group. Remember to
label the CDFs, display a legend, and label the axes. Write a few
sentences that describe and interpret the results.

\hypertarget{modeling-distributions}{%
\section{Modeling distributions}\label{modeling-distributions}}

Some distributions have names. For example, you might be familiar with
the normal distribution, also called the Gaussian distribution or the
bell curve. And you might have heard of others like the exponential
distribution, binomial distribution, or maybe Poisson distribution.

These ``distributions with names'' are called \textbf{analytic} because
they are described by analytic mathematical functions, as contrasted
with empirical distributions, which are based on data.

It turns out that many things we measure in the world have distributions
that are well approximated by analytic distributions, so these
distributions are sometimes good models for the real world.\\
In this context, what I mean by a ``model'' is a simplified description
of the world that is accurate enough for its intended purpose.

In this section, we'll compute the CDF of a normal distribution and
compare it to an empirical distribution of data. But before we get to
real data, we'll start with fake data.

The following statement uses NumPy's \passthrough{\lstinline!random!}
library to generate 1000 values from a normal distribution with mean
\passthrough{\lstinline!0!} and standard deviation
\passthrough{\lstinline!1!}.

\begin{lstlisting}[language=Python,style=source]
np.random.seed(17)
\end{lstlisting}

\begin{lstlisting}[language=Python,style=source]
sample = np.random.normal(size=1000)
\end{lstlisting}

Here's what the empirical distribution of the sample looks like.

\begin{lstlisting}[language=Python,style=source]
cdf_sample = Cdf.from_seq(sample)
cdf_sample.plot(label='Random sample')

plt.xlabel('x')
plt.ylabel('CDF')
plt.legend();
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_112_0.pdf}
\end{center}

If we did not know that this sample was drawn from a normal
distribution, and we wanted to check, we could compare the CDF of the
data to the CDF of an ideal normal distribution, which we can use the
SciPy library to compute.

\begin{lstlisting}[language=Python,style=source]
from scipy.stats import norm

xs = np.linspace(-3, 3)
ys = norm(0, 1).cdf(xs)
\end{lstlisting}

First we import \passthrough{\lstinline!norm!} from
\passthrough{\lstinline!scipy.stats!}, which is a collection of
functions related to statistics.

Then we use \passthrough{\lstinline!linspace()!} to create an array of
equally-spaced points from -3 to 3; those are the
\passthrough{\lstinline!x!} values where we will evaluate the normal
CDF.

Next, \passthrough{\lstinline!norm(0, 1)!} creates an object that
represents a normal distribution with mean \passthrough{\lstinline!0!}
and standard deviation \passthrough{\lstinline!1!}.

Finally, \passthrough{\lstinline!cdf!} computes the CDF of the normal
distribution, evaluated at each of the \passthrough{\lstinline!xs!}.

I'll plot the normal CDF with a gray line and then plot the CDF of the
data again.

\begin{lstlisting}[language=Python,style=source]
plt.plot(xs, ys, color='gray', label='Normal CDF')
cdf_sample.plot(label='Random sample')

plt.xlabel('x')
plt.ylabel('CDF')
plt.legend();
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_116_0.pdf}
\end{center}

The CDF of the random sample agrees with the normal model. And that's
not surprising because the data were actually sampled from a normal
distribution. When we collect data in the real world, we do not expect
it to fit a normal distribution as well as this. In the next exercise,
we'll try it and see.

\textbf{Exercise:} Is the normal distribution a good model for the
distribution of ages in the U.S. population?

To answer this question:

\begin{itemize}
\item
  Compute the mean and standard deviation of ages in the GSS dataset.
\item
  Use \passthrough{\lstinline!linspace!} to create an array of equally
  spaced values between 18 and 89.
\item
  Use \passthrough{\lstinline!norm!} to create a normal distribution
  with the same mean and standard deviation as the data, then use it to
  compute the normal CDF for each value in the array.
\item
  Plot the normal CDF with a gray line.
\item
  Plot the CDF of the ages in the GSS.
\end{itemize}

How well do the plotted CDFs agree?

\textbf{Exercise:} In many datasets, the distribution of income is
approximately \textbf{lognormal}, which means that the logarithms of the
incomes fit a normal distribution. We'll see whether that's true for the
GSS data.

\begin{itemize}
\item
  Extract \passthrough{\lstinline!REALINC!} from
  \passthrough{\lstinline!gss!} and compute its logarithm using
  \passthrough{\lstinline!np.log10()!}. Hint: Replace the value
  \passthrough{\lstinline!0!} with \passthrough{\lstinline!NaN!} before
  computing logarithms.
\item
  Compute the mean and standard deviation of the log-transformed
  incomes.
\item
  Use \passthrough{\lstinline!norm!} to make a normal distribution with
  the same mean and standard deviation as the log-transformed incomes.
\item
  Plot the CDF of the normal distribution.
\item
  Compute and plot the CDF of the log-transformed incomes.
\end{itemize}

How similar are the CDFs of the log-transformed incomes and the normal
distribution?

\hypertarget{probability-density-functions}{%
\section{Probability Density
Functions}\label{probability-density-functions}}

We have seen two ways to represent distributions, PMFs and CDFs. Now
we'll learn another way: a probability density function, or PDF. The
\passthrough{\lstinline!norm!} function, which we used to compute the
normal CDF, can also compute the normal PDF:

\begin{lstlisting}[language=Python,style=source]
xs = np.linspace(-3, 3)
ys = norm(0,1).pdf(xs)
plt.plot(xs, ys, color='gray', label='Normal PDF')

plt.xlabel('x')
plt.ylabel('PDF')
plt.title('Normal density function')
plt.legend();
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_121_0.pdf}
\end{center}

The normal PDF is the classic ``bell curve''.

It is tempting to compare the PMF of the data to the PDF of the normal
distribution, but that doesn't work. Let's see what happens if we try:

\begin{lstlisting}[language=Python,style=source]
plt.plot(xs, ys, color='gray', label='Normal PDF')

pmf_sample = Pmf.from_seq(sample)
pmf_sample.plot(label='Random sample')

plt.xlabel('x')
plt.ylabel('PDF')
plt.title('Normal density function')
plt.legend();
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_123_0.pdf}
\end{center}

The PMF of the sample is a flat line across the bottom. In the random
sample, every value is unique, so they all have the same probability,
one in 1000.

However, we can use the points in the sample to estimate the PDF of the
distribution they came from. This process is called \textbf{kernel
density estimation}, or KDE. It's a way of getting from a PMF, a
probability mass function, to a PDF, a probability density function.

To generate a KDE plot, we'll use the Seaborn library, which I'll import
as \passthrough{\lstinline!sns!}. Seaborn provides
\passthrough{\lstinline!kdeplot!}, which takes the sample, estimates the
PDF, and plots it.

\begin{lstlisting}[language=Python,style=source]
import seaborn as sns

sns.kdeplot(sample, label='Estimated sample PDF')

plt.xlabel('x')
plt.ylabel('PDF')
plt.title('Normal density function')
plt.legend();
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_125_0.pdf}
\end{center}

Now we can compare the KDE plot and the normal PDF.

\begin{lstlisting}[language=Python,style=source]
plt.plot(xs, ys, color='gray', label='Normal PDF')
sns.kdeplot(sample, label='Estimated sample PDF')

plt.xlabel('x')
plt.ylabel('PDF')
plt.title('Normal density function')
plt.legend();
\end{lstlisting}

\begin{center}
\includegraphics[scale=0.75]{08_distributions_files/08_distributions_127_0.pdf}
\end{center}

The KDE plot matches the normal PDF pretty well, although the
differences look bigger when we compare PDFs than they did with the
CDFs. That means that the PDF is a more sensitive way to look for
differences, but often it is too sensitive.\\
It's hard to tell whether apparent differences mean anything, or if they
are just random, as in this case.

\textbf{Exercise:} In a previous exercise, we asked ``Is the normal
distribution a good model for the distribution of ages in the U.S.
population?'' To answer this question, we plotted the CDF of the data
and compared it to the CDF of a normal distribution with the same mean
and standard deviation.

Now we'll compare the estimated density of the data with the normal PDF.

\begin{itemize}
\item
  Again, compute the mean and standard deviation of ages in the GSS
  dataset.
\item
  Use \passthrough{\lstinline!linspace!} to create an array of values
  between 18 and 89.
\item
  Use \passthrough{\lstinline!norm!} to create a normal distribution
  with the same mean and standard deviation as the data, then use it to
  compute the normal PDF for each value in the array.
\item
  Plot the normal PDF with a gray line.
\item
  Use \passthrough{\lstinline!sns.kdeplot!} to estimate and plot the
  density of the ages in the GSS.
\end{itemize}

Note: Seaborn can't handle NaNs, so use \passthrough{\lstinline!dropna!}
to drop them before calling \passthrough{\lstinline!kdeplot!}.

How well do the PDF and KDE plots agree?

\textbf{Exercise:} In a previous exercise, we used CDFs to see if the
distribution of income fits a lognormal distribution. We can make the
same comparison using a PDF and KDE.

\begin{itemize}
\item
  Again, extract \passthrough{\lstinline!REALINC!} from
  \passthrough{\lstinline!gss!} and compute its logarithm using
  \passthrough{\lstinline!np.log10()!}.
\item
  Compute the mean and standard deviation of the log-transformed
  incomes.
\item
  Use \passthrough{\lstinline!norm!} to make a normal distribution with
  the same mean and standard deviation as the log-transformed incomes.
\item
  Plot the PDF of the normal distribution.
\item
  Use \passthrough{\lstinline!sns.kdeplot()!} to estimate and plot the
  density of the log-transformed incomes.
\end{itemize}

\hypertarget{summary}{%
\section{Summary}\label{summary}}

In this chapter, we've seen three ways to visualize distributions, PMFs,
CDFs, and KDE.

In general, I use CDFs when I am exploring data. That way, I get the
best view of what's going on without getting distracted by noise.

Then, if I am presenting results to an audience unfamiliar with CDFs, I
might use a PMF if the dataset contains a small number of unique values,
or KDE if there are many unique values.

As an example, see my article about the Inspection Paradox at
\url{https://towardsdatascience.com/the-inspection-paradox-is-everywhere-2ef1c2e9d709}.
I wrote it for a general audience, so I use KDE to present and compare
distributions.


%-----------------------------------------------------------------------------
\section{Evolution Equations}

The compressible and low Mach number formulations of the governing
equations both share the unapproximated continuity and momentum equations
shown here (where $p = p_0 + \pi$).
\begin{eqnarray}
\frac{\partial(\rho X_k)}{\partial t} &=& -\nabla\cdot(\rho X_k\Ub) +
\rho\omegadot_k,\label{enth:eq:species}\\
\frac{\partial\Ub}{\partial t} &=& -\Ub\cdot\nabla\Ub  -
  \frac{1}{\rho}\nabla\pi -
  \frac{\rho-\rho_0}{\rho} g\eb_r,\label{eq:momentum}
\end{eqnarray}

In the compressible formulation we complete the system with an energy
equation as well as an equation of state; in the low Mach number
formulation we can derive a constraint on the velocity by setting
$p_{EOS} = p(\rho,h,X_k) = p_0$ and differentiating the equation of
state along particle paths.  In this case adding the energy equation
would over-constrain the system, but its solution can help us in
providing a diagnostic capability for the solution.  We can write the
energy equation in terms of internal energy, $e$, or enthalpy, $h$;
these two equations are analytically equivalent.
\begin{eqnarray}
\frac{\partial(\rho h)}{\partial t} + \nabla\cdot(\rho h\Ub)
 &=& \frac{Dp}{Dt} + \rho\Hnuc \nonumber \\
%
\frac{\partial(\rho e)}{\partial t} + \nabla\cdot(\rho e \Ub)
 &=& - p\nabla\cdot \Ub + \rho\Hnuc \nonumber
\end{eqnarray}
In low Mach number combustion, $Dp/Dt = 0,$ which makes the enthalpy
equation preferable to the internal energy equation because we don't
need to evaluate $p \nabla \cdot \Ub.$

%-----------------------------------------------------------------------------
\section{Derivation of Velocity Constraint}

Differentiating the equation of state, written in the form, $p =
p(\rho,T,X_k),$ along particle paths, we can write
\begin{equation}
\frac{D p}{Dt}  = p_\rho \frac{D \rho}{Dt} + p_T \frac{D T}{Dt} + \sum_k p_{X_k} \frac{D X_k}{Dt}
\end{equation}
Then, by rearranging the terms, we get
\begin{equation}
\frac{D \rho}{Dt}  = -\rho \nabla \cdot \Ub =
    \frac{1}{p_\rho}
    \left( \frac{D p}{Dt} - p_T \frac{D T}{Dt}
                          - \sum_k p_{X_k} {\omegadot}_k \right) \enskip ,
\end{equation}
with $p_\rho = \left.\partial p/\partial \rho\right|_{X_k,T}$,
$p_{X_k} = \left.\partial p/\partial X_k \right|_{T,\rho,(X_j,j\ne k)}$,
and $p_T = \left.\partial p/\partial T\right|_{\rho,X_k}$.

\subsection{Using the Enthalpy Equation}

Now writing $h = h(p,T,X_k)$ and expanding $Dh/Dt$ and using the
enthalpy evolution equation:
\begin{equation}
\rho \frac{D h}{Dt}  = \rho \left( h_p \frac{D p}{Dt} + c_p \frac{D T}{Dt} + \sum_k h_{X_k} \frac{D X_k}{Dt} \right)
                     = \frac{D p}{D t} + \rho \Hnuc
\end{equation}
we can express $DT/Dt$ in terms of
\begin{equation}
\frac{DT}{Dt} = \frac{1}{\rho c_p} \left( (1 - \rho h_p) \frac{D p}{D t}
- \sum_k \rho \xi_k \omegadot_k + \rho \Hnuc \right) \enskip , \label{eq:dTdt}
\end{equation}
where $c_p = \left.\partial h/\partial T\right|_{p,X_k}$ is the
specific heat at constant pressure,
$\xi_k = \left.\partial h/\partial X_k \right|_{p,T}$,
and $h_p = \left.\partial h/\partial p\right|_{T,X_k}.$

We could then write,
\begin{eqnarray*}
\nabla \cdot \Ub &=& \frac{1}{\rho p_\rho} \left(
- \frac{D p}{D t} + \frac{p_T}{\rho c_p}
  \left( (1 - \rho h_p) \frac{D p}{D t} - \rho \sum_k \xi_k \omegadot_k + \rho \Hnuc \right)
+ \sum_k p_{X_k} \omegadot_k \right)  \\
                 &=& \frac{1}{\rho p_\rho}
  \left( \frac{p_T}{\rho c_p}(1  - \rho h_p) - 1 \right) \frac{D p}{D t}
 + \frac{1}{\rho p_\rho} \left(
  \frac{p_T}{\rho c_p} (\rho \Hnuc - \rho \sum_k \xi_k   \omegadot_k)
                               + \sum_k p_{X_k} \omegadot_k \right) \enskip .
\end{eqnarray*}
When we derived this expression we explicitly retained the dependence
of $h$ on $p$, as shown by the presence of the $h_p$ term.

Then, replacing $p$ by $p_0(r)$, $Dp/Dt$ becomes $\Ub \cdot
\nabla p_0$, and the divergence constraint can be written
\begin{equation}
\nabla \cdot \Ub + \alpha \Ub \cdot \nabla p_0 =
\frac{1}{\rho p_\rho} \left(
   \frac{p_T}{\rho c_p} \left(
  - \sum_k\rho  \xi_k \omegadot_k + \rho \Hnuc \right)
 + \sum_k p_{X_k} \omegadot_k \right)  \equiv \tilde{S} \enskip \label{eq:full_divu_constraint} ,
\end{equation}
where we define
\begin{equation}
\alpha(\rho,T) \equiv - \left( \frac{(1 - \rho h_p )p_T - \rho c_p}{\rho^2
  c_p p_\rho} \right) \enskip . \label{eq:alphadef}
\end{equation}

\subsection{Using the Energy Equation}

Now writing $e = e(p,T,X_k)$ and expanding $De/Dt$ and using the
energy evolution equation:
\begin{equation}
\rho \frac{D e}{Dt}  = \rho \left( e_p \frac{D p}{Dt} + e_T \frac{D T}{Dt} + \sum_k e_{X_k} \frac{D X_k}{Dt} \right)
                     = -p \nabla \cdot \Ub + \rho \Hnuc
\end{equation}
we can express $DT/Dt$ in terms of
\begin{eqnarray*}
\frac{DT}{Dt} &=& \frac{1}{\rho e_T} \left(
                  -p \nabla \cdot \Ub + \rho \Hnuc
                 - \rho e_p \frac{D p}{Dt}
                 - \rho \sum_k e_{X_k} \omegadot_k   \right)
\end{eqnarray*}
where $e_T = \left.\partial e/\partial T\right|_{p,X_k}$
and $e_p = \left.\partial e/\partial p\right|_{T,X_k}.$
Then
\begin{equation}
-\rho \nabla \cdot \Ub =
    \frac{1}{p_\rho} \left( \frac{D p}{Dt}
    - \frac{p_T}{\rho e_T} \left(
                 -p \nabla \cdot \Ub + \rho \Hnuc
                 - \rho e_p \frac{D p}{Dt}
                 - \rho \sum_k e_{X_k} \omegadot_k   \right)
    - \sum_k p_{X_k} {\omegadot}_k \right) \enskip ,
\end{equation}
which leads to
\begin{equation}
\left(-\rho -  \frac{p p_T}{\rho e_T p_\rho} \right) \nabla \cdot \Ub =
    \frac{1}{p_\rho} \left( (1 + \frac{e_p p_T}{e_T}) \frac{D p}{Dt}
                           - \frac{p_T}{\rho e_T} \left(
                                        \rho \Hnuc
                                        - \rho \sum_k e_{X_k} \omegadot_k   \right)
    - \sum_k p_{X_k} {\omegadot}_k \right) \enskip ,
\end{equation}
Note that we can replace $p$ by $p_0$ in the coefficient on the
l.h.s. as well as on the r.h.s.

%-----------------------------------------------------------------------------
\section{Comparison of Constraints}

If we set $\omegadot_k = \Hnuc = 0$ for simplicity, then the
constraint as derived using $h$ can be written
\begin{equation}
\nabla \cdot \Ub
+ \left( \frac{(1 - \rho h_p )p_T - \rho c_p}{\rho^2
  c_p p_\rho} \right) \frac{D p_0}{D t} = 0
\end{equation}
and the constraint derived using $e$ can be written
\begin{equation}
\nabla \cdot \Ub
+ \left( \frac{\rho e_T + \rho e_p p_T}{\rho^2 e_T p_\rho + p p_T} \right) \frac{D p_0}{D t} = 0
\end{equation}

We note that if we evaluate both constraints for $p = \rho R T,$ with
$h = c_p T,$ $e = c_v T,$ $c_p = c_v + R$ and $\gamma = c_p / c_v,$
then both constraints reduce to
\begin{equation}
\nabla \cdot \Ub + \frac{1}{\gamma p} \frac{D p_0}{D t} = 0
\end{equation}

%-----------------------------------------------------------------------------
\section{Enthalpy vs Energy Equation}

The full enthalpy equation, with no approximations, appears as:
\begin{equation}
\frac{\partial(\rho h)}{\partial t} = -\nabla\cdot(\rho h\Ub) +
  \frac{Dp}{Dt} + \rho\Hnuc \label{eq:enthalpy}
\end{equation}
Here, $h = e + p/\rho$ is the specific enthalpy, with $e$ the specific
internal energy.  In the low Mach number formulation, we replace $p$
with $p_0$ in the $Dp/Dt$ term, however, the definition of enthalpy
implicitly contains a pressure.  When calling the equation of state,
we take $h$ and $\rho$ as inputs.  The equation of state is expressed
in terms of $T$ and $\rho$, so it iterates until it finds the $h$ that
we desire.  This $h$ will be of the form $h = e + p_\mathrm{EOS}/\rho$,
where $p_\mathrm{EOS}$ is the pressure returned from the EOS.  Note that
$p_\mathrm{EOS}$ may not be equal to $p_0$---this may be what
causes us to drive off of the constraint.

The mismatch between the pressure implicit in the definition of $h$
and $p_0$ can be seen by substituting $h = e + p/\rho$ into the
enthalpy equation, where we replace $p$ with $p_0$ in the $Dp/Dt$ term:
\begin{eqnarray}
\frac{\partial(\rho h)}{\partial t} &=& -\nabla\cdot(\rho h\Ub) +
  \frac{Dp_0}{Dt} + \rho\Hnuc \nonumber \\
%
\frac{\partial(\rho e)}{\partial t} + \frac{\partial p}{\partial t} &=&
 -\nabla\cdot(\rho e\Ub) -\nabla\cdot(p\Ub) + \frac{Dp_0}{Dt} + \rho\Hnuc \nonumber \\
%
\frac{\partial(\rho e)}{\partial t} &=&
 -\nabla\cdot(\rho e\Ub) - p\nabla\cdot\Ub + \rho\Hnuc +
  \left \{ \frac{Dp_0}{Dt} - \frac{Dp}{Dt} \right \} \nonumber
\end{eqnarray}
However, if we solve the evolution equation for $e$ we would
substitute $p_0$ for $p$ in this equation as well.  Thus, we can pose
the situation as the following.  If we solve the evolution equation
for $h$ then we effectively are solving
\begin{equation}
\frac{\partial(\rho e)}{\partial t} +
  \nabla\cdot(\rho e\Ub) = -p \; \nabla\cdot\Ub + \rho\Hnuc +
  \left \{ \frac{Dp_0}{Dt} - \frac{Dp}{Dt} \right \} \nonumber
\end{equation}
but if we solve the evolution equation for $e$ we are effectively solving
\begin{equation}
\frac{\partial(\rho e)}{\partial t} +
  \nabla\cdot(\rho e\Ub) = p_0 \nabla\cdot\Ub + \rho\Hnuc \nonumber
\end{equation}

The second equation subtracted from the first gives:
\begin{equation} \label{eq:difference between h and e equations}
  \frac{D (p_0 - p)}{Dt} - (p_0 - p)  \nabla \cdot \Ub = 0,
\end{equation}
but this equation is only true, in general, if $p=p_0$.

Suppose we solve the current enthalpy equation, but when we call the
EOS, we subtract $p_0$ from $\rho h$ and then call the EOS with $e$
instead of $h$.  This is equivalent to:
\begin{eqnarray}
\frac{\partial(\rho h)}{\partial t} &=& -\nabla\cdot(\rho h\Ub) +
  \frac{Dp_0}{Dt} + \rho\Hnuc \nonumber \\
%
\frac{\partial(\rho e)}{\partial t} + \frac{\partial p_0}{\partial t} &=&
 -\nabla\cdot(\rho e\Ub) -\nabla\cdot(p_0 \Ub) + \frac{Dp_0}{Dt} + \rho\Hnuc \nonumber \\
%
\frac{\partial(\rho e)}{\partial t} &=&
 -\nabla\cdot(\rho e\Ub) - p_0 \nabla\cdot\Ub + \rho\Hnuc  \nonumber
\end{eqnarray}
which is identical to solving the energy equation with $p\to p_0$.
This option is enabled in \maestro\ via {\tt
  use\_eos\_e\_instead\_of\_h = T}.

\subsection{Constant $\gamma$ Gas}
Going back to the constant $\gamma$, ideal gas EOS, we can rewrite the
enthalpy equation as a pressure evolution equation
\begin{eqnarray}
  \frac{\partial\rho h}{\partial t} + \nabla\cdot\left(\rho h\Ub\right) &=& \frac{Dp_0}{Dt} + \rho H {} \nonumber\\
  \frac{\gamma}{\gamma-1}\frac{\partial p}{\partial t} + \frac{\gamma}{\gamma-1}\nabla\cdot\left(p\Ub\right) &=& \frac{Dp_0}{Dt} + \rho H {} \nonumber\\
  \frac{Dp}{Dt} &=& -p\nabla\cdot\Ub + \frac{\gamma}{\gamma-1}\left(\frac{Dp_0}{Dt}+\rho H\right) \label{eq:H:pressure evolution}.
\end{eqnarray}
Similarly, we can derive a pressure evolution equation from the energy equation
\begin{equation}
  \frac{Dp}{Dt} = -p\nabla\cdot\Ub - \left(\gamma-1\right)p_0\nabla\cdot\Ub + \rho H \label{eq:e:pressure evolution}
\end{equation}
Now, if we further make the assumption that $p_0$ is constant,
$Dp_0/Dt = 0$, then the divergence constraint for such a gas reduces
to
\begin{equation}
\nabla\cdot\Ub = \frac{\gamma-1}{\gamma p_0}\rho H \label{eq:div constraint for constant gamma}.
\end{equation}
Plugging this back into either of \eqref{eq:H:pressure evolution} or
\eqref{eq:e:pressure evolution} gives
\begin{equation}
  \frac{Dp}{Dt} = \frac{\gamma-1}{\gamma}\left(1-\frac{p}{p_0}\right)\rho H. \label{eq:pressure evolution constant gamma}
\end{equation}
If $p_0$ is assumed constant and using \eqref{eq:div constraint for
  constant gamma}, the difference between the enthalpy equation and
the energy equation, \eqref{eq:difference between h and e equations},
can be rewritten as
\[
-\frac{Dp}{Dt} - \frac{\gamma}{\gamma-1}\left(1-\frac{p}{p_0}\right)\rho H = 0,
\]
where the equality holds from \eqref{eq:pressure evolution constant
  gamma}. In other words, for the constant $\gamma$ gas we have
$p=p_0$ as expected.

%-----------------------------------------------------------------------------
\section{Outstanding Questions}

\begin{enumerate}
\item Why do we want to start with enthalpy instead of internal energy?

   We believe that the original desire stems from our experience with
   smallscale combustion.  There, stratification is not important and
   $Dp_0/Dt = 0$, so the enthalpy equation becomes a conservation
   equation for $(\rho h)$.

\item Should we call the EOS with $h$ as is, or call the EOS with $e =
  h - p_0 / \rho$?

\item When we stay on the constraint, i.e. $p_{EOS} = p_0$, then the
  equations for $e$ and for $h$ are identical.  However, once we are
  off the constraint, do the terms in the current evolution equation
  for $h$ serve to drive us back to the constraint?  Recall our
  current "volume discrepancy factor" acts as a source term which
  modifies the divergence constraint, which effectively modifies both
  $\rho$ and $T$ (or $e$ or $h$).  The term in the enthalpy equation
  only modifies $\rho.$ Is this relevant and/or useful??  Recall that
  the current "volume discrepancy factor" takes the form of adding to
  the r.h.s. of the constraint:
\begin{equation}
\nabla \cdot(\beta_0\Ub) = \beta_0\left(S-\frac{1}{\overline{\Gamma_1} p_0}
       \frac{\partial p_0}{\partial t} - \frac{f}{\overline{\Gamma_1} p_0}
       \frac{p_0-p_\text{EOS}}{\dt}\right)
\end{equation}

%\item We substitute $p$ for $p_0$ in two places: 1) in the derivation of the
%constraint, 2) in the enthalpy evolution equation.   Are these substitutions
%consistent with each other?  If they are not consistent, which of these
%substitutions is responsible for filtering the acoustics from the system?

\item Suppose we corrected the $h$ (or $e$ equation) by using the full
  $p$ instead of $p_0$?  Would this be more or less consistent (one
  could imagine doing this as a correction after solving for $\pi$
  earlier in the timestep).

%\item In the EOS, we take $h$ and construct $e$ by subtracting off
%  $p$---depending on the inputs to the EOS, we treat the `$p$' part of $h$
%  differently.
%
%   if we come into the EOS with $\rho$, $T$, then we get $p$ and $e$ from
%   the EOS and compute $h$.
%
%   if we come into the EOS with $\rho$, $h$, then we find a $T$ such that
%   $e + p(\rho,T)/\rho = h$, where $p(\rho,T)$ is the pressure from the EOS.
%   It is not necessarily equal to $p_0$.
%
%   if we come into the EOS with $\rho$, $p_0$, then find a $T$ that matches
%   $p_0$, and then $h = e + p_0/\rho$.  In this case, there is no mismatch
%   in the enthalpy equation.

\item In computing the thermodynamic coefficients in $S$ for the
  projection, don't we need these to be in terms of $p_0$ instead of
  $p(h,\rho)$?

\end{enumerate}


\documentclass[]{article}

%opening
\title{Mobile Weka}
\author{Matthew Swartwout \and James Zhang}
\begin{document}
\maketitle

\section{Introduction of Algorithms}
\subsection{J48}
J48 is an implementation of the C4.5 algorithm. This algorithm builds decision trees to classify inputs. These decision trees check each attribute and calculate how well the data set can be split using that attribute. Once the best attribute for splitting has been decided, this is added as a decision node to the tree. This then moves onto the next feature and adds that node as a child of the tree based on the best split. Once all the splits have been determined, the final decision tree can be used to classify the data.

\subsection{SVM}
SVM stands for Support Vector Machine. The basis of this is that there should be a linear function that can separate the data set the best. There are an infinite number of possible functions that can separate the functions. Thus we look for the function that has the largest margin. This can be done by transposing the data into a higher dimensional space, which is often done by using the dot product. Once you've found the function with the highest margin, this can be used to classify the input data.

\subsection{Naive Bayes}
Naive Bayes is an classification algorithm based off applying Bayes' theorem and assuming independence between all of the labels. Bayes' theorem says that P(A|B) = P(A)P(B|A)/P(B). The input to Naive Bayes' is our data, which is represented as a vector of n features (x), and we are trying to calculate the probability that these features will result in a given class (C), e.g. P(C|x). This can be rewritten with Bayes' Theorem as P(C)P(x|C)/p(x). Each of these three probabilities can be easily calculated with the provided data set. Doing this over each of the n features included in x and we can obtain the probability that a certain set of features.

\subsection{RBF Network}
RBF Network stands for Radial Basis Function Network. This is a type of neural network that can be used for classification tasks. The network is made of three layers: an input layer, a hidden layer with a non-linear activation function, and a linear output layer. The input vector is our vector with n features. This is given to every neuron in the input layer. Each neuron contains one of the vectors from the training set. It then outputs a value between 0 and 1 depending on how similar the two vectors are. Each node in the output layer represents one of the possible classification categories. Each output node compares the outputs from the activation functions and creates a score for that class. Looking at the value of the output nodes, the network can classify the input.

\section{Data Preprocessing}

\section{Experimental Results}
\subsection{Experiment 1}
\begin{tabular}{c | c | c| c | c}
Algorithm & Recall & Precision & Correctly Classified & Incorrectly Classified \\ \hline
J48 & 1.0 & 0.9375 & 428 & 10 \\
SVM & 0.0133 & 1.0 & 290 & 148 \\
Naive Bayes & 1.0 & 0.9375 &  428 & 10 \\
RBF Network & 1.0 & 0.9554 & 431 & 7 \\
\end{tabular}
\subsection{Experiment 2}
\begin{tabular}{c | c | c| c | c}
Algorithm & Recall & Precision & Correctly Classified & Incorrectly Classified \\ \hline
J48 & 1.0 & 1.0 & 1355 & 0 \\
SVM & 0.0 & 0.0 & 456 & 899 \\
Naive Bayes & 0.9652 & 0.7296 & 1234 & 121 \\
RBF Network & 0.9340 & 0.8432 & 1239 & 116 \\
\end{tabular}
\subsection{Experiment 3}
\subsubsection{Experiment 1}
\begin{tabular}{c | c | c| c | c}
	Algorithm & Recall & Precision & Correctly Classified & Incorrectly Classified \\ \hline
	J48 & 1.0 & 1.0 & 70 & 0 \\
	SVM & 1.0 & 1.0 & 70 & 0 \\
	Naive Bayes & 1.0 & 1.0 & 70 & 0 \\
	RBF Network & 1.0 & 1.0 & 70 & 0 \\
\end{tabular}
\subsubsection{Experiment 2}
\begin{tabular}{c | c | c| c | c}
	Name & Algorithm & Recall & Precision & Correctly Classified & Incorrectly Classified \\ \hline
	Matt & J48 & 1.0 & 1.0 & 101 & 0 \\
	Matt & SVM & 0.08 & 1.0 & 29 & 72 \\
	Matt & Naive Bayes & 1.0 & 1.0 & 101 & 0 \\
	Matt & RBF Network & 1.0 & 1.0 & 101 & 0 \\
	James & J48 & 1.0 & 1.0 & 101 & 0 \\
	James & SVM & 0.08 & 1.0 & 29 & 72 \\
	James & Naive Bayes & 1.0 & 1.0 & 101 & 0 \\
	James & RBF Network & 1.0 & 1.0 & 101 & 0 \\
\end{tabular}
\section{Simple Results Analysis}

\section{Project Contributions}
\end{document}


\subsection{Theoretical prediction of the channel capacity}
\label{sec_channcap}

We now turn our focus to the channel capacity, which is a metric by which we can
quantify the degree to which cells can measure the environmental state (in this
context, the inducer concentration). The channel capacity is defined as the
mutual information $I$ between input and output (\eref{eq_mutual_info}),
maximized over all possible input (IPTG) distributions $P(c)$. If used as a
metric of how reliably a signaling system can infer the state of the external
signal, the channel capacity, when measured in bits, is commonly interpreted as
the logarithm of the number of states that the signaling system can properly
resolve. For example, a signaling system with a channel capacity of $C$ bits is
interpreted as being able to resolve $2^C$ states, though channel capacities
with fractional values are allowed. We therefore prefer the Bayesian
interpretation that the mutual information quantifies the improvement in the
inference of the input when considering the output compared to just using the
prior distribution of the input by itself for prediction \cite{Voliotis2014a,
Bowsher2014}. Under this interpretation a channel capacity of a fractional bit
still quantifies an improvement in the ability of the signaling system to infer
the value of the extracellular signal compared to having no sensing system at
all.

Computing the channel capacity implies optimizing over an infinite space of
possible distributions $P(c)$. For special cases in which the noise is small
compared to the dynamic range, approximate analytical equations have been
derived \cite{Tkacik2008a}. But given the high cell-to-cell variability that
our model predicts, the conditions of the so-called small noise approximation
are not satisfied. We therefore appeal to a numerical solution known as the
Blahut-Arimoto algorithm \cite{Blahut1972} (See \siref{supp_channcap} for
further details). \fref{fig5_channcap}(A) shows zero-parameter fit predictions
of the channel capacity as a function of the number of repressors for different
repressor-DNA affinities (solid lines). These predictions are contrasted with
experimental determinations of the channel capacity as inferred from
single-cell fluorescence intensity distributions taken over 12 different
concentrations of inducer. Briefly, from single-cell fluorescence measurements
we can approximate the input-output distribution $P(p \mid c)$. Once these
conditional distributions are fixed, the task of finding the input distribution
at channel capacity becomes a computational optimization routine that can be
undertaken using conjugate gradient or similar algorithms. For the particular
case of the channel capacity on a system with a discrete number of inputs and
outputs the Blahut-Arimoto algorithm is built in such a way that it guarantees
the convergence towards the optimal input distribution (See
\siref{supp_channcap} for further details). \fref{fig5_channcap}(B) shows
example input-output functions for different values of the channel capacity.
This illustrates that having access to no information (zero channel capacity)
is a consequence of having overlapping input-output functions (lower panel). On
the other hand, the more separated the input-output distributions are (upper
panel) the higher the channel capacity can be.

All theoretical predictions in \fref{fig5_channcap}(A) are systematically above
the experimental data. Although our theoretical predictions in
\fref{fig5_channcap}(A) do not numerically match the experimental inference of
the channel capacity, the model does capture interesting qualitative features of
the data that are worth highlighting. On one extreme, for cells with no
transcription factors, there is no information processing potential as this
simple genetic circuit would be constitutively expressed regardless of the
environmental state. As cells increase the transcription factor copy number, the
channel capacity increases until it reaches a maximum before falling back down
at high repressor copy number since the promoter would be permanently repressed.
The steepness of the increment in channel capacity as well as the height of the
maximum expression is highly dependent on the repressor-DNA affinity. For strong
binding sites (blue curve in \fref{fig5_channcap}(A)) there is a rapid increment
in the channel capacity, but the maximum value reached is smaller compared to a
weaker binding site (orange curve in \fref{fig5_channcap}(A)). In
\siref{supp_empirical} we show using the small noise approximation
\cite{Tkacik2008, Tkacik2008a} that if the systematic deviation of our
predictions on the cell-to-cell variability was explained with a multiplicative
constant, i.e. all noise predictions can be corrected by multiplying them by a
single constant, we would expect the channel capacity to be off by a constant
additive factor. This factor of $\approx 0.43$ bits can recover the agreement
between the model and the experimental data.

\begin{figure}[h!]
	\centering \includegraphics
  {./fig/main/fig5_channcap.pdf}
	\caption{\textbf{Comparison of theoretical and experimental channel
	capacity.} (A) Channel capacity as inferred using the Blahut-Arimoto
	algorithm \cite{Blahut1972} for varying number of repressors and
	repressor-DNA affinities. All inferences were performed using 12 IPTG
	concentrations as detailed in the Methods. Curves represent zero-parameter
	fit predictions made with the maximum entropy distributions as shown in
	\fref{fig4_maxent}. Points represent inferences made from single cell
	fluorescence distributions (See \siref{supp_channcap} for further details).
	Theoretical curves were smoothed using a Gaussian kernel to remove
	numerical precision errors. (B) Example input-output functions in opposite
	limits of channel capacity. Lower panel illustrates that zero channel
	capacity indicates that all distributions overlap. Upper panel illustrates
	that as the channel capacity increases, the separation between
	distributions increases as well. Arrows point to the corresponding channel
	capacity computed from the predicted distributions.}
  \label{fig5_channcap}
\end{figure}


\documentclass{article}

\usepackage{fancyhdr}
\usepackage{extramarks}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{amsfonts}
\usepackage{tikz}
\usepackage{physics}
\usepackage[plain]{algorithm}
\usepackage{algpseudocode}
\usepackage{hyperref}

\usetikzlibrary{automata,positioning}

%
% Basic Document Settings
%

\topmargin=-0.45in
\evensidemargin=0in
\oddsidemargin=0in
\textwidth=6.5in
\textheight=9.0in
\headsep=0.25in

\linespread{1.1}

\pagestyle{fancy}
\lhead{\hmwkAuthorName}
\chead{\hmwkClass\ : \hmwkTitle}
\rhead{\firstxmark}
\lfoot{\lastxmark}
\cfoot{\thepage}

\renewcommand\headrulewidth{0.4pt}
\renewcommand\footrulewidth{0.4pt}

\setlength\parindent{0pt}

%
% Create Problem Sections
%
\newcommand{\be}{\begin{equation}}
\newcommand{\ee}{\end{equation}}
\newcommand{\bes}{\begin{equation*}}
\newcommand{\ees}{\end{equation*}}
\newcommand{\bea}{\begin{flalign*}}
\newcommand{\eea}{\end{flalign*}}

\newcommand{\enterProblemHeader}[1]{
    \nobreak\extramarks{}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
}

\newcommand{\exitProblemHeader}[1]{
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \stepcounter{#1}
    \nobreak\extramarks{Problem \arabic{#1}}{}\nobreak{}
}

\setcounter{secnumdepth}{0}
\newcounter{partCounter}
\newcounter{homeworkProblemCounter}
\setcounter{homeworkProblemCounter}{1}
\nobreak\extramarks{Problem \arabic{homeworkProblemCounter}}{}\nobreak{}

%
% Homework Problem Environment
%
% This environment takes an optional argument. When given, it will adjust the
% problem counter. This is useful for when the problems given for your
% assignment aren't sequential. See the last 3 problems of this template for an
% example.
%
\newenvironment{homeworkProblem}[1][-1]{
    \ifnum#1>0
        \setcounter{homeworkProblemCounter}{#1}
    \fi
    \section{Problem \arabic{homeworkProblemCounter}}
    \setcounter{partCounter}{1}
    \enterProblemHeader{homeworkProblemCounter}
}{
    \exitProblemHeader{homeworkProblemCounter}
}

%
% Homework Details
%   - Title
%   - Due date
%   - Class
%   - Section/Time
%   - Instructor
%   - Author
%

\newcommand{\hmwkTitle}{Assignment\ \#1}
\newcommand{\hmwkDueDate}{Due on 18th January, 2019}
\newcommand{\hmwkClass}{Advanced Statistical Mechanics}
\newcommand{\hmwkClassTime}{}
\newcommand{\hmwkClassInstructor}{}
\newcommand{\hmwkAuthorName}{\textbf{Aditya Vijaykumar}}

%
% Title Page
%

\title{
    %\vspace{2in}
    \textmd{\textbf{\hmwkClass:\ \hmwkTitle}}\\
    \normalsize\vspace{0.1in}\small{\hmwkDueDate\ }\\
%    \vspace{3in}
}

\author{\hmwkAuthorName}
\date{}

\renewcommand{\part}[1]{\textbf{\large Part \Alph{partCounter}}\stepcounter{partCounter}\\}

%
% Various Helper Commands
%

% Useful for algorithms
\newcommand{\alg}[1]{\textsc{\bfseries \footnotesize #1}}

% For derivatives
\newcommand{\deriv}[1]{\frac{\mathrm{d}}{\mathrm{d}x} (#1)}

% For partial derivatives
\newcommand{\pderiv}[2]{\frac{\partial}{\partial #1} (#2)}

% Integral dx
\newcommand{\dx}{\mathrm{d}x}

% Alias for the Solution section header
\newcommand{\solution}{\textbf{\large Solution}}

% Probability commands: Expectation, Variance, Covariance, Bias
\newcommand{\E}{\mathrm{E}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\Cov}{\mathrm{Cov}}
\newcommand{\Bias}{\mathrm{Bias}}

\begin{document}

\maketitle
(\textbf{Acknowledgements} - I would like to thank Aditya Sharma and Junaid Majeed for discussions.)

\begin{homeworkProblem}[1]
	\textbf{Part (a)}\\
	Given that
	\begin{align*}
	X &= \dfrac{\sum_{i=1}^N X_i}{\sigma \sqrt{N}} \\
	\ev{X} &= \dfrac{\sum_{i=1}^N \ev{X_i}}{\sigma \sqrt{N}} = \dfrac{\sum_{i=1}^N 0}{\sigma \sqrt{N}} = 0\\
	\sqrt{\ev{X^2}} &= \dfrac{\sqrt{\sum_{i=1}^N \sum_{j=1}^N \ev{X_i X_j}}}{\sigma \sqrt{N}} \\
	&=  \dfrac{\sqrt{\sum_{i=1}^N \sum_{j=1}^N \ev{X_i^2} \delta_{ij}}}{\sigma \sqrt{N}} \impliedby \qq{independent variables, hence covariance is zero}\\
	&= \dfrac{\sqrt{\sum_{i=1}^N\sigma^2 }}{\sigma \sqrt{N}}\\
	\sigma_X &= \dfrac{\sqrt{N \sigma^2 }}{\sigma \sqrt{N}} = 1
	\end{align*}

	\textbf{Part (b)}\\
	Let $ x, y - x, 1- y $ be the lengths of the 3 sticks after breaking. Triangle inequality gives us the following conditions on the stick,
	\begin{align*}
	x < 1-x \implies x < \dfrac{1}{2}\\
	y - x < x + 1 - y \implies y -x > \dfrac{1}{2}\\
	1-y < y \implies y > \dfrac{1}{2}
	\end{align*}
	In the unit square of the $ x-y $ plane, we need to find the intersection of the above regions. The first and last regions have an intersection of are$ \dfrac{1}{4} $ in the plane, and the second region automatically includes this intersection. Hence the required probability is $ \dfrac{1}{4}$.

	\textbf{Part (c)}\\
	The histograms are plotted below,
	\begin{figure}[!h]
		\centering
		\includegraphics[scale=0.5]{trace_u.png}
		\includegraphics[scale=0.5]{d.png}
		\caption{L : Trace Distribution, R: Eigenvalue Spacing Distribution for Uniform case}
	\end{figure}

	\begin{figure}[!h]
		\centering
		\includegraphics[scale=0.5]{distr_n.png}
		\includegraphics[scale=0.5]{trace_n.png}
		\caption{L : Trace Distribution, R: Eigenvalue Spacing Distribution for Normal case}
	\end{figure}

	\textbf{Part (d)}\\
	In our case, in time interval $ dt $, the walker can go left with probability $ \alpha dt$, right with probability $ \alpha dt$ and can stay at the same position with probability $ 1 - 2 \alpha dt $. So then, at position $ i $ and time $ t + dt $,
	\begin{align*}
	P(i, t + dt) &= P(i,t) (1 - 2 \alpha dt) + (P(i+1,t) + P(i-1,t)) \alpha dt\\
	\pdv{P(i, t)}{t} &= - 2 P(i,t) \alpha   + (P(i+1,t) + P(i-1,t)) \alpha \\
	\end{align*}
	We solve the above equation by writing down the Fourier series. Every step away from $ i $th site will introduce factors of $ e^{jk} $ in the Fourier space,
	\begin{align*}
	\pdv{P(k,t)}{t} &= (-2 \alpha + e^{jk} + e^{-jk}) P(k,t)
	\end{align*}
	We know that $ P(x,t=0) = \delta_{x,0} \implies P(k,t=0) = 1   $ and then the solution to the above equation is,
	\begin{align*}
	P(k,t) &= e^{ (-2 \alpha + e^{jk} + e^{-jk})t}\\
	&= e^{- 2 \alpha t} \sum_{x=-\infty}^{\infty } e^{2jkx \cos k}\\
	P(k,t)&= e^{- 2 \alpha t} \sum_{x=-\infty}^{\infty } e^{2jkx} I_x(t)
	\end{align*}
	From which we can see by inverse Fourier,  $ P(x,t) =  e^{- 2 \alpha t} I_x(t) $, where $ I_x(t) $ is the Bessel function of the first kind.

	\textbf{Part (e)}\\
	Let's say the random walker takes $ x $ steps rightward and $ y $ steps leftward. For this walker to be at $ r $ after $ N $ steps, $ x + y = N $ and $ x-y = r $ $ \implies x = \dfrac{N+r}{2} $ and $ y = \dfrac{N - r}{2} $. The probability $ P(r,N) $ is then,
	\begin{align*}
	P(r,N) &= {N \choose r} \dfrac{1}{2^{x}} \dfrac{1}{2^{y}} \\
	&= {N \choose x} \dfrac{1}{2^{N}}
	\end{align*}

	For very large $ N $,
	\begin{align*}
	{n \choose x} = \dfrac{N!}{x! y!} &= \dfrac{N!}{\qty(\dfrac{N+r}{2})!\qty(\dfrac{N-r}{2})!}\\
	&=\dfrac{e^{-N} N^N}{(N+r)^{(N+r)/2} (N-r)^{(N-r)/2} 2^{-N} e^{-N}}\\
	&= \dfrac{N^N}{(1+r/N)^{(N+r)/2} (1-r/N)^{(N-r)/2} 2^{-N} N^N }\\
	P(r,N) &= \dfrac{1}{(1+r/N)^{(N+r)/2} (1-r/N)^{(N-r)/2}}\\
	P(r,N) &= [{(1+r/N)^{(1+r/N)} (1-r/N)^{(1-r/N)}}]^{-N/2}
	\end{align*}
	Comparing with the form $ P(r,N) = e^{-N \phi(r/N)} $, we get,
	\begin{equation*}
	\phi(x) =\dfrac{ (1+x) \ln (1+x) + (1 -x ) \ln (1-x)}{2}
	\end{equation*}

\end{homeworkProblem}

\begin{homeworkProblem}[2]
	\textbf{Part (a)}\\
	For constant number of particles,
	\begin{equation*}
	dU = -PdV + TdS + hdM
	\end{equation*}
	The enthalpy $ E $ is defined as $E = U + PV$,
	\begin{align*}
	dE &= dU + PdV + VdP = -PdV + TdS + hdM + PdV + VdP \\
	&=  TdS + hdM  + VdP
	\end{align*}
	The Helmholtz Potential $ A $ is defined as $A = U - TS$,
	\begin{align*}
	dA &= dU - TdS - SdT  = -PdV + TdS + hdM - TdS - SdT  \\
	&= -PdV + hdM  - SdT
	\end{align*}
	The Gibbs Potential $ G = E - TS $,
	\begin{align*}
	dG &=  TdS + hdM  + VdP - TdS - SdT\\
	&=   hdM  + VdP - SdT
	\end{align*}
	As all the quantities are related by Legendre Transforms, knowledge of one of the quantities is enough to calculate all the others.

	\textbf{Part (b)}\\
	$ C_x $ and $ \kappa_x $ are defined as follows,
	\begin{equation*}
	C_x = \eval{\dv{Q}{T}}_{x = const.} \qq{} \kappa_x = -\eval{\dfrac{1}{V}\dv{V}{P}}_{x = const.}
	\end{equation*}
	Consider the following,
	\begin{align*}
	C_P = T \eval{\dv{S}{T}}_{P = const} = -T \pdv[2]{G}{T} \implies \pdv[2]{G}{T} < 0 \implies G(T) \text{ is concave}\\
	C_V = T \eval{\dv{S}{T}}_{V = const} = -T \pdv[2]{A}{T} \implies \pdv[2]{A}{T} < 0 \implies A(T) \text{ is concave}\\
	\kappa_T = -\eval{\dfrac{1}{V}\dv{V}{P}}_{T = const.} = -\dfrac{1}{V} \pdv[2]{G}{P} \implies \pdv[2]{G}{P} < 0 \implies G(P) \text{ is concave}\\
	\kappa_T = -\eval{\dfrac{1}{V}\dv{V}{P}}_{T = const.} = \dfrac{1}{V} \dfrac{1}{\pdv[2]{A}{V} }\implies \pdv[2]{A}{V} > 0 \implies A(V) \text{ is convex}\\
	\end{align*}
\end{homeworkProblem}

\begin{homeworkProblem}[3]
	Given $ S_{Gibbs} = - \sum_a p_\alpha \log p_\alpha $, which we have to maximize under $ \sum_\alpha p_\alpha = 1 $. We use the method of Lagrange multipliers, $ f(p_\alpha) = - \sum_a p_\alpha \log p_\alpha + \lambda (\sum_\alpha p_\alpha - 1) $,
	\begin{align*}
	\dv{f}{p_\gamma} &= - \log p_\gamma - 1 + \lambda = 0 \\
	\implies \lambda &= 1 + \log p_\gamma \implies p_\gamma = constant = \dfrac{1}{N}
	\end{align*}
	where $ N $ is the number of microstates. Hence, we have shown that the all microstates are equally likely in microcanonical ensemble.

	Similarly, we proceed to carry out the calculation for canonical ensemble and grand canonical ensemble. In this case, $ f(p_\alpha) = - \sum_a p_\alpha \log p_\alpha + \lambda_1 (\sum_\alpha p_\alpha - 1) +  \lambda_2 (\sum_\alpha p_\alpha E_\alpha - \ev{E}) $,
	\begin{align*}
	\dv{f}{p_\gamma} &= - \log p_\gamma - 1 + \lambda_1 + \lambda_2 E_\gamma = 0\\
	\implies p_\gamma &= \exp(\lambda_1 - 1) \exp(\lambda_2 E_\gamma)
	\end{align*}
	which is the expression for probability in the canonical ensemble \textit{ie}. some normalization times $ \exp(\beta E_\gamma) $.

	For the grand canonical ensemble, $ f(p_\alpha) = - \sum_a p_\alpha \log p_\alpha + \lambda_1 (\sum_\alpha p_\alpha - 1) +  \lambda_2 \qty(\sum_\alpha p_\alpha E_\alpha - \ev{E}) +\lambda_3 (\sum_\alpha N_\alpha p_\alpha - \ev{N})$,
	\begin{align*}
	\dv{f}{p_\gamma} &= - \log p_\gamma - 1 + \lambda_1 + \lambda_2 E_\gamma + \lambda_3 N_\gamma = 0\\
	\implies p_\gamma &= \exp(\lambda_1 - 1) \exp(\lambda_2 E_\gamma + \lambda_3 N_\gamma)
	\end{align*}
	which is the expression for probability in the canonical ensemble \textit{ie}. some normalization times $ \exp(\beta E_\gamma + \mu N_\gamma) $.

	\textbf{Part (b)}\\
	We first write down the partition function for a classical ideal gas,
	\begin{align*}
	Z &= \dfrac{1}{h^{3N} N!} \prod_{i=1}^{N} \int d^3q_i d^3 p_i \exp(-\beta p_i^2/2m)\\
	&= \dfrac{1}{h^{3N} N!} \prod_{i=1}^{N} V \qty(\dfrac{2 \pi m}{\beta})^{3/2}\\
	&= \dfrac{ V^N}{h^{3N} N!} \qty(\dfrac{2 \pi m}{\beta})^{3N/2}
	\end{align*}

	So then, the probability is given by,
	\begin{align*}
	P(N=N) &= \sum_{\epsilon_i} \dfrac{e^{\beta \mu N} e^{-\beta \epsilon_i}}{Z}\\
	&= \dfrac{e^{\beta \mu N}}{e^{\ev{N}}} \dfrac{1}{N!} \qty(\dfrac{V}{\lambda^3})^N\\
	&= \dfrac{e^{-\ev{N}} \ev{N}^N}{N!}
	\end{align*}
\end{homeworkProblem}

\begin{homeworkProblem}[4]
	\begin{align*}
	f_\nu (z ) &= \dfrac{1}{\Gamma(\nu)} \int_{0}^{\infty}  \dd x \dfrac{x^{\nu - 1}}{z^{-1} e^x + 1}\\
	&= \dfrac{1}{\Gamma(\nu)} \qty[ \eval{\dfrac{x^{\nu}}{\nu(z^{-1} e^x + 1)}}_{0}^{\infty} - \int_{0}^{\infty}  \dd x \dfrac{x^{\nu}}{\nu } \dv{z^{-1} e^x + 1}{x}]\\
	&= -\dfrac{1}{\Gamma(\nu+1)} \qty[ \dd x {x^{\nu}} \dv{z^{-1} e^x + 1}{x}]
 	\end{align*}
 	We change the variables to $ x=  \ln z + t$ and get,
 	\begin{align*}
 	f_\nu(z) &\approx -\dfrac{1}{\Gamma(\nu + 1)} \int_{-\infty}^\infty dt (\ln z + t )^\nu \dv{t} \qty(\dfrac{1}{z^{-1} z e^t + 1})\\
 	&\approx -\dfrac{1}{\Gamma(\nu + 1)} \int_{-\infty}^\infty dt \sum_m \dfrac{\nu!}{m! (\nu -m)!} t^m ( \ln z)^{\nu - m}  \dv{t} \qty(\dfrac{1}{e^t + 1})\\
 	&\approx \dfrac{1}{\Gamma(\abs{\nu - m} + 1)} \int_{-\infty}^\infty dt \sum_m ( \beta \mu)^{\nu - m}  I_m
 	\end{align*}

 	\textbf{Part (b)}\\
 	\begin{align*}
 	N &= \dfrac{V}{\lambda^3} f_{3/2} \\
 	&= \dfrac{V}{\lambda^3} \qty(\dfrac{(\beta \mu)^{3/2}}{\Gamma(5/2)}  +  \qty(\dfrac{(\beta \mu)^{-1/2}}{\Gamma(3/2)} 2 f_2^- (1) ))\\
 	&=\dfrac{V \beta^{-3/2}}{(2 \pi m)^{-3/2}} \qty(\dfrac{4\mu^3/2}{3} + 4 \mu^{-1/2} \beta^2 f_2^-(1))\\
 	&= \dfrac{4V \mu^{3/2}}{3 \pi (2 m \pi)^{3/2}} + \dfrac{4 V \mu^{-1/2} \beta f_2(1)}{\pi (2 m \pi )^{3/2}}
 	\end{align*}

 	\begin{align*}
 	E &= \dfrac{3 V (\beta)^{-5/2} }{2 \lambda^3 } f_{5/2}(z)\\
 	&= \dfrac{3V \beta^{-5/2}}{2 (2 \pi m)^{-3/2}} \qty[\dfrac{(\beta \mu)^{5/2}}{\Gamma (7/2)} + \dfrac{(\beta \mu)^{1/2}}{\Gamma (3/2)} (2 f_2^-(1))]\\
 	&= \dfrac{3V }{2 (2 \pi m)^{-3/2}} \qty[\dfrac{(\mu)^{5/2}}{\Gamma (7/2)} + \dfrac{(\beta)^{-2} (\mu)^{1/2}}{\Gamma (3/2)} (2 f_2^-(1))]
 	\end{align*}
 	\begin{equation*}
 	C_V = \lim\limits_{T \rightarrow 0} \pdv{E}{T} = \dfrac{3V \mu^{1/2} (2 \pi m)^{3/2}}{\Gamma(3/2)} k_B^2 T
 	\end{equation*}
\end{homeworkProblem}

\begin{homeworkProblem}[5]
	\textbf{Part (b)}\\
	We are given,
	\begin{equation*}
	H_N = \sum_{i=1}^N \dfrac{p_i^2}{2m} + \sum_{i=1}^{N-1} u(x_i - x_{i-1}) +v (x_1) + v(x_N)
	\end{equation*}
	One can write the expression of the partition function as follows,
	\begin{align*}
	Z &= \dfrac{1}{N!} \prod_{i}^N \int \int \dfrac{\dd{p_i} \dd{x_i}}{h} \exp(-\beta H_N) \\
	&= \dfrac{1}{N!h^N} \prod_{i}^N \int  \dd{p_i} \exp(-\beta p_i^2/2m) \prod_{i} \int \dd{x_i} \exp(-\beta \sum_{i=1}^{N-1} u(x_i - x_{i-1})  )\\
	&= \dfrac{1}{N!h^N} \qty(\dfrac{2 \pi m}{\beta})^{N/2} \prod_{i} \int \dd{x_i} \exp(-\beta \sum_{i=1}^{N-1} u(x_i - x_{i-1})  )
	\end{align*}
	Consider,
	\begin{align*}
	\prod_{i} \int \dd{x_i} \exp(-\beta \sum_{i=1}^{N-1} u(x_i - x_{i-1})  ) &= \prod_{i \ne 1} \int \dd{x_i} \exp(-\beta \sum_{i=1}^{N-1} u(x_i - x_{i-1})  + v(x_i) ) \int_{-\infty }^\infty \dd x_1 e^{-\beta u(x_2 - x_1) + v(x_1)}\\
	&= \prod_{i \ne 1} \int \dd{x_i} \exp(-\beta \sum_{i=1}^{N-1} u(x_i - x_{i-1})  + v(x_i) ) \int_{a/2 }^{x_2 -a } \dd x_1\\
	&= \prod_{i \ne 1} \int \dd{x_i} \exp(-\beta \sum_{i=1}^{N-1} u(x_i - x_{i-1})  + v(x_i) ) (x_2 - 1.5 a)
	\end{align*}
	We have done one integral, and we now similarly separate out integrals over the other $ x_i $'s. The constant terms will integrate to zero, while the terms involving $ x_i $'s can be simply integrated as power functions. After $ N $ integrals, one will be left with,
	\begin{align*}
	\int_{(N - a/2 )}^{L  - a/2} \dd x_N \dfrac{1}{(N-1)!} \exp(\beta u(x_N - x_{N-1}) - \beta v (x_N)) \qty(x_N - \dfrac{N-1}{2}a)^{N-1}\\
	&= \int_{(N - a/2 )}^{L  - a/2} \dd x_N \dfrac{1}{(N-1)!} \qty(x_N - (N-0.5)a)^{N-1}\\
	&=\dfrac{1}{N!} \qty(L - \dfrac{a}{2} - Na + \dfrac{a}{2})^N\\
	&= \dfrac{1}{N!} (L- Na )^N
	\end{align*}
	Hence, the full partition function of the problem,
	$ Z_N = \dfrac{1}{\lambda^N N!^2} (L-Na)^N$

	Now that we have the partition function, $ A = -k_B T \log Z_N $,
	\begin{align*}
	\therefore P &= -\pdv{A}{L} = \dfrac{k_B T}{Z_N} \pdv{Z_N}{L}\\
	&= \dfrac{N k_B T}{L- Na}\\
	&= \dfrac{n k_B T}{1 - na}
	\end{align*}
	where $ n= \dfrac{N}{L} $.
\end{homeworkProblem}

\begin{homeworkProblem}[6]
	A $ s $-particle density is defined as,
	\begin{equation*}
	f_s (\forall \va{p}_i, \forall \va{q}_i, t)= \dfrac{N!}{(N-s)!} \int \prod_{i=x+1}^N \dd V_i \rho(\va{p}, \va{q},t
	\end{equation*}

	Now one needs to use the defined $ s $-particle density with the Hamiltonian of the form,
	\begin{align*}
	H &= \sum_{i=1}^{N} \qty[\dfrac{p_i^2}{2m} + U(\va{q_i})] + \sum_{i,j = 1}^{N}\dfrac{1}{2} V(\va{q}_i - \va{q}_j)\\
	&= \qty[\sum_{i=1}^{s} \qty[\dfrac{p_i^2}{2m} + U(\va{q_i})] + \sum_{i,j = 1}^{s}\dfrac{1}{2} V(\va{q}_i - \va{q}_j)] + \qty[\sum_{i=s+1}^{N} \qty[\dfrac{p_i^2}{2m} + U(\va{q_i})] + \sum_{i,j = s+1}^{N}\dfrac{1}{2} V(\va{q}_i - \va{q}_j)] \\& \qq{ }+ \qty[\sum_{i= 1}^s \sum_{j= s+1}^N V (\va{q}_i - \va{q}_j)]\\
	&= H_s + H_{N-s} + H'
	\end{align*}
	We know from Liouville Theorem that,
	\begin{align*}
	\pdv{\rho_s}{t} &= \int \prod_{i=s+1}^{N} \dd V_i \pdv{\rho}{t}\\
	&= -\int \prod_{i=s+1}^{N} \dd V_i \pb{\rho}{H_s + H_{N-s} + H'}
	\end{align*}
	We now note the following,
	\begin{align*}
	\int \prod_{i=s+1}^{N} \dd V_i \pb{\rho}{H_s} &= \pb{ \qty(\prod_{i=s+1}^N \dd V_i \rho)}{ H_s} = \pb{\rho_s}{H_s} \\
	-\int \prod_{i=s+1}^{N} \dd V_i \pb{\rho}{H_{N-s}} &= \int \prod_{i=s+1}^{N} \dd V_i \sum_{i=s+1}^N \qty[\pdv{\rho}{\va{p}_j} \pdv{H_{N-s}}{\va{q}_j} - \pdv{\rho}{\va{q}_j} \pdv{H_{N-s}}{\va{p}_j}] \\
	&= \int \prod_{i=s+1}^{N} \dd V_i \sum_{i=s+1}^N \qty[\pdv{\rho}{\va{p}_j} \qty(\pdv{U}{\va{q}_j} + \sum_{j=s+1}^{N} \pdv{V}{\va{q}_j})- \pdv{\rho}{\va{q}_j} \dfrac{\va{p}_j}{m}] \\
	\int \prod_{i=s+1}^{N} \dd V_i \pb{\rho}{H_{N-s}} &= \int \prod_{i=s+1}^{N} \dd V_i \sum_{n=1}^s \qty[\pdv{\rho}{\va{p}_n} \qty( \sum_{j=s+1}^{N} \pdv{V}{\va{q}_j})] - \qty[\sum_{n=1}^{s} \pdv{\rho}{\va{p}_j} \sum_{n=1}^s \pdv{V}{\va{q}_j}]
	\end{align*}
	In the last expression, we can integrate out the second term. The first term has $ N-s $ equal terms (by symmetry),
	\begin{equation*}
	(N-s) \int \prod_{i=s+1}^{N} \dd V_i \sum_{n=1}^s \pdv{V}{\va{q}_n} \pdv{\rho}{\va{p}_n} = (N-s) \sum_{n=1}^s \dd V_{s+1} \pdv{V}{\va{q}_n} \vdot \pdv{\va{p}_n} \qty[\int \prod_{i=s+2}^{N} \dd V_i \rho]
	\end{equation*}

	Adding up the three terms above, we get,
	\begin{equation*}
	\pdv{f_s}{t} - \pb{H_s}{f_s} = \sum_{n=1}^{s} \int \dd V_{s+1} \pdv{V(q_n - q_{s+1})}{\va{q}_n} \vdot \pdv{\va{p}_n} \qty[\int \prod_{i=s+2}^N \dd V_i \rho]
	\end{equation*}

	\textbf{Part (b)}\\
	\begin{equation*}
	\dfrac{f_{s+1}}{f_s} = \dfrac{(N-s)!}{(N-s-1)!} \rho_1 (x_{s+1})
	\end{equation*}
	We then have, from the BBGKY hierarchy,
	\begin{align*}
	\qty[\pdv{t} + \sum_{n=1}^s \qty(\dfrac{\va{p}_n}{m} \pdv{\va{q}_n} - \pdv{U}{\va{q}_n} \pdv{\va{p}_n}) ] f_s &\approx \sum_{n=1}^s\int \dd V_{s+1} \pdv{V}{\va{q_n}} \pdv{\va{p}_n} (N-s) f_s \rho_1 (x_{s+1}) \\
	&\approx \sum_{n=1}^s  \pdv{\va{q_n}}\qty[\int \dd V_{s+1} \rho_1(x_{s+1}) V N] \pdv{\va{p}_n} f_s \\
	\implies \qty[\pdv{t} + \sum_{n=1}^s \qty(\dfrac{\va{p}_n}{m} \pdv{\va{q}_n} - \pdv{U_{eff}}{\va{q}_n} \pdv{\va{p}_n}) ] &= 0
	\end{align*}
	where $ U_{eff} = U(\va{q}) + N \int \dd V' V(\va{q} - \va{q}') f_1(\va{x'},t) $.
\end{homeworkProblem}

\end{document}