% !TEX root = ./busty_transcription.tex
\section{Mean Gene Expression}\label{section_02_means}

As noted in the previous section, there are two broad classes of models in play
for computing the input-output functions of regulatory architectures as shown in
Figure~\ref{fig1:means_cartoons}. In both classes of model, the promoter is
imagined to exist in a discrete set of states of occupancy, with each such state
of occupancy accorded its own rate of transcription -- including no
transcription for many of these states. The models are probabilistic with each
state assigned some probability and the overall rate of transcription given by
\begin{equation}
\mbox{average rate of transcription} = \sum_i r_i p_i,
\label{eq:transcrip_prop_pbound}
\end{equation}
where $i$ labels the distinct states, $p_i$ is the probability of the
$i^{\text{th}}$ state, and $r_i$ is the rate of transcription of that state.
Ultimately, the different models differ along several key axes: what states to
consider and how to compute the probabilities of those states.

The first class of models that are the focus of the present section on
predicting the mean level of gene expression, sometimes known as thermodynamic
models, invoke the tools of equilibrium statistical mechanics to compute the
probabilities~\cite{Ackers1982, Shea1985, Buchler2003, Vilar2003a, Vilar2003b,
Bintu2005a, Bintu2005c, Gertz2009, Sherman2012, Saiz2013}. As seen in
Figure~\ref{fig1:means_cartoons}(B), even within the class of thermodynamic
models, we can make different commitments about the underlying microscopic
states of the promoter.  Indeed, the list of options considered here does not at
all exhaust the suite of different microscopic states we can assign to the
promoter.

The second class of models that allow us to access the mean gene expression use
chemical master equations to compute the probabilities of the different
microscopic states ~\cite{Ko1991, Peccoud1995, Record1996, Kepler2001,
Sanchez2008, Shahrezaei2008, Sanchez2011, Michel2010}. As seen in
Figure~\ref{fig1:means_cartoons}(C), we consider a host of different
nonequilibrium models, each of which will have its own result for both the mean
(this section) and noise (next section) in gene expression.

%The second class of models based on chemical master equations compute the time-evolution
%of the probability distribution for mRNA.

\subsection{Fold-changes are indistinguishable across models}
As a first stop on our search for the ``right'' model of simple repression, let
us consider what we can learn from theory and experimental measurements on the
average level of gene expression in a population of cells. One experimental
strategy that has been particularly useful (if incomplete since it misses out on
gene expression dynamics) is to measure the fold-change in mean expression. The
fold-change is defined as
\begin{equation}
\text{fold-change}
= \frac{\langle \text{gene expression with repressor present} \rangle}
        {\langle \text{gene expression with repressor absent} \rangle}
= \frac{\langle m (R > 0) \rangle}{\langle m (R = 0) \rangle}
\label{eq:fc_def}
\end{equation}
where angle brackets $\left\langle \cdot \right\rangle$ denote the average over
a population of cells and mean mRNA $\langle m\rangle$ is viewed as a function
of repressor copy number $R$. What this means is that the fold-change in gene
expression is a relative measurement of the effect of the transcriptional
repressor ($R > 0$) on the gene expression level compared to an unregulated
promoter ($R = 0$). The second equality in Eq.~\ref{eq:fc_def} follows from assuming
that the translation efficiency, i.e., the number of proteins translated per
mRNA, is the same in both conditions. In other words, we assume that mean
protein level is proportional to mean mRNA level, and that the proportionality
constant is the same in both conditions and therefore cancels out in the ratio.
This is reasonable since the cells in the two conditions are identical except
for the presence of the transcription factor, and the model assumes that the
transcription factor has no direct effect on translation.

Fold-change has proven a very convenient observable in past
work~\cite{Garcia2011a, Brewster2014, Razo-Mejia2018, Chure2019}. Part of its
utility in dissecting transcriptional regulation is its ratiometric nature,
which removes many secondary effects that are present when making an absolute
gene expression measurement. Also, by measuring otherwise identical cells with
and without a transcription factor present, any biological noise common to both
conditions can be made to cancel away.
% \mmnote{I'd love to include the factoid below but I can't find
% a cite to back it up...}
% In addition, the presence of different
% operator sites, i.e. binding sites for the repressor, is known to change the
% basal expression level in the absence of repressor~\mmnote{look up
% cite}\mrm{does reg-seq mention anything like this in their SI? I definitely saw
% that on my own experiments, but didn't mention it in the paper}. Computing
% fold-changes neatly cancels this effect as well.

\fig{fig1:means_cartoons} depicts a smorgasbord of mathematicized cartoons for
simple repression using both thermodynamic and kinetic models that have appeared
in previous literature. For each cartoon, we calculate the fold-change in mean
gene expression as predicted by that model, deferring some algebraic details to
Appendix~\ref{sec:non_bursty}. What we will find is that all cartoons collapse
to a single master curve, shown in \fig{fig1:means_cartoons}(D), which contains
just two parameters. We label the parameters $\Delta F_R$, an effective free
energy parametrizing the repressor-DNA interaction, and $\rho$, which subsumes
all details of transcription in the absence of repressors. We will offer some
intuition for why this master curve exists and discuss why at the level of the
mean expression, we are unable to discriminate ``right'' from ``wrong'' cartoons
given only measurements of fold-changes in expression.

\subsubsection{The Two-State Equilibrium Model}
In this simplest model, depicted as (1) in Figure~\ref{fig1:means_cartoons}(B),
the promoter is idealized as existing in one of two states, either repressor
bound or repressor unbound. The rate of transcription
is assumed to be proportional to the fraction of time spent
in the repressor unbound state.
From the relative statistical weights listed in
Figure~\ref{fig1:means_cartoons}, the probability $p_U$ of being in the
unbound state is
\begin{equation}
p_U = \left(1 + \frac{R}{N_{NS}} e^{-\beta\Delta\varepsilon_R}\right)^{-1}.
\end{equation}
The mean rate of transcription is then given by $r p_U$, as assumed by
Eq.~\ref{eq:transcrip_prop_pbound}. The mean number of mRNA is
set by the balance of average mRNA transcription and degradation rates,
so it follows that the mean mRNA level is given by
\begin{equation}
\langle m \rangle = \frac{r}{\gamma}
        \left(1 + \frac{R}{N_{NS}} e^{-\beta\Delta\varepsilon_R}\right)^{-1},
\end{equation}
where $r$ is the transcription rate from the repressor unbound state, $\gamma$
is the mRNA degradation rate, $R$ is repressor copy number, $N_{NS}$ is the
number of nonspecific binding sites in the genome where repressors spend most
of their time when not bound to the operator, $\beta \equiv 1/k_BT$, and
$\Delta\varepsilon_R$ is the binding energy of a repressor to its operator site.
The derivation of this result is deferred to Appendix~\ref{sec:non_bursty}.

The fold-change is  found as the ratio of mean mRNA with and without repressor
as introduced in Eq.~\ref{eq:fc_def}. Invoking that definition results in
\begin{equation}
\text{fold-change}
= \left(1 + \frac{R}{N_{NS}} e^{-\beta\Delta\varepsilon_R}\right)^{-1},
\end{equation}
which matches the form of the master curve in
Figure~\ref{fig1:means_cartoons}(D) with $\rho=1$ and $\Delta F_R =
\beta\Delta\varepsilon_r - \log (R / N_{NS})$.

In fact it was noted in~\cite{Chure2019} that this two-state model can be viewed
as the coarse-graining of any equilibrium promoter model in which no
transcriptionally active states have transcription factor bound, or put
differently, when there is no overlap between
transcription factor bound states and transcriptionally
active states. We will see this explicitly in the 3-state equilibrium model
below, but perhaps surprising is that an analogous result carries over even to
the nonequilibrium models we consider later.

\subsubsection{The Three-State Equilibrium Model}
Compared to the previous model, here we fine-grain the repressor unbound state
into two separate states: empty, and RNAP bound as shown in (2) in
Figure~\ref{fig1:means_cartoons}(B). This picture was used in~\cite{Garcia2011a}
as we use it here, and in~\cite{Razo-Mejia2018} and~\cite{Chure2019} it was
generalized to incorporate small-molecule inducers that bind the repressor.
The effect of this generalization is, roughly speaking,
simply to rescale $R$ from the total number of repressors to a smaller
effective number of available repressors which are unbound by inducers.
We point out that the same generalization can be incorporated quite easily
into any of our models in Figure~\ref{fig1:means_cartoons} by simply
rescaling the repressor copy number $R$ in the equilibrium models, or
equivalently $k_R^+$ in the nonequilibrium models.

The mean mRNA copy number, as derived in Appendix~\ref{sec:non_bursty}
from a similar enumeration of states and weights as the previous model, is
\begin{equation}
\langle m \rangle = \frac{r}{\gamma}
\frac{\frac{P}{N_{NS}} e^{-\beta\Delta\varepsilon_P}}
        {
        1 + \frac{R}{N_{NS}} e^{-\beta\Delta\varepsilon_R}
        + \frac{P}{N_{NS}} e^{-\beta\Delta\varepsilon_P}
        },
\end{equation}
where the new variables are $\Delta\varepsilon_P$, the difference in RNAP
binding energy to its specific site (the promoter) relative to an average
nonspecific background site, and the RNAP copy number, $P$. The fold-change
again follows immediately as
\begin{align}
\text{fold-change}
&= \frac{\frac{P}{N_{NS}} e^{-\beta\Delta\varepsilon_P}}
        {
        1 + \frac{R}{N_{NS}} e^{-\beta\Delta\varepsilon_R}
        + \frac{P}{N_{NS}} e^{-\beta\Delta\varepsilon_P}
        }
\frac{1 + \frac{P}{N_{NS}} e^{-\beta\Delta\varepsilon_P}}
        {\frac{P}{N_{NS}} e^{-\beta\Delta\varepsilon_P}}
\\
&= \left(
1 + \frac{\frac{R}{N_{NS}} e^{-\beta\Delta\varepsilon_R}}
        {1 + \frac{P}{N_{NS}} e^{-\beta\Delta\varepsilon_P}}
\right)^{-1}
\\
&= (1 + \exp(-\Delta F_R - \log\rho))^{-1},
\end{align}
with $\Delta F_R = \beta\Delta\varepsilon_R - \log(R/N_{NS})$ and $\rho = 1 +
\frac{P}{N_{NS}}\mathrm{e}^{-\beta\Delta\varepsilon_P}$ as shown
in~\fig{fig1:means_cartoons}(B). Thus far, we see that the two thermodynamic
models, despite making different coarse-graining commitments, result in the same
functional form for the fold-change in mean gene expression.  We now explore how
kinetic models fare when faced with computing the same observable.

\subsubsection{The Poisson Promoter Nonequilibrium Model}
For our first kinetic model, we  imitate the states considered in the Two-State
Equilibrium Model and consider the simplest possible picture with only two
states, repressor bound and unbound. This is exactly the model used for the main
results of~\cite{Jones2014}. In this picture, repressor association and
dissociation rates from its operator site, $k_R^+$ and $k_R^-$, respectively,
govern transitions between the two states. When the system is in the unbound
state, transcription initiates at rate $r$, which represents a coarse-graining
of all the downstream processes into a single effective rate. mRNA is degraded
at rate $\gamma$ as already exploited in the previous models.

Let $p_R(m,t)$ denote the joint probability of finding the system in the
repressor bound state $R$ with $m$ mRNA molecules present at time $t$. Similarly
define $p_U(m,t)$ for the repressor unbound state $U$. This model is governed
by coupled master equations giving the time evolution of $p_R(m,t)$ and
$p_U(m,t)$~\cite{Sanchez2008, Sanchez2011, Phillips2019} which we can write as
\begin{align}
\begin{split}
\deriv{t}p_R(m,t) =&
- \overbrace{k_R^- p_R(m,t)}^{R \rightarrow U}
+ \overbrace{k_R^+ p_U(m,t)}^{U \rightarrow R}
+ \overbrace{(m+1)\gamma p_R(m+1,t)}^{m + 1 \rightarrow m}
- \overbrace{\gamma p_R(m,t)}^{m \rightarrow m - 1}
\\
\deriv{t}p_U(m,t) =&\;
\overbrace{k_R^- p_R(m,t)}^{R \rightarrow U}
- \overbrace{k_R^+ p_U(m,t)}^{U \rightarrow R}
+ \overbrace{rp_U(m-1,t)}^{m-1 \rightarrow m}
- \overbrace{rp_U(m,t)}^{m \rightarrow m + 1}
\\
&+ \overbrace{(m+1)\gamma p_U(m+1,t)}^{m + 1 \rightarrow m}
- \overbrace{\gamma p_U(m,t)}^{m \rightarrow m - 1},
\label{eq:poisson_promoter_cme}
\end{split}
\end{align}
where each term on the right corresponds to a transition between two states of
the promoter as indicated by the overbrace label. In each equation, the first
two terms describe transitions between promoter states due to repressors
unbinding and binding, respectively. The final two terms describe degradation of
mRNA, decreasing the copy number by one, and the terms with coefficient $r$
describe transcription initiation increasing the mRNA copy number by one.
We direct the reader to Appendix~\ref{sec:cme_from_cartoon} for a careful
treatment showing how the form of this master equation follows from the
corresponding cartoon in Figure~\ref{fig1:means_cartoons}.

We can greatly simplify the notation, which will be especially useful for the
more complicated models yet to come, by re-expressing the master equation in
vector form~\cite{Phillips2012}. The promoter states are collected into a vector
and the rate constants are collected into matrices as
\begin{equation}
\vec{p}(m) = \begin{pmatrix} p_R(m) \\ p_U(m) \end{pmatrix},\
\mathbf{K} = \begin{pmatrix} -k_R^- & k_R^+ \\ k_R^- & -k_R^+ \end{pmatrix},\
\mathbf{R} = \begin{pmatrix} 0 & 0 \\ 0 & r \end{pmatrix},\
\label{eq:2state_cme_matrices}
\end{equation}
so that the master equation may be condensed as
\begin{equation}
\deriv{t}\vec{p}(m,t) =
\left( \mathbf{K} - \mathbf{R} - \gamma m \mathbf{I} \right) \vec{p}(m,t)
                + \mathbf{R} \vec{p}(m-1,t)
                + \gamma (m+1) \mathbf{I} \vec{p}(m+1,t),
\label{eq:2state_rep_cme}
\end{equation}
where $\mathbf{I}$ is the identity matrix. Taking steady state by setting time
derivatives to zero, the mean mRNA can be found to be
\begin{equation}
\langle m \rangle = \frac{r}{\gamma}
        \left(1 + \frac{k_R^+}{k_R^-}\right)^{-1},
\label{eq:mean_m_model1}
\end{equation}
with the algebra details again deferred to Appendix~\ref{sec:non_bursty}. Recall
$k_R^+$ is proportional to the repressor copy number, so in computing
fold-change, absence of repressor corresponds to $k_R^+\rightarrow0$. Therefore
fold-change in this model is simply
\begin{equation}
\text{fold-change} = \left(1 + \frac{k_R^+}{k_R^-}\right)^{-1},
\end{equation}
again matching the master curve of~\fig{fig1:means_cartoons}(D) with $\rho=1$.

\subsubsection{Nonequilibrium Model Two - RNAP Bound and Unbound States}
Our second kinetic model depicted in Figure~\ref{fig1:means_cartoons}(C) mirrors
the second equilibrium model of Figure~\ref{fig1:means_cartoons}(B) by
fine-graining  the repressor unbound state of nonequilibrium model 1, resolving
it into an empty promoter state and an RNAP-bound state. Note in this picture,
in contrast with model 4 below, transcription initiation is accompanied by a
promoter state change, in keeping with the interpretation as RNAP-bound and
empty states: if an RNAP successfully escapes the promoter and proceeds to
elongation of a transcript, clearly it is no longer bound at the promoter.
Therefore another RNAP must bind before another transcript can be initiated.

The master equation governing this model is analogous
to Eqs.~\ref{eq:2state_cme_matrices}-\ref{eq:2state_rep_cme} for model 1 above. The
main subtlety arises since transcription initiation accompanies a promoter state
change. This can be understood by analogy to $\mathbf{K}$. The off-diagonal and
diagonal elements of $\mathbf{K}$ correspond to transitions arriving at or
departing from, respectively, the promoter state of interest. If transcription
initiation is accompanied by promoter state changes, we must have separate
matrices for arriving and departing transcription events since the arriving and
departing transitions have different initial copy numbers of mRNA, unlike for
$\mathbf{K}$ where they are the same (see Appendix~\ref{sec:non_bursty}). The
master equation for this model is
\begin{equation}
\deriv{t}\vec{p}(m,t) =
\left( \mathbf{K} - \mathbf{R_D} - \gamma m \mathbf{I} \right) \vec{p}(m,t)
                + \mathbf{R_A} \vec{p}(m-1,t) +
                \gamma (m+1) \mathbf{I} \vec{p}(m+1,t),
\label{eq:3state_rep_cme}
\end{equation}
with the state vector and promoter transition matrix defined as
\begin{equation}
\vec{p}(m) = \begin{pmatrix} p_R(m) \\ p_E(m) \\ p_P(m) \end{pmatrix},\
\mathbf{K} = \begin{pmatrix} -k_R^- & k_R^+ & 0 \\
                        k_R^- & -k_R^+ -k_P^+ & k_P^- \\
                        0 & k_P^+ & -k_P^-
                \end{pmatrix},
\label{eq:3state_cme_matrices_pt1}
\end{equation}
and the initiation matrices given by
\begin{equation}
\mathbf{R_A} = \begin{pmatrix}
                0 & 0 & 0 \\
                0 & 0 & r \\
                0 & 0 & 0
                \end{pmatrix},\
\mathbf{R_D} = \begin{pmatrix}
                0 & 0 & 0 \\
                0 & 0 & 0 \\
                0 & 0 & r
                \end{pmatrix}.
\label{eq:3state_cme_matrices_pt2}
\end{equation}
The elements of $\vec{p}(m)$ encode the probabilities of having $m$ mRNA present
along with the promoter having repressor bound ($R$), being empty ($E$), or
having RNAP bound ($P$), respectively. $\mathbf{R_A}$ describes probability flux
\textit{arriving} at the state $\vec{p}(m)$ from a state with one fewer mRNA,
namely $\vec{p}(m-1)$, and $\mathbf{R_D}$ describes probability flux
\textit{departing} from the state $\vec{p}(m)$ for a state with one more mRNA,
namely $\vec{p}(m+1)$. $\mathbf{K}$ is closely analogous to model 1.

Mean mRNA at steady state is found analogously to model 1, with the result
\begin{equation}
\langle m\rangle = \frac{r}{\gamma}
        \frac{k_R^- k_P^+}
        {k_R^- k_P^+ + k_R^- (k_P^- + r) + k_R^+ (k_P^- + r)},
\label{eq:model2_meanm}
\end{equation}
and with details again deferred to Appendix~\ref{sec:non_bursty}.
Fold-change is again found from the ratio prescribed by Eq.~\ref{eq:fc_def}, from
which we have
\begin{align}
\text{fold-change}
&=      \frac{k_R^- k_P^+}
        {k_R^- k_P^+ + k_R^- (k_P^- + r) + k_R^+ (k_P^- + r)}
        \frac{k_P^+ + k_P^- + r}{k_P^+}
\\
&=      \left(1 + \frac{k_R^+}{k_R^-}
                \frac{k_P^- + r}{k_P^+ + k_P^- + r}
        \right)^{-1}
\\
&=      \left(1 + \frac{k_R^+}{k_R^-}
        \left(1 + \frac{k_P^+}{k_P^- + r}\right)^{-1}
        \right)^{-1},
\end{align}
which follows the master curve of~\fig{fig1:means_cartoons}(D) with $\rho = 1 +
k_P^+/(k_P^- + r)$ as claimed.

\subsubsection{Nonequilibrium Model Three - Multistep Transcription Initiation
and Escape}
One might reasonably complain that the first two ``nonequilibrium'' models we
have considered are straw men. Their steady states necessarily satisfy detailed
balance which is equivalent to thermodynamic equilibrium. Why is this the case?
At steady state there is by definition no net probability flux in or out of each
promoter state, but since the promoter states form a linear chain, there is only
one way in or out of the repressor bound and RNAP bound states, implying each
edge must actually have a net zero probability flux, which is the definition of
detailed balance (usually phrased as equality of forward and reverse transition
fluxes).

Now we consider model 3 in Figure~\ref{fig1:means_cartoons}(C) which allows the
possibility of true nonequilibrium steady-state fluxes through the promoter
states. We point out that this model was considered previously
in~\cite{Mitarai2015} where a comparison was made with model 1 as used
in~\cite{Jones2014}. The authors of~\cite{Mitarai2015} argued that the
additional complexity is essential to properly account for the noise in the mRNA
distribution. We will weigh in on both models later when we consider observables
beyond fold-change.

The master equation governing this model is identical in form to model 2 above,
namely
\begin{equation}
\deriv{t}\vec{p}(m,t) =
\left( \mathbf{K} - \mathbf{R_D} - \gamma m \mathbf{I} \right) \vec{p}(m,t)
                + \mathbf{R_A} \vec{p}(m-1,t) +
                \gamma (m+1) \mathbf{I} \vec{p}(m+1,t),
\end{equation}
but with a higher-dimensional state space and different matrices. The
state vector and promoter transition matrix are now
\begin{equation}
\vec{p}(m) = \begin{pmatrix} p_R(m) \\ p_E(m) \\
                             p_C(m) \\ p_O(m)\end{pmatrix},\
\mathbf{K} = \begin{pmatrix} -k_R^- & k_R^+ & 0 & 0\\
                        k_R^- & -k_R^+ -k_P^+ & k_P^- & 0 \\
                        0 & k_P^+ & -k_P^- - k_O & 0 \\
                        0 & 0 & k_O & 0
                \end{pmatrix},
\end{equation}
with the four promoter states, in order, being repressor bound ($R$), empty
($E$), RNAP closed complex ($C$), and RNAP open complex ($O$). Besides
increasing dimension by one, the only new feature in $\mathbf{K}$ is the
rate $k_O$, representing the rate of open complex formation from
the closed complex, which we assume for simplicity to be irreversible
in keeping with some~\cite{Mitarai2015} but not all~\cite{DeHaseth1998}
past literature. The initiation matrices are given by
\begin{equation}
\mathbf{R_A} = \begin{pmatrix}
        0 & 0 & 0 & 0 \\
        0 & 0 & 0 & r \\
        0 & 0 & 0 & 0 \\
        0 & 0 & 0 & 0
                \end{pmatrix},\
\mathbf{R_D} = \begin{pmatrix}
        0 & 0 & 0 & 0 \\
        0 & 0 & 0 & 0 \\
        0 & 0 & 0 & 0 \\
        0 & 0 & 0 & r
                \end{pmatrix},
\end{equation}
again closely analogous to nonequilibrium model 2.

The expression for mean mRNA is substantially more complicated now, as worked
out in Appendix~\ref{sec:non_bursty} where we find
\begin{equation}
\langle m\rangle = \frac{r}{\gamma}
        \frac{k_R^- k_P^+ k_O}
        {k_R^- [(k_P^+ (k_O + r) + r(k_P^- + k_O)] + k_R^+ r(k_P^- + k_O)},
\label{eq:model3_mean_m}
\end{equation}
which can be simplified to
\begin{equation}
\langle m\rangle
= \frac{r}{\gamma}
\frac{\frac{k_P^+ k_O}{r(k_O + k_P^-)}}
        {1 + \frac{k_P^+ (k_O + r)}{r(k_O + k_P^-)} + \frac{k_R^+}{k_R^-}}.
\end{equation}
The strategy is to isolate the terms involving the repressor, so that now the
fold-change is seen to be simply
\begin{align}
\text{fold-change}
&= \frac{\frac{k_P^+ k_O}{r(k_O + k_P^-)}}
        {1 + \frac{k_P^+ (k_O + r)}{r(k_O + k_P^-)} + \frac{k_R^+}{k_R^-}}
        \frac{1 + \frac{k_P^+ (k_O + r)}{r(k_O + k_P^-)}}
                {\frac{k_P^+ k_O}{r(k_O + k_P^-)}}
\\
&= \left(
        1 + \frac{k_R^+}{k_R^-}
        \left(1 + \frac{k_P^+ (k_O + r)}{r(k_O + k_P^-)}\right)^{-1}
\right)^{-1},
\end{align}
surprisingly reducing to the master curve of~\fig{fig1:means_cartoons}(D) once
again, with $\rho = 1 + \frac{k_P^+ (k_O + r)}{r(k_O + k_P^-)}$.

This example hints that an arbitrarily fine-grained model of downstream
transcription steps may still be collapsed to the form of the master curve for
the means given in Figure~\ref{fig1:means_cartoons}(D), so long as the repressor
binding is exclusive with transcriptionally active states. We offer this as a
conjecture, and we suspect that a careful argument using the King-Altman diagram
method~\cite{King1956, Hill1966} might furnish a ``proof.'' Our focus here is
not on full generality but rather to survey an assortment of plausible models
for simple repression  that have been proposed in the literature.

\subsubsection{Nonequilibrium Model Four - ``Active'' and ``Inactive'' States}
Model 4 in Figure~\ref{fig1:means_cartoons}(C) is at the core of the theory
in~\cite{Razo-Mejia2020}. At a glance the cartoon for this model may appear very
similar to model 2, and mathematically it is, but the interpretation is rather
different. In model 2, we interpreted the third state literally as an RNAP-bound
promoter and modeled initiation of a transcript as triggering a promoter state
change, making the assumption that an RNAP can only
make one transcript at a time. In contrast, in the present model the promoter
state does \textit{not} change when a transcript is initiated. So we no longer
interpret these states as literally RNAP bound and unbound but instead as
coarse-grained ``active'' and ``inactive'' states, the details of which we leave
unspecified for now. We will comment more on this model below when we discuss
Fano factors of models.

Mathematically this model is very similar to models 1 and 2. Like model 1, the
matrix $R$ describing transcription initiation is diagonal, namely
\begin{equation}
\mathbf{R} = \begin{pmatrix}
                0 & 0 & 0 \\
                0 & 0 & 0 \\
                0 & 0 & r
        \end{pmatrix}.
\end{equation}
The master equation takes verbatim the same form as it did for model 1,
Eq.~\ref{eq:2state_rep_cme}. Meanwhile the promoter transition
matrix $\mathbf{K}$ is the same as Eq.~\ref{eq:3state_cme_matrices_pt1}
from model 2, although we relabel the
rate constants from $k_P^\pm$ to $k^\pm$ to reiterate that these are not simply
RNAP binding and unbinding rates.

Carrying out the algebra, the mean mRNA can be found to be
\begin{equation}
\langle m\rangle = \frac{r}{\gamma}
\frac{k_R^- k^+}
{k_R^- k^+ + k_R^- k^- + k_R^+ k^-},
\end{equation}
and the fold-change readily follows,
\begin{align}
\text{fold-change}
&=      \frac{k_R^- k^+}{k_R^- k^+ + k_R^- k^- + k_R^+ k^-}
        \frac{k_R^- k^+ + k_R^- k^-}{k_R^- k^+}
\\
&=      \left(1 + \frac{k_R^+}{k_R^-}
                \left(1 + \frac{k^+}{k^-}\right)^{-1}
        \right)^{-1},
\end{align}
from which we see $\rho = 1 + k^+/k^-$ as shown in~\fig{fig1:means_cartoons}(C).

\subsubsection{Nonequilibrium Model Five - Bursty Promoter}
The final model we consider shown in Figure~\ref{fig1:means_cartoons}(C) is an
intuitive analog to model 1, with just two states, repressor bound or unbound,
and transition rates between them of $k_R^+$ and $k_R^-$. In model 1, when in
the unbound state, single mRNA transcripts are produced as a Poisson process
with some characteristic rate $r$. The current model by contrast produces, at
some Poisson rate $k_i$, \textit{bursts} of mRNA transcripts.
The burst sizes are assumed to be geometrically distributed with
a mean burst size $b$, which we will motivate in Section~\ref{sec:beyond_means}
when we derive this model as a certain limiting case of model 4.

From this intuitive picture and by analogy to model 1, then, it should be
plausible that the mean mRNA level is
\begin{equation}
\langle m\rangle = \frac{k_i b}{\gamma}
        \left(1 + \frac{k_R^+}{k_R^-}\right)^{-1},
\end{equation}
which will turn out to be correct from a careful calculation. For now, we simply
note that just like model 1, the fold-change becomes
\begin{equation}
\text{fold-change} = \left(1 + \frac{k_R^+}{k_R^-}\right)^{-1}
\end{equation}
with $\rho=1$ also like model 1.
We will also see later how this model emerges as a natural limit of model 4.

\subsection{Discussion of Results Across Models for Fold-Changes in Mean
Expression}
The key outcome of our analysis of the models in
Figure~\ref{fig1:means_cartoons} is the existence of a master curve shown
in Figure~\ref{fig1:means_cartoons}(D) to which the fold-change predictions
of all the models collapse. This master curve is parametrized by only two
effective parameters: $\Delta F_R$, which characterizes the number of repressors
and their binding strength to the DNA, and $\rho$, which characterizes all other
features of the promoter architecture. The key assumption underpinning this
result is that no transcription occurs when a repressor is bound to its
operator. Note, however, that we are agnostic about the molecular mechanism
which achieves this; steric effects are one plausible mechanism, but, for
instance, ``action-at-a-distance'' mediated by kinked DNA due to repressors
bound tens or hundreds of nucleotides upstream of a promoter is plausible as
well.

Why does the master curve of Figure~\ref{fig1:means_cartoons}(D) exist at all?
This brings to mind the deep questions posed in,
e.g.,~\cite{Frank2013} and~\cite{Frank2014a}, suggesting we consider multiple
plausible models of a system and search for their common patterns to tease
out which broad features are and are not important.
In our case, the key feature seems to be the
exclusive nature of repressor and RNAP binding, which allows the parameter
describing the repressor, $\Delta F_R$, to cleanly separate from all other
details of the promoter architecture, which are encapsulated in $\rho$.
Arbitrary nonequilibrium behavior can occur on the rest of the
promoter state space, but it may all be swept up in the effective
parameter $\rho$, to which the repressor makes no contribution.
We point the interested reader to~\cite{Gunawardena2012}
and~\cite{Ahsendorf2014} for an interesting analysis of similar problems
using a graph-theoretic language.

As suggested in~\cite{Chure2019}, we believe this master curve should generalize
to architectures with multiple repressor binding sites, as long as the
exclusivity of transcription factor binding and transcription initiation is
maintained. The interpretation of $\Delta F_R$ is then of an effective free
energy of all repressor bound states. In an equilibrium picture this is simply
given by the log of the sum of Boltzmann weights of all repressor bound states,
which looks like the log of a partition function of a subsystem. In a
nonequilibrium picture, while we can still mathematically gather terms and give
the resulting collection the label $\Delta F_R$, it is unclear if the physical
interpretation as an effective free energy makes sense. The problem is that free
energies cannot be assigned unambiguously to states out of equilibrium because
the free energy change along a generic path traversing the state space is path
dependent, unlike at equilibrium. A consequence of this is that, out of
equilibrium, $\Delta F_R$ is no longer a simple sum of Boltzmann weights.
Instead it resembles a restricted sum of King-Altman diagrams~\cite{King1956,
Hill1966}. Following the work of Hill~\cite{Hill1989}, it may yet be possible to
interpret this expression as an effective free energy, but this remains unclear
to us. We leave this an open problem for future work.

If we relax the requirement of exclusive repressor-RNAP binding, one could
imagine models in which repressor and RNAP doubly-bound states are allowed,
where the repressor's effect is to \text{reduce} the transcription rate rather
than setting it to zero. Our results do not strictly apply to such a model,
although we note that if the repressor's reduction of the transcription rate is
substantial, such a model might still be well-approximated by one of the models
in~\fig{fig1:means_cartoons}.

One may worry that our ``one curve to rule them all'' is a mathematical
tautology. In fact we \textit{agree} with this criticism if $\Delta F_R$ is
``just a fitting parameter'' and cannot be meaningfully interpreted as a real,
physical free energy. An analogy to Hill functions is appropriate here. One of
their great strengths and weaknesses, depending on the use they are put to, is
that their parameters coarse-grain many details and are generally not
interpretable in terms of microscopic models, for deep reasons discussed at
length in~\cite{Frank2013}. By contrast, our master curve claims to have the
best of both worlds: a coarse-graining of all details besides the repressor into
a single effective parameter $\rho$, while simultaneously retaining an
interpretation of $\Delta F_R$ as a physically meaningful and interpretable free
energy. Our task, then, is to prove or disprove this claim.

How do we test this and probe the theory with fold-change measurements? There is
a fundamental limitation in that the master curve is essentially a one-parameter
function of $\Delta F_R + \log\rho$. Worse, there are many \textit{a priori}
plausible microscopic mechanisms that could contribute to the value of $\rho$,
such as RNAP binding and escape kinetics~\cite{DeHaseth1998, Mitarai2015},
and/or supercoiling accumulation and release~\cite{Chong2014, Sevier2016},
and/or, RNAP clusters analogous to those observed in
eukaryotes~\cite{Cisse2013, Cho2016} and recently also observed in
bacteria~\cite{Ladouceur2020}. Even if $\Delta F_R$ is measured to high
precision, inferring the potential microscopic contributions to $\rho$, buried
inside a log no less, from fold-change measurements seems beyond reach. As a
statistical inference problem it is entirely nonidentifiable, in the language
of~\cite{Gelman2013}, Section 4.3.

If we cannot simply infer values of $\rho$ from measurements of fold-change, can
we perturb some of the parameters that make up $\rho$ and measure the change?
Unfortunately we suspect this is off-limits experimentally: most of the
potential contributors to $\rho$ are global processes that affect many or all
genes. For instance, changing RNAP association rates by changing RNAP copy
numbers, or changing supercoiling kinetics by changing topoisomerase copy
numbers, would massively perturb the entire cell's physiology and confound any
determination of $\rho$.

One might instead imagine a bottom-up modeling approach, where we mathematicize
a model of what we hypothesize the important steps are and are not, use \textit{in vitro}
data for the steps deemed important, and \textit{predict} what $\rho$ should be.
But again, because of the one-parameter nature of the master curve, many
different models will likely make indistinguishable predictions, and without any
way to experimentally perturb \textit{in vivo}, there is no clear way
to test whether the modeling assumptions are correct.

In light of this, we prefer the view that parameters and rates are not directly
comparable between cartoons in~\fig{fig1:means_cartoons}. Rather, parameters in
the simpler cartoons represent coarse-grained combinations of parameters in the
finer-grained models. For instance, by equating $\rho$ between any two models,
one can derive various possible correspondences between the two models'
parameters. Note that these correspondences are clearly not unique, since many
possible associations could be made. It then is a choice as to what microscopic
interpretations the model-builder prefers for the parameters in a particular
cartoon, and as to which coarse-grainings lend intuition and which seem
nonsensical. Indeed, since it remains an open question what microscopic features
dominate $\rho$ (as suggested above, perhaps RNAP binding and escape
kinetics~\cite{DeHaseth1998, Mitarai2015}, or supercoiling accumulation and
release~\cite{Chong2014, Sevier2016}, or, something more exotic like RNAP
clusters~\cite{Cisse2013, Cho2016, Ladouceur2020}), we are hesitant to put too
much weight on any one microscopic interpretation of model parameters that make
up $\rho$.

One possible tuning knob to probe $\rho$ that would not globally perturb the
cell's physiology is to manipulate RNAP binding sites. Work such
as~\cite{Kinney2010} has shown that models of sequence-dependent RNAP affinity can be
inferred from data, and the authors of~\cite{Brewster2012} showed that the model
of~\cite{Kinney2010} has predictive power by using the model to \textit{design}
binding sites of a desired affinity. But for our purposes, this begs the
question: the authors of~\cite{Kinney2010} \textit{assumed} a particular model
(essentially our 3-state equilibrium model but without the repressor), so it is
unclear how or if such methods can be turned around to \textit{compare}
different models of promoter function.

Another possible route to dissect transcription details without a global
perturbation would be to use phage polymerase with phage-specific promoters.
While such results would carry some caveats, e.g., whether the repression of
the phage polymerase is a good analog to the repression of the native RNAP,
it could nevertheless be worthy of consideration.

We have already pointed out that the master curve of~\fig{fig1:means_cartoons}
is essentially a one-parameter model, the one parameter being $\Delta F_R +
\log\rho$. By now the reader may be alarmed as to how can we even determine
$\Delta F_R$ and $\rho$ independently of each other, never mind shedding a lens
on the internal structure of $\rho$ itself. A hint is provided by the weak
promoter approximation, invoked repeatedly in prior studies~\cite{Bintu2005c,
Garcia2011a, Razo-Mejia2018} of simple repression using the 3-state equilibrium
model in~\fig{fig1:means_cartoons}(B). In that picture, the weak promoter
approximation means $\frac{P}{N_{NS}}\exp(-\beta\Delta\varepsilon_P) \ll 1$,
meaning therefore $\rho\approx1$.  This approximation can be well justified on
the basis of the number of RNAP and $\sigma$ factors per cell
and the strength of binding of RNAP to
DNA at weak promoters. This is suggestive, but how can we be sure that $\rho$ is
not, for instance, actually $10^2$ and that $\Delta F_R$ hides a compensatory
factor? A resolution is offered by an independent inference of $\rho$ in the
absence of repressors. This was done in~\cite{Razo-Mejia2020} by fitting
nonequilibrium model 4 in~\fig{fig1:means_cartoons}(C), with zero repressor
(looking ahead, this is equivalent to model 4
in Figure~\ref{fig2:constit_cartoons}(A)),
to single-cell mRNA counts data from~\cite{Brewster2014}. This provided a
determination of $k^+$ and $k^-$, from which their ratio is estimated to be no
more than a few $10^{-1}$ and possibly as small as $10^{-2}$.

The realization that $\rho\approx1$ to an excellent approximation,
\textit{independent} of which model in~\fig{fig1:means_cartoons} one prefers,
goes a long way towards explaining the surprising success of equilibrium models
of simple repression. Even though our 2- and 3-state models get so many details
of transcription wrong, it does not matter because fold-change is a cleverly
designed ratio. Since $\rho$ subsumes all details except the repressor, and
$\log\rho\approx0$, fitting these simple models to fold-change measurements can
still give a surprisingly good estimate of repressor binding energies. So the
ratiometric construction of fold-change fulfills its intended purpose of
canceling out all features of the promoter architecture except the repressor
itself. Nevertheless it is perhaps surprising how effectively it does so:
\textit{a priori}, one might not have expected $\rho$ to be quite so close to 1.

We would also like to highlight the relevance of~\cite{Landman2019} here.
Landman et.\ al.\ reanalyzed and compared \textit{in vivo} and
\textit{in vitro} data on the lacI repressor's binding affinity to its
various operator sequences. (The \textit{in vivo}
data was from, essentially, fitting our master curve to expression
measurements.) They find broad agreement between the
\textit{in vitro} and \textit{in vivo}
values. This reinforces the suspicion that the equilibrium $\Delta\varepsilon_R$
repressor binding energies do in fact represent real physical free energies.
Again, \textit{a priori} this did not have to be the case, even knowing that
$\rho\approx1$.

In principle, if $\Delta F_R$ can be measured to sufficient precision, then
deviations from $\rho=1$ become a testable matter of experiment. In practice, it
is probably unrealistic to measure repressor rates $k_R^+$ or $k_R^-$ or
fold-changes in expression levels (and hence $\Delta\varepsilon_R$) precisely
enough to detect the expected tiny deviations from $\rho=1$. We can estimate the
requisite precision in $\Delta F_R$ to resolve a given $\Delta\rho$ by noting,
since $\rho\approx1$, that $\log(1+\Delta\rho)\approx \Delta\rho$, so
$\Delta(\Delta F_R) \approx \Delta\rho$. Suppose we are lucky and $\Delta\rho$
is $\sim0.1$, on the high end of our range estimated above. A
determination of $\Delta\varepsilon_R/k_BT$ with an uncertainty of barely 0.1
was achieved in the excellent measurements of~\cite{Razo-Mejia2018}, so this
requires a very difficult determination of $\Delta F_R$ for a very crude
determination of $\rho$, which suggests, to put it lightly, this is not a
promising path to pursue experimentally. It is doubtful that inference of
repressor kinetic rates would be any easier.%\footnote{\mmnote{In fact the
% uncertainties quoted in Hammar et.\ al.~\cite{Hammar2014} from Elf's group are
% small enough to do this calculation, but HJ \& I think their error bars are
% insanely optimistic. Not sure I want to blatantly call them out on it though??}}

Moving forward, we have weak evidence supporting the interpretation of $\Delta
F_R$ as a physically real free energy~\cite{Landman2019} and other work casting
doubt~\cite{Hammar2014}. How might we resolve the confusion? If there is no
discriminatory power to test the theory and distinguish the various models with
measurements of fold-changes in means, how do we probe the theory? Clearly to
discriminate between the nonequilibrium models in~\fig{fig1:means_cartoons}, we
need to go beyond means to ask questions about kinetics, noise and even full
distributions of mRNA copy numbers over a population of cells. If the
``one-curve-to-rule-them-all'' is more than a mathematical tautology, then the
free energy of repressor binding inferred from fold-change measurements should
agree with repressor binding and unbinding rates. In other words, the
equilibrium and nonequilibrium definitions of $\Delta F_R$ must agree, meaning
\begin{equation}
\Delta F_R = \beta\Delta\varepsilon_R - \log(R/N_{NS})
        = - \log(k_R^+/k_R^-),
\label{eq:deltaFR_eq_noneq_equiv}
\end{equation}
must hold, where $\beta\Delta\varepsilon_R$ is inferred from the master curve
fit to fold-change measurements, and $k_R^+$ and $k_R^-$ are inferred in some
orthogonal manner. Single molecule measurements such as from~\cite{Hammar2014}
have directly observed these rates, and in the remainder of this work we explore
a complementary approach: inferring repressor binding and unbinding rates
$k_R^+$ and $k_R^-$ from single-molecule measurements of mRNA population
distributions.


\documentclass{article}
\usepackage{fullpage}
\usepackage{nopageno}
\usepackage{amsmath}
\usepackage{amssymb}
\allowdisplaybreaks

\newcommand{\abs}[1]{\left\lvert #1 \right\rvert}

\begin{document}
\title{Notes}
\date{February 10, 2014}
\maketitle
\section*{homework}
for number 35 write down subsets in lexicographic order, and complements (this will be reverse lexicographic order)

number 33 has a theorem in the book
\section*{pigeonhole principle}
this probably won't be on the test, but "it will blow my mind"

so if we have 11 cups and 12 markers than distributing the markers we have at least one cup with 2 markers. or if you have $n$ holes and $n+1$ pigeons then you will have at least one hole with more than one pigeon.
\subsection*{aside}
we have 24 people in the class. what are the chances that anyone has the same birthday?
\subsection*{in math mathyness}
\subsubsection*{abstract pigeonhole principle}
let $x$ and $y$ be finite sets and $f:x\Rightarrow y$. If $|x|>|y|$ then $f$ is not injective. If $|x|=|y|$ then $f$ is surjective if and only if $f$ is injective (that is bijective).
\subsection*{example}
given a sequence$a_1,\dots,a_m,a_i\in\mathbb{N}\quad \forall i$. $\exists$ consecutive $a_i$'s whose sum is divisible by $m$.
\subsubsection*{proof}
consider $a_1,a_1+a_2,a_1+a_2+a_3,\dots,a_1+a_2+\dots+a_m$. Suppose none of these are divisible by $m$. So the remainders of each sum when divided by m are in $\{1,\dots,m-1\}$. So $\exists k<l$ such that $a_1+\dots+a_k$ and $a_1+\dots+a_l$have the same remainder $r$. So $\exists b,c\in\mathbb{N}$ such that
\begin{align*}
  a_1+\dots+a_l&=cm+r\\
  -(a_1+\dots+a_k&=bm+r)\\
  a_{k+1}+\dots+a_l&=(c-b)m
\end{align*}
So $a_{k+1}+\dots+a_l$ is divisible by $m$.
\subsubsection*{note}
what if $a_i$'s$\in\mathbb{Z}$?
\subsection*{back to the birthday problem}
use the subtraction principle. The number of ways for 24 people to have a pair with the same birthday is the total number of birthday distributions minus the number of ways to have distinct birthdays. So $365^{24}$ is the total number of was for people to have birthdays. And $\frac{365!}{(365-24)!}$ is the number of ways to have distinct birthdays. And the number of ways for 24 people to have a pair of the same birthday is $365^{24}-\frac{365!}{(365-24)!}$.
\end{document}


% !TeX root = ../solution.tex

\hypertarget{he22.34}{%
\chapter{[HE22.34] AES Burgers}\label{he22.34}}

\begin{marginfigure}
	\includegraphics[width=49mm]{level8/challenge34.jpg}
\end{marginfigure}
\section{Intro}
Welcome to AES Burgers - where the patty is tatty™ !

\noindent Connect to our server, and place your order!

\noindent \verb+nc 46.101.107.117 2207+

\noindent File: \verb+aesburgers.py+

\section{Solution}\label{hv22.34solution}

The program \verb+aesburgers.py+ supplied does one thing: it asks for a bun and a number of patties, builds a burger and then encrypts the burger with AES in electronic cookbook mode (ECB).  The relevant two functions are:

{\small
\begin{minted}{python}
def encrypt(plain):
    cipher = AES.new(key.encode(), AES.MODE_ECB)
    plain += b' ' * ((16 - (len(plain) % 16)) % 16)
    enc = cipher.encrypt(plain)
    return ''.join('%02x' % b for b in enc)

def makeBurger(bun, patties):
    burger = b''
    burger += bun[::-1] # the bottom bun, flipped
    burger += patties * flag # the patties
    burger += bun # the top bun
    return burger
\end{minted}
}

\noindent
Since the burger is encrypted using ECB, we can craft a known plaintext attack:
In ECB, a given 16-byte-aligned 16-byte-block is always encrypted to the same
result.  If we can construct a bun in such a way that the reversed bun
corresponds to a given block of the burger, then we can brute force the flag.

To illustrate the principle, assume a bun \verb+0123456789ABCDEF+ and assume
that the number of patties times the flag-length is $n\times 16 + 1$.  Then the
constructed burger looks like
\noindent
\begin{verbatim}
FEDCBA9876543210
he2022{...
...
}0123456789ABCDE
F
\end{verbatim}
\noindent
Using the bun, we have full control over the first 16-bytes of the burger and we know 15
out of the second to last 16-byte block.  The one, unknown byte is determined by the flag.

If we are now choosing a bun like \verb+AAAAAAAAAAAAAAA.+, then the
first block becomes \verb+.AAAAAAAAAAAAAAA+, the second last will be
\verb+}AAAAAAAAAAAAAAA+.  This allows us to try all possible buns by
replacing the \verb+.+ with any printable character and check if the encrypted
blocks match.  If they match, we know the last character of the flag (which we
assume to be \verb+}+).  Once we have one character, we try to brute force
the next character by changing the number of patties and constructing a new
bun \verb+AAAAAAAAAAAAAA}.+

The first step is to figure out the number of characters in the flag.  Try all
allowed values for the number of patties, observe the length of the returned,
encrypted burger and solve for the smallest possible length of the flag.  Can
be done by thinking or just by brute forcing, and find 35 as flag length.

With this we have all information to start the brute force attack.  Once 16
characters have been found, the third to last block has to be considered and
the bun is just the reversed 16 bytes of the flag considered.

{\small
\begin{minted}{python}
from pwn import *
IP = '46.101.107.117'
PORT = 2207

def patties(flag_len, offset):
    for n in range(1,25):
        if (n*flag_len)%16 == offset%16:
            return n
    print(f'[x] cannot find number of patties')
    exit(-1)

p = remote(IP, PORT)

bun = b'1'*16
flag_len = 35
flag = ''

for j in range(1,flag_len+1):
    n = patties(flag_len, j)
    print(f'[ ] need {n} patties for char {j}')

    for i in range(33,127):
        bun = chr(i) + flag
        if len(bun) < 16:
            bun += (16-j)*'a'
        else:
            bun = bun[:16]
        bun = bun[::-1].encode()
        p.recvuntil(b'How many patties: ')
        p.sendline(bytes(str(n), encoding='ASCII'))

        p.recvuntil(b'Which bun? ')
        p.sendline(bun)
        p.recvuntil(b"Heres your order, enjoy!\n")
        burger = p.recvuntil(b'\n')
        cnub = burger[:32]
        shift = ((j-1)//16)*32
        cbunf = burger[-65-shift:-33-shift]
        if cnub == cbunf:
            flag = chr(i) + flag
            print(f'[+] found {j}th character ', chr(i), ' flag: ', flag)
            break
p.close()
\end{minted}
}
Flag: \verb+he2022{w3_luv_junk_f00d_s0m3t1m3s!}+


\documentclass{scrartcl}

\usepackage{amsmath}
\usepackage{hyperref}
\usepackage{pdfpages}
\usepackage[]{microtype}
\usepackage{float} %required for the placement specifier H

\usepackage{siunitx}
\hypersetup{colorlinks=true}

\setlength{\parskip}{1em}
\setlength{\parindent} {6ex}

\date{\today}
\author{Jake Rose \\ jacob.rose@mymail.champlain.edu}
\title{CSI 370 Computer Architecture \\ Research Project}
\subtitle{Arduino WS2812B Addressable LEDs}
\begin{document}
\maketitle

\section{Introduction}
In my apartment, I have a large array of lights that are Addressable LEDs. Addressable LEDs allow the color to be changed on a per-light basis, allowing for unique effects that can be much more unique than a standard LED strip.

I have achieved this already using an Arduino and the FastLED library, but I am hoping to gain more control and get faster speeds than the library can provide. That requires programming the protocol based directly off of the data sheet specification for WS2812B LED standard, the type of LED strip I am using for this project. Knowing the hardware is capable of this output is useful, but I want to see how fast I can make the refresh rate to create unique animations that are satisfying to the eye.

I want to create a very fluid bouncing dot using these LEDs.
I also will be creating an iteration with the similar array for color structure like FastLED.\linebreak

\centerline{\href{https://cdn-shop.adafruit.com/datasheets/WS2812B.pdf}{Full WS2812B Datasheet}}

I will include direct images from this document to explain the process for the Arduino when useful. \pagebreak

\section{Enviorment}
The light array is currently up to 300 LEDs. This has some unfortunate side effects, such as power consumption going over the USB 5v connector, and I do not have a 12v connector.

The specific micro-controller I will be using will be an Arduino Uno running at 16MHz. 16MHz when calculated out equals:

\begin{equation}
    \centering
    16MHz = 62.5\: nanoseconds\: per\: cycle
    \label{eq:ns_per_cycle}
\end{equation}

The WS2812B requires only three cables connected to the strip. The pin layout will be shown is the next section. The JST JM connector is the standard connector type used for these LED strips.

\begin{figure}[h]
    \centering
    \includegraphics[scale=0.20]{arduino-layout.jpeg}
    \caption{Arduino Layout for Project | Using an extension board that routes the positive to the red/yellow and ground to green, with the white wire routed to pin 13.}
    \label{fig:arduino_;ayout}
\end{figure}
\pagebreak

\section{WS2812B Specifications}

The WS2812B LED Strip uses only three wires, and each LED has four pins as shown below:
\begin{figure}[h]
    \centering
    \includegraphics[scale=0.6]{pinstructure.PNG}
    \caption{WS2812B Pin Configuration Specifications | The data is transferred through to each LED by daisy chaining off of the previous lights, with one input and one output per light. The light modifies the output based on the input as well to allow the separation of color per light.}
    \label{fig:pin_structure}
\end{figure}

\pagebreak
Using this singular wire, the specification gives guidelines on how to send color values on a per light basis. This is defined in the next three figures below.
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.8]{datatransfertime.PNG}
    \caption{WS2812B Byte Time Specifications | Proper timing is required as we are sending a stream of data instead of a singular bit. These are the timings that will be used in figure \ref{fig:timing_order}.
    Notice that we have a error leniency of 150ns, or around two full cycles of leeway in timing per variable.}
    \label{fig:byte_timing}
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[scale=0.8]{dataorderTLTH.PNG}
    \caption{WS2812B Timing Order | The values from \ref{fig:byte_timing} are used here to show how a singular bit value of zero or one will be sent. It will be useful to note here that for both of these situations, one bit sent equals 1.25µs. }

    \label{fig:timing_order}
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[scale=0.75]{bitorder.PNG}
    \caption{WS2812B Bit Order per LED | each color can have 7 bits, thus each light has 255 possible values, similar to most RGB standards.}
    \label{fig:bitorder}
\end{figure}
\pagebreak

\section{Development Cycle}

\subsection{First Approach | Start High Level}

\subsubsection{Strategy}
The first approach for this project was to start high level and to add in assembly after the fact to improve optimization. That proved to not be the case, as the smallest delay that can be created without assembly is one microsecond, which is just slightly too large to get the time differences required for the WS2812B data transfer.

\subsubsection{Progress Made}
We got the lights to turn on at full brightness, but that is the only brightness currently. The number of lights we control is also easily set, as there is a 50microsecond delay between each LED's bits being sent as specified in the sheet.

Assembly is going to be required, and using the nop command, the no-operation command, or any other operations that require one cycle, will be the only way to properly send the data.

\subsection{Second Approach | Some Assembly Required}

\subsubsection{Strategy}
The next approach was to create my own functions that would write zero and write one in a properly timed fashion. I chose this method as I was taking into considerations the few conclusions I made in the previous step.

\begin{enumerate}
    \item Each Bit can have a bit of space between them, and this can be useful for the for loops and subroutine calls required as I am still using high-level code for the majority of this.
    \item I could not create a custom delay as the calling of a function itself would use clock cycles.
    \item The time per clock cycle allows two full cycles before we are outside the error leniency.
\end{enumerate}

Using these findings, I figured that creating a writeZero() and writeOne() function in the code was going to be the best bridge between the high-level code and the low-level assembly.

\subsubsection{Progress Made}
    Using these properly timed delays, I was able to get the lights to be fully controllable on a per-LED basis. This allowed any RGB value to be inputted into the lights and it would reliably show up, and it also could work on any amount of LEDs.

    One of the largest setbacks and time sinks of this project occured at this step. The lights were completely addressable, but it seems there was an issue with the writeZero() function. The writeZero() function works about 95\% of the time from my estimation. This is clearly the right step, but if 5\% of the bits are ones instead of zero, there is a notable flicker among random lights, I would say around 10 per second. Since this is updating so quickly, I would assume for each loop through the LEDs, around 1 or two LED's have a one where a zero should have been. The flicker is thus less noticeable when the light is set to a higher brightness.

    There must be some issue either with the math behind my code, or a misreading of the specifications. I attempted every mix and match of nop operators on the writeZero() function, and even tried messing with the writeOne() function as well to see if it was having an effect, but this came to no noticeable cause.

    If I were to hypothesize, I would guess that the higher level code is throwing off some timing in sending the bits, and that it can add up every now and then and the lights interpret the data as a one instead of a zero. However, after many hours attempting fixes, I wanted to finish the project with this flaw instead of getting obsessed with fixing the flicker.

\subsection{Final Step | Fluid Light Bounce}

It took me a while to come to the conclusion that I must move on with my life and that I needed to show off the control that was there in the program. I started by creating a simple light bouncing, setting an integer increment per LED, and it looked exactly as expected, not good.

The second attempt I used a float and incremented that by a smaller value. The benefit of doing this is that you don't need to use a delay on a per-frame basis, and that lets you get higher frame rates than the previous method.

\subsection{If I Had More Time}

If I had to guess, my next step would be to put the whole program in assembly. I also think I could rearrange the delays and put some before the Arduino starts outputting the bits, but the real slowdown I believe is in the readBit() function, which I think I would need to rewrite in assembly.

I also did not complete the full project in assembly, and I know half of those operations could be minimized if written in assembly, and those noops I use in the writeZero() and writeOne() could be used for other purposes instead of wasting time. This would be the best option, but I do not have time to go to such extremes. I could have also only updated the pixels that actually change per frame, which is reasonable but it gets harder and harder for each animation you add.

\section{Limitations}

\subsection{Timing}
Arduino has a very poor sense of time, as well as a slow refresh rate. They are cheap, and the fact that I can push out these frame rates is insane, however any additional math equations for other patterns is going to slow down the frame rate considerably. There is also no proper time check equation, making counting how much time has passed is impossible to a reliable degree. The onkly way we can measure time is from the clock speed, and keeping track of the number of clock cycles is the only way to properly measure time in this system. The problem is, counting the number of clock cycles dynamically requires using clock cycles, so we need to create these sections for writeZero() and writeOne() in order to get one full bit sent in the proper timing, with it ensuring that it leaves the pin power set to low when it ends the function. Any time the pin is turned on, there is a set number of clock cycles written in Arduino to clear the bit again.

\subsection{Max Refresh Rate}
125ms per byte is required according to figure \ref{fig:byte_timing}

\begin{gather*}
        Time~per~LED = (1.25µs~to~send~byte * 24 bits\:per\:led)\\
        + 50µs~per~rest~between~LEDs = \\
        80~microseconds~per~LED
\end{gather*}

However, that is just the maximum theoretical input that these LEDs could handle, and that is just the sending of the color. We also need to assume that per LED, there are going to be clock cycles dedicated to calculating the color for the pixel (for the animation we program). These are just some approximations of the number of cycles for certain processes, but this will give a good sense of the magnitude of this project.

\begin{gather*}
        Time~per~LED~Calculation = 62.4~ns * (10~cycles~per~bit~check +\\
        12~cycles~per~increment~and~checks+\\
        (12 * 3)~cycles~to~calculate~light~values) = \\
        3.619 ms
\end{gather*}

This is just for checking and calculating the light values. It is hard to account for the high-level code written and the number of clock cycles, but each of these added checks, for loops, and calculations are going to affect the maximum framerate.

\section{Conclusion}
I have a theory that we need to send the data and handle the loops using an array and assembly to get rid of the flickering issue, and each send we need to send all the colors at the same time, but that would require a substantial amount of assembly knowledge that was not possible in the time. However, I am very happy that I got a large amount of control with these lights and that they actually work almost perfectly except for the flicker. I got a very fluid animation out of the Arduino using floats as well, further improving the results.\linebreak

\centerline{\href{https://youtu.be/4BZP083g6_s}{Video Example Of LEDs Working}}

\end{document}


\chapter{Course Introduction}

The concepts and techniques in this course are probably the most useful in engineering. A \textit{signal} is a function of one or more independent variables conveying information about a physical (or virtual) phenomena. A \textit{system} may respond to signals to produce other signals, or produce signals directly.

\begin{center}
  \begin{tikzpicture}[auto, node distance=3cm,>=latex']
    \node [input, name=input] {};
    \node [block, right of=input] (system) {System $T$};
    \node [output, right of=system] (output) {};

    \draw [draw,->] (input) -- node {Input $x$} (system);
    \draw [->] (system) -- node {Output $y$} (output);
  \end{tikzpicture}
\end{center}

This course is about the mathematical models and related techniques for the design and understanding of systems as signal transformations. We focus on a broadly useful class of systems, known as {\it linear, time-invariant systems}. You will learn about:

\begin{itemize}
\item the representation and analysis of signals as information carrying channels
\item and how to analyze and implement linear, time-invariant systems to transform those signals.
\end{itemize}

\section{Example Signals and Systems}

\begin{example} Electrical Circuits. This is a Sallen-Key filter, a second-order system commonly use to select frequencies from a signal:
  \begin{center}
    \begin{circuitikz}[american voltages,scale=0.8, every node/.style={transform shape}]
      \draw
      (7,3.5) node[op amp] (opamp1) {}
      (0,4) to[R,l=$R_1$,o-] (2,4)
      (2,4) to[short] (2,5)
      (2,5) to[C,l=$C_1$] (8.2,5)
      (8.2,5) to[short] (opamp1.out)
      (2,4) to[R, l=$R_2$] (4,4)
      (4,4) to[C, l=$C_2$] (4,0)
      (0,0) to[short,o-o] (12,0)
      (4,4) to[short] (opamp1.-)
      (opamp1.+) to[short] (5.8,1.75)
      (5.8,1.75) to[short] (8.2,1.75)
      (opamp1.out) to[R, l=$(1-\beta)R$] (8.2,1.75)
      (8.2,1.75) to[R, l=$\beta R$] (8.2,0)
      (opamp1.out) to[short, -o] (12,3.5)
      (0,4) to[open, v=$x(t)$] (0,0)
      (12,3.5) to[open, v=$y(t)$] (12,0);
    \end{circuitikz}
  \end{center}
  There are two signals we can easily identify, the input signal as the voltage applied across $x(t)$, and the output voltage measured across $y(t)$. We build on your circuits course by viewing this circuit as an implementation of a more abstract linear system. We see how it can be viewed as a frequency selective filter. We will see how to answer questions such as:
  how do we choose the values of the resistors and capacitors to select the frequencies we are interested in? and how do we determine what those frequencies are?
\end{example}

\begin{example} Robotic Joint. This is a Linear, Time-Invariant model of a DC motor, a mixture of electrical and mechanical components.

  \begin{center}
    \tikzstyle{block} = [draw, fill=gray!20, rectangle,
      minimum height=3em, minimum width=3em]
    \tikzstyle{sum} = [draw, fill=gray!20, circle, node distance=1cm]
    \tikzstyle{input} = [coordinate]
    \tikzstyle{output} = [coordinate]
    \tikzstyle{pinstyle} = [pin edge={to-,thin,black}]

    \begin{tikzpicture}[auto, node distance=2cm,>=latex, scale=0.9, every node/.style={transform shape}]

      \node [sum] at (2.5,0) (vsum) {$\Sigma$};
      \node [block] at (4,0) (arm) {$\frac{1}{R_a + L_a s}$};
      \node [block] at (6,0) (torque) {$K_i$};
      \node [sum] at (8,0) (tsum) {$\Sigma$};
      \node [block] at (9.5,0) (motor) {$\frac{1}{B_m + J_m s}$};
      \node [block] at (12,0) (int) {$\frac{1}{s}$};
      \node [block] at (6,-2) (backemf) {$K_b$};

      \draw [->] (1.5,0) node [left] {$e_a$} -- node[pos=0.8] {$+$} (vsum);
      \draw [->] (vsum) -- (arm);
      \draw [->] (arm) -- (torque) node [midway,above] {$i_a$};
      \draw [->] (torque) -- node[pos=0.8] {$+$} (tsum) node [midway,above] {$\tau_m$};
      \draw [->] (tsum) -- (motor);
      \draw [->] (8,1) node[above] {$\tau_L$} -- node[pos=0.8] {$-$} (tsum);
      \draw [->] (motor) -- (int) node [midway,above] {$\omega_m$};
      \draw [->] (11,0) |- (backemf);
      \draw [->] (backemf) -| node[midway] {$e_b$} node[pos=0.95] {$-$} (vsum);
      \draw [->] (int) -- (13,0) node[right] {$\theta_m$};
    \end{tikzpicture}
  \end{center}

  How do we convert the motor into a servo for use in a robotic joint? What are its characteristics (e.g.\ how fast can it move)?
\end{example}

\begin{example} Audio Processing. Suppose you record an interview for a podcast, but during an important part of the discussion, the HVAC turns on and there is an annoying noise in the background.

  \begin{center}
    \includegraphics[scale=0.5]{graphics/noisysignal.pdf}
  \end{center}

  How could you remove the noise minimizing distortion to the rest of the audio?
\end{example}

\begin{example} Communications. Consider a wireless sensor, that needs to transmit to a base station, e.g.\ a wireless mic system.
  \begin{center}
    \tikzset{block/.style = {draw, fill=white, rectangle,
        minimum height=3em, minimum width=2cm},
      input/.style = {coordinate},
      output/.style = {coordinate},
      pinstyle/.style = {pin edge={to-,t,black}},
      radiation/.style={decorate,decoration={expanding waves,angle=12,segment length=4pt}}
    }
    \begin{tikzpicture}[auto, node distance=2cm,>=latex']
      \node[block](tx){Sensor Node};
      \node[antenna] at (tx.east) {};
      \node[block,right = 5cm of tx](rx){Base Station};
      \node[antenna,xscale=-1] at (rx.west) {};

      \draw[radiation] ([shift={(1cm,2cm)}]tx.east)-- node [above=5mm] {} ([shift={(-1cm,2cm)}]rx.west);
    \end{tikzpicture}
  \end{center}

  How should the signal be processed so it can be transmitted? How should the received signal be processed?
\end{example}

\section{Types of Problems}
Applications of this material occur in all areas of science and engineering. When we have a measured output but are unsure what combination of inputs and system components could have produced it, we have a \textit{modeling} problem.

\begin{center}
  \begin{tikzpicture}[auto, node distance=3cm,>=latex']
    \node [input, name=input] {};
    \node [block, right of=input] (system) {?};
    \node [output, right of=system] (output) {};

    \draw [draw,->] (input) -- node {Input?} (system);
    \draw [->] (system) -- node {Output $y$} (output);
  \end{tikzpicture}
\end{center}

Models are the bedrock of the scientific method and are required to apply the concepts of this course to engineering problems.

When we know the input and the system description and desire to know the output we have an \textit{analysis} problem.

\begin{center}
  \begin{tikzpicture}[auto, node distance=3cm,>=latex']
    \node [input, name=input] {};
    \node [block, right of=input] (system) {System $T$};
    \node [output, right of=system] (output) {};

    \draw [draw,->] (input) -- node {Input $x$} (system);
    \draw [->] (system) -- node {Output?} (output);
  \end{tikzpicture}
\end{center}

Analysis problems are the kind you have encountered most often already. For example, given an electrical circuit and an applied voltage or current, what are the voltages and currents across and through the various components.

When we know either the input and desired output and seek the system to perform this transformation,
\begin{center}
  \begin{tikzpicture}[auto, node distance=3cm,>=latex']
    \node [input, name=input] {};
    \node [block, right of=input] (system) {System?};
    \node [output, right of=system] (output) {};

    \draw [draw,->] (input) -- node {Input $x$} (system);
    \draw [->] (system) -- node {Output $y$} (output);
  \end{tikzpicture}
\end{center}

or we know the system description and output and desire the input that would generate the output,

\begin{center}
  \begin{tikzpicture}[auto, node distance=3cm,>=latex']
    \node [input, name=input] {};
    \node [block, right of=input] (system) {System $T$};
    \node [output, right of=system] (output) {};

    \draw [draw,->] (input) -- node {Input?} (system);
    \draw [->] (system) -- node {Output $y$} (output);
  \end{tikzpicture}
\end{center}

we have a \textit{design problem}.

This course focuses on modeling and analysis with applications to electrical circuits and devices for measurement and control of the physical world and is broadly applicable to all ECE majors. Some Examples:

\begin{itemize}
\item Controls, Robotics, \& Autonomy: LTI systems theory forms the basis of perception and control of machines.
\item Communications \& Networking: LTI systems theory forms the basis of transmission and reception of signals, e.g. AM and FM radio.
\item Machine Learning: LTI systems are often used to pre-process samples or to create basis functions to improve learning.
\item Energy \& Power Electronic Systems: linear circuits are often modeled as LTI systems.
\end{itemize}

Subsequent courses, e.g. ECE 3704, focus more on analysis and design.

\section{Learning Objectives}

The learning objectives (LOs) for the course are:
\begin{enumerate}
\item[LO-1] Describe a given system using a block-level description and identify the input/output signals.
\item[LO-2] Mathematically model continuous and discrete linear, time-invariant systems using differential and difference equations respectively.
\item[LO-3] Analyze the use of filters and their interpretation in the time and frequency domains and implement standard filters in hardware and/or software.
\item[LO-4] Apply computations of the four fundamental Fourier transforms to the analysis and design of linear systems.
\item[LO-5] Communicate solutions to problems and document projects within the domain of signals and systems through formal written documents.
\end{enumerate}

These are broken down further into the following topic learning objectives (TLOs).  The TLOs generally map onto one class meeting but are used extensively in later TLOs.

\begin{enumerate}
\item[TLO-1] Course introduction (OW Forward and \S 1.0) TODO add brief summary?
  \begin{enumerate}
  \item Signals as models
  \item Systems as transformation of signals
  \item Prerequisites
  \end{enumerate}

\item[TLO-2] Continuous-time (CT) signals (OW \S 1.1 through 1.4 and 2.5): A continuous-time (CT) signal \index{CT Signal} is a function of one or more independent variables conveying information about a physical phenomena. This lecture gives an introduction to continuous-time signals as functions. You learn how to characterize such signals in a number of ways and are introduced to two very important signals: the unit impulse and the complex exponential.
  \begin{enumerate}
  \item Continuous-time signals as functions $\mathbb{R}\mapsto\mathbb{C}$
  \item Transformations of time
  \item Characterizing signals
    \begin{enumerate}
    \item periodic/aperiodic
    \item even/odd
    \item energy or/nor power
    \end{enumerate}
  \item Impulse function
  \item Step function
  \item Complex exponential
  \end{enumerate}

\item[TLO-3] Discrete-time (DT) signals (OW \S 1.1 through 1.4)
  \begin{enumerate}
  \item Discrete-time signals as functions $\mathbb{Z}\mapsto\mathbb{C}$
  \item Transformations of time index
  \item Characterizing signals
    \begin{enumerate}
    \item periodic/aperiodic
    \item even/odd
    \item energy or/nor power
    \end{enumerate}
  \item Impulse function
  \item Step function
  \item Complex exponential
  \end{enumerate}

\item[TLO-4] CT systems as linear constant coefficient differential equations (OW \S 2.4.1)
  \begin{enumerate}
  \item LCCDE and their solution (1st and 2nd order)
  \item impulse response from LCCDE
  \end{enumerate}

\item[TLO-5] DT systems as linear constant coefficient difference equations (OW \S 2.4.2)
  \begin{enumerate}
  \item LCCDE and their solution (1st and 2nd order)
  \item impulse response from LCCDE
  \end{enumerate}

\item[TLO-6] Linear time invariant CT systems (OW \S 1.5, 1.6, 2.3)
  \begin{enumerate}
  \item Memory
  \item Invertability
  \item Causality
  \item Stability
  \item Time-invariance
  \item Linearity
  \item Define LTI system
  \end{enumerate}

\item[TLO-7] Linear time invariant DT systems (OW \S 1.5, 1.6, 2.3)
  \begin{enumerate}
  \item Memory
  \item Invertability
  \item Causality
  \item Stability
  \item Time-invariance
  \item Linearity
  \item Define LTI system
  \end{enumerate}

\item[TLO-8] CT convolution (OW \S 2.2)
  \begin{enumerate}
  \item Review CT LTI systems and superposition property
  \item CT Convolution Integral
  \item Properties of convolution
    \begin{enumerate}
    \item communative
    \item distributive
    \item associative
    \end{enumerate}
  \item Determining system response using convolution with impulse response
  \end{enumerate}

\item[TLO-9] DT convolution (OW \S 2.1)
  \begin{enumerate}
  \item Review DT LTI systems and superposition property
  \item DT Convolution Sum
  \item Properties of convolution
    \begin{enumerate}
    \item communative
    \item distributive
    \item associative
    \end{enumerate}
  \item Determining system response using convolution with impulse response
  \end{enumerate}

\item[TLO-10] CT block diagrams (OW \S 1.5.2 and 2.4.3)
  \begin{enumerate}
  \item blocks represented by impulse response
  \item series and parallel connections, reductions
  \item scale, sum, and integrator blocks
  \item equivalence of LCCDE's and block diagrams
  \item first-order differential equation as feedback motif
  \item second-order differential equation as a feedback motif
  \item implementing a LCCDE using adders, multipliers, and integrators
  \end{enumerate}

\item[TLO-11] DT block diagrams (OW \S 1.5.2 and 2.4.3)
  \begin{enumerate}
  \item blocks represented by impulse response
  \item series and parallel connections, reductions
  \item scale, sum, and unit delay blocks
  \item equivalence of LCCDE's and block diagrams
  \item first-order difference equation as feedback motif
  \item second-order difference equation as a feedback motif
  \item implementing a LCCDE using adders, multipliers, and delays
  \end{enumerate}

\item[TLO-12] Eigenfunctions of CT systems (OW \S 3.2 and 3.8)
  \begin{enumerate}
  \item Eigenfunction $e^{st}$
  \item Transfer Function $H(s)$
  \item Stability and Frequency Response (FR)  $H(j\omega)$
  \item How this is useful - decomposition of input signal into complex exp
  \item What signals can be decomposed this way, foreshadow Fourier Analysis
  \end{enumerate}

\item[TLO-13] Eigenfunctions of DT systems (OW \S 3.2 and 3.8)
  \begin{enumerate}
  \item Eigenfunction $z^{n}$
  \item Transfer Function $H(z)$
  \item Stability and Frequency Response (FR) $H\left(e^{j\omega}\right)$
  \item How this is useful - decomposition of input signal into complex exp
  \item What signals can be decomposed this way, foreshadow Fourier Analysis
  \end{enumerate}

\item[TLO-14] CT Fourier Series representation of signals (OW \S 3.3 through 3.5)
  \begin{enumerate}
  \item review CT periodic functions
  \item harmonic sums
  \item derive synthesis equation
  \item derive analysis equation
  \item spectrum plots
  \item define mean-square convergence
  \item truncated CT FS
  \item stable LTI system response using CTFS
  \item example of the impulse train (for sampling theory later)
  \item formal Dirichlet conditions
  \item properties of CT FS
  \end{enumerate}

\item[TLO-15] DT Fourier Series representation of signals  (OW \S 3.6 and 3.7)
  \begin{enumerate}
  \item review DT periodic functions
  \item harmonic sums
  \item derive synthesis equation
  \item derive analysis equation
  \item spectrum plots
  \item stable LTI system response using DTFS
  \item properties of DT FS
  \end{enumerate}

\item[TLO-16] CT Fourier Transform (OW \S 4.0 through 4.7)
  \begin{enumerate}
  \item derive the CTFT pair from the CTFS
  \item Dirichlet existence conditions
  \item CTFT of the CTFS
  \item Properties of the CT Fourier Transform
    \begin{enumerate}
    \item linearity
    \item time shift
    \item conjugacy
    \item integration and differentiation: application to LCCDE $\mapsto$ CTFR
    \item time scaling
    \item duality
    \item convolution: stable LTI system response using CTFT
    \item multiplication/modulation
    \item application of the properties in combination
    \end{enumerate}
  \end{enumerate}

\item[TLO-17] DT Fourier Transform (OW \S 5.0 though 5.8)
  \begin{enumerate}
  \item derive the DTFT from DTFS
  \item DTFT of DTFS
  \item Properties of the DT Fourier Transform
    \begin{enumerate}
    \item periodicity
    \item linearity
    \item index-shift: application to LCCDE $\mapsto$ DTFR
    \item frequency shift
    \item conjugation
    \item finite difference and accumulation
    \item interpolation /index expansion
    \item frequency differentiation
    \item Parseval's
    \item convolution: stable LTI system response using DTFT
    \item multiplication/modulation
    \item application of the properties in combination
    \end{enumerate}
  \end{enumerate}

\item[TLO-18] CT Frequency Response (OW \S 6.1, 6.2, 6.5)
  \begin{enumerate}
  \item review CTFR as CTFT of impulse response
  \item review CTFR to/from LCCDE
  \item review CTFR to/from block diagram
  \item magnitude-phase representation of the frequency response
  \item frequency response acting on sinusoids
  \item Bode plots
    \begin{enumerate}
    \item why plot it this way: dB units and log time axis
    \item how to read them (\textbf{not} construct them manually)
    \item Bode plots in software, e.g. Matlab/Python/Julia
    \end{enumerate}
  \item CTFR of first and second order systems
  \end{enumerate}

\item[TLO-19] DT Frequency Response (OW \S 6.1, 6.2, 6.6)
  \begin{enumerate}
  \item review DTFR as DTFT of impulse response
  \item review DTFR to/from LCCDE
  \item review DTFR to/from block diagram
  \item magnitude-phase representation of the frequency response
  \item frequency response acting on sinusoids
  \item DTFR plots
    \begin{enumerate}
    \item periodicity
    \item dB units
    \item DTFR plots in software, e.g. Matlab/Python/Julia
    \end{enumerate}
  \item DTFR of first and second order systems
  \end{enumerate}

\item[TLO-20] Frequency Selective Filters in CT (OW \S 3.9, 3.10, 6.3, 6.4)
  \begin{enumerate}
  \item ideal low-pass
  \item ideal high-pass
  \item ideal bandpass
  \item ideal notch/bandstop
  \item practical filters
  \item transformations
  \item first and second order systems as building blocks
    \begin{enumerate}
    \item review LCCDE representation
    \item review block diagram representation
    \item review CTFR representation
    \item CT 1st order RC+buffer
    \item CT Sallen-key
    \end{enumerate}
  \end{enumerate}

\item[TLO-21] Frequency Selective Filters in DT (OW \S 3.11, 6.3, 6.4)
  \begin{enumerate}
  \item ideal low-pass
  \item ideal high-pass
  \item ideal bandpass
  \item ideal notch/bandstop
  \item practical filters
  \item transformations
  \item first and second order systems as building blocks
    \begin{enumerate}
    \item review LCCDE representation
    \item review block diagram representation
    \item review DTFR representation
    \item DT 1st order implementation in code
    \item DT 2nd order implementation in code
    \end{enumerate}
  \end{enumerate}

\item[TLO-22] The Discrete Fourier Transform
  \begin{enumerate}
  \item time window the DTFT to get the DFT
  \item interpreting the index axis as DT and CT frequency
  \item zero-padding
  \item offline or batched filtering using the DFT
  \item briefly mention fast algorithms to compute the DFT = FFT
  \end{enumerate}

\item[TLO-23] Sampling (OW \S 7.1, 7.3, 7.4)
  \begin{enumerate}
  \item sampling using the impulse train
  \item derive the Nyquist rate
  \item effects of aliasing
  \item practical ADC (sample and hold, SAR, bit-width)
  \item designing anti-aliasing filters
  \end{enumerate}

\item[TLO-24] Reconstruction (OW \S 7.2)
  \begin{enumerate}
  \item reconstruction as removal of images
  \item reconstruction as interpolation
  \item practical DAC: R-2R ladder
  \item designing reconstruction filters
  \end{enumerate}
\end{enumerate}

\section{Graphical Outline}

\begin{center}
  \includegraphics[scale=1]{graphics/system-representations-fig.pdf}
\end{center}
\begin{center}
  \includegraphics[scale=1]{graphics/analysis-methods-fig.pdf}
\end{center}


\chapter*{Abstract}

This is a report on advancements in type theory and formalized proof-based programming languages often referred to as verified programming languages. I have tried to explain the importance of mathematical modelling and logic system development based on the ancient works of mathematicians like Per-Martin Lof by the applications in present programming toolchain and possible future aspects like applications in a quantum  programming toolchain.

Things are explained in chapter-wise manner and sufficient effort has been put to properly introduce things making understanding things easy.The chapters deal with mathematical logic systems first and then proceed to explain how this has been used and developed in real systems. With the current developments that are going on in this field, I have tried to explain the very possible usage in modelling a quantum computer's theoretical behaviour and applications in Machine Learning.\\

This area of study comes under the umbrella term of \underline{Programming Language Research} and scientists all over have been  using these concepts to do explain the language semantics of any new programming language. This works using a system of inductive logics and as in abstract algebra, various operations on a given mathematical structure (\textit{example : Rings, Groups, Sets etc.}) a computer program is thought of being an operation on the a given type system. Type theory in its most literal meaning deals with the abstract idea of Types as a fundamental mathematical structures. Classical programming languages deals with data types and now some functional languages like \textit{Haskell, Agda, Ocaml etc} consider operations too as a type. Debugging has been made so easy because of the type systems and test driven development. Consider the case where it is just some control signals flipping the states of bits and without a proper analytical typed contraint. The system's behaviour in this case will be completely unpredictable program-wise. Thus I would assume readers to believe with me that we would all like to have programs check that our programs are correct. Today most people who write software ,people from both academia and industry assume that the costs of formal verification of program outweighs the benefits. One simple example case is of \textit{javascript} programming language which is very informally developed thus has a very weak type system that often leads to bugs.  \textit{Haskell} on other hand does not allow programs to disobey the type system used for that program.

\section*{Categories}
[ \textbf{Operating Systems} ]:  Reliability—Verification, [\textbf{Software Engineering}]: Software/Program Verification

\section*{Key Words}
Verification, Design, COQ, PLT (Programming Language Theory)


%!TEX root =  ../main.tex

\mychapters{Parents}{parents}{\chapdir/pics/Elakala_Waterfalls_Swirling_Pool_Mossy_Rocks}

Why is most of our universe so easy to model?
Is it valid to divide the world into dimensions,
or are those purely human constructs with no external reality?
Is it necessary to think in more
than one dimension at once?  More than two?
What good is instantaneous information when
time continues to flow and never stops anyway?

Some of the simplest functions to deal with are lines, which is why your education
up to this point as largely focuses on them.  Lines through the origin model directly
proportional variables.  Their opposite --- you recall --- are inversely proportional
variables, and produce a weird looking graph called a hyperbola.  But even hyperbolas
look like lines, if you zoom in far enough.  So do parabolas.  And roots.  Lastly,
we will see what a how velocity information can tell us about position ...
and what it can't.

\newpage
\chapterminitoc

%							3 - 1
\newpage
\invisiblesection{Linear}
\marginlessinput{\chapdir/0301p}
\newpage
\input{\chapdir/0301}
\newpage
\input{\chapdir/0301x}

%							3 - 2
\newpage
\invisiblesection{1/x}
\marginlessinput{\chapdir/0302p}
\newpage
\input{\chapdir/0302}
~\vfill
\newpage
\subsection{Exercises}
Kuta

%							3 - 3
\newpage
\invisiblesection{Quadratic}
\marginlessinput{\chapdir/0303p}
\newpage
\input{\chapdir/0303}
%\newpage
\subsection{Exercises}
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0303xA.pdf}}
\newpage
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0303xB.pdf}}
\newpage
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0303xC.pdf}}

%							3 - 4
\newpage
\invisiblesection{Radical}
\marginlessinput{\chapdir/0304p}
\newpage
\input{\chapdir/0304}
~\vfill
\subsection{Exercises}
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0304xA.pdf}}
newpage
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0304xB.pdf}}
\newpage
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0304xC.pdf}}
\newpage
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0304xD.pdf}}

%							3 - 5
%\newpage
\invisiblesection{Derivatives}
%\subsection{Problems}
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0305p.pdf}}
\newpage
\input{\chapdir/0305}
~\vfill
%\newpage
\subsection{Exercises}
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0305xA.pdf}}
\newpage
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0305xB.pdf}}
\newpage
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0305xC.pdf}}

\newpage

\section{Review}
\subsection{Chapter Review}
We have covered a lot of ground.  The fundamental linearity of many functions serves
as justification for us using the difference quotient on them.  This created new functions which
described the lines tangent to them at any point.  We have only seen this on function which are
powers of $x$, which can be written as $x^n$, and we saw that the rule of thumb for the
derivative of such equations is $n\cdot x^{n-1}$.
\subsection{Chapter Test}