The \code{sp} subcommand optimizes the location of sensors in a
water distribution network to minimize the impact of potential
contamination incidents. The \code{sp} subcommand has a rich interface
that supports a variety of optimization formulations, and it
integrates a wide range of optimization solvers.
An impact file is used to define the ensemble of contamination
incidents. By default, sensors can be placed at any feasible junction in
the network, but fixed and infeasible locations can be
specified within the \code{sp} subcommand.

A flowchart representation of the \code{sp} subcommand is shown in Figure \ref{fig:sp-flowchart}.

\begin{figure}[h!]
  \centering
  \includegraphics[scale=0.80]{graphics/sp_flowchart.pdf}
  \caption{Sensor placement flowchart.}
  \label{fig:sp-flowchart}
\end{figure}

The required input for the \code{sp} subcommand is an impact file and
sensor characteristics. The impact file could be created by the \code{sim2Impact} subcommand,
or through some non-WST impact calculation process.
Multiple impact files can be used as input to the sensor placement problem. The other
required input is the sensor characteristics. These characteristics are supplied through
the \code{sp} WST configuration file as well as additional files that provide details
on the cost and failure rates of the sensors.

Advanced features of the \code{sp} subcommand are discussed in
Section~\ref{chap:pmedian}. This includes a discussion of how to specify
feasible sensor locations and evaluation techniques. The section
also includes advanced sensor placement methods for computing a
bound on the quality of sensor networks, and techniques for minimizing
memory used during sensor placement: (a) aggregation of scenarios
and (b) skeletonization of the water distribution system network
model.

\section{Sensor Placement Formulations}\label{sp_formulation}

Several sensor placement optimization formulations are available
in the \code{sp} subcommand. The following formulations are described
below: expected-\/impact, robust optimization, side-constrained
and minimum cost formulations. WST also supports several sensor
placement formulations that are not documented yet: multi-stage
sensor placement and imperfect sensor models.

\subsection{Expected-\/Impact Formulation}\label{sp:average_formulation}

The most widely studied sensor placement formulation for a contamination warning system
(CWS) design is to minimize the expected impact of an ensemble of contamination incidents
given a fixed number of sensors. This formulation has also become the standard formulation
in the \code{sp} subcommand, because it can be effectively used to select sensor placements
in large water distribution networks.

A mixed-integer programming (MIP) formulation for expected-\/impact sensor placement is (eSP):
\begin{align}
\textrm{minimize} \qquad & \sum_{a \in A} \alpha_a \sum_{i \in {\cal L}_a} d_{ai} x_{ai} \label{eqn:eSP} \\
\textrm{subject to} \qquad &\sum_{i\in {\cal L}_a} x_{ai} = 1 &&\forall a \in A\\
&x_{ai} \le s_i &&\forall a \in A, i \in {\cal L}_a\\
&\sum_{i \in L} c_i s_i \le p\\
&s_i \in \{0,1\} &&\forall i \in L\\
&0 \leq x_{ai} \leq 1 &&\forall a \in A, i \in {\cal L}_a
\end{align}

This MIP minimizes the expected impact of a set of contamination incidents defined by $A$.
For each incident $a \in A$, $\alpha_a$ is the weight of incident $a$, which is typically a
probability. This formulation integrates contamination impact calculations, which
are reported at a set of locations from the full set, denoted $L$,
where a location refers to a network node. For each incident $a$, ${\cal L}_a \subseteq L$
is the set of locations that can be contaminated by $a$. Thus, a sensor at a
location $i \in {\cal L}_a$ can detect contamination from incident $a$ at the time
contamination first arrives at location $i$. Each incident is witnessed
by the first sensor to see it. For each incident $a \in A$ and location
$i \in {\cal L}_a$, $d_{ai}$ defines the impact of the contamination incident $a$
if it is witnessed by location $i$. This impact metric assumes that as soon as a
sensor witnesses contamination, then any further contamination impacts are mitigated
(perhaps after a suitable delay that accounts for the response time of the water utility).
The $s_i$ variables indicate where sensors are placed in the network, $c_i$ is
the cost of placing a sensor at location $i$ and $p$ is the budget.

The $x_{ai}$ variables indicate whether incident $a$ is witnessed by a sensor at
location $i$. They are defined as continuous variables between 0 and 1.
In practice, there is always an optimal solution where $x_{ai}$ is binary.
If $x_{ai}$ is fractional, then two or more equally good locations have sensors
\citet{TEVASPOTCompendium10}. A given set of sensors might not be able to witness
all contamination incidents. To account for this, $L$ contains a dummy location, $q$.
The dummy location is assigned the impact if contamination was not detected at any node location.
The dummy location is not a physical location in the network.
This dummy location is in all sets ${\cal L}_a$. If the dummy location witnesses
an incident, it generally means that no real sensor can detect that incident. The
impact for this location is the impact of the contamination incident after the entire
contaminant transport simulation has finished, which estimates the impact that
would occur without an online sensor network. The impact of a dummy detection is greater
than all other impacts for each incident, so the witness variable $x_{ai}$ for the
dummy will only be selected if no sensors have been placed that can detect this
incident with smaller impact.

For examples on expected-\/impact sensor placement (eSP), see Example
\ref{sp_example1}, \ref{sp_example2} and \ref{sp_example3}.
\citet{BerHarPhiUbeWat06} describe eSP, and they note that this
formulation is identical to the well-\/known $p$-\/median facility
location problem \citep{MirFra90} when $c_i=1$. In the $p$-\/median
problem, $p$ facilities (e.g., central warehouses) are to be located
on $m$ potential sites such that the sum of distances $d_{ai}$
between each of $n$ customers (e.g., retail outlets) and the nearest
facility $i$ is minimized. In comparing eSP and $p$-\/median problems,
there is equivalence between (1) sensors and facilities, (2)
contamination incidents and customers and (3) contamination impacts
and distances. While eSP allows placement of at most $p$ sensors,
$p$-\/median formulations generally enforce placement of all $p$
facilities; in practice, the distinction is irrelevant unless $p$
approaches the number of possible locations. This equivalence enables the application of $p$-\/median
solvers to eSP (see Example~\ref{sp_example3}).

\subsection{Robust Formulations}\label{sp:robust_formulation}

The eSP model described in Section \ref{sp:average_formulation} can
be viewed as optimizing one particular statistic of the distribution
of impacts defined by the contaminant transport simulations. However,
other statistics might provide more robust solutions that are
less sensitive to changes in this distribution~\citep{WatHarMur06a, WatMurHar09}.
Consider the following generalization of eSP:
\begin{align}
\textrm{minimize} \qquad &\textrm{Impact}_{f}(\alpha,d,x) \\
\textrm{subject to} \qquad &\sum_{i\in {\cal L}_a} x_{ai} = 1 &&\forall a \in A\\
&x_{ai} \le s_i &&\forall a \in A, i \in {\cal L}_a\\
&\sum_{i \in L} c_i s_i \le p\\
&s_i \in \{0,1\} &&\forall i \in L\\
&0 \leq x_{ai} \leq 1 &&\forall a \in A, i \in {\cal L}_a
\end{align}

The function $\textrm{Impact}_ {f}(\alpha,d,x)$ computes a statistic of the impact
distribution. The following functions supported in WST have been developed by
researchers to find robust solutions to optimization problems \citep{WatHarMur06a, WatMurHar09}:
\begin{itemize}
\item {\bfseries Mean:} This is the statistic used in eSP (Equation \ref{eqn:eSP})
\item {\bfseries VaR:} Value-\/at-\/Risk (VaR) is a percentile-\/based metric.
Given a confidence level $\beta\in(0,1)$, the VaR is the value of the distribution
at the $1-\beta$ percentile \citep{TopVlaZen02}. The value of VaR is less than
the TCE value (see below).
Mathematically, suppose a random variable $W$ describes the distribution of possible
impacts. The probability that $W$ is less than a value $w$ is denoted as $\P[W \le w]$. Then
\begin{equation}
{\rm VaR}(W,\beta) = \min \{ w \mid \P[W \le w] \ge \beta \}
\end{equation}
Note that the distribution $W$ changes with each sensor placement. Further, VaR
can be computed using the $\alpha$, $d$ and $x$ values.
\item {\bfseries TCE:} The Tail-\/Conditioned Expectation (TCE) is a related
metric that measures the conditional expectation of impact exceeding VaR at a
given confidence level. Given a confidence level $1-\beta$, TCE is the expectation
of the worst impacts whose likelihood sums to $\beta$. This value is between VaR and
the worst-\/case value.
Mathematically, then
\begin{equation}
{\rm TCE}(\beta) = {\rm E}\left[ W \mid W \ge {\rm VaR}(\beta) \right]
\end{equation}
\item {\bfseries CVar:} The Conditional Value-\/at-\/Risk (CVaR) is a
linearization of TCE investigated by \citet{rockafellar02cvar}. CVaR
approximates TCE with a continuous, piecewise-\/linear function of $\beta$,
which enables the use of CVaR in a MIP model.
\item {\bfseries Worst:} The worst impact value can be easily computed, since a
finite number of contamination incidents are simulated. However, this statistic
is sensitive to changes in the number of contamination incidents that are simulated;
adding additional contamination incidents could significantly impact this statistic.
\end{itemize}

WST includes robust MIP reformulations of eSP for the the worst and CVar statistics.
The reformulation to minimize the worst contamination impact is (wSP):
\begin{align}
\textrm{minimize} \qquad & w \label{eqn:wSP} \\
\textrm{subject to} \qquad & \alpha_a \sum_{i \in {\cal L}_a} d_{ai} x_{ai} \leq w && \forall a \in A\\
&\sum_{i\in {\cal L}_a} x_{ai} = 1 &&\forall a \in A\\
&x_{ai} \le s_i &&\forall a \in A, i \in {\cal L}_a\\
&\sum_{i \in L} c_i s_i \le p\\
&s_i \in \{0,1\} &&\forall i \in L\\
&0 \leq x_{ai} \leq 1 &&\forall a \in A, i \in {\cal L}_a
\end{align}
This is a standard formulation for the $p$-\/center problem \citep{Daskin95,ElloumiEtAl04}.

Similarly, the reformulation to minimize CVaR is (cvarSP):
\begin{align}
\textrm{minimize} \qquad & v + \frac{1}{\beta} \sum_{a \in A} \alpha_a y_a \label{eqn:cvarSP} \\
\textrm{subject to} \qquad & y_a \geq \sum_{i \in {\cal L}_a} d_{ai} x_{ai} - v && \forall a \in A\\
&y_q \geq 0 &&\forall a \in A \\
&\sum_{i\in {\cal L}_a} x_{ai} = 1 &&\forall a \in A\\
&x_{ai} \le s_i &&\forall a \in A, i \in {\cal L}_a\\
&\sum_{i \in L} c_i s_i \le p\\
&s_i \in \{0,1\} &&\forall i \in L\\
&0 \leq x_{ai} \leq 1 &&\forall a \in A, i \in {\cal L}_a
\end{align}
The variable $v$ represents $\rm VaR$, which is implicitly computed when this model is solved.
See \citet{rockafellar02cvar} for further discussion of CVaR formulations.

Note that these formulations share a core set of constraints and
variables with eSP. The difference in these models is how the
objective is expressed.
For examples on robust sensor placement (wSP and cvarSP), see Example \ref{sp_example4} and \ref{sp_example5}.

\subsection{Side-\/Constrained Formulation}\label{sp:sc_formulation}

Another natural generalization of eSP is to consider the addition
of side constraints that represent bounds on alternate objectives
or statistics. For example, consider a simple extension of eSP that
includes a single side-constraint (scSP):
\begin{align}
\textrm{minimize} \qquad & \sum_{a \in A} \alpha_a \sum_{i \in {\cal L}_a} d_{ai} x_{ai} \label{eqn:scSP} \\
\textrm{subject to} \qquad &\sum_{i\in {\cal L}_a} x_{ai} = 1 &&\forall a \in A\\
&x_{ai} \le s_i &&\forall a \in A, i \in {\cal L}_a\\
&\sum_{i \in L} c_i s_i \le p\\
&s_i \in \{0,1\} &&\forall i \in L\\
&0 \leq x_{ai} \leq 1 &&\forall a \in A, i \in {\cal L}_a\\
& \sum_{a \in A} \alpha_a \sum_{i \in {\cal L}_a} \hat{d}_{ai} x_{ai} \leq G
\end{align}

The last constraint in this formulation bounds the value of an
impact statistic $\hat{d}_{ai}$. Note that this statistic could
easily be used as the objective for (eSP). This can be
viewed as a goal constraint. Iteratively solving scSP for different
goals, $G$, provides an assessment of the trade-off between the impact
statistics in the objective and this constraint. Hence, the scSP formulation
provides a mechanism for analyzing trade-\/offs between different objectives.

WST provides general support for side-constraints beyond what is
represented in scSP, since multiple side-constraints can
be specified. Additionally, robust statistics can be specified.
For example, WST can express sensor placement formulations where
the mean impact is minimized while the worst-case impact is
constrained.
See Example \ref{sp_example6} for a demonstration of side-\/constrained sensor placement (scSP).

\subsection{Min-\/Cost Formulation}\label{sp:minCost_formulation}

The eSP model described in Section~\ref{sp:average_formulation}
minimizes expected contamination impact subject to a cost constraint
on the number of sensors that are installed. A related sensor
placement formulation is to minimize the cost of installing sensors
while constraining the contamination impact to be below a specified
threshold, $u$.

For example, eSP can be reformulated to minimize cost (mcSP):
\begin{align}
\textrm{minimize} \qquad & \sum_{i \in L} c_i s_i \label{eqn:ceSP} \\
\textrm{subject to} \qquad &\sum_{i\in {\cal L}_a} x_{ai} = 1 &&\forall a \in A\\
&x_{ai} \le s_i &&\forall a \in A, i \in {\cal L}_a\\
&\sum_{a \in A} \alpha_a \sum_{i \in {\cal L}_a} d_{ai} x_{ai} \leq u\\
&s_i \in \{0,1\} &&\forall i \in L\\
&0 \leq x_{ai} \leq 1 &&\forall a \in A, i \in {\cal L}_a
\end{align}

See Example \ref{sp_example7} for a demonstration of min-\/cost sensor placement (mcSP).

\if 0
\subsection{Multiple Objective Formulation}\label{sp:multiObj_formulation}

CWS design generally requires the evaluation and optimization of a variety of
performance objectives. Some performance objectives cannot be simultaneously optimized,
and thus a CWS design must be selected from a trade-\/off between these
objectives~\cite{WatGreHar04a}.

The \code{sp} subcommand supports the analysis of these trade-\/offs with the specification of additional
constraints on impact measures. For example, a user can minimize the expected
extent of contamination (EC) while constraining the worst-\/case time to
detection (TD). The \code{sp} subcommand allows for the specification of more than one impact
constraint. However, the sensor placement solvers cannot reliably optimize formulations with
more than one impact constraint.

\subsection{Imperfect Sensors Formulation}\label{sp:imperfect_formulation}

The previous sensor placement formulations make the implicit assumption that
sensors work perfectly. That is, they never fail to detect a contaminant when it
exists, and they never generate an erroneous detection when no contaminant exists.
In practice, sensors are imperfect, and they generate these types of errors.

The \code{sp} subcommand addresses this issue by supporting a formulation that models simple sensor
failures~\cite{BerryCHLPW09}. Each sensor, $s_i$, has an associated probability
of failure, $p_i$. With these probabilities, the probability
that a contamination incident will be detected by a particular sensor can be easily assessed. Thus, it
is straightforward to compute the expected impact of a contamination incident.

This formulation does not explicitly allow for the specification of probabilities
of false detections. These probabilities do not impact the performance of a CWS
during a contamination incident. Instead, they impact the day-\/to-\/day maintenance
and use of the CWS; erroneous detections create work for the CWS users, which is an
ongoing cost. The overall likelihood of false detections is simply a function of the
sensors that are selected. In cases where every sensor has the same likelihoods, this
implies a simple constraint on the number of sensors.
\fi

\section{Sensor Placement Solvers}

The \code{sp} subcommand performs optimization using a solver specified in the
configuration file. All of the solvers supported by the \code{sp} subcommand are practical
for small-\/sized water distribution networks, and heuristic
solvers can be used to find sensor placements for very large networks.

The \code{sp} subcommand interfaces with a variety of external
solvers that can be used to perform sensor placement. Several
different MIP solvers can be used to find a globally optimal solution
for the eSP MIP formulation. However, this might be a computationally
expensive process (especially for large problems), and the size of
the MIP formulation can become prohibitively large in some cases.
A variety of public-domain and commercial solvers can be used by
the \code{sp} subcommand, including GLPK, CBC, PICO, CPLEX, GUROBI and
XPRESS.

% TODO - verify this text.  How do we name these solvers?

A greedy randomized adaptive sampling process (GRASP) heuristic
performs sensor placement optimization without
explicitly creating a MIP formulation. Thus, this solver uses much
less memory, and it usually runs very quickly. Although the GRASP
heuristic does not guarantee that a globally optimal solution is
found, it has proven effective at finding optimal solutions to a
variety of large-scale applications. Two different implementations
of the GRASP solvers can be used: an AT\&T commercial solver (att\_grasp)
or an open-source implementation of this solver (snl\_grasp).

% TODO - verify this text.  What is the status of this solver?

The Lagrangian heuristic uses the structure of the $p$-\/median MIP
formulation (eSP) to find near-optimal solutions while computing a
lower bound on the best possible solution.

\section{\code{sp} Subcommand}

The \code{sp} subcommand is executed with the following command:

\begin{unknownListing}
wst sp <configfile>
\end{unknownListing}

where \code{configfile} is a WST configuration file in the YAML format.

The \code{---help} option prints information about this subcommand, such as usage,
arguments and a brief description:

\begin{unknownListing}
wst sp --help
\end{unknownListing}

Two other options can be used to print help information.
The \code{---help-problems} option prints a table of the different types of
optimization problems that can be solved with the \code{sp} subcommand. For example,
the following is a description of the standard problem solved by the \code{sp} subcommand (eSP):

\begin{unknownListing}
default, p-median, average-case perfect-sensor
----------------------------------------------
mean obj
0 constraints
perfect
single obj
1 stage
exact
\end{unknownListing}

The first row lists the different names that can be used to specify
this problem type. These are synonyms, since the standard problem,
eSP, is a $p$-\/median problem where the objective is an average statistic.
This is also the standard perfect-sensor formulation, where all
sensors are assumed to work perfectly without failures.
The six rows following the dashed lines are different characteristics of this problem:
\begin{enumerate}
\item the type of objective used
\item the number of side-constraints
\item specify whether sensors are perfect (i.e., without failures) or whether they can fail
\item the number of objectives
\item the number of stages (i.e., time steps) in the sensor placement formulation
\item the type of solution that is required (e.g., an exact solution versus any feasible solution).
\end{enumerate}
These six characteristics are used by the \code{sp} subcommand
to verify the suitability of solvers that are specified for optimization.

The \code{---help-solvers} option prints a table of the different solvers that can
be applied to perform optimization. For example, the following is a description
of the solvers that can be used to optimize \code{average-case perfect-sensor} problems (which is the default):

\begin{unknownListing}
Problem Type                  Solver        Modeling Language
======================================================================
average-case perfect-sensor   *att_grasp     none
average-case perfect-sensor   *cbc           pyomo
average-case perfect-sensor   *cplex         pyomo
average-case perfect-sensor   *glpk          pyomo
average-case perfect-sensor    gurobi        pyomo
average-case perfect-sensor   *lagrangian    none
average-case perfect-sensor    pico          ampl
average-case perfect-sensor    pico          pyomo
average-case perfect-sensor   *snl_grasp     none
average-case perfect-sensor    xpress        pyomo
\end{unknownListing}

Solvers highlighted with an asterisk are available in the current
installation of WST. The modeling language indicates whether AMPL
\citep{AMPL}, Pyomo \citep{PYOMO} or neither is used to solve sensor
placement optimization problem. Note that GLPK includes a modeling
tool that includes a subset of AMPL. Thus, problems that require
AMPL can be solved if either AMPL or GLPK is installed.

\subsection{Configuration File}

The \code{sp} subcommand generates a template configuration file using the following command line:
\begin{unknownListing}
wst sp --template <configfile>
\end{unknownListing}

The template configuration file for the \code{sp} subcommand is shown in Figure \ref{fig:sp_template}.
Brief descriptions of the options are included in the template after the \# character.

\begin{figure}[h!]
	\unknownInputListing{examples/sp_config.yml}{}{1}{24}
	\caption[The \code{sp} configuration template file]{The \code{sp} configuration template file (\textit{continued in Figure \ref{fig:sp_template_ctd}}).}
	\label{fig:sp_template}
\end{figure}

\begin{figure}[h!]
	\unknownInputListing{examples/sp_config.yml}{}{25}{67}
	\caption[The \code{sp} configuration template file (ctd.)]{The \code{sp} configuration template file (\textit{continued from Figure \ref{fig:sp_template}}).}
	\label{fig:sp_template_ctd}
\end{figure}

\clearpage
\subsection{Configuration Options}

Full descriptions of the WST configuration options used by the \code{sp} subcommand are listed below.
\input{examples/sp_config.tex}

\subsection{Subcommand Output}

The \code{sp} subcommand creates several output files.
The YAML file called <output prefix>sp\_output.yml contains the sensor locations,
final impact metric, the run date and CPU time. For some solvers, the
lower and upper bound on the objective is reported.
The log file called <output prefix>sp\_output.log contains basic debugging information.
A visualization YAML configuration file called
<output prefix>sp\_output\_vis.yml is also created and can be used to generate
network graphics of the sensor placement solution using the \code{visualization}
subcommand. The correct EPANET INP file must be included in the visualization YAML configuration
file for the graphic to display properly.

The \code{sp} subcommand also outputs information a file named <output prefix>\_evalsensor.out
That file includes the following data:
\begin{itemize}
\item {\bfseries Objective:} The impact metric value achieved with the sensor network design.
\item {\bfseries Lower bound:} The lower bound on the impact metric with the sensor network design.
\item {\bfseries Upper bound:} The upper bound on the impact metric with the sensor network design.
\item {\bfseries Solutions:} The internal node indices used by \code{sp} for the sensor network design.
\item {\bfseries Locations:} The EPANET junction labels for the sensor placement locations.
\item {\bfseries Sensor placement ID:} An integer ID used to distinguish the sensor network design.
\item {\bfseries Number of sensors:} The number of sensors in the sensor network design.
\item {\bfseries Total cost:} The cost of the sensor network design, which could be non-zero if cost data is provided.
\item {\bfseries Sensor node IDs:} The internal node indices used by \code{sp} for the sensor network design. The same as Solutions.
\item {\bfseries Sensor junctions:} The EPANET junction labels for the sensor placement locations. The same as Locations.
\item {\bfseries Impact file:} The name of the impact file used in the sensor network design.
\item {\bfseries Number of events:} The number of contamination scenarios that were simulated.
\end{itemize}

The performance of the sensor network design is summarized for each impact data file
specified in the configuration file. The impact statistics included are:
\begin{itemize}
\item {\bfseries Min impact:} The minimum impact over all contamination
incidents simulated. Assuming that a sensor protects the node
at which it is placed, this statistic will typically be zero.
\item {\bfseries Mean impact:} The mean (or average) impact over all
contamination incidents simulated.
\item {\bfseries Lower quartile impact:} 25\% of the contamination incidents,
weighted by their likelihood, have an impact value less than this quartile.
\item {\bfseries Median impact:} 50\% of the contamination incidents simulated, weighted
by their likelihood, have an impact value less than this quartile.
\item {\bfseries Upper quartile impact:} 75\% of the contamination incidents simulated,
weighted by their likelihood, have an impact value less than this quartile.
\item {\bfseries Value at Risk (VaR):} VaR is the minimum value
for which $100\times(1-\beta)$\% of the contamination incidents simulated have a smaller
impact, in which $\beta$ is a user-defined percentage between $ 0.0 < \beta < 1.0$.
\item {\bfseries TCE:} The tailed-\/conditioned expectation (TCE)
is the mean value of the impacts that are greater than or equal to VaR.
\item {\bfseries Worst impact:} The worst impact over all contamination
incidents simulated.
\end{itemize}

If the \code{[compute greedy ranking]} option is used, a greedy sensor placement is printed
to <output prefix>\_evalsensor.out. The greedy ranking places sensors one-\/at-\/a-\/time at the locations
in the optimal sensor network design by consecutively minimizing the mean impact
of placing each sensor. This analysis gives a sense of the relative priorities for these
sensors. The greedy ranking is listed in terms of the sensor node IDs used by the
\code{sp} subcommand. The corresponding EPANET node ID is listed at the top of the file.

\section{Sensor Placement Examples}\label{sp_example}

The following examples illustrate common ways that the \code{sp} subcommand can be used.
Additional examples using the \code{sp} subcommand are provided in Section~\ref{chap:pmedian}.

\subsection{Example 1: Solving eSP with a MIP Solver}
\label{sp_example1}

The first example uses the configuration file, sp\_ex1.yml, shown in Figure \ref{fig:sp_ex1}.
It specifies the impact file as {\outputprefix}\_ec.impact created by the \code{sim2Impact} subcommand
using the extent of contamination (EC) impact metric and EPANET Example Network 3.
The objective is to minimize the mean EC over all contamination incidents
simulated while limiting the number of sensors (NS) to no more than five.
The solver is the GLPK mixed-integer programming (MIP) solver,
which finds globally optimal sensor placements. In addition, the greedy ranking
option is used.

\begin{figure}[h]
  \unknownInputListing{../../examples/sp_ex1.yml}{}{1}{27}
  \caption{The \code{sp} configuration file for example 1.}
  \label{fig:sp_ex1}
\end{figure}

The \code{sp} subcommand is executed using the following command line:

\begin{unknownListing}
wst sp sp_ex1.yml
\end{unknownListing}

The \code{sp} subcommand for example 1 generates the YAML output file, {\outputprefix}sp\_output.yml,
which summarizes the sensor placement results (see Figure \ref{fig:sp_ex1_yml}).
The sensor network design places sensors at nodes 113, 121, 141, 163 and 209 to achieve
an objective value of approximately 8655 of pipe feet contaminated.

\begin{figure}[h]
  \unknownInputListing{examples/sp/sp_ex1_output.yml}{}{1}{14}
  \caption{The \code{sp} YAML output file for example 1.}
  \label{fig:sp_ex1_yml}
\end{figure}

The {\outputprefix}\_evalsensor.out file for the first \code{sp} subcommand example is
shown in Figure \ref{fig:sp_ex1_evalsensor}. It displays the
same sensor network design and impact value as in the YAML output file.
It also includes the greedy ranking of the sensor network design, in which
a sensor at node 163 would provide the greatest reduction in the impact
followed by sensors at nodes 209, 113, 141 and 121. The first value in the
greedy ranking is -1 (or the dummy location value), which gives the impact if no
sensors are placed in the network.

\begin{figure}[h]
  \unknownInputListing{examples/sp/sp_ex1_evalsensor.out}{}{1}{27}
  \caption{The evalsensor output for \code{sp} example 1.}
  \label{fig:sp_ex1_evalsensor}
\end{figure}

\FloatBarrier
\subsection{Example 2: Evaluating Solutions to eSP with Multiple Impact Files}
\label{sp_example2}

The \code{sp} subcommand can also be configured to evaluate a sensor network design using
impact data not used for optimization. The second example uses the configuration
file, sp\_ex2.yml, shown in Figure \ref{fig:sp_ex2}. Several impact files,
{\outputprefix}\_ec.impact and {\outputprefix}\_mc.impact, are defined in the
impact block of the WST configuration file. The objective of the sensor placement
optimization is to minimize the mean EC impact metric. The optimal sensor
network design is then evaluated against the mass consumed (MC) impact metric.

\begin{figure}[h]
  \unknownInputListing{../../examples/sp_ex2.yml}{}{1}{30}
  \caption{The \code{sp} configuration file for example 2.}
  \label{fig:sp_ex2}
\end{figure}

The \code{sp} subcommand is executed using the following command line:

\begin{unknownListing}
wst sp sp_ex2.yml
\end{unknownListing}

All of the impact files specified in the configuration file are
used when evaluating the sensor placement, and a greedy sensor
placement is generated for each (see Figure \ref{fig:sp_ex2_evalsensor}).
The sensor network design optimized for EC has a mean MC
impact of 56320 mg, while the EC impact is 8655 feet.
The greedy ranking of the sensors is different for
the two different impact metrics. A sensor at node 209 would be the
first sensor placed for the MC impact metric compared to a sensor
at node 163 for the EC impact metric. The greedy ranking is a cumulative
effect of adding more sensors, so the -1 value is the impact of having no
sensors in the network and the next sensor location is the impact of adding
one sensor and so forth down the list.

\begin{figure}[h]
  \unknownInputListing{examples/sp/sp_ex2_evalsensor.out}{}{1}{47}
  \caption{The evalsensor output for \code{sp} example 2.}
  \label{fig:sp_ex2_evalsensor}
\end{figure}

\FloatBarrier
\subsection{Example 3: Solving eSP with a GRASP Solver}
\label{sp_example3}

The third example illustrates the use of a heuristic solver for sensor placement.
A GRASP heuristic iteratively applies local search to adaptively sample locations.
Two GRASP heuristics are provided in WST. The AT\&T GRASP solver is based on the
AT\&T \code{popstar} software, and it can be used for research
purposes. The SNL GRASP solver is a new implementation of
the GRASP algorithm in popstar. Example 3 uses the configuration file, sp\_ex3.yml, which is
shown in Figure \ref{fig:sp_ex3}. The sensor placement parameters are the same as
in Example~\ref{sp_example1}; the only difference is that the SNL GRASP solver is used instead of GLPK.

\begin{figure}[h]
  \unknownInputListing{../../examples/sp_ex3.yml}{}{1}{27}
  \caption{The \code{sp} configuration file for example 3.}
  \label{fig:sp_ex3}
\end{figure}

The \code{sp} subcommand is executed using the following command line:

\begin{unknownListing}
wst sp sp_ex3.yml
\end{unknownListing}

The {\outputprefix}\_evalsensor.out file for example 3 is shown in Figure~\ref{fig:sp_ex3_evalsensor}.
The SNL GRASP heuristic solver finds the same solution found by
the GLPK solver in example 1. In addition, the solver found
another solution during the search with the same performance. Since GRASP is a heuristic solver,
it is not guaranteed to find sensor placements with the globally optimal value.
However, GRASP has proven capable of finding optimal or near-optimal solutions
even for large sensor placement problems. While the MIP solver in example 1
provided an upper and lower bound on the value of the solution, the GRASP solver
does not generate these bounds since it is a heuristic.

GRASP is a heuristic search that is partly dependent on a random
number generator. Consequently, users should not expect the solver
to give identical results when run on different machines or operating
systems. Thus, the solution shown in Figure \ref{fig:sp_ex3_evalsensor}
might not be the solution on everyone's machine.

\begin{figure}[h]
  \unknownInputListing{examples/sp/sp_ex3_evalsensor.out}{}{1}{27}
  \caption{The evalsensor output for \code{sp} example 3.}
  \label{fig:sp_ex3_evalsensor}
\end{figure}

\FloatBarrier
\subsection{Example 4: Solving wSP with a MIP Solver}
\label{sp_example4}
The subsequent examples illustrate the use of WST to solve more
complex sensor placement problems. In most cases, these problems are
significantly harder to solve than eSP. MIP solvers are used in the following
examples to ensure consistency in the solution, but the time required to solve
these problems is non-trivial.

The fourth example uses the configuration file, sp\_ex4.yml, shown
in Figure \ref{fig:sp_ex4}. The objective is to minimize the
worst-case expected contamination over all contamination incidents
simulated while limiting the number of sensors to no more than
five. The solver is the CBC solver, which finds globally optimal
sensor placements. In addition, the greedy ranking option is used.

\begin{figure}[h]
  \unknownInputListing{../../examples/sp_ex4.yml}{}{1}{27}
  \caption{The \code{sp} configuration file for example 4.}
  \label{fig:sp_ex4}
\end{figure}

The \code{sp} subcommand is executed using the following command line:

\begin{unknownListing}
wst sp sp_ex4.yml
\end{unknownListing}

The {\outputprefix}\_evalsensor.out file for this example is shown in Figure
\ref{fig:sp_ex4_evalsensor}. Compared with the results from example 1, this
solution has a lower maximum impact for EC and a higher mean impact
for EC. This reflects the difference in the objectives of these two
problems.

\begin{figure}[h]
  \unknownInputListing{examples/sp/sp_ex4_evalsensor.out}{}{1}{27}
  \caption{The evalsensor output for \code{sp} example 4.}
  \label{fig:sp_ex4_evalsensor}
\end{figure}

\FloatBarrier
\subsection{Example 5: Solving cvarSP with a MIP Solver}
\label{sp_example5}

Example 5 uses the configuration file, sp\_ex5.yml, shown in Figure
\ref{fig:sp_ex5}. The objective is to minimize the conditional
value-at-risk (CVaR) over all contamination incidents simulated
while limiting the number of sensors to no more than five. The
parameter $\gamma = 0.05$ specifies the weight of the tail that is
used to measure CVaR (CVaR approximates the mean impact of the
$5\%$ worst scenarios). Hence, minimizing CVaR is similar to
minimizing the worst-case; the difference is that minimizing CVaR
reduces the impact of all of the worst $5\%$ of the scenarios.
The GLPK solver and the greedy ranking option are used.

\begin{figure}[h]
  \unknownInputListing{../../examples/sp_ex5.yml}{}{1}{28}
  \caption{The \code{sp} configuration file for example 5.}
  \label{fig:sp_ex5}
\end{figure}

The \code{sp} subcommand is executed using the following command line:

\begin{unknownListing}
wst sp sp_ex5.yml
\end{unknownListing}

The {\outputprefix}\_evalsensor.out file for this example is shown in Figure
\ref{fig:sp_ex5_evalsensor}. Compared with the results of example 1, this
solution has a lower TCE for EC and a higher mean impact for EC.
Compared with the results of example 4, this solution has the same maximum
impact for EC and a lower TCE impact for EC. Both of these comparisons
reflect the difference in the objectives of these problems.

\begin{figure}[h]
  \unknownInputListing{examples/sp/sp_ex5_evalsensor.out}{}{1}{27}
  \caption{The evalsensor output for \code{sp} example 5.}
  \label{fig:sp_ex5_evalsensor}
\end{figure}

\FloatBarrier
\subsection{Example 6: Solving scSP with a MIP Solver}
\label{sp_example6}

This example considers two sensor placement problems where a side constraint
is used to limit the feasible sensor networks.
Example 6a uses the configuration file, sp\_ex6a.yml, shown in Figure
\ref{fig:sp_ex6a}. The objective is to minimize the mean contamination
impact for EC over all contamination incidents simulated
while limiting (1) the number of sensors to no more than five and (2) the
mean contamination impact for MC to no more than 50000.0.
The GLPK solver and the greedy ranking option are used.

\begin{figure}[h]
  \unknownInputListing{../../examples/sp_ex6a.yml}{}{1}{36}
  \caption{The \code{sp} configuration file for example 6a.}
  \label{fig:sp_ex6a}
\end{figure}

The \code{sp} subcommand is executed using the following command line:

\begin{unknownListing}
wst sp sp_ex6a.yml
\end{unknownListing}

The {\outputprefix}\_evalsensor.out file for this example is shown in Figure
\ref{fig:sp_ex6a_evalsensor}. By comparison with example 1, this
solution has a higher mean impact for EC and a lower mean impact for MC.
This comparison reflects how adding constraints to the formulation in
example 1 leads to a worse solution for the objective while satisfying a
side constraint.

\begin{figure}[h]
  \unknownInputListing{examples/sp/sp_ex6a_evalsensor.out}{}{1}{47}
  \caption{The evalsensor output for \code{sp} example 6a.}
  \label{fig:sp_ex6a_evalsensor}
\end{figure}

Example 6b illustrates that a different impact statistic can be
used in a side-constraint than is used in the objective.  Note that
the solution in Figure~\ref{fig:sp_ex6a_evalsensor} has a worst-case
impact statistic of $143999.0$ The configuration file in
Figure~\ref{fig:sp_ex6a} uses a side-constraint where the MC impacts
are constrained by the worst-case value of $150000.0$.

\begin{figure}[h]
  \unknownInputListing{../../examples/sp_ex6b.yml}{}{1}{36}
  \caption{The \code{sp} configuration file for example 6b.}
  \label{fig:sp_ex6a}
\end{figure}

Figure~\ref{fig:sp_ex6b_evalsensor} shows the solution to Example
6b.  This solution is similar to the solution in Example 6a.  Although
the solution has the same worst-case value for MC impacts, the mean
MC impacts are larger.

\begin{figure}[h]
  \unknownInputListing{examples/sp/sp_ex6b_evalsensor.out}{}{1}{47}
  \caption{The evalsensor output for \code{sp} example 6b.}
  \label{fig:sp_ex6b_evalsensor}
\end{figure}

\FloatBarrier
\subsection{Example 7: Solving mcSP with a MIP Solver}
\label{sp_example7}

Example 7 uses the configuration file, sp\_ex7.yml, shown in Figure
\ref{fig:sp_ex7}. The objective is to minimize the number of sensors
while limiting the mean contamination impact for EC over all
contamination incidents to no more than 5000.0. The
CBC solver and the greedy ranking option are used.

\begin{figure}[h]
  \unknownInputListing{../../examples/sp_ex7.yml}{}{1}{27}
  \caption{The \code{sp} configuration file for example 7.}
  \label{fig:sp_ex7}
\end{figure}

The \code{sp} subcommand is executed using the following command line:

\begin{unknownListing}
wst sp sp_ex7.yml
\end{unknownListing}

The {\outputprefix}\_evalsensor.out file for this example is shown in Figure
\ref{fig:sp_ex7_evalsensor}. By comparison with example 1, this
solution uses 11 sensors to find a lower mean impact for EC.
Both the mcSP and eSP formulations can be used to explore the
trade-off between number of sensors and contamination impact. However,
the eSP formulation is much easier to solve, especially for large-scale
sensor placement problems.

\begin{figure}[h]
  \unknownInputListing{examples/sp/sp_ex7_evalsensor.out}{}{1}{33}
  \caption{The evalsensor output for \code{sp} example 7.}
  \label{fig:sp_ex7_evalsensor}
\end{figure}


\newcommand{\Val}{\fun{Val}}
\newcommand{\POV}[2]{\ensuremath{\mathsf{PresOfVal}(#1, \mathsf{#2})}}

\section{Properties}
\label{sec:properties}

The properties of this section are modifications and additions to the properties of the Shelley ledger. See \cite{shelley_spec} for definitions used here. We need to amend the definition of $\Val$ to the following:

\begin{equation*}
    \Val(x \in \Value) = x
\end{equation*}
\begin{equation*}
    \Val(x \in \Coin) = \fun{inject}~x
\end{equation*}
\begin{equation*}
    \Val((\wcard\mapsto (y \in \Value))^{*}) = \sum y
\end{equation*}

\begin{lemma}
  \label{lemma:utxo-pres-of-value}
  For all environments $e$, transactions $t$, and states $s$, $s'$, if
  \begin{equation*}
    e\vdash s\trans{utxo}{t}s'
  \end{equation*}
  then
  \begin{equation*}
    \Val(s) + wm = \Val(s')
  \end{equation*}
  where $wm = \fun{inject} (\fun{wbalance}~(\fun{txwdrls}~{t})) + \fun{mint}~(\fun{txbody}~{t})$.
\end{lemma}
\begin{proof}
  The proof is identical to the corresponding one in the previous
  specification, except that unfolding $\fun{consumed}$ gives an
  additional $\fun{mint}~{txb}$ term and all $\Coin$ quantities have
  to be converted to $\Value$.
\end{proof}

We also need to track the sum of the $\fun{mint}$ fields of all
transactions in a block, for which we write $\fun{mint}(b)$.

\begin{theorem}[Preservation of Value]
  \label{thm:chain-pres-of-value}
  For all environments $e$, blocks $b$, and states $s$, $s'$, if
  \begin{equation*}
    e\vdash s\trans{chain}{b}s'
  \end{equation*}
  then
  \begin{equation*}
    \Val(s) + \fun{mint}(b) = \Val(s')
  \end{equation*}
\end{theorem}
\begin{proof}
  Similar to the corresponding proof in the Shelley specification,
  for a given $x$ and transition $\mathsf{TR}$, let \POV{x, TR}
  be the statement:

  \begin{tabular}{l}
    for all environments $e$, signals $\sigma$, and states $s$, $s'$,
    $$
    $e\vdash s\trans{tr}{\sigma}s'~\implies~\Val(s) + x = \Val(s')$.
    $$
  \end{tabular}
  \noindent
  Then, \POV{\fun{mint}~{txb}}{LEDGER} follows from
  Lemma~\ref{lemma:utxo-pres-of-value}.  By induction, we have \\
  \POV{\fun{mint}(b)}{LEDGERS}. The rest of the proof works as
  previously, using \POV{0}{TR} or \POV{\fun{mint}(b)}{TR} as needed.
\end{proof}

\begin{theorem}[Preservation of Ada]
  \label{thm:chain-pres-of-ada}
  For all environments $e$, blocks $b$, and states $s$, $s'$, if
  \begin{equation*}
    e\vdash s\trans{chain}{b}s'
  \end{equation*}
  then
  \begin{equation*}
    \fun{coin}(\Val(s)) = \fun{coin}(\Val(s'))
  \end{equation*}
\end{theorem}
\begin{proof}
  The hypothesis implies that for all transactions $tx$ included in
  $b$, there exist some $e_{tx}, s_{tx}$ and $s'_{tx}$ such that
  $e_{tx}\vdash s_{tx}\trans{utxo}{tx}s'_{tx}$, so
  $\fun{coin}(\fun{mint}~tx) = 0$ for all transactions included
  in $b$. This implies $\fun{coin}(\fun{mint}(b)) = 0$, so the
  claim follows by Lemma~\ref{thm:chain-pres-of-value}.
\end{proof}


 %++++++++++++++++++++++++++++++++++++++++
\documentclass[letterpaper,12pt]{article}
\usepackage{tabularx}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage[margin=1in,letterpaper]{geometry}
\usepackage{cite}
\usepackage[final]{hyperref}
\hypersetup{
    colorlinks=true,
    linkcolor=blue,
    citecolor=blue,
    filecolor=magenta,
    urlcolor=blue
}
%++++++++++++++++++++++++++++++++++++++++

\begin{document}

\title{A Brief Exploration of the NBA Draft}
\author{Mike "dirty-mike" Neuder}
\date{\today}
\maketitle

\begin{abstract}
This quick analysis of the NBA draft was inspired and written for Phil Derbesy and Michael Chiappini of the exceptional podcast, "The Basketball Guyaries". During Episode 4, they are exploring the complexity of the current NBA draft, and ask for a bit of clarification on the probabilities going on to determine the order of picks. With the 2017 Draft just occurring, it seemed appropriate to take a glance at the process and learn more about how 14 ping-pong balls can change everything.
\end{abstract}

\section{Current Lottery System}
The current weighted lottery system was implemented in 1990 and reformed in 1993. The system was changed due to teams allegedly losing games intentionally during the near the end of the season in order to secure the first pick in the draft, as the team with the worst record of the season got the first pick. In the new weighted lottery system, the teams with the worst records still have the highest chance to pick early in the draft, however there is an element of randomness introduced in order to make it less appealing to be the worst team in the NBA. Currently, with 30 teams in the NBA, 16 will qualify for the playoffs and the remaining 14 are eligible to participate in the lottery. Since 1987, only the first three picks are determined by the lottery, and the remaining picks are done in reverse record order. Under this current system, the worst team of the previous season at least would be picking fourth in the draft, while the second worst team would choose at latest fifth and so forth. Now that the stage is set, lets look at how it works.

\section{A Bit of Math}
The first three picks are determined by a random selection of ping-pong balls numbered one through fourteen.  These balls are put into a lottery machine, and four are selected at random. Because order doesn't matter, (i.e. $5,6,7,8 = 5,7,6,8 = 5,8,7,6 = etc$), the total number of combinations is 1001. We can calculate by recalling our Algebra 2 formula for combinations, where $n$ is the total number of items, and $r$ is how many we want to select while ignoring order.
$$ \binom{n}{r} = \frac{n!}{r!\cdot(n-r)!}$$ \newpage
In this context, we say we have 14 choose 4 combinations of ping-pong balls and by using our handy formula we can conclude that the number of combinations is,
$$\binom{14}{4} = \frac{14!}{4!\cdot(14-4)!} = \frac{87178291200}{24 \cdot 3628800} = 1001.$$
Out of all these combinations, only one isn't assigned to a team ($11,12,13,14$). The remaining 1000 combinations are distributed among the 14 non-playoff teams based on their regular season record according to the table below. Note that the rank column is in reverse order (worst team is ranked 1 and so forth).

\begin{center}
\begin{tabular}{ |c|c|c| }
 \hline
 rank & number of combinations & probability of being drawn \\
 \hline
 1 & 250 & 0.2498 \\
 2 & 199 & 0.1988 \\
 3 & 156 & 0.1558 \\
 4 & 119 & 0.1189 \\
 5 & 88 & 0.0879 \\
 6 & 63 & 0.0629 \\
 7 & 43 & 0.0429 \\
 8 & 28 & 0.0280 \\
 9 & 17 & 0.0170 \\
 10 & 11 & 0.0110 \\
 11 & 8 & 0.0080 \\
 12 & 7 & 0.0070 \\
 13 & 6 & 0.0060 \\
 14 & 5 & 0.0050 \\
 \hline
\end{tabular}
\end{center}

It is clear that as the better teams in the lottery have a pretty slim chance at landing a top pick. Another interesting thing to note is that if the 14th worst team in the league doesn't get one of the first three picks, they are locked into receiving the 14th pick, due to the fact that after the first three picks the selections are given in reverse record order. One last thing to examine is the probabilities of each team earning a specific pick. Note that in the table below, a cell with a '-' implies that this team cannot receive this pick.
\begin{center}
\includegraphics[scale=0.6]{../images/table.png}
\end{center} \newpage

In order to better visualize these probabilities I created a couple plots displayed below. The first plot shows a given teams probability of receiving any of the first fourteen draft slots.
\begin{center}
\includegraphics[scale=0.5]{../images/plot1.png}
\end{center}

I found it interesting that as you move farther into the draft the teams are have more certainty around which pick they will be receiving. The next plot I created is quite similar, but instead of plotting the probability of a team receiving a pick, it displays the probability that a team will have picked by the $n^{th}$ slot of the draft. For example, team 1 will have .250 chance of having picked after round 1, and after round two they will have $.250 + .215 = .465$ chance of having picked.

\begin{center}
\includegraphics[scale=0.5]{../images/plot2.png}
\end{center} \newpage

As expected, one team per round will reach a probability having picked of 1. This is due to the fact that the worst slot the rank 1 team can be assigned is fourth, and the result follows for the remaining teams.

\section{Conclusions}
In a sport dominated by superstars, securing strong draft picks is hugely important for NBA teams. Though the process is complicated and long, it is nice to know that in a world where control is paramount, a little bit of randomness can turn expectation on its head. Thanks for reading.\begin{footnote}{ if you are interested in seeing how i collected and plotted the data or want to play around with any of the files they are being hosted \href{https://github.com/michaelneuder/nba_draft}{here}. if you find any errors or have any questions feel free to get in touch.}\end{footnote} \\ \\
-dirty mike

\newpage
\begin{thebibliography}{99}

\bibitem{melissinos}
\url{https://en.wikipedia.org/wiki/NBA_draft_lottery}

\bibitem{Cyr}
\url{http://www.nba.com/news/draft/nba-draft-lottery-what-will-happen-2016/}

\bibitem{Wiki}
\url{https://www.sbnation.com/nba/2017/4/13/15268144/nba-draft-lottery-odds-2017-lakers-celtics-suns}

\end{thebibliography}

\end{document}


\hypertarget{legacy-code-legacy.ipynb}{%
\section{Legacy code (Legacy.ipynb)}\label{legacy-code-legacy.ipynb}}

\begin{lstlisting}[language=Python]
%%HTML
<style>
.container { width:100% }
</style>
\end{lstlisting}

\begin{lstlisting}[language=Python]
def alphabeta_max(state, alpha, beta, depth):
    global alphabeta_best_move
    if(state.game_over):
        return terminal_utility(state)
    if(depth >= ALPHABETA_DEPTH_LIMIT):
        return heuristic_utility(state)
    max_utility = -math.inf
    for move in state.get_possible_moves():
        tmp_state = copy.deepcopy(state)
        tmp_state.move(move[0], move[1])
        tmp_utility = alphabeta_min(tmp_state, alpha, beta, depth + 1)
        if(tmp_utility > max_utility):
            max_utility = tmp_utility
            if(depth == 0):
                alphabeta_best_move = move
        if(max_utility >= beta):
            return max_utility
        alpha = max(alpha, max_utility)
    return max_utility

def alphabeta_min(state, alpha, beta, depth):
    global alphabeta_best_move
    if(state.game_over):
        return -terminal_utility(state)
    if(depth >= ALPHABETA_DEPTH_LIMIT):
        return -heuristic_utility(state)
    min_utility = math.inf
    for move in state.get_possible_moves():
        tmp_state = copy.deepcopy(state)
        tmp_state.move(move[0], move[1])
        tmp_utility = alphabeta_max(tmp_state, alpha, beta, depth + 1)
        if(tmp_utility < min_utility):
            min_utility = tmp_utility
            if(depth == 0):
                alphabeta_best_move = move
        if(min_utility <= alpha):
            return min_utility
        beta = min(beta, min_utility)
    return min_utility
\end{lstlisting}

\begin{lstlisting}[language=Python]
ALPHABETA_DEPTH_LIMIT = 4

#TODO: What if one player has to play twice in a row?
def alphabeta_negamax(state, depth, alpha, beta):
    global alphabeta_best_move
    if(state.game_over):
        return terminal_utility(state) * state.turn
    if(depth == 0):
        if(state.turn == WHITE):
            return WHITE_PLAYER_HEURISTIC(state)
        else:
            return BLACK_PLAYER_STRAT_HEURISTIC(state) * state.turn
    utility = -math.inf
    for move in get_possible_moves(state, state.turn):
        tmp_state = make_move(state, move[0], move[1])
        tmp_utility = -alphabeta_negamax(tmp_state, depth - 1, -beta, -alpha)
        if(tmp_utility > utility):
            utility = tmp_utility
            if(depth == ALPHABETA_DEPTH_LIMIT):
                alphabeta_best_move = move
        if(utility >= beta):
            return utility
        alpha = max(alpha, utility)
    return utility

def alphabeta_ai_make_move(state):
    alphabeta_negamax(state, ALPHABETA_DEPTH_LIMIT, -math.inf, math.inf)
    state = make_move(state, alphabeta_best_move[0], alphabeta_best_move[1])
\end{lstlisting}


%\section{Physical Human Factors Report: Tristan Griffith}
\rhead{\today}
\begin{center}
{\large  Tristan Griffith}\\
\vspace{2mm}
{\large Dr. James Hubbard Jr.}
\noindent\rule{\textwidth}{2pt}
\end{center}
\setcounter{section}{1}

%\begin{wrapfigure}{r}{0.45\textwidth}
%\centering
%\includegraphics[scale=.3]{../../../figures/demo_map.png}
%\caption{Modal Heatmap for Subject 32}
%\end{wrapfigure}
\subsection{Motivation}
In the past months, we have become very comfortable converting EEG signals into state space models using output only system identification techniques. As it stands, these models are somewhat useful for biometric classification and analysis, but have not been extended to the quantum domain, which is a desired outcome of our research. The principle barrier preventing the application of quantum probability to these models is the restrictive formulation of the quantum Hilbert space itself, which among other things, requires the state matrix to take the form $-iH$, where $H$ is a self-adjoint matrix. The self-adjoint structure is especially important in quantum probability, because it preserves the norm of the initial condition $\big(||x_0||=||x(t)||\big)$ as the system evolves, ensuring the state vector is still a probability distribution over the possible collapse states.

Physical systems of the form $\dot{x}=Ax+Bu $ rarely have this much structure in their state matrices, so modern system identification techniques do not provide means to directly estimate a self-adjoint matrix from a set of black box measurements. An additional technique is required to convert models generated from these system identification techniques to quantum models robustly, with minimal loss of information. We previously found that generating the self-adjoint matrix as $A_h=\frac{1}{2}(A \pm A^T)$  was removing as much as half the information in the original matrix $A$ as measured by the induced metric.
\subsection{Problem Statement}
Modern system identification techniques do not allow for the discovery of quantum-like models as the eigenvalues identified from the black box system are not restricted to quantum eigenvalues. Therefore, additional methods are needed to convert these existing system identification state space models into quantum state space models with the accompanying mathematical structure.
\subsection{The Hadamard Transform Applied to EEG Modes}
\subsubsection{Transform Basics}
\label{sec:basics}
The Hadamard transform is the most common method for decomposing signals into a set of orthogonal functions that are not trigonometric which are a specific form of Walsh function. The Hadamard transform is analogous to the Fourier transform, but uses orthogonal square waves instead of sines and cosines. A comparison of the first eight Fourier functions and first eight Hadamard functions is in Figure \ref{fig:hf_comp}.
\begin{figure}[]
    \centering
    \includegraphics[scale=0.5]{../../../figures/func_comp.png}
    \caption{Fourier vs. Hadamard Base Functions}
    \label{fig:hf_comp}
\end{figure}
The Hadmard transform has been popular in algorithmic applications, because the nature of square waveforms can be described by a recursive relationship that is easily carried out by computers. The Hadamard transform requires less computational complexity to decompose a given signal into its Walsh functions. The Hadamard transform may be computed with the following recursive function:
\begin{align}
H_1 &\equiv \frac{1}{\sqrt{2}}\begin{bmatrix}
1 & 1\\ 1 & -1
\end{bmatrix} \label{eqn:hada1} \\
H_n=H_1 \otimes H_{n-1}=&\frac{1}{\sqrt{2}}\begin{bmatrix}
H_{n-1} & H_{n-1}\\ H_{n-1} & -H_{n-1}
\end{bmatrix}
\end{align}
Intuitively, equation \ref{eqn:hada1} represents the first two functions in the Walsh function set for the Hadamard transform. The first is a straight line with value one. It has no transition across the origin from 1 to -1, so it does not have any sign changes in its column. The second column corresponds to a function with a single transition from 1 to -1, represented by a single sign change in the column. Higher and higher order Hadamard transforms can be easily determined. The Hadamard matrix corresponding to Figure \ref{fig:hf_comp} is:
\begin{align}
H_8=\left(\begin{array}{cccccccc} 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1\\ 1 & -1 & 1 & -1 & 1 & -1 & 1 & -1\\ 1 & 1 & -1 & -1 & 1 & 1 & -1 & -1\\ 1 & -1 & -1 & 1 & 1 & -1 & -1 & 1\\ 1 & 1 & 1 & 1 & -1 & -1 & -1 & -1\\ 1 & -1 & 1 & -1 & -1 & 1 & -1 & 1\\ 1 & 1 & -1 & -1 & -1 & -1 & 1 & 1\\ 1 & -1 & -1 & 1 & -1 & 1 & 1 & -1 \end{array}\right) \label{eqn:h8}
\end{align}
Notice that the ordering of the columns in equation \ref{eqn:h8} is not ordered from least to most transitions, but that the constant 1 function is immediately followed by the square wave with the maximum number of transitions. These columns may be reordered for convenience or interpretation. Ordering in terms of number of transitions is known as sequency ordering, as in Figure \ref{fig:hf_comp}, and is most commonly found in signal processing applications.  Hadamard ordering, which is used in controls applications, arranges them as 0, 4, 6, 2, 3, 7, 5, 1. Dyadic or gray code ordering, which is used in mathematics, arranges them as 0, 1, 3, 2, 6, 7, 5, 4.
\begin{figure}[]
    \centering
    \includegraphics[scale=0.5]{../../../figures/wht_waveform_1.png}
    \caption{Hadamard vs. Sequency Ordering}
    \label{fig:ordering}
\end{figure}
Keeping track of which ordering method an algorithm uses is critical. The Discrete Walsh Hadamard Transform (DWHT) had been widely used for biometric identification, data compression, and image segmentation. Figure \ref{fig:comp} demonstrates how the DWHT, along with the Discrete Cosine Transform (DCT), are better suited to image compression that traditional methods centered on the Discrete Fourier Transform (DFT) \cite{bull2014communicating}.
\begin{figure}[]
    \centering
    \includegraphics[scale=0.7]{../../../figures/hada_example.jpg}
    \caption{Comparison of Decomposition Methods}
    \label{fig:comp}
\end{figure}
Notice that the DWHT and DCT concentrate their high energy areas in a single corner, unlike the DFT. This makes it much easier to set a cutoff point for image compression.
\subsubsection{Hadamard Logic Gates}
Nothing in the previous section indicates that the Hadamard transform is better suited to application on quantum cognitive models than any other decomposition method. The Hadamard transform piqued our interest due to its relevance in both signal processing and quantum logic gate operations. Hadamard Gates are often used as the first step in a proposed quantum algorithm. They have the neat property of smearing the initial condition into a superposition of all possible states evenly in a given quantum probability space.

For example, if we are working in a two dimensional space, such as that of qubits, we may have an initial state:
\begin{align}
x_0=\begin{bmatrix}
1 \\ 0
\end{bmatrix}
\end{align}
If we pass this initial state through a Hadamard Gate, it becomes:
\begin{align}
H x_0=x_0'\\
\frac{1}{\sqrt{2}}\begin{bmatrix}
1 & 1\\1 & -1
\end{bmatrix}\begin{bmatrix}
1 \\ 0
\end{bmatrix}=\frac{1}{\sqrt{2}}\begin{bmatrix}
1 \\ 1
\end{bmatrix}
\end{align}
Before passing the initial condition through the Hadamard gate, the system was in a prepared state. After passing through the gate, the probability distribution is evenly distributed over both possible measurement outcomes. Notice that it is very important to carry the normalizing term along with the Hadamard matrix so that it is a unitary transform. This is true for systems of higher order, of course:
\begin{align}
x_0=\begin{bmatrix}
0 \\ 1 \\ 0 \\0
\end{bmatrix}\\
x_0'=\frac{1}{2}\left(\begin{array}{cccc} 1 & 1 & 1 & 1\\ 1 & -1 & 1 & -1\\ 1 & 1 & -1 & -1\\ 1 & -1 & -1 & 1 \end{array}\right)\begin{bmatrix}
0 \\ 1 \\ 0 \\0
\end{bmatrix}=\frac{1}{2}\begin{bmatrix}
1 \\ -1 \\1\\-1
\end{bmatrix}
\end{align}
And so on and so on. For algorithms where we expect a certain outcome regardless of initial conditions, this is a quick method to remove potential bias in the preparation of the quantum state. Notice further that this process is reversible, and we can recover $x_0$ from $x_0'$.
\subsubsection{``Quantumness'' of OMA and Hadamard Modes}
So we've established an intuitive connection to the Hadamard transform. It's useful in signal processing and in quantum probability, so it may get us closer to a probabilistic model of cognition. We may look at the resultant state space model from our operational modal analysis on EEG data $A_{OMA}$. $A_{OMA}$ has complex eigevalues and eigenvectors. Further, the eigenvectors are not orthogonal. Quantum state Hamiltonian matricies are known to have only real valued eigenvalues with corresponding orthogonal eigenvectors which represent the basic observables in the quantum model. We may Hadamard transform $A_{OMA}$:
\begin{align}
A_{h}=HA_{OMA}
\end{align}
The resultant matrix has real valued eigenvalues, which gets us a little closer to a quantum Hamiltonian. Regardless of the ordering, however, the eigenvectors of $A_h$ are not orthogonal unless the original eigenvalues of $A_{OMA}$ were orthogonal to begin with.

A computational example illustrates this point. The identify matrix has orthogonal eigenvectors beforehand:
\begin{align}
\phi_0(I)=\begin{bmatrix}
1 \\0
\end{bmatrix},\begin{bmatrix}
0 \\ 1
\end{bmatrix}\\
I_{h}=HI\\
I_{h}=\frac{1}{\sqrt{2}}\begin{bmatrix}
1 & 1\\1 & -1
\end{bmatrix}\begin{bmatrix}
1 & 0\\0 & 1
\end{bmatrix}\\
\lambda_{I_h}=\pm 1\\
\phi=\begin{bmatrix}
1 \pm \sqrt{2} \\ 1
\end{bmatrix}
\end{align}
While $I_h$ has different eigenvectors than $I$, they remain orthogonal. Conversely, if we consider an example 2x2 matrix from our OMA analysis:
\begin{align}
A_{OMA}=\left(\begin{array}{cc} 0.728 & 0.345\\ -0.378 & 0.923 \end{array}\right)\\
\phi_0(A_{OMA})=\left(\begin{array}{cc} 0.186-0.665{}\mathrm{i} & 0.186+0.665{}\mathrm{i}\\ 0.723 & 0.723 \end{array}\right) \label{eqn:phiA}
\end{align}
From equation \ref{eqn:phiA}, it can be seen that the eigenvectors are not orthogonal.
\begin{align}
A_{h}=HA_{OMA}\\
A_{h}=\frac{1}{\sqrt{2}}\begin{bmatrix}
1 & 1\\1 & -1
\end{bmatrix}A_{OMA}\approx \left(\begin{array}{cc} 0.248 & 0.896\\ 0.782 & -0.409 \end{array}\right)\\
\lambda_{A_h}=0.819,-0.980\\
\phi(A_h)=\left(\begin{array}{cc} 0.843 & -0.59\\ 0.537 & 0.808 \end{array}\right)
\end{align}
A few things to note here:
\begin{itemize}
\item Neither $A_{OMA}$, or $A_h$ have orthogonal eigenvectors
\item $A_h$ has eigenvalues in the right half plane (unstable poles)
\item Unlike $A_{OMA}$, notice that $A_h\approx A_h^*$
\end{itemize}
Therefore, we have to conclude that the Hadamard transform doesn't get us all the way to a quantum Hamiltonian. However, I notice that it provides wholly real eigenvalues, which is a step closer to quantum. Further, the state matrix is almost self-adjoint, and looks to me like some of the noisy state matrices I used to develop the earlier \href{https://drive.google.com/file/d/17kZl22Yy5hwzDv9F1psR9IAmViYwAzik/view?usp=sharing}{least squares SINDy like Hamiltonian estimator}. Perhaps we can go one more step, using the SINDy algorithm to get to a quantum Hamiltonian.
\subsubsection{Least Squares to Nearest Hamiltonian}
So far, we've shown that the Hadamard transform, while getting us closer to a quantum Hamiltonian, does not totally complete the path from EEG signals to quantum probability space.

\textcolor{red}{Ok. So this was a nice idea. I could be further developed. However, as I'm looking at the linear systems side of things, we have issues.} Primarily, with the transform taking the form $A_{h}=HA_{OMA}$, the transform is operating on the state matrix, not on the vector. Normally a transform is defined as:
\begin{align}
\bar{x}=Tx\\
\bar{\dot{x}}=T\dot{x}\\
\dot{x}=Ax\\
T^{-1}\bar{\dot{x}}=AT^{-1}\bar{x}\\
\bar{\dot{x}}=TAT^{-1}\bar{x}\\
\end{align}
However, I did something different:
\begin{align}
\bar{x}=HA\dot{x}
\end{align}
Since the operation was applied on the state matrix and not on the state vector, the thing you are differentiating in $\dot{x}$ and the result of $\bar{x}$ aren't the same. After consulting with Dr. Balas (2020-11-05), I'm not aware of other transforms like that, and I've destroyed the meaning or use of the state transition matrix $\Phi$. Even though $\bar{x}$ is in a quantum space, $\dot{x}$ is still in the measure space. I may abandon this for the time being and take a look at some other methods. Argh so close.
\subsubsection{Reachability?}
I would have shown that the use of the least squares estimator averages the distance between the off diagonal components, and that introduces an interval for reachability.
\subsection{Results}
\subsubsection{A Quantum Hamiltonian}
\subsubsection{Interpretation of Model}

\subsection{Discussion}

%\begin{displayquote}
%The class of monotone DNF expressions is learnable via an algorithm $B$ that uses $L=L(h,d)$ calls of examples and $dt$ calls of oracle, where $d$ is the degree of the DNF expression $f$ to be learnt and $t$ the number of variables.
%\end{displayquote}

%\begin{wrapfigure}{r}{0.55\textwidth}
%\centering
%\includegraphics[scale=.5]{../figures/complexity.png}
%\caption{The Error-Complexity Trade Off}
%\end{wrapfigure}


\hypertarget{control-flow}{%
\section{Control Flow}\label{control-flow}}

Control flow statements are used to determine whether and how many times
a selected piece of code is executed. These include:

\begin{itemize}

\item
  \texttt{if} - Selectively execute a piece of code if a condition is
  met.
\item
  \texttt{else} - Execute a different block of code if the test in an
  \texttt{if} statement fails.
\item
  \texttt{for} - Repeatedly execute a section of code with a counter
\item
  \texttt{while} - Repeatedly execute a section of code while a
  condition is true.
\end{itemize}

\hypertarget{if}{%
\section{If}\label{if}}

If allows you to selectively execute a section of code depending on
whether a condition is met. The simplest version looks like this:

\begin{lstlisting}
if (x<1) print x
\end{lstlisting}

where the body of the loop, \texttt{print\ x}, is only executed if x is
less than 1. The body can be a code block to accommodate longer sections
of code:

\begin{lstlisting}
if (x<1) {
    ... // do something
}
\end{lstlisting}

If you want to choose between two alternatives, use \texttt{else}:

\begin{lstlisting}
if (a==b) {
    // do something
} else {
    // this code is executed only if the condition is false
}
\end{lstlisting}

You can even chain multiple tests together like this:

\begin{lstlisting}
if (a==b) {
    // option 1
} else if (a==c) {
    // option 2
} else {
    // something else
}
\end{lstlisting}

\hypertarget{while}{%
\section{While}\label{while}}

While loops repeat a section of code while a condition is true. For
example,

\begin{lstlisting}
var k=1
while (k <= 4) { print k; k+=1 }
       ^cond   ^body
\end{lstlisting}

prints the numbers 1 to 4. The loop has two sections: \texttt{cond} is
the condition to be executed and \texttt{body} is the section of code to
be repeated.

Simple loops like the above example, especially those that involve
counting out a sequence of numbers, are more conveniently written using
a \texttt{for} loop,

\begin{lstlisting}
for (k in 1..4) print k
\end{lstlisting}

Where \texttt{while} loops can be very useful is where the state of an
object is being changed in the loop, e.g.

\begin{lstlisting}
var a = List(1,2,3,4)
while (a.count()>0) print a.pop()
\end{lstlisting}

which prints 4,3,2,1.

\hypertarget{do}{%
\section{Do}\label{do}}

A \texttt{do}\ldots{}\texttt{while} loop repeats code while a condition
is true---similar to a \texttt{while} loop---but the test happens at the
end:

\begin{lstlisting}
var k=1
do {
  print k;
  k+=1
} while (k<5)
\end{lstlisting}

which prints 1,2,3,4

Hence this type of loop executes at least one interation

\hypertarget{for}{%
\section{For}\label{for}}

For loops allow you to repeatedly execute a section of code. They come
in two versions: the simpler version looks like this,

\begin{lstlisting}
for (var i in 1..5) print i
\end{lstlisting}

which prints the numbers 1 to 5 in turn. The variable \texttt{i} is the
\emph{loop variable}, which takes on a different value each iteration.
\texttt{1..5} is a range, which denotes a sequence of numbers. The
\emph{body} of the loop, \texttt{print\ i}, is the code to be repeatedly
executed.

Morpho will implicitly insert a \texttt{var} before the loop variable if
it's missing, so this works too:

\begin{lstlisting}
for (i in 1..5) print i
\end{lstlisting}

If you want your loop variable to count in increments other than 1, you
can specify a stepsize in the range:

\begin{lstlisting}
for (i in 1..5:2) print i
               ^step
\end{lstlisting}

Ranges need not be integer:

\begin{lstlisting}
for (i in 0.1..0.5:0.1) print i
\end{lstlisting}

You can also replace the range with other kinds of collection object to
loop over their contents:

\begin{lstlisting}
var a = Matrix([1,2,3,4])
for (x in a) print x
\end{lstlisting}

Morpho iterates over the collection object using an integer
\emph{counter variable} that's normally hidden. If you want to know the
current value of the counter (e.g.~to get the index of an element as
well as its value), you can use the following:

\begin{lstlisting}
var a = [1, 2, 3]
for (x, i in a) print "${i}: ${x}"
\end{lstlisting}

Morpho also provides a second form of \texttt{for} loop similar to that
in C:

\begin{lstlisting}
for (var i=0; i<5; i+=1) { print i }
     ^start   ^test ^inc.  ^body
\end{lstlisting}

which is executed as follows: start: the variable \texttt{i} is declared
and initially set to zero. test: before each iteration, the test is
evaluated. If the test is \texttt{false}, the loop terminates. body: the
body of the loop is executed. inc: the variable \texttt{i} is increased
by 1.

You can include any code that you like in each of the sections.

\hypertarget{break}{%
\section{Break}\label{break}}

Break is used inside loops to finish the loop early. For example

\begin{lstlisting}
for (i in 1..5) {
    if (i>3) break // --.
    print i        //   | (Once i>3)
}                  //   |
...                // <-'
\end{lstlisting}

would only print 1,2 and 3. Once the condition \texttt{i\textgreater{}3}
is true, the \texttt{break} statement causes execution to continue after
the loop body.

Both \texttt{for} and \texttt{while} loops support break.

\hypertarget{continue}{%
\section{Continue}\label{continue}}

Continue is used inside loops to skip over the rest of an iteration. For
example

\begin{lstlisting}
for (i in 1..5) {     // <-.
    print "Hello"          |
    if (i>3) continue // --'
    print i
}
\end{lstlisting}

prints ``Hello'' five times but only prints 1,2 and 3. Once the
condition \texttt{i\textgreater{}3} is true, the \texttt{continue}
statement causes execution to transfer to the start of the loop body.

Traditional \texttt{for} loops also support \texttt{continue}:

\begin{lstlisting}
                // v increment
for (var i=0; i<5; i+=1) {
    if (i==2) continue
    print i
}
\end{lstlisting}

Since \texttt{continue} causes control to be transferred \emph{to the
increment section} in this kind of loop, here the program prints 0..4
but the number 2 is skipped.

Use of \texttt{continue} with \texttt{while} loops is possible but isn't
recommended as it can easily produce an infinite loop!

\begin{lstlisting}
var i=0
while (i<5) {
    if (i==2) continue
    print i
    i+=1
}
\end{lstlisting}

In this example, when the condition \texttt{i==2} is \texttt{true},
execution skips back to the start, but \texttt{i} \emph{isn't}
incremented. The loop gets stuck in the iteration \texttt{i==2}.


\documentclass[11pt]{article}
\usepackage{setspace}
\usepackage{pxfonts}
\usepackage{graphicx}
\usepackage{geometry}

\geometry{letterpaper,left=.5in,right=.5in,top=0in,bottom=.75in,headsep=5pt,footskip=20pt}

\title{Problem Set 3 -- Integrate-and-fire neuron model}
\author{Computational Neuroscience Summer Program}
\date{June, 2011}

\begin{document}
\maketitle

In this problem set you will be building a simple integrate-and-fire
neuron.  You should assume a specific membrane capacitance of
$c_m = 10$ nF/mm\textsuperscript{2}, a specific membrane resistance of
$r_m = 1$ M$\Omega\cdot$mm\textsuperscript{2}, a resting membrane
potential of $E = -70$ mV, a reset potential of $V_{reset} = -80$ mV,
an action potential threshold of $V_{threshold} = -55$ mV, and a cell
surface area of $A = 0.025$ mm\textsuperscript{2}.  Write up your results in
a text editor of your choosing.  Include any relevant figures, your
Matlab code, and any other calculations related to the problem set.  You may work individually or in groups, but each student should hand in their own report.

\subsection*{Equations}

\begin{center}
\begin{tabular}{ll}
$C_m = A \cdot c_m$ & $R_m = \frac{r_m}{A}$\tabularnewline
$\tau_m = c_m \cdot r_m$ & $V_\infty = E + R_mI_{ext}$\tabularnewline
$V(t) = V_\infty + (V(0) - V_\infty)e^\frac{-t}{\tau_m}$ & $r_{isi} =
(\tau_m \mathrm{ln}(\frac{R_mI_{ext} + E - V_{reset}}{R_mI_{ext} + E -
  V_{threshold}}))^{-1}$\tabularnewline
$\tau_m\frac{dV}{dt} = E - V(t-1) + R_mI_{ext}$ & \tabularnewline
\end{tabular}
\end{center}

\subsection*{Problems}

\paragraph{1.} Model an integrate-and-fire neuron using the equations
above and the following rule: when the neuron's membrane voltage exceeds
$V_{threshold}$, set the voltage in that timestep to $V_{peak} = 40$ mV, and in the
next timestep set the voltage to $V_{reset}$.  Set $dt = 0.1$ ms.  Apply a square pulse of
0.5 nA from $t = 250$ ms until $t = 750$ ms in your simulation.  Use
Matlab's subplot command to plot the membrane voltage over time in the
top panel and $I_{ext}$ in the bottom panel (use the same time scale
for the horizontal axis of both plots).  No text is required for this
question; just include a plot.

\paragraph{2.} Compute the average firing rate (spikes per second) of the integrate-and-fire neuron
for the pulse interval you used in quesion 1 (500 ms).  Now plot simulated firing rate vs $r_{isi}$ for several values of $I_{ext}$ (use $I_{ext}$ between 0 and 1 nA).  How does the firing rate of the modeled neuron compare to the estimated firing
rate given by $r_{isi}$?  Note: the equation for $r_{isi}$ only holds if $V_\infty > V_{thresh}$; otherwise, $r_{isi} = 0$.

\paragraph{3.}  Starting from 0 nA, gradually increase the amount of external current
injected into the integrate-and-fire neuron in steps of 0.01 nA.  Keep the pulse duration constant at 500 ms.  What is the smallest amount of current you
can inject which will still result in an action potential?  Is there a
maximum firing rate this neuron can achieve?  Why or why not?

\paragraph{4.  Challenge problem.} Compute firing rate as a function of pulse duration, $I_{duration}$, using 20 durations between 10 and 500 ms.  Repeat this for several different values of $I_{ext}$ (try using 10 log-spaced values between 0.1 and 5 nA).  Explain what you see.  In particular, are the firing rate curves smooth or jagged?  Why?

\paragraph{5.} Vary the resting potential, specific
capacitance, specific resistance, and surface area variables.  How do
increases or decreases in these values affect firing rate of the integrate-and-fire neuron?  Explain
(try to stay at or under 1-2 sentences per variable).  Include plots for each of these variables.

\end{document}


\section{Introduction}

\begin{fullwidth}

Pursuing abstractness and generality, a number of books and articles in
econometrics rely heavily on standard algebraic proofs.
However, most of the theorems proved in such a way have a strong geometric appeal.
This paper demonstrates how shorter, less technical and even more beautiful the proofs
can be when they are based on geometric theorems.
We show that this technique can be extended to explain
concepts in probability and statistics.

Although most of the theorems and ideas are not completely new and
there are even a few works on the geometry in econometrics and statistics,
this paper introduces alternative explanations and more general results in some cases.
Further, there are algebraic proofs provided parallel to geometric ones
and illustrations which are our own work.
The illustrations are published at \url{https://github.com/olyagnilova/gauss-markov-pythagoras} and licensed under the
Creative Commons Attribution 4.0. % todo: wiki
They are free to use and have the potential to serve as a pedagogical tool
in explaining material for students.

Other researchers have done similar work, an especially
thought-provoking and motivating paper was
\citeauthor{cobb2011teaching}.
The geometric proof of the Gauss-Markov theorem and the introduction of the Herschel-Maxwell approach are inspired by
\citeauthor{cobb2011teaching}'s work.
\citeauthor{jacobson}'s (\citeyear{jacobson}) thesis \citetitle{jacobson}
is a comprehensive overview of a geometric approach in linear models including
statistical foundations.
\citeauthor{gmt_blue}’s (\citeyear{gmt_blue}) text \citetitle{gmt_blue} and
\citetitle{gmt_american_statistician} by
\citeauthor{gmt_american_statistician} (\citeyear{gmt_american_statistician}) were helpful when thinking over
alternative ways of proving this central theorem of econometrics.
The works that deepened our understanding of instrumental variables included
\citeauthor{Butler2016}'s (\citeyear{Butler2016}) \citetitle{Butler2016}
and \href{http://web.hku.hk/~pingyu/6005/6005.htm}{lecture notes} on Econometric theory 1 course by Ping Yu.
The first work that proves the Frisch-Waugh-Lovell theorem using pure geometry
is \citetitle{fwl} by
\citeauthor{fwl} (\citeyear{fwl}).
Finally, there were several works that apply the geometric approach to
hypothesis testing. They are
\citetitle{Langsrud2004}
by \citeauthor{Langsrud2004} (\citeyear{Langsrud2004}),
\citetitle{Siniksaran2005}
by \citeauthor{Siniksaran2005} (\citeyear{Siniksaran2005}),
and one which uses an unusual approach —
\citetitle{friendly2013}
by \citeauthor{friendly2013} (\citeyear{friendly2013}).

The paper has four parts. In the first the fundamentls of the geometry
of random variables are introduced with examples.
In part two, we develop the ideas related to linear regressions
from the ideas which need almost no assumptions to more sophisticated theorems
and concepts.
The third part is devoted to partial correlations. It contains a proof of
a newly-introduced fact about the partial correlation and the correlation
between the residuals in  alinear regression model.
Finally, probability distributions are introduced from a geometric perspective
and hypothesis tests are illustrated.

\end{fullwidth}


\section{\label{sec:selection}Computing an Execution Schedule}

Predictions of solver performance are useful only if they can be used to
execute more appropriate solvers more often. To describe the algorithm
portfolio situation in decision-theoretic terms, we take our set of past
observations---in this case, the solvers already executed on this problem
instance, and their success or failure---as our \emph{belief state}. The
Bellman equation describes the expected reward of an optimal policy, \[ V^*(s)
= R(s) + \max_a \gamma \sum_{s'}P(s' | s, a) V^*(s'), \] where $s$ is a
particular belief state, $R(s)$ describes the reward associated with a state
(here, let $1$ be the reward of any state in which any solver has been
successful, and $0$ the reward otherwise), $\gamma$ is an arbitrary discount
factor (which can be set lower to prefer quickly-obtained solutions more
strongly), $a$ is an action (the execution of some solver for some amount of
time), and $P(s'|s,a)$ is the probability of arriving in state $s'$ after
taking action $a$ in state $s$.  Since the number of possible belief states
grows quickly as actions are taken, the optimal policy is practical to compute
only if the portfolio is limited to a short action sequence.

Just such an optimal short sequence of actions was computed offline, using a
learned DCM model to define $P$; the {\tt borg-sat-10.06.07} solver then
follows that sequence when solving any new problem instance.


\section{Notation}
\label{sec:notation-shelley}

\begin{description}
  \item[$\mathbb{N}$] The (canonical) symbol for the natural numbers
  \item[$\mathbb{H}$] The type of byte strings
  \item[Aggregated Addition] Given a type
  $\type{FM}~\in~ \powerset(\type{X} \times \type{Y})$,
    where addition is defined on terms of type $\type{Y}$, and a term $\var{fm} \in \type{FM}$,
    we overload the $\sum$ notation as follows:
    \[\sum_{(x, y)\in\var{fm}} (x,y) :=
    \{ x\mapsto \sum_{(x,y)\in\var{fm}} y \} \]

    In the case $\type{Y}~=~\type{A}\mapsto\type{B}$ is itself a finite map,
    and addition is defined on terms of type $\type{B}$,
    we interpret
    \[\sum_{(x, (a\mapsto b))\in\var{fm}} (x,a\mapsto b) :=
    \{ x \mapsto (a\mapsto \sum_{(x,a\mapsto b)\in\var{fm}} b) \} \]

    We define $\sum$ on a set of the form
    \[\type{FM} \subseteq \{ x \mapsto y \vert x \in \mathsf{X}, y \in \mathsf{Y} \} \]

    in a similar way,
    \[\sum_{(x\mapsto y)\in\var{fm}} x \mapsto y :=
    \{ x\mapsto \sum_{x\mapsto y\in\var{fm}} y \} \]

    We use the $+$ to denote this overloaded addition operation also.

  \item[Other Operations] Similar to the definition of aggregated addition,
  other scalar operations are defined on finite maps. This includes multiplication,
  floor, etc.

  \item[$\leq$] This symbol is overloaded to represent the conjunction of the
  pairwise $\leq$ comparison of entries with at same index in a list or a vector.
  Every other comparison symbol is overloaded in a similar way.

\end{description}
\clearpage


%!TEX root = ../PhD_thesis__Lilian_Besson
% ******************************* Thesis Appendix B ****************************
% \chapter{About another publication: ``What Doubling-Trick Can and Can't Do for Multi-Armed Bandits''}
\chapter{About Doubling Tricks for Multi-Armed Bandits}
\label{app:2:DoublingTricks}

This appendix quickly presents the contributions of another publication that was not presented in the main text of this thesis.
We studied doubling tricks and their possible uses for multi-armed bandits, between fall 2017 and spring 2018, and we wrote an article \cite{Besson2018DoublingTricks} which was rejected at the COLT 2018 conference, and unfortunately we lacked the time to improve it and resubmit it to another conference.

% \TODOL{
% I cannot include it raw without more work on the maths, we have to write only the best versions of our results.

% But I am interested in including this article in my thesis, mainly because GLRklUCB in Chapter 6 is not anytime!

% I need to try DoublingTrick + GLRklUCB, at least experimentally!
% For the theoretical analysis, it will probably be a huge failure.
% }

\paragraph{Motivations for anytime algorithms.}
%
As introduced in Chapter~\ref{chapter:2},
an online reinforcement learning algorithm is \emph{anytime} if it does not need to know in advance the horizon $T$ of the experiment.
%
Depending on the context of the practical application of interest, it might be unrealistic to assume to know in advance $T$. We give two examples to illustrate where this prior knowledge can be realistic or not.
On the one hand, consider clinical trials: it is likely that the number of patients is known before starting the trial, and for this model we refer to all the research that studies the setting of fixed-time best-arm identification \cite{audibert2010best,Garivier16BAI}.
On the second hand, consider cognitive radio and decentralized MAB learning implemented on IoT devices (like we present it in Chapter~\ref{chapter:4}): usually a learning step corresponds to an up-link and then down-link message sent and received by the IoT device, and so the time horizon $T$ denotes the total number of such messages. While it can be assumed that the device will run for instance for $10$ years (as it is advertised by some companies \cite{Centenaro16}), many kinds of application such as medical sensors or connected fields cannot predict the number of total communications before setting up the device.

For this later range of applications, it is of highest interest to be able to use low-cost MAB algorithms that do not rely on prior knowledge of the problem for which they will be applied, and especially do not need to know the horizon $T$.
In this thesis,
while we proposed in Chapter~\ref{chapter:5} any-time algorithms solving the presented problem of stationary multi-players bandit, our solution for the problem of non-stationary MAB problem studied in Chapter~\ref{chapter:6} does rely on a prior knowledge of the horizon $T$.
It is an interesting direction of research to know what is the best approach to fix this weakness: work more and design algorithms that are inherently any-time, or use a generic technique to avoid depending on a prior knowledge of $T$, and automatically transform our non-anytime approach to make it anytime.

\paragraph{Solution to ``patch'' a non-anytime algorithm.}
%
A well-known technique to obtain an anytime algorithm from any non-anytime algorithm is the ``Doubling Trick'', as first introduced in \cite{CesaLugosi06} and used for instance in \cite{Auer10,AuerGajaneOrtner18}.
%
In the context of adversarial or stochastic multi-armed bandits,
the performance of an algorithm is measured by its regret,
and we study two families of sequences of growing horizons (geometric and exponential)
to generalize previously known results that certain doubling tricks can be used to conserve certain regret bounds.
In a broad setting, we prove that a geometric doubling trick can be used to conserve (minimax) bounds in $R_T = \cO(\sqrt{T})$ but \emph{cannot} conserve (distribution-dependent) bounds in $R_T = \cO(\log(T))$.
We give insights as to why exponential doubling tricks may be better, as they conserve bounds in $R_T = \cO(\log(T))$, and are close to conserving bounds in $R_T = \cO(\sqrt{T})$.
Interestingly, we prove that they conserve bounds of the mixed form $R_T = \cO(T^{\gamma} (\log(T))^{\delta})$, for $0<\gamma<1$ and $\delta>0$, and so an exponential doubling trick could be used to obtain an anytime version of the algorithm we propose for non-stationary bandits, \GLRklUCB{} in Chapter~\ref{chapter:6}.
However, our study was only focusing on stationary bandit, and it is left as a future work to explore the harder case of piece-wise stationary problems.

In our article \cite{Besson2018DoublingTricks}, we also present numerical experiments in the case of stationary MAB problems, to compare the performance of efficient anytime algorithms, like \klUCB, against efficient non-anytime algorithms, like \KLUCBpp{} from \cite{Menard17}, made anytime with different choices of doubling-trick.
We conclude that for such problems, if $T$ is not known before, it is always more efficient to use policies that were designed to be anytime that to use a doubling trick.
For example, an applicative paper written in 2018, \cite{li2019useDoublingTrick}, only tested the use of an exponential doubling trick combined with \KLUCBpp, but most surely the \klUCB{} algorithm would have given better empirical performance as well as more robust results\ldots

To conclude this work, we would need to complete the study of the doubling tricks, and instead of focusing on two families (geometric and exponential), we need to completely characterize the doubling tricks that allow to preserve certain regret bounds.
Such doubling is given either by the function mapping the current time $t$ to the current estimate of the horizon $T(t)$, or the sequence $(T_i)_{i\in\N^*}$ of successive estimates of the horizons.
We are interested in pursuing this work, in the hope of finding an intermediate sequence, growing faster than a geometric but slower than an exponential doubling sequence, that can preserve both worlds, problem-dependent bounds in $\cO(\log(T))$ and problem-independent bounds in $\cO(\sqrt{T})$.