\label{bdoodle:chapter}

Scheduling an event for a group of invitees is a frustrating task; it tends to be tedious and time consuming.
A typical scheduling process can be described as iterative approval voting.
First, an event organizer selects a candidate set of date/time options, and asks invitees to respond with their availability.
Given the responses, the organizer then chooses an agreeable option and announces it, or she may repeat the process by proposing another set of options if no feasible option is found.
Naturally the organizer and her invitees wish to reach an agreement within a small number of iterations and proposed options -- the more iterations and proposed options there are, the more laborious a scheduling process becomes.

There exist several software tools that are designed to help an event organizer handle a scheduling process more efficiently -- one of the most well-known tools is Doodle~\footnote{http://www.doodle.com}.
In Doodle, an organizer can simply list as many date/time options as she likes,
and each invitee is asked to respond with her availability.
Essentially, invitees participate in approval voting on all options proposed by the organizer.
Hence if too many options are proposed, then invitees are given the burden of answering them all.
This often leads to undesired behaviors of invitees such as herding or procrastination, instead of honest, quick responses~\cite{zou2015strategic}.
On the other hand, if the organizer proposes only few options, there may not exist an agreeable outcome after all, which may result in another iteration of proposals and responses.
In fact, surveys find that the most challenging part of group scheduling is due to ``chasing people who do not answer'' and ``finding a suitable time.''~\footnote{http://en.blog.doodle.com/2012/07/26/new-findings-a-small-number-of-initiators-organize-most-of-the-meetings/}

Doodle has many practical advantages.
One of them is its simplicity -- invitees simply need to approve a subset of options based on their availability.
Another is a short duration of scheduling process -- the duration it takes until
the last invitee responds and the meeting time is settled. Each invitee
needs to respond only once, which limits the degree to which the
process is hijacked by invitees' delayed responses. But Doodle has many
drawbacks, even in this idealized setting. In particular, Doodle
forces the invitees to examine many potential date/time options. This can be
inconvenient not so much due to the effort involved (though that is a
factor), but mostly because invitees need to block their available slots
off until a option is announced -- in a way, their `free time' is being hostaged until the final schedule is set.
To quantify this inconvenience, we use the expected number of time
slots floated by the organizer as a proxy.

In order to avoid incurring much inconvenience,
the organizer can select just a handful number of options,
and poll the invitees about those. If a feasible option is not found
among them, then repeat with another batch of date/time options.
We call this broad class of polling mechanisms \emph{B-Doodle} (for ``Batched Doodle").
Clearly, Doodle is a special case with one batch consisting of all
options. Another extreme case is the OAAT (one-at-a-time)
mechanism, in which the organizer tests a single option at each
iteration. Doodle minimizes the total number of iterations, whereas
OAAT minimizes (expected) inconvenience caused by the scheduling process.
In-between lie many other mechanisms, with different
batching schemes, that trade off time against inconvenience
differently.

Figure~\ref{bdoodle:fig:Pareto_scatter} illustrates this via a simple example.
In this scenario there are six options, four invitees, each of whom
is available at each option independently with probability $.8$.
The figure depicts a scatter plot of all 32 different B-Doodle mechanisms,
including Doodle and OAAT. The vertical axis depicts the expected
number of rounds to determine the option, and the horizontal axis
the expected inconvenience.

\begin{figure}[h!] \small
\centering
\includegraphics[scale=0.48]{plots/pareto_scatter_fin.eps}
\caption{A scatter plot of B-Doodle Mechanisms given four invitees and six options.}
\label{bdoodle:fig:Pareto_scatter}
\end{figure}

We can clearly observe a time-inconvenience Pareto frontier in Figure~\ref{bdoodle:fig:Pareto_scatter}.
In additino, if there is an overall cost function combining time
and inconvenience, one can identify an optimal B-Doodle mechanism
along this frontier as illustrated in
Figure~\ref{bdoodle:fig:Pareto_objective}.
If we assume that the overall cost is defined as 3$\cdot$\Time+\Inconveniences (which is a linear combination of the two), the optimal mechanism happens to be the ``Half-n-half" mechanism; this mechanism
sends out $3$ options in the first batch, and if no feasible time
slot is found then sends out the remaining $3$ options in the next
batch.

\begin{figure}[h!] \small
\centering
\includegraphics[scale=0.48]{plots/pareto_objective_fin.eps}
\caption{Pareto-frontier and objective function.}
\label{bdoodle:fig:Pareto_objective}
\end{figure}

In this chapter we will investigate the difficulty of finding an optimal B-Doodle mechanism, and to what degree it improves on the simple Doodle in realistic scenarios.
Under the assumption that the set of events in which an invitee is available for a particular option is mutually independent,
we provide an efficient recursive algorithm for computing an optimal polling mechanism for a broad class of objective functions.

In addition we also  assume that the event organizer is given probability estimates on availability of agents, but it is not clear how one can obtain such probabilities.
 Probability estimation is an interesting and challenging research question on its own, and we do not attempt to solve the question in this work. However, we provide several plausible methods for estimating the probabilities in the context of group scheduling, which can enable our model and algorithm to be deployed as a real-world application in the future.
 In the psychology literature, Mann et al. found that cultural differences between the Western, individualistic countries (such as the United States) and the Eastern, collectivistic countries (such as China and Japan) lead to different behaviors of respondents when it comes to a group-decision making process~\cite{mann1998cross}.
 More recently, Reinecke et al. analyzed more than 1.5 million Doodle date/time polls from 211 countries, and confirmed similar findings regarding time perception and group's behavior~\cite{reinecke2013doodle}. Among others, they found that ``in comparison to predominantly individualist societies, poll participants from collectivist countries respond earlier, agree to fewer options but find more consensus,'' which agrees with the findings of Mann et al.
 Besides the cultural differences, Doodle's own surveys on event scheduling found that people tend to respond to the scheduling surveys on Mondays, while Monday is the least popular day for having a meeting.~\footnote{http://en.blog.doodle.com/2012/05/23/mondays-for-planning-busy-weekends/}
 We believe that these studies and findings can be used to design a reasonable estimator for availability of agents, by utilizing the features that are known to be crucial -- such as demographics of the group and purpose of the event being scheduled.
 Recent work by Zou et al. analyzed over 340,000 Doodle polls data to study behavioral patterns of the users, and they were able to identify response functions that match the response patterns observed in the real data~\cite{zou2015strategic}. We believe that a similar approach can be taken to tackle the problem of probability estimation in the context of group scheduling.

\paragraph{Related Work.}
Group scheduling is of tremendous practical importance, and much research has been devoted to it. Early work in AI had a heuristic, systems-oriented flavor to it. For example, Jennings et al.~\cite{Jennings95agent} proposed the design of an agent-based meeting scheduling system, in which an autonomous agent negotiates with other agents on behalf of its human user. The emphasis was on the system description, rather than on theoretical or empirical results. Somewhat more formal work was done by Sen and Durfee~\cite{sen1998formal}, which focused on heuristic negotiation-strategies for group scheduling. Ephrati et al.~\cite{ephrati1994non} tackled incentive issues, adopting a game-theoretic approach. It proposed three monetary-based meeting systems, in which invitees bid their preferences using monetary ``points". They extended the Vickrey-Clarke-Groves mechanism (at the time not as widely known in computer science as today), preventing manipulative behavior by the invitees assuming that the host has an access to the calendars of invitees -- the authors called this the ``open calendar" system, but in our work we do not assume that the organizer has an access to the calendars (or availability) of invitees.

Others have focused on applying machine learning to group scheduling. Mitchell et al.~\cite{mitchell1994experience} and Maes~\cite{maes1994agents} respectively focused on learning user preferences and reducing the amount of work that needs to be done by human users during the scheduling process. More recently, Crawford and Veloso~\cite{crawford2005learning} approached the negotiation-based group scheduling problem by training agents to learn about the negotiation strategies of other agents. In addition to these strands of domain-independent group scheduling, there has been work on group scheduling problem in specific domains,  including sales summit scheduling (Cowling et al.~\cite{Cowling}), airport scheduling (Chia et al.~\cite{chia1998coordinating} and Neiman et al.~\cite{neiman1994exploiting}), and medical scheduling (Decker and Li~\cite{decker1998coordinated} and Hannebauer and M{\"u}ller~\cite{hannebauer2001distributed}).

All of the above directions are relevant to the general problem of group scheduling, but not directly to the setting that is being considered in this chapter. The most closely related work of which we are aware is done by Franzin et al.~\cite{franzin04} and Garrido and Sycara~\cite{garrido1996multi} respectively. These papers are superficially different, as their main focus is the impact of privacy considerations (specifically, different levels of calendar sharing among agents) on time and quality, whereas we concentrate on the interaction between time spent and inconvenience caused during the scheduling process. But this is not the main differentiator, since their notion of privacy level is tied to the number of slots examined before a schedule is found, which is close to our inconvenience measure. A bigger difference lies in the mechanisms considered; we explore the space of Single-proposer Mechanisms in which the sole organizer is trying to schedule a meeting, whereas they explore two specific mechanisms that fall outside the scope of SPMs as any agent can propose a schedule to others (this falls into Multi-proposer Mechanisms). Another crucial difference lies in the nature of the results; neither of the two papers proposes optimal solutions, and do not compare their proposed solutions to common web-based solutions such as Doodle (nor could they have, since no web-based solutions existed at the time).

\section{Notation and Definitions}

We mentioned two dimensions of optimality in the scheduling process: \Times and \Inconvenience.
\Times captures the duration of the scheduling process and \Inconveniences measures how much inconvenience is caused for each agent during the process.
In this section we formally define the class of B-Doodle mechanisms, the Batched Doodle Problem (\BDP), and other technical terms and notations.

\subsection{Problem Setting}
Consider $n$ invitees denoted by $\{a_1, \dots, a_n\}$ and $s$ time slots denoted by $\{1, 2, \dots, s\}$.
For each agent $a_i$ and each time slot $t$, there is a known prior probability, $p_{i, t}$, such that the agent is available at time $t$ with probability $p_{i, t}$ and unavailable with probability $(1-p_{i, t})$.
While the organizer knows all such priors (i.e. all $p_{i,t}$'s), she does not know the realization of availability of agents -- it is private information of each agent.
The organizer is trying to find a \emph{feasible} time slot by asking invitees to reveal their availability; we assume that the organizer has a \emph{feasibility threshold}, $f$, such that a feasible time slot requires at least $\lceil f \cdot n \rceil$ agents be available. The organizer iteratively sends out a batch of time slots until a feasible time slot is found. In our setting \Times spent by the scheduling process is measured by the number of iterations and \Inconveniences caused is measured by the number of time slots that have been sent out by the organizer.

This setting is general enough to capture many realistic scenarios of group scheduling. For instance in a corporate setting, a busy manager may have low probability of being available for all time slots while other employees may be less busy and have high probability of being available. Or, if students are trying to decide on the meeting time for a group project, their availability is expected to be equally likely because the schedules of their classes are similar.

Let us define a class of B-Doodle mechanisms that describe how the organizer sends out time slots in each iteration.
\begin{definition}[B-Doodle Mechanism]
Let $S = \{1, 2, \dots, s\}$ be a set of $s$ time slots. We define a B-Doodle mechanism for $S$ as an ordered partition of $S$.
Let $B = \langle S_1, S_2, \dots, S_m \rangle$ be a partition of $S$ into $m$ subsets such that $S_j \neq\emptyset$, $S_j \cap S_k = \emptyset$ for $j \neq k$, and $\cup_{l=1}^{m} S_l = S$ where $1 \leq j,k \leq m$. We define $b_j = |S_j|$ for all $j \leq m$ and call $b_j$ the size of the $j$-th batch.
We write $B_m$ to emphasize that $B$ has $m$ batches.
\end{definition}
We interpret a B-Doodle mechanism $B$ as follows: the organizer sends out time slots in $S_1$ during the first iteration. If a feasible time slot is found, the process ends. Otherwise she sends out the next batch, $S_2$, and so on.
Note that there exist exponentially many B-Doodle mechanisms for any fixed $S$ (exponential in cardinality of $S$).
Doodle is a special case of B-Doodle mechanism and can be described as $\langle S \rangle$.

Our objective is to find a B-Doodle mechanism that minimizes the expected cost of scheduling process, given an objective function of \Times and \Inconvenience.  Earlier we considered a simple cost function that is a linear combination of the two.  While this cost function fits in many realistic situations, we want to explore a larger class of cost functions. We define a class of cost functions that depend on the number of iterations and batch sizes.
\begin{definition}[Cost function] \label{bdoodle:def:cost-function}
A cost function $c$ takes two integers $j$ and $b$ as arguments, and $c(j, b) > 0$ describes the aggregate cost of \Times and \Inconveniences that is incurred by sending out a batch of $b$ time slots during the $j$-th iteration. We assume that cost is additive so that the overall cost of executing the first $j$ iterations of ${B}_m = \langle S_1, S_2, \dots, S_m \rangle$ is simply the sum, $\sum_{k=1}^{j} c(k, b_k)$, which we denote by $C(j, {B}_m)$; recall that $b_k = |S_k|$.
A cost function $c$ is said to be $\theta$-\emph{simple} if there exists some constant $\theta>0$ such that $c(j, b) = \theta \cdot c(j-1, b)$ for all $j > 1$ and for all $b \geq 1$. We often drop $\theta$ and just state that a cost function is \emph{simple}, for brevity.
\end{definition}
It is reasonable to assume that cost is additive (with respect to iterations) because the time spent and inconvenience caused for the agents all add up.
Note that $C(j, B_{m})$ is strictly increasing in $j$ for any fixed $B_{m}$ because $c(j, b) > 0$ for any $j$ and $b$. While the class of simple cost functions may seem too restrictive, we present a few natural choices of cost functions that belong to the class of simple cost functions. In this chapter we assume that the underlying cost function is simple; further we assume that $c(j, b)$ for all $1 \leq j, b \leq s$ can be computed in polynomial time with respect to $s$.

$\theta$-simple cost functions form a broad class of cost functions, and can model many different settings.
For instance, when $\theta = 1$, it means that the cost of sending out a batch of a certain size is the same regardless of during which iteration it is being sent. When $\theta > 1$, it means that a later iteration costs more than an earlier iteration if the size of a batch is the same. Lastly, when $\theta \in (0, 1)$, it is the opposite in that an earlier iteration costs more than a later iteration given the same batch size -- this last case may be rare in real life, but our definition naturally allows for such scenarios.
The following example shows this by various simple cost functions.

\begin{example}[Simple cost functions] \label{bdoodle:eg:costFunctions}
The simplest choice of a cost function is a linear combination of \Times and \Inconvenience, parameterized by some constant $\alpha > 0$; we call this a \emph{linear} cost function. This cost function is $1$-simple. Notice that the first term captures \Times and the second captures \Inconveniences in both expressions.
\begin{equation} \label{bdoodle:eqn:linear_cost_function}
c_{\alpha}(j, b_j) = \alpha + b_j, ~~ C_{\alpha}(j, {B}_m) = \left(\alpha \cdot j\right) + \left(\sum_{k=1}^{j} b_k\right).
\end{equation}
In some cases the organizer may want to penalize the mechanism for executing too many iterations by making later iterations to cost more than earlier iterations. The following describes such cost function, parameterized by some constant $\beta > 1$; we call this a \emph{time-averse} cost function. Notice that the term $\beta^j$ is strictly increasing as $j$ increases. This cost function is $\beta$-simple.
\begin{equation} \label{bdoodle:eqn:time_averse_cost_function}
c_{\beta}(j, b_j) = \beta^j \cdot b_j, ~~ C_{\beta}(j, {B}_m) = \left(\sum_{k=1}^{j} \beta^k \cdot b_k \right).
\end{equation}
In contrast if the organizer is interested in reducing the size of each batch because it may cause too much inconvenience, then the following cost function could be used, which is parameterized by some constant $\gamma > 1$; we call this an \emph{inconvenience-averse} cost function which is $1$-simple.
\begin{equation} \label{bdoodle:eqn:inconvenience_averse_cost_function}
c_{\gamma}(j, b_j) = \gamma^{b_j}, ~~ C_{\gamma}(j, {B}_m) = \left(\sum_{k=1}^{j} \gamma^{b_k} \right).
\end{equation}
\end{example}

We now formally define the Batched Doodle Problem.
\begin{definition}[Batched Doodle Problem] \label{bdoodle:def:bdp}
An instance of the Batched Doodle Problem (\BDP) is a tuple $(N, S, f, c, P)$ where $N = \{a_1, a_2, \dots, a_n\}$ is a set of $n$ agents , $S = \{1, 2, \dots, s\}$ is a set of time $s$ slots, $f$ is the feasibility threshold ($0 \leq f \leq 1$), $c$ is a cost function, and $P$ is a probability distribution matrix of availability ($P = \{p_{i, t}\}$ with $0 \leq p_{i,t}\leq 1$ for all $i, t$).

The objective in \BDPs is to find an optimal partition of $S$, $B^*$, among all B-Doodle mechanisms that partition $S$, such that $B^*$ minimizes the expected cost of the scheduling process (expectation with respect to $P$), given $(N, S, f, c, P)$.
\end{definition}
Since there are exponentially many B-Doodle mechanisms, it is impractical to enumerate each B-Doodle and compute its expected cost in order to find an optimal one. In what follows we make some simplifying assumptions that are assumed throughout this paper, and we present an efficient algorithm to solve \BDPs in next section.

\subsection{Technical Assumptions}
As we briefly mentioned in the intro, we assume that the availability of agents for each time slot is independent of their availability at other time slots, and of the availability of other agents. That is, the set of events in which agents are available for certain time slots is mutually independent.
We discuss the relaxation of this assumption in Section~\ref{bdoodle:sec:Discussion}. The independence assumption allows us to compute the likelihood of each time slot being feasible, as the following lemma states.
\begin{lemma} \label{bdoodle:lemma:q_t_polytime}
Given $(N, S, f, P)$, let $q_t$ for each $t\in S$ be the probability that time slot $t$ is $f$-feasible. Under the independence assumption, $q_t$ can be computed in polynomial time.
\end{lemma}
\begin{proof}
For any fixed $t$, let us define $V_t(i, z)$ to be the probability that among the agents $\{a_1, a_2, \dots, a_i\}$ exactly $z$ agents are available at time slot $t$. $V_t(i, z)$ is well-defined where $1 \leq i \leq n$ and $0 \leq z \leq i$. We further define $V_t(i, z)$ for the following degenerate cases:
\begin{equation*}
V_t(i, z) =
\begin{cases}
	1 & \mbox{if~} i = z = 0 \\
	0 & \mbox{if~} z = -1 \mbox{~or~} (i = 0 \land z > 0)
\end{cases}
\end{equation*}
Then we can compute $V_t(i, z)$ using a simple dynamic programming algorithm according to the following recurrence relation where $1 \leq i \leq n$ and $0 \leq z \leq i$:
\begin{equation} \label{bdoodle:eqn:vt_precompute}
V_t(i, z) = V_t(i-1, z-1) \cdot p_{i,t} + V_t(i-1, z) \cdot (1- p_{i,t})
\end{equation}
It is easy to verify that the recurrence relation is correct. For the degenerate cases (or base cases) when $i = z = 0$ it is always (with probability 1) the case that no agents are available. When $z = -1$ or $(i=0 \land z>0)$, it is never (with probability 0) the case that $z$ agents are available. For the main recurrence relation, agent $a_i$ is either available or unavailable, and we compute $V_t(i, z)$ by considering both cases. For each fixed $t$, there are $O(n^2)$ entries of $V_t$ to be computed, and computing each entry takes $O(1)$; therefore it takes $O(sn^2)$ to compute $V_t(i, z)$ for all $t, i, z$.

Finally we can compute $q_t$ after we compute $V_t(i, z)$ for all $i,z$, as follows:
\begin{equation}
q_t = \sum_{z = \lceil f \cdot n \rceil}^{n} V_t(n, z).
 \end{equation}
\end{proof}

Lemma~\ref{bdoodle:lemma:q_t_polytime} allows us to describe a B-Doodle mechanism in a simpler form, along with the following theorem.
\begin{theorem} \label{bdoodle:thm:swap_argument}
Consider a B-Doodle mechanism $B$ described by $\langle S_1, S_2, \dots, S_m \rangle$. Suppose that there is some time slot $t \in S_l$ and $t'\in S_{l'}$ with $q_t < q_{t'}$ and $l < l'$. Then $B$ is not an optimal in the sense that we can find another mechanism $B^*$ whose expected cost is strictly less than the expected cost of $B$.
\end{theorem}
\begin{proof}
Given $B$, let us construct another B-Doodle mechanism, $B^*$, as follows: $B^* = \langle S^*_1, S^*_2, \dots, S^*_m \rangle$, which is the same as $B$ except that we swap $t$ and $t'$. That is, $S^*_i = S_i$ for all $i \neq l$ and $i\neq l'$, and $S^*_{l} = (S_l \cup t') \setminus t$ and $S^*_{l'} = (S_{l'} \cup t) \setminus t'$. Note that the two mechanisms have the same batch sizes ($b_j = b^*_j$ for all $j$).

Let $w_j$ be the probability that the scheduling process ends after the $j$-th iteration if we use $B$, and $w_j^*$ if we use $B^*$. Then $w_j = w_j^*$ for all $j < l$ because $S_j = S^*_j$ for all $j$ with $1 \leq j < l$. Clearly it holds that $w_l < w_l^*$ because of the swap of $t$ and $t'$. For $j$ with $l < j \leq l'$, it holds that $w_j > w_j^*$; this is not obvious, but the intuition is that if $w_l^*$ increases then the subsequent $w_j^*$'s with $l < j \leq l'$ must decrease as they depend on $1-w_l^*$, until this effect is canceled out in the $l'$-th batch. Since the probability of finding no feasible time slots during $l'$ iterations is the same in both cases (because $\cup_{i=1}^{l'} S_i = \cup_{i=1}^{l'}S^*_i$) it holds that $w_j = w_j^*$ for all $j > l'$.

Let $\Exp_B$ be the expected cost of $B$ and $\Exp_{B^*}$ of $B^*$. Since $b_j = b^*_j$ for all $j$, it holds that $C(j, B) = C(j, B^*)$ for all $j$, and we have:
\begin{eqnarray}
	\Exp_B - \Exp_{B^*}
	&=& (w_l - w_l^*) C(l, B) + \sum_{j=l+1}^{l'} (w_j - w_j^*)C(j, B) \\
	&=& \sum_{j=l+1}^{l'} (w_j - w_j^*)(C(j, B) - C(l, B))
\end{eqnarray}
The first inequality holds by definition of the expected cost and canceling out some terms; the second inequality holds because $(w_l - w_l^*) + \sum_{j=l+1}^{l'} (w_j - w_j^*) = 0$ (as probabilities must add up to one). Since $w_j > w_j^*$ and $C(j, B) > C(l, B)$ for all $j$ (because $C(j, B)$ is an increasing function in $j$ for fixed $B$), we conclude that $\Exp_{B} > \Exp_{B^*}$.
\end{proof}
Theorem~\ref{bdoodle:thm:swap_argument} implies that an optimal B-Doodle that minimizes the expected cost must have the following property: for any two batches $S_j$ and $S_k$ with $j < k$, it must hold that $q_t \geq q_{t'}$ for all $t\in S_j$ and $t' \in S_k$ (otherwise we can apply the theorem and swap the two time slots to obtain a mechanism with smaller expected cost).
Therefore we can limit our attention to the sub-class of B-Doodle mechanisms that only describe batch sizes, but not explicitly which time slots in each batch. Due to Lemma~\ref{bdoodle:lemma:q_t_polytime} we can compute $q_t$ for each $t$ in polynomial time, and thus we can simply sort time slots by $q_t$ in non-increasing order as a pre-processing step.

Therefore we can focus on the following sub-class of \emph{simplified} B-Doodle mechanisms:
\begin{definition}[Simplified B-Doodle Mechanism]
Let $S = \{1, 2, \dots, s\}$ be a set of $s$ time slots.
A simplified B-Doodle mechanism $B$ for $S$ is a vector of integers, described as $B = \langle b_1, b_2, \dots, b_m\rangle$ where $(b_j \geq 1$ for all $j \leq m)$ and $(\sum_{k=1}^{m} b_k = s)$.
We write $B_m$ to emphasize that $B$ has $m$ batches.
\end{definition}
From now on we use the definition of a simplified B-Doodle, and simply write a B-Doodle mechanism to mean a simplified B-Doodle mechanism but this should cause no confusion.

\section{Main Algorithm for the Batched Doodle Problem} \label{bdoodle:sec:Algorithm}

In this section we present an algorithm that finds an optimal B-Doodle mechanism that minimizes the expected cost, given an instance of \BDP. Let us first describe how one can express the expected cost of a B-Doodle mechanism.

Given an instance $(N, S, f, c, P)$, consider some B-Doodle mechanism $B_{m} = \langle b_1, b_2, \dots, b_m \rangle$ with $\sum_{k=1}^{m} b_k = |S|$. Let $Pr(j)$ be the probability that the scheduling process ends after $j$-th iteration. Let $\Exp[B_{m} | c]$ denote the expected cost of $B_{m}$ given cost function $c$, which can be expressed as follows:
\begin{equation} \label{bdoodle:eqn:expCost}
	\Exp[B_{m} | c] = \sum_{j=1}^{m} Pr(j) \cdot C(j, B_{m})  =  \sum_{j=1}^{m} Pr(j) \cdot \left(\sum_{k=1}^{j} c(k, b_k)\right).
\end{equation}%

While one can compute $Pr(j)$ in polynomial time, it is not necessary for our algorithm. Instead we present an important lemma that leads us to an efficient algorithm. The following lemma states that if $c$ is simple, then we can compute $\Exp[B_{m}|c]$ in a recursive manner.

\begin{lemma} \label{bdoodle:lemma:recurrence}
	Consider any mechanism $B_m = \langle b_1, b_2, \dots, b_m \rangle$. Let us denote another mechanism that is obtained after removing the first batch ($b_1$) from $B_m$, as $\hat{B}_{m-1}$ (i.e. $\hat{B}_{m-1} = \langle b_2, b_3, \dots, b_m \rangle$). For each time slot $t \in S$, let $q_t$ be the probability that $t$ is $f$-feasible.

If $c$ is a $\theta$-simple cost function with with $\theta > 0$, then the following equality holds:
\begin{equation} \label{bdoodle:eqn:recurrence}
\Exp[B_{m}|c] = c(1, b_1) +  \left(\prod_{t=1}^{b_1} \left(1 - q_t\right)\right) \cdot \theta \cdot \Exp[\hat{B}_{m-1}|c]
\end{equation}
\end{lemma}
\begin{proof}
Let us first provide an intuitive way to understand the recurrence relation given by Equation~\ref{bdoodle:eqn:recurrence}. The first term $c(1, b_1)$ captures the cost of sending out the first batch; regardless of when the scheduling process ends, this cost incurs with probability of $1$. If it turns out that the first batch does not contain a feasible time slot with probability of $(1 - Pr(1))$, then the organizer must send out the remaining batches, which is precisely $\hat{B}_{m-1}$. It is easy to verify that the product term in the equation is equal to $(1 - Pr(1))$. The expected cost of $\hat{B}_{m-1}$ is adjusted by a factor of $\theta$ in the equation because the cost function is $\theta$-simple.

We now formally prove the claim. Given $B_{m}$ as described above, for each batch $j \in \{1, 2, \dots, m\}$, let $r_j$ be the probability that the $j$-th batch has at least one $f$-feasible time slot. Let us define $v_0 = 0$ and $v_j = \sum_{k=1}^{j} b_k$ (i.e. $v_j$ is the number of time slots in batches $1$ through $j$).
We can express $r_j$ as:
\begin{equation} \label{bdoodle:eqn:r_j}
r_j = 1 - \prod_{t=v_{j-1} + 1}^{v_j} (1 - q_t)
\end{equation}
$Pr(j)$ is the probability that the scheduling process ends after $j$-th iteration (with $Pr(m) = 1 - \sum_{k=1}^{m-1} Pr(k)$). For $1 \leq j < m$, $Pr(j)$ is given by:
\begin{equation} \label{bdoodle:eqn:Pr_j}
Pr(j) =  r_j  \prod_{k=1}^{j-1} (1 - r_k)
\end{equation}

Recall that $\hat{B}_{m-1} = \langle b_2, b_3, \dots, b_{m} \rangle$.
For clarity, let us express it as $\hat{B}_{m-1} = \langle \hat{b}_1, \hat{b}_2, \dots, \hat{b}_{m-1}\rangle$ with $\hat{b}_j = b_{j+1}$ for all $j\geq 1$. Let $\hat{r}_j$ be the probability that a feasible time slot exists in $j$-th batch of $\hat{B}_{m-1}$, which is equal to $r_{j+1}$ for all $j \geq 1$. Let $\hat{Pr}(j)$ be the probability that scheduling ends after the $j$-th iteration of $\hat{B}_{m-1}$:
\begin{equation}
	\hat{Pr}(j)
	= \hat{r}_j \prod_{k=1}^{j-1} (1 - \hat{r}_k)
	= r_{j+1} \prod_{k=2}^{j} (1 - r_{k})  \label{bdoodle:eqn:pr_hat_b}
\end{equation}
We can express $\Exp[\hat{B}_{m-1}|c]$ as follows:
\begin{eqnarray*}
\Exp[\hat{B}_{m-1}|c]
&=& \sum_{j=1}^{m-1} \hat{Pr}(j) \left( \sum_{k=1}^{j} c(k, \hat{b}_k) \right) \\
&=& \sum_{j=1}^{m-1} r_{j+1} \left(\prod_{k=2}^{j} (1 - r_{k})\right) \left(\sum_{k=2}^{j+1} c(k-1, b_{k+1})\right) \\
&=& \sum_{j=2}^{m} r_{j} \left(\prod_{k=2}^{j-1} (1 - r_{k})\right) \left(\sum_{k=2}^{j} c(k-1, b_{k+1})\right)
\end{eqnarray*}
The first equality holds by definition. The second equality is due to Equation~\ref{bdoodle:eqn:pr_hat_b} and because $\hat{b}_k = b_{k+1}$. The third equality is by changing the range of $j$ in the summation.

If we multiply $\Exp[\hat{B}_{m-1}|c]$ by $(1 - r_1)\theta$, we get the following:
\begin{equation} \label{bdoodle:eqn:ExpCost_Bhat}
(1-r_1)\theta \Exp[\hat{B}_{m-1}|c] = \sum_{j=2}^{m} r_{j} \left(\prod_{k=1}^{j-1} (1 - r_{k})\right) \left(\sum_{k=2}^{j} c(k, b_{k+1})\right)
\end{equation}
Notice that the product term now runs from $k = 1$ to $j-1$ as we multiply by $(1-r_1)$ and the inner-most summation has $c(k, b_{k+1})$ as we multiply by $\theta$ (recall that $c$ is $\theta$-simple).

Finally we can express $\Exp[B_m|c]$ in terms of $\Exp[\hat{B}_{m-1}|c]$ as follows (where $c_1 = c(1, b_1)$ for brevity):
\begin{eqnarray*}
\Exp[B_{m}|c]
&=& \sum_{j=1}^{m} Pr(j) \left(\sum_{k=1}^{j} c(k, b_k)\right) \\
&=& c_1 + \left(\sum_{j=2}^{m} r_j \left(\prod_{k=1}^{j-1} (1 - r_k)\right) \left(\sum_{k=2}^{j} c(k, b_k)\right) \right) \\
&=& c_1 + (1-r_1)\theta \Exp[\hat{B}_{m-1}|c] \\
&=& c(1, b_1) + \left(\prod_{t=1}^{b_1} (1-q_t) \right)\cdot \theta \cdot \Exp[\hat{B}_{m-1}|c]
\end{eqnarray*}
The first equality holds by definition of $\Exp[B_{m}|c]$. The second equality is obtained by taking $c(1, b_1)$ out from the summation (and note that $Pr(j)$ adds up to 1) first and then applying Equation~\ref{bdoodle:eqn:Pr_j}. The last two inequalities hold due to Equation~\ref{bdoodle:eqn:ExpCost_Bhat} and \ref{bdoodle:eqn:r_j}, respectively.
The last expression exactly  matches Equation~\ref{bdoodle:eqn:recurrence} in the lemma.
\end{proof}

Using the recurrence relation in Lemma~\ref{bdoodle:lemma:recurrence} and the computing method in Lemma~\ref{bdoodle:lemma:q_t_polytime}, we can now design an efficient recursive algorithm finds the optimal B-Doodle mechanism. Our algorithm is presented in Algorithm~\ref{bdoodle:alg:recursive} as a recursive method $Rec(x)$. We assume that the values of $q_t$ have been computed as a pre-processing step prior to running our algorithm, using the method in Lemma~\ref{bdoodle:lemma:q_t_polytime}. We further assume that time slots are sorted in decreasing order of $q_t$ (i.e. $q_1 \geq q_2 \geq \dots \geq q_{s}$); therefore we simply refer to the time slot by its index (i.e. $t = 3$ refers to the third time slot in the sorted list of time slots).

Given $1 \leq x \leq s$, $Rec(x)$ returns optimal B-Doodle ($B^*$) that consists of time slots $\{x, x+1, \dots, s\}$ and its expected cost ($EC^*$). To solve for a given instance of the problem, we simply call the method $Rec(x)$ with $x = 1$.

\begin{algorithm}
\caption{Recursive Algorithm: $Rec(x)$}
\label{bdoodle:alg:recursive}
\begin{algorithmic}[1]
	\State ${B}^* \gets \langle s-x+1 \rangle$, $EC^* \gets c(1, s-x+1)$
	\For{$b := 1, 2, \dots, s-x$}
		\State $({B}, EC) \gets Rec( x+b )$
		\State $EC_b \gets c(1, b) + \left(\prod_{t=x}^{x+b-1} (1 - q_t)\right)\cdot \theta \cdot EC$
		\If {$EC^* > EC_b$}
			\State $EC^* \gets EC_b$
			\State $B^* \gets \langle b, {B} \rangle$
		\EndIf
	\EndFor
	\State \textbf{return} $({B}^*, EC^*)$
\end{algorithmic}
\end{algorithm}

\begin{theorem} \label{bdoodle:thm:recursive_algo}
Algorithm~\ref{bdoodle:alg:recursive} runs in polynomial time, and returns an optimal B-Doodle mechanism that minimizes the expected cost, given an instance $(N, S, f, c, P)$ of \BDPs when $c$ is $\theta$-simple.
\end{theorem}
\begin{proof}
Our recursive method is very simple: given time slots $\{x, x+1, \dots, s\}$, it iteratively considers the case of sending out $b$ time slots in the first iteration, and computes the expected cost $EC_b$ of doing so, for each $b$ with $1 \leq b \leq s-x+1$.  In line 1, our method checks for the trivial case when $b = s-x+1$ (i.e. all time slots are sent in a single batch); we store this mechanism in $B^*$ and its expected cost in $EC^*$. Then we iterate $b$ from $1$ to $s-x$ and compute the expected cost $EC_b$, and compare with the optimal expected cost found so far (lines 2-9). For each $b$, we first recursively compute the expected cost for the remaining time slots (namely, $\{x+b, x+b+1, \dots, s\}$) by calling our method $Rec(x+b)$, and store the expected cost in $EC$ (line 3). Using Lemma~\ref{bdoodle:lemma:recurrence} we can compute $EC_b$ as in line 4 (note that the first batch contains $b$ time slots from $x$ to $x+b-1$). In lines 5-8, we simply compare $EC_b$ with the current optimal, $EC^*$, and update if necessary; $\langle b, B \rangle$ should be interpreted as the concatenation of $b$ and $B$ into a single vector of integers. Finally in line 10, our method returns the optimal B-Doodle found, $B^*$ and its expected cost $EC^*$.

We prove correctness of the algorithm by induction on $x$ (from $x = s$ to $x = 1$). The base case is trivial: if $x = s$ the only B-Doodle mechanism is $\langle 1 \rangle$ and our method finds it in line $1$ and returns it in line $10$. Suppose our method correctly returns the optimal B-Doodle mechanism (and its expected cost) for all $x > k$ for some $k$ (the inductive hypothesis), and we prove for the case of $x = k$. When $x = k$, there are precisely $(s-k+1)$ time slots that are to be sent, and the first batch can contain any number of time slots between $1$ and $(s-k+1)$, inclusive. In lines 1-9 our method considers all such cases, and for each case it computes the expected cost correctly (in line 4) due to our inductive hypothesis and Lemma~\ref{bdoodle:lemma:recurrence}. Therefore our method finds the optimal B-Doodle for all $x$ with $1 \leq x \leq s$; in particular when $x = 1$, it returns the desired optimal B-Doodle mechanism for the given problem instance.

Our recursive method runs in polynomial time when we use memoization on $Rec(x)$ with respect to $x$; that is, for each $x$ we cache what $Rec(x)$ returns for the first time, and for any subsequence calls to $Rec(x)$ we simply return the cached values. Therefore lines 1-9 are executed at most once for each $x$ with $1 \leq x \leq s$. Also note that $Rec(x)$ makes a call to $Rec(x + b)$ with $b \geq 1$ and $x+b \leq s$, which means there are no infinite loops. Once $Rec(x)$ is computed for all $x > k$, it takes $O(|S|^2)$ time to compute $Rec(k)$, and thus the overall running time of our algorithm is $O(|S|^3)$ when we start with $Rec(1)$. Pre-processing steps (of computing $q_t$) also run in polynomial time due to Lemma~\ref{bdoodle:lemma:q_t_polytime}.
\end{proof}

\section{Experimental Results} \label{bdoodle:sec:Results}

We showed that we can find an optimal B-Doodle mechanism that minimizes the expected cost. But a very important question remains: Is our optimal B-Doodle substantially better than Doodle in realistic settings? If the answer is no, we do not have much incentive to discard the simplest mechanism, Doodle, over using our sophisticated algorithm. Intuitively we expect Doodle to be inefficient if there are many time slots (i.e., $s$ is large) because it causes much inconvenience for agents to examine the options. Reasoning further, we can also see that inefficiency of Doodle depends on the probability of each time slot being feasible (i.e. $q_t$ values), which in turn depends on $p_{i, t}$ values. If agents are relatively free, then a small number of time slots would be sufficient for finding a feasible time slot, which makes Doodle inefficient. Last but not least, the underlying cost function plays an important role as well. These all sound plausible, and we validate our intuition with simulation results.

In our setting there are many experimental choices one can choose from. In what follows we look at a representative example in which each agent is available for each time slot with the same probability of $p$; that is, $p_{i, t} = p$ for some constant $p$. (While in realistic situations the $p_{i, t}$'s may all differ, we  note that in our simulations  our results seem to carry over to these more general settings as well.) We present experimental results for each of the three cost functions discussed in Example~\ref{bdoodle:eg:costFunctions}: The linear cost function, the time-averse cost function, and the inconvenience-averse function.

% ===================== Linear Cost Function (Below)

\subsection{Linear Cost Functions} \label{bdoodle:sec:result_linear_cost}
Recall that the linear cost function is parameterized by some constant $\alpha>0$ and that $c_{\alpha}(j, b) = \alpha + b$. We used $\alpha = 2$ as an experimental choice, and we later discuss what we observed for different values of $\alpha$.

We reasoned that Doodle becomes suboptimal when the number of time slots, $s$, gets large. Therefore it is interesting to know for what ranges of $s$, Doodle is suboptimal. For some fixed $(n, s, p)$, let $C^D(n, s, p)$ be the cost of Doodle and $C^*(n, s, p)$ be the expected cost of $B^*$. We want to find the smallest critical point $S^*(n, p)$ such that for any $s \geq S^*(n, p)$ we have $C^D(n, s, p) > C^*(n, s, p)$.

In Table~\ref{bdoodle:table:DoodleSuboptimal}, we show $S^*$ for various $(n, p)$ when $f = 1$ (i.e. the organizer requires everyone be available). For instance we find $S^*(6, .8) = 5$, which implies that Doodle is suboptimal for all $s \geq 5$ given that $n=6$ and $p=.8$. The smaller $S^*$ is, the less practical Doodle is for the corresponding $(n, p)$.
Across the table we can observe that $S^*$ is small when $n$ is small and/or when $p$ is high -- this agrees with our intuition.
We highlighted $8$ entries in boldface to emphasize that $S^* \leq 15$; for the corresponding $(n, p)$ values, Doodle is suboptimal if there are $15$ or more time slots being considered.
\begin{table}[h!]
	\small
\centering
\begin{tabular}{|c|c|c|c|c|c|}
	\hline
	$S^*$ & $n = 2$ & $n = 4$ & $n = 6$ & $ n = 10 $ & $n = 15$ \\ \hline
	$p = .8$ & \textbf{3} & \textbf{4} & \textbf{5} & \textbf{8} & \textbf{15} \\ \hline
	$p = .5$ & \textbf{5} & \textbf{11} & 22 & 90 & $>300$\\ \hline
	$p = .2$ & \textbf{14} & 70 & $>300$ & $>300$ & $>300$ \\ \hline
\end{tabular}
\caption{Critical point $S^*$ is shown for various values of $(n, p)$ when $f = 1, \alpha = 2$. Entries in boldface emphasize that $S^* \leq 15$.} \label{bdoodle:table:DoodleSuboptimal}
\end{table}

Not only do we want to know when Doodle is suboptimal, but we also want to know how inefficient Doodle is in realistic situations. For every fixed $(n, s, p)$, we can define {\em efficiency of Doodle}, $e_{D}(n,s,p)$, as the ratio of the optimal expected cost to the cost of Doodle: $e_{D}(n,s,p) = C^*(n, s, p) / C^D(n, s, p)$. The smaller this value is, the less efficiency Doodle is compared to the optimal Batched Doodle, for any given $(n, s, p)$.

In Table~\ref{bdoodle:table:DoodleEfficiency} we show $e_{D}$ for the same settings of $(n, p)$ we used in Table~\ref{bdoodle:table:DoodleSuboptimal}. For this experiment we used $s = 15$ and $f = 1$. If $e_{D} = 1$ Doodle is optimal, and if $e_{D}$ is close to zero then Doodle is very inefficient. We find that $e_{D}(2, .8) = .270$, which implies that Doodle is very inefficient in this case; on the other hand $e_{D}(10, .5) = 1$, in which case Doodle is optimal. Across the table we observe the same pattern we observed before -- for small $n$ and high $p$, Doodle is substantially inefficient.
\begin{table}[h]
	\small
\centering
\begin{tabular}{|c|c|c|c|c|c|c|}
	\hline
	$e_{D}$ & $n = 2$ & $n = 4$ & $n = 6$ & $ n = 10 $ & $n = 15$ \\ \hline
	$p = .8$ & \textbf{.270} & \textbf{.361} & \textbf{.486} & .777 & .986 \\ \hline
	$p = .5$ & \textbf{.502} & .904 & 1 & 1 & 1  \\ \hline
	$p = .2$ & .970 & 1 & 1 & 1 & 1\\ \hline
\end{tabular}
\caption{Efficiency of Doodle $e_{D}$ is shown for various values of $(n, p)$ when $f = 1, \alpha=2, s = 15$. Entries in boldface emphasize that $e_{D} < .750$. } \label{bdoodle:table:DoodleEfficiency}
\end{table}

Inefficiency of Doodle is more pronounced when the organizer has a lower feasibility threshold such as $f = .7$; in such case, only a fraction of agents need to be available.
We can clearly observe the worsened inefficiency of Doodle in Table~\ref{bdoodle:table:DoodleEfficiency-lower-attendance}.
Here we show $e_{D}$ values for the same set of $(n, p)$ as in Table~\ref{bdoodle:table:DoodleEfficiency} but with $f = .7$.
Previously we highlighted four entries with $e_{D} < .750$ when $f = 1$ in Table~~\ref{bdoodle:table:DoodleEfficiency}; when $f = .7$ the number of entires with $e_{D} < .750$ doubled to eight.
Note that the first column is identical between the two tables; this is because $f = .7$ still requires both agents to be available when $n = 2$.
Also notice that $e_{D}$ is surprisingly low in the first row of Table~\ref{bdoodle:table:DoodleEfficiency-lower-attendance} across all columns. This shows that regardless of the number of invitees, Doodle is significantly inefficient when the invitees are highly available ($p \geq .8$) and the organizer has a relaxed feasibility threshold.
\begin{table}[h]  \small
\centering
\begin{tabular}{|c|c|c|c|c|c|c|}
	\hline
	$e_{D}$ & $n = 2$ & $n = 4$ & $n = 6$ & $ n = 10 $ & $n = 15$ \\ \hline
	$p = .8$ & \textbf{.270} & \textbf{.215} & \textbf{.267} & \textbf{.201} & \textbf{.211} \\ \hline
	$p = .5$ & \textbf{.502} & \textbf{.434} & .772 & \textbf{.628} & .913  \\ \hline
	$p = .2$ & .970 & 1 & 1 & 1 & 1\\ \hline
\end{tabular}
\caption{Efficiency of Doodle $e_{D}$ is shown for various values of $(n, p)$ when $f = .7, \alpha=2, s = 15$.  Entries in boldface emphasize that $e_{D} < .750$. } \label{bdoodle:table:DoodleEfficiency-lower-attendance}
\end{table}

We ran experiments with different values of $(n, s, p, f, \alpha)$, and observed the same  trends that are presented here.

% ===================== Other Cost Functions (Below)

%\balancecolumns
\subsection{Other Cost Functions}
We now consider the time-averse cost function and the inconvenience-averse cost function. As the names suggest, the former places more weight on optimizing \Time, while the latter on optimizing \Inconvenience.  For the time-averse cost function (recall $c_{\beta}(j, b) = \beta^j \cdot b$), we chose $\beta = 2$ as an experimental choice. Notice that the cost increases exponentially for later iterations, which forces the organizer to send out a small number of batches (as in Doodle). We ran the same set of experiments as before to measure efficient of Doodle, $e_{D}$.

The result is summarized in Table~\ref{bdoodle:table:DoodleEfficiency-lower-attendance_time_averse}, which shows the same trends as we observed in previous experiments. Notice that in the first row ($p = .8$), as $n$ increases we expect $e_{D}$ to decrease, but $e_{D}$ fluctuates while decreasing in general. The fluctuation is due to the rounding of attendance requirement ($\lceil a \cdot n \rceil$). For instance when $n = 4$, $\lceil a \cdot n \rceil$ = 3 which effectively requires 75\% of attendees be available.
\begin{table}[h]  \small
\centering
\begin{tabular}{|c|c|c|c|c|c|c|}
	\hline
	$e_{D}$ & $n = 2$ & $n = 4$ & $n = 6$ & $ n = 10 $ & $n = 15$ \\ \hline
	$p = .8$ & \textbf{.180} & \textbf{.104} & \textbf{.175} & \textbf{.088} & \textbf{.099} \\ \hline
	$p = .5$ & \textbf{.561} & \textbf{.457} & .876 & \textbf{.726} & .987  \\ \hline
	$p = .2$ & 1 & 1 & 1 & 1 & 1\\ \hline
\end{tabular}
\caption{Efficiency of Doodle $e_{D}$ is shown for various values of $(n, p)$ when $f = .7, \beta=2, s = 15$.
Entries in boldface emphasize that $e_{D} < .750$.
} \label{bdoodle:table:DoodleEfficiency-lower-attendance_time_averse}
\end{table}

For the inconvenience-averse cost function (recall $c_{\gamma}(j, b) = \gamma^{b}$), we chose $\gamma = 1.1$ as an experimental choice. Because the cost function is an exponential function of $b$, an optimal B-Doodle must send out small-size batches even though $\gamma$ is small. We ran the same set of experiments as before to measure efficiency of Doodle, $e_{D}$.

The result is summarized in Table~\ref{bdoodle:table:DoodleEfficiency-lower-attendance_inconvenience_averse}. Notice that $e_{D}$ is not equal to $1$ even when $p = .2$ in this setting, which we did not observe with the other cost functions previously. Due to integer-rounding, we again observe some fluctuations in $e_{D}$ across columns within the same row, but the general trend is that $e_{D}$ decreases as $n$ increases.
\begin{table}[h]   \small
\centering
\begin{tabular}{|c|c|c|c|c|c|c|}
	\hline
	$e_{D}$ & $n = 2$ & $n = 4$ & $n = 6$ & $ n = 10 $ & $n = 15$ \\ \hline
	$p = .8$ & \textbf{.333} & \textbf{.299} & \textbf{.329} & \textbf{.294} & \textbf{.298} \\ \hline
	$p = .5$ & \textbf{.499} & \textbf{.450} & \textbf{.695} & \textbf{.592} & .799  \\ \hline
	$p = .2$ & .850 & .887 & .974 & .976 & .980 \\ \hline
\end{tabular}
\caption{Efficiency of Doodle $e_{D}$ is shown for various values of $(n, p)$ when $f = .7, \gamma=1.1, s = 15$.
Entries in boldface emphasize that $e_{D} < .750$.
} \label{bdoodle:table:DoodleEfficiency-lower-attendance_inconvenience_averse}
\end{table}

%\balancecolumns

\subsection{Summary of Experiments}
Although we only presented experimental results with specific values of $(n, s, p, f)$ and parameters $(\alpha, \beta, \gamma)$, we observed that Doodle is in general substantially inefficient, including (but not limited to) when one or more of the following conditions hold:
\begin{itemize}
	\item There is a relatively small number of agents ($n \leq 10$).
	\item There is a large number of time slots ($s \geq 15$).
	\item Agents are relatively free ($p > .5$).
	\item Feasibility threshold is relaxed ($f < .8$).
	\item The cost function places more weight on \Inconveniences than \Times ($\alpha < 20$, $\beta < 5$, or $\gamma > 1.05$).
\end{itemize}

Intuitively the first four conditions affect $q_t$ (the probability that time slot $t$ is feasible) in the same way. If $q_t$ is higher, then Doodle is more likely to become more inefficient because there is no need to poll everyone about every date/time option -- a subset of the options would suffice. In such cases B-Doodle has advantages in that it reduces inconvenience of all agents. The last condition is independent of $q_t$, yet it is clear that Doodle should be inefficient if the cost function we are optimizing favors reducing \Inconveniences over reducing \Time. Given our experimental results, it is clear that under the right circumstances, B-Doodle is superior in optimizing \Times and \Inconvenience.

\section{Discussion} \label{bdoodle:sec:Discussion}

We identified and formally defined two dimensions of optimality in
group scheduling : \Times and \Inconvenience. We generalized the popular
Doodle mechanism to a class of B-Doodle mechanisms that partition time
slots into batches. We showed an example of the Pareto-frontier of
B-Doodle mechanisms on the \Time-\Inconveniences dimensions. We then
described an efficient algorithm for finding an optimal B-Doodle
mechanisms, given a simple cost function that aggregates \Times and
\Inconvenience, assuming probabilistic independence among availability
of agents. We showed in simulations that an optimal B-Doodle mechanism
is superior to Doodle in realistic situations, sometimes greatly so.

While useful in and of itself, our work described in this chapter leaves open many questions.
Here we focused on a class of B-Doodle mechanisms which allow the event organizer to poll all invitees on a batch of date/time options. In some settings, it may be more desirable for the organizer to poll some invitees on some date/time options -- for instance, when scheduling a defense of dissertation in the computer science department of Stanford University, it is desirable to poll availability of the advisor and committee chair before polling other participants because the advisor and committee chair must be physically present while others can join remotely. In the very next chapter, we generalize the Batched Doodle Problem by considering such settings.

In addition, we implicitly assumed that agents prefer all time slots equally (by
allowing agents to specify their availability but not preferences),
but people often have specific preferences. Thus it will be important
to study the trade-off between optimizing the cost of scheduling
process and finding a `good' schedule.

Finally, we assumed honesty, promptness, and trust of invitees, but
these are strong assumptions. In particular, promptness assumes that
invitees do not procrastinate and do not deliberately delay responses.
It is arguably true that if the scheduling process is efficient then
it will indeed reduce the procrastination of invitees. However in
practice an invitee may have an incentive to delay her response or to
lie about her availability or preferences. Practical mechanisms should
not be vulnerable to such strategic behavior of invitees, and this is
yet another direction for future work.

\section*{Publications}
Some of the contents of this chapter were published and included in the proceedings of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS) as extended abstracts~\cite{lee2014algorithmic,lee14doodle}.


%%
%% Author: novitoll
%% 3/8/18
%%

% Preamble
\documentclass[11pt]{article}

% Packages
\usepackage{amsmath}
\usepackage{enumitem}

\title{CVT: Lecture 5}
\date{2018-03-08}
\author{Novitoll}

\begin{document}
    \maketitle
    \pagenumbering{arabic}

    \section{Taylor formula} \label{sec:taylorformula}

    \begin{align}
        f(x) = f(0) + O(1)
    \end{align}

    Given parameter, compute its derivative and so you can "convert" the original function to Taylor
    \[
        f'(x) = f'(0) + O(1) --> \int_{0}^{x} ... dx
    \]

    So basic Taylor formula is:
    \begin{multline*}
        f(x) = f(0) + f'(0)x + f''(0)\frac{x^2}{2} \\
            + f'''(0)\frac{x^3}{6} + f^{IV}(0)\frac{x^4}{24} + f^n(0)\frac{x^n}{n!} + O(x^n)
    \end{multline*}

    Example:
    \begin{gather*}
        f(x) = x^2 + 3x + 6 \\
        f(0) = 6; f'(0) = 3; f''(0) = 2; f'''(0) = 0; f^n(0) = 0, n > 2 \\
        Taylor: f(x) = 6 + 3x + 2\frac{x^2}{2}
    \end{gather*}

    \section{Optical flow in general} \label{sec:optflowgeneral}

    Given x, y coordinate of some pixels in image (assuming that we are dealing with the only RGB pixels) in given t time,
    we can process the next image's pixels x, y coordinate in some $t + 1$ time

    \begin{enumerate}
        \item Constant color
    \end{enumerate}

    \begin{gather*}
        I_1(x, y, t) \\
        I_2(x + u, y + v, t + 1) \\
        O = I(x + u, y + v, t + 1) - I(x, y, t)
    \end{gather*}

    Let's put it to Taylor formula, assuming that we will neglect with high order derivatives due to non-significant bias of pixels.

    \begin{enumerate}[resume]
        \item Small motion
    \end{enumerate}

    \begin{gather*}
        I(x + u, y + v) = I(x, y) + \frac{\partial{I}}{\partial{x}}u + \frac{\partial{I}}{\partial{y}}v
    \end{gather*}
    \begin{multline}
        O \approx I(x, y, t + 1) + \frac{\partial{I}}{\partial{x}}u + \frac{\partial{I}}{\partial{y}}v - I(x, y, t) \\
            \approx [I(x, y, t + 1) - I(x, y, t)] + .. \\
            \approx \frac{\partial{I}}{t} + \frac{\partial{I}}{\partial{x}}u + \frac{\partial{I}}{\partial{y}}v
    \end{multline}

    , which can be also exposed as $\frac{\partial{I}}{t} + \det{I} \times <u, v>$,
    e\.g\. time derivative + multiplication of image gradient and vector of motion u, v per x, y axis.

    \section{Lucas-Kanade optical flow (1981)} \label{sec:lucaskanade}

    Adding here additional assumption:

    \begin{enumerate}[resume]
        \item Uniform motion of pixels
    \end{enumerate}

    Assume that you have a pixel x and you take it as 3x3 matrix with surrounded other pixels,
    assuming that they are similar (in real life this is true for the object, probably not true for micro objects like
    bacteria)

    , then your formula (2) is true for 3x3 matrix, e.g.:

    \[
        \begin{bmatrix}
            I_x(p_1) & \dots & I_y(p_1) \\
            I_x(p_2) & \dots & I_y(p_2) \\
            \vdots & \ddots & \vdots \\
            I_x(p_9) & \dots & I_y(p_9)
        \end{bmatrix}
        \times
        \begin{bmatrix}
            u & v
        \end{bmatrix}
        = -
        \begin{bmatrix}
            I_t(p_1) \\
            I_t(p_2) \\
            \vdots   \\
            I_t(p_9)
        \end{bmatrix}
    \]

    Let's name the matrix in A, d, b:

    \[
        A_{9x2} \times d_{2x1} = B_{9x1}
    \]

    and now we want to

    \[
        minimize => \lVert{Ad - b}\rVert^2
    \]

    \subsection{Eigenvalue and eigenvector} \label{subsec:eigenvv}

    Recap eigenvalue and eigenvector of linear transformation:
    \begin{gather*}
        T(\vec{v}) = \lambda\vec{v} \\
        , where \ \vec{v} - eigenvector, \ \lambda - eigenvalue, e.g. some scalar
    \end{gather*}

\end{document}


\documentclass[11pt,a4paper]{article}

% These are extra packages that you might need for writing the equations:
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{booktabs}
\usepackage{hyperref}
\usepackage{listings}
\usepackage{xcolor}
\usepackage{graphicx}
\usepackage{subfig}
\usepackage{float}

\lstset {language=C++,
		 basicstyle=\ttfamily,
         keywordstyle=\color{blue}\ttfamily,
         stringstyle=\color{red}\ttfamily,
         commentstyle=\color{purple}\ttfamily,
         morecomment=[l][\color{magenta}]{\#},
       	 basicstyle=\tiny}

% You need the following package in order to include figures in your report:
\usepackage{graphicx}

% With this package you can set the size of the margins manually:
\usepackage[left=2cm,right=2cm,top=2cm,bottom=2cm]{geometry}

\begin{document}

% Enter the exercise number, your name and date here:
\noindent\parbox{\linewidth}{
 \parbox{.25\linewidth}{ \large HPCSE I, Exercise 06 }\hfill
 \parbox{.5\linewidth}{\begin{center} \large Beat Hubmann \end{center}}\hfill
 \parbox{.2\linewidth}{\begin{flushright} \large Nov 09, 2018 \end{flushright}}
}
\noindent\rule{\linewidth}{2pt}

\section{Question 1: Linear Layer}

Done as instructed; tests passed.

% \begin{figure}[ht]
% \begin{center}
% \includegraphics[scale=0.5]{results.png}
% \end{center}
% \caption{Parallel Monte Carlo integration on Euler compute cluster.}
% \label{fig1}
% \end{figure}

\section{Question 2: Optimization Algorithm}

Done as instructed including the L2 penalization.\\
Finished with: Training set MSE:13.520405, Test set MSE:13.381790.
The 10 principal components of the MNIST data set obtained by using a single linear hidden layer are shown in figure~\ref{fig:1}.
These principal components clearly show the main 'activity' in the center of the figures
where all digits have 'active' regions.

\begin{figure}[H]
\begin{tabular}{ccccc}
\subfloat[Component 0]{\includegraphics[width = 1.1in]{lin_component_0.eps}} &
\subfloat[Component 1]{\includegraphics[width = 1.1in]{lin_component_1.eps}} &
\subfloat[Component 2]{\includegraphics[width = 1.1in]{lin_component_2.eps}} &
\subfloat[Component 3]{\includegraphics[width = 1.1in]{lin_component_3.eps}} &
\subfloat[Component 4]{\includegraphics[width = 1.1in]{lin_component_4.eps}} \\
\subfloat[Component 5]{\includegraphics[width = 1.1in]{lin_component_5.eps}} &
\subfloat[Component 6]{\includegraphics[width = 1.1in]{lin_component_6.eps}} &
\subfloat[Component 7]{\includegraphics[width = 1.1in]{lin_component_7.eps}} &
\subfloat[Component 8]{\includegraphics[width = 1.1in]{lin_component_8.eps}} &
\subfloat[Component 9]{\includegraphics[width = 1.1in]{lin_component_9.eps}}
\end{tabular}
\caption{10 Principal components of MNIST data set obtained from single linear hidden layer.}
\label{fig:1}
\end{figure}

\section{Question 3: Non-linearity}

Done as instructed; tests passed.\\
Finished with: Training set MSE:13.538446, Test set MSE:13.395468.
The 10 principal components of the MNIST data set obtained by using three hidden layers with $\tanh$ activation function are shown in figure~\ref{fig:2}.
Now certain elements of specific digits are clearly identifiable among the principal components,
which is clearly owed to the much larger network (2 additional hidden layers with 100 neurons each) as well
as the non-linear activation function.

\begin{figure}[H]
\begin{tabular}{ccccc}
\subfloat[Component 0]{\includegraphics[width = 1.1in]{component_0.eps}} &
\subfloat[Component 1]{\includegraphics[width = 1.1in]{component_1.eps}} &
\subfloat[Component 2]{\includegraphics[width = 1.1in]{component_2.eps}} &
\subfloat[Component 3]{\includegraphics[width = 1.1in]{component_3.eps}} &
\subfloat[Component 4]{\includegraphics[width = 1.1in]{component_4.eps}} \\
\subfloat[Component 5]{\includegraphics[width = 1.1in]{component_5.eps}} &
\subfloat[Component 6]{\includegraphics[width = 1.1in]{component_6.eps}} &
\subfloat[Component 7]{\includegraphics[width = 1.1in]{component_7.eps}} &
\subfloat[Component 8]{\includegraphics[width = 1.1in]{component_8.eps}} &
\subfloat[Component 9]{\includegraphics[width = 1.1in]{component_9.eps}} i\\
\subfloat[-Component 0]{\includegraphics[width = 1.1in]{component_10.eps}} &
\subfloat[-Component 1]{\includegraphics[width = 1.1in]{component_11.eps}} &
\subfloat[-Component 2]{\includegraphics[width = 1.1in]{component_12.eps}} &
\subfloat[-Component 3]{\includegraphics[width = 1.1in]{component_13.eps}} &
\subfloat[-Component 4]{\includegraphics[width = 1.1in]{component_14.eps}} \\
\subfloat[-Component 5]{\includegraphics[width = 1.1in]{component_15.eps}} &
\subfloat[-Component 6]{\includegraphics[width = 1.1in]{component_16.eps}} &
\subfloat[-Component 7]{\includegraphics[width = 1.1in]{component_17.eps}} &
\subfloat[-Component 8]{\includegraphics[width = 1.1in]{component_18.eps}} &
\subfloat[-Component 9]{\includegraphics[width = 1.1in]{component_19.eps}}
\end{tabular}
\caption{10 Principal components of MNIST data set obtained from three hidden layers with $\tanh$ activation function.}
\label{fig:2}
\end{figure}

\section{Question 4: Parallelization}

Most parallelization loops can be done in a straightforward way based on
\texttt{\#pragma omp parallel for}. A few loops however required special care:

\begin{itemize}
    \item In \texttt{LinearLayer::bckward}, the loops aren't perfectly nested and therefore can't be collapsed. Thus, only the inner loop was OMP'ed.
    \item In \texttt{main\_linear.cpp} starting on line 88, the operation on \texttt{std::vector<int> sample\_ids} is not thread-safe. This was solved by declaring a private variable and using a \texttt{critical} section for \texttt{pop\_back}.
    \item In \texttt{main\_linear.cpp} starting on lines 97 and 128, updating \texttt{epoch\_mse} (or \texttt{test\_mse}, respectively) leads to a data race. This was solved with a \texttt{reduction(+: \_\_ \_mse)} statement.
\end{itemize}

I tested my implementation for correctness with different numbers of threads (especially the edge cases
of one single and the maximum number of threads). While there were no errors (passed all test; achieved same training/test errors), the parallelization
did not lead to any noticeable speedups. I imagine this is due to the comparably small batch sizes
and both small images and overall small size of the neural network.

\end{document}


\input{permve-ntnu-latex-assignment.tex}

\usepackage{float}

\title{
\normalfont \normalsize
\textsc{Norwegian University of Science and Technology\\IT3105 -- Artificial Intelligence Programming}
\horrule{0.5pt} \\[0.4cm]
\huge Module 1:\\Implementing and Testing the\\A* Algorithm on a Navigation Task\\
\horrule{2pt} \\[0.5cm]
}

\author{Per Magnus Veierland\\permve@stud.ntnu.no}

\date{\normalsize\today}

\newacro{BFS}{Breadth First Search}
\newacro{DFS}{Depth First Search}

\begin{document}

\fancyfoot[C]{}
\maketitle

\newpage
\fancyfoot[C]{\thepage~of~\pageref{LastPage}} % Page numbering for right footer
\setcounter{page}{1}

\section*{Search agenda}

The search agenda, also known as the ``open list'' is used to maintain generated search nodes which have not yet been expanded. When popping a search node from the ``open list'' it is checked whether the state associated with the search node is a goal state. If it is a goal state the search completes; otherwise the search node is expanded and successor states are generated -- adding unique successor states to the ``open list''. Search nodes which have been expanded are added to a ``closed list'' to keep track of previously examined states.

The important operations relating to the ``open list'' are:
\begin{enumerate*}
\item testing if a state exists in the list
\item adding new search nodes to the list
\item removing the search node with the lowest $f$-value
\end{enumerate*}.

To implement these operations efficiently both a hash map and a heap queue are maintained to keep track of the ``open list''. It is required that objects representing state implement a hashing operation and an operation to test for equality. The hash map is used to allow testing if a state exists in the ``open list'' in O$(1)$ time; while the heap queue is used to efficiently keep track of the search node with the lowest $f$-value. The heap queue allows insertion of a search node and extraction of the search node with lowest $f$-value in O$(\log(n))$ time.

An important note is that when a better path is found to an existing search node; the path cost of the existing search node must be updated and propagated. Since this may change the $f$-value of search nodes on the ``open list'' the heap queue must be re-heapified; an operation which takes O$(n)$ time.

The ``closed list'' does not need to keep track of $f$-values and must only support testing for membership and adding new search nodes efficiently. It is maintained using a hash map.

Partly for the assignment and partly for fun; the strategy employed by the \texttt{BestFirst} search class can be changed mid-search between \ac{BFS}, \ac{DFS}, Dijkstra's algorithm and A*. When using \ac{BFS} and \ac{DFS} the ``open list'' is maintained as a double ended queue, also known as a deque, to allow adding and removing elements from both ends of the queue in O$(1)$ time. The search nodes on the ``open list'' will be correctly moved between the priority queue and the double ended queue used in the ``open list'' when switching strategies.

\section*{Generality of program}

When starting work on the three first modules in the \textsc{IT3105} course I decided to do all software in the Python language as it allows for easy prototyping and development as well as the ability to write readable software. The codebase is split into a file hierarchy with some of the classes central to the module~1 problem shown in Figure~\ref{figure:vi_classes}.

Any problem can be used with the \texttt{BestFirst} search class if it provides the following interface:

\begin{itemize}
\item \texttt{goal\_test(state)} -- Evaluates whether the given search space state is a goal state. Must return a truthy type. This method is called directly by the \texttt{BestFirst} class.
\item \texttt{heuristic(state)} -- \textit{(Optional)} A method calculating the estimated heuristic value for the provided search space state. This method is only called from the search node class. If this method is not available heuristic values will always be zero.
\item \texttt{initial\_node()} -- Returns a search node representing the initial search space state. This method is called directly by the \texttt{BestFirst} class. This method may return \texttt{None} if no valid first search space state is available for the \texttt{Problem}.
\item \texttt{solution(node)} -- \textit{(Optional)} Returns a \texttt{Solution} object built from the given search node. When this method is not provided by \texttt{Problem} class, the \texttt{BestFirst} will default to a \texttt{Solution} class which builds a solution path based on the assumption that each search node has an action and a parent search node.
\item \texttt{successors(node)} -- Generates all successors for a given node by applying all valid operators. This method is called directly by the \texttt{BestFirst} class.
\end{itemize}

The domain-specific \texttt{Problem} class is responsible for instantiating all search nodes, successor objects and solution objects. If it is necessary for the problem; custom implementations can be used for all of these objects as long as they follow the same interface as the \texttt{vi.search.graph.Node}, \texttt{vi.search.graph.Successor} and \texttt{vi.search.graph.Solution} classes.

\begin{figure}[H]
\includegraphics[scale=0.7]{images/vi_class_hierarchy}
\caption{VI Python library classes}
\label{figure:vi_classes}
\end{figure}

\section*{Heuristic function}

The central component of the A* search is the heuristic function which estimates the path cost from a given search space state to the goal state. To guarantee that an A* search finds an optimal solution, the heuristic function must be admissible, meaning that it must never overestimate the path cost to the goal state.

Two popular heuristics for the grid search problem is the Manhattan distance and the Euclidean distance. The Manhattan distance is given by $\textit{abs}(g_x - s_x) + \textit{abs}(g_y - s_y)$ where $g$ is the goal state and $s$ is the given state. Given our grid problem where only vertical and horizontal movement is allowed, the Manhattan distance is not only an admissible heuristic, but also a perfect heuristic since its value will always be the shortest path between a cell in the grid and the goal cell. However since our problem also includes obstacles, the heuristic estimate will not always match the actual path cost as it assumes that all grid cells are traversable.

The Euclidean heuristic uses the straight-line distance between two grid cells as the estimate to the goal. It is given by $\sqrt{(g_x - s_x)^2 + (g_y - s_y)^2}$. It is an admissible heuristic for the problem, but it will underestimate the cost to the goal whenever the starting cell is on a different row or column than the goal.

Table~\ref{table:vi_generated_nodes} shows the Manhattan heuristic outperforming the Euclidean heuristic in all scenarios. If the problem was modified to allow diagonal movements the heuristics would be more equal in their performance.

\begin{table}
\centering
\begin{tabular}{c|ccccc}
Scenario & BFS & DFS & Dijkstra & A* Manhattan & A* Euclidian \\
\hline
0        &  73 &  73 &       73 &           73 &           73 \\
1        & 291 & 296 &      293 &          190 &          256 \\
2        & 344 & 282 &      344 &          215 &          341 \\
3        &  57 &  50 &       56 &           44 &           50 \\
4        &  62 &  58 &       62 &           58 &           59 \\
5        & 273 & 227 &      271 &          233 &          246 \\
\end{tabular}
\caption{Nodes generated for different search strategies}
\label{table:vi_generated_nodes}
\end{table}

\section*{Generating successor states}

``A.I. -- A Modern Approach'' suggests an interface between the search and the problem consisting of the functions \texttt{problem.ACTIONS(node.STATE)}, \texttt{problem.RESULT(parent.STATE, action)} and \texttt{problem.STEP-COST(parent.STATE, action)}. After initially attempting to follow the same abstraction I found it problematic since the work involved in the operations can for many problems be overlapping. For example generating the list of valid actions for a state in the grid problem requires generating the resulting states.

My attempt at a more general interface is a single \texttt{problem.SUCCESSORS} function which does not return a list of successor nodes; but which generates \texttt{Successor} objects representing each valid successor to the given search node. The \texttt{Successor} object contains all information necessary to construct the corresponding \texttt{Node} object, as well as the step cost from the parent node to the successor node.

The \texttt{Successor} abstraction makes it possible to avoid building \texttt{Node} objects when the successor state is not unique. When adding the newly created or existing node object representing the successor state as a child in the parent node; the action and step cost needed to reach the child are also stored. Whenever a child node is later reattached to a parent node the step cost and action information will already be available.

Building the right trade-offs in the interface between what is computationally efficient and memory-efficient will depend on the problem. The focus in this abstraction has been on semantic correctness and avoiding redundant computation.

\end{document}


\documentclass[a4paper]{article}

\input{temp}

\begin{document}

\title{Principle of Statistics (continued) }

\maketitle

\newpage

\tableofcontents

\newpage

In the previous lecture, we saw that for a bootstrap sample $(X_1^b,...,X_n^b)$ drawn from $\P_n( \mid X_1,...,X_n)$, we have $\sup_{t \in \R} | \P_n(\sqrt{n} (\bar{X}_n^b - \bar{X}_n) \leq t(X_1,...,X_n) - \Phi(t) | \xrightarrow{a.s.} 0$ as $n \to \infty$, where $\Phi(t) = \P(Z \leq t)$, $Z \sim N(0,\sigma^2)$.

\begin{rem}
As in the B.vM theorem, this theorem can be used to show that $\P(\mu \in \mathcal{C}_n) \to 1-\alpha$ when $n \to \infty$.
\end{rem}

Idea: For fiexd $(X_i)_{i \geq 1}$, $\P_n(\mid X_1,...,X_N)$ is a sequence of distributions. Considering the $X_i$ as independent random variables drawn from $P$ allows us to make statements "with randomness". The idea is to fix a sequence $X_i$ (equivalent to fix $\omega$ in the original probability space) if we can show that $\sup_{t \in \R} |\P_n(...\mid X_1,...,X_n) - F(t)| \to 0$ "for almost all $\omega"$, then we have almost sure convergence.

\begin{lemma}
If $A_n \sim f_n \xrightarrow{d} A \sim f$, and $F$ is continuous (c.d.f of $f$), then $\sup|F_n(t) - F(t)| \to 0$ as $n \to \infty$.
\begin{proof}
By continuity of $F$, there exists points $-\alpha_0 = x_0 < x_1 < ... < x_k = +\infty$ such that $F(x_i) = \frac{i}{k}$. Then for every $x \in [x_{i-1},x_i]$, $F_n(x) - F(x) \leq F_n(x_i) - F(x_{i-1}) = F_n(xi)-F(x_i) + \frac{1}{k}$, and $F_n(x) - F(x) \geq F_n(x_{i-1}) - F(x_i) = F_n(x_{i-1}) - F(x_{i-1}) - \frac{1}{k}$. For $k$ large enough, $\frac{1}{k} < \frac{\varepsilon}{2}$, for $n$ large enough (dependes on $k$), we have $\max_{0 \leq i \leq k} |F_n(x_i) - F(x_i)| < \varepsilon/2$ (pointwise convergence) (dependes on $k$), we have $$\max_{0 \leq i \leq k} |F_n(x_i) - F(x_i)| < \varepsilon/2$$ (pointwise convergence). As a consequence, $$\sup_{x \in \R} |F_n(x) - F(x)| \leq \max_{0 \leq i \leq k} |F_n(x_i) - F(x_i)| + \frac{1}{k}<\varepsilon$$
\end{proof}
\end{lemma}

\begin{defi}
The sequence $(Z_{n,i},i=1,...,n)_{n \geq 1}$ is a triangular array of i.i.d. random variables if:\\
$\bullet$ For all $n \geq 1$, $(Z_{n,1},...,Z_{n,i},...,Z_{n,n})$ is a sequence of i.i.d. random variables;\\
For example, $Z_{11} = (Z_{1,i})$,\\
$Z_{21},Z_{22} = (Z_{2,i})$,\\
...,\\
$Z_{n,1},...,Z_{n,n} = (Z_{n,i})$.\\
We need independence on each line, but not across the lines. We don't need even need the same distribution at each line.
\end{defi}

\begin{prop} (CLT for triangular arrays)\\
Let $(Z_{n,i};i=1,...,n)$ be a triangular array of iid random variables, each with finite variance. We have $Var_{Q_n}(Z_{n,i}) = \sigma_n^2 \to \sigma_2$ as $n \to \infty$, each line consists of $n$ independent draws from $Q_n$. Then, under the following hypotheses (1-3), we have $$\sqrt{n} (\frac{1}{n} \sum_{i=1}^n Z_{n,i} - \E_{Q_n} [Z_{n,i}])\xrightarrow{d} N(0,\sigma^2)$$
(1) $\forall \delta>0$, $n Q_n (|Z_{n,1}|>\sqrt{n}\delta) \to 0$ as $n \to \infty$;\\
(2) $Var(Z_{n,1} 1\{|Z_{n,1}|\leq \sqrt{n}\}) \to \sigma^2$ as $n \to \infty$;\\
(3) $\sqrt{n} \E [Z_{n,1} 1\{|Z_{n,1}| > \sqrt{n}\}] \to 0$ as $n \to \infty$.\\
(The statement of these assumptions is not examinable.)
\begin{proof} (of the main theorem)\\
Fix $(X_i)_{i \geq 1}$ (equivalent to fix $\omega$ in the original probability space). Under $Q_n = \P_n(\cdot \mid X_1,...,X_n)$, $Z_{n,i} = X_i^{b(n)}$ (bootstrap on $n$ observations), $\E_n[Z_{n,i}] = \E_{\P_n} [X_i^{b(n)}] = \bar{X}_n$. Then, the $(X_i^{b(n)};i=1,...,n)_{n \geq 1}$ are a triangular array of i.i.d. variables. We have that $$Var_{\P_n}(X_i^{b(n)}) = \E_{\P_n} [X_i^{b(n)^2}] - (\E_{\P_n} [X_i^{b(n)}])^2 = \frac{1}{n}\sum_{i=1}^n X_i^2-(\frac{1}{n}\sum_{i=1}^n X_i)^2 = \sigma_n^2$$ by definition of $\P_n(\mid X_1,...,X_n)$. For almost all the $\omega$, or almost all infinite sequences, $\sigma_n^2 \to \sigma^2$, and hypotheses (1-3).\\
$\bullet$ By the CLT for triangular arrays, we have that $$\sqrt{n} (\bar{X}_n^{b(n)} - \bar{X}_n) \xrightarrow{d} N(0,\sigma^2)$$ as $n \to \infty$ 'for almost all $\omega$'.\\
$\bullet$ By lemma, $$\sup_{t \in \R} | \P_n (\sqrt{n} (\bar{X}_n^{b(n)} - \bar{X}_n) \leq t) - \Phi(t)| \geq 0$$ as $n \to \infty$ 'for almost all $\omega$', meaning that it $\xrightarrow{a.s.} 0$.
\end{proof}
\end{prop}

\begin{rem}
$\bullet$ This shows the validity of the bootstrap confidence interval for the mean.\\
$\bullet$ In genral, this can be extended to estimation of $\theta$: Sampling from $\P_n$ as for the mean, and compute the bootstrap MLE $\hat{\theta}_n^b = \hat{\theta} (X_1^b,...,X_n^b)$ then using $\sqrt{n}(\hat{\theta}_n^b - \hat{\theta}_n)$ as a proxy for $\sqrt{n} (\hat{\theta}_n - \theta_0)$, we will have similar results that show that taking $R_n$ such that $$\P_n(||\hat{\theta}_n^b - \hat{\theta}_n|| \leq \frac{R_n}{\sqrt{n}} \mid X_1,...,X_n) = 1-\alpha$$ can be used to construct a valid confidence region.\\
This approach is known as the non-parametric bootstrap.\\
Another approach is to do the same thing with $X_1^b,...,X_n^b \sim \P_{\hat{\theta}_n}$, same types of results will hold.
\end{rem}

Last lecutre on Monday. \emph{No} Lecture on Wednesday!

\subsection{Monte-Carlo methods}
In statistics, we often cannot explicityl compute expectation/integrals, which can be problematic when:\\
$\bullet$ We want to obtain a posterior distribution;\\
$\bullet$ We want to obtain a posterior mean;\\
$\bullet$ We want to compute quantiles of a distribution.

One of the ideas of Monte-Carlo methods is to replace explicitly computation by \emph{simulations}Last lecutre on Monday. \emph{No} Lecture on Wednesday!

\subsection{Monte-Carlo methods}
In statistics, we often cannot explicityl compute expectation/integrals, which can be problematic when:\\
$\bullet$ We want to obtain a posterior distribution;\\
$\bullet$ We want to obtain a posterior mean;\\
$\bullet$ We want to compute quantiles of a distribution.

One of the ideas of Monte-Carlo methods is to replace explicitly computation by \emph{simulations}. One of the first challenges is to draw from a given, general distribution.

\begin{defi}
A pseudorandom generator provides independent $U_i^{*} \sim Unif(0,1)$.
\end{defi}

\begin{rem}
They are generated such that for all practical uses, $P(U_1^* \leq u_1,...,U_N^* \leq u_n) = \prod_{i=1}^N u_i$ (up to 'machine precision'). Here it can be thought of as a blackbox that outputs i.i.d. uniform numbers -- this can be used as a starting point to generate other variables.
\end{rem}

\begin{prop}
The random variables $K_i = \sum_{k=1}^n k 1_{(\frac{k-1}{n},\frac{k}{n}]} (U_i^*)$ are i.i.d. uniform on $\{1,...,n\}$.
\begin{proof}
$K_i$ is clearly uniform on $\{1,...,n\}$, as each segment has length $\frac{1}{n}$. They are independent as functions of independent random variables $U_i^*$.
\end{proof}
\end{prop}
\begin{rem}
Assigining other values to each of the intervals with uniform distribution on any set of size $n$. In particular, we can simulate bootstrap samples by writing $$X_i^b = \sum_{k=1}^n X_k 1_{(\frac{k-1}{n},\frac{k}{n}]} (U_i^*)$$ If the intervals are chosen with different lengths, we can generate any discrete distribution. For a general distribution with cdf $F$, we can generalize this idea.
\end{rem}

\begin{defi}
For a general cdf $F$, we define the generalized inverse of $F$ as $$F^-(u) = \inf \{x: u \leq F(x)\}$$
\end{defi}

\begin{rem}
For a fixed value $t \in \R$, the function $F$ gives $F(t) \in [0,1]$ a probability $\P(X \leq t)$. For a fixed value $u \in [0,1]$, the function $F$ gives $t = F^- (u)$ such that approximately $\P(X \leq t) = u$.
\end{rem}

\begin{prop}
$X = F^-(u)$ for $U \sim Unif (0,1)$ has a distribution cdf $F$.
\begin{proof} (Example sheet)\\
$\P(X \leq t) = \P(F^-(U) \leq t) = ... = F(t)$, and use that $\P(U \leq z) = z$ for $z \in (0,10$.
\end{proof}
\end{prop}
Conclusion (of the first part): If $F$ is known, explicit, we can generated $(X_1^*,...,X_N^*) = (F^-(U_1^*),...,F^-(U_N^*))$ that are i.i.d., each with cdf $F$. If we want to compute $\E_{X \sim f} [g(X)]$, we can approximate it by $\frac{1}{N} \sum_{i=1}^N g(X_i^*)$, and use the fact that $\frac{1}{N} \sum_{i=1}^N g(X_i^*) \xrightarrow{a.s.} \E[g(X)]$ by the LLN.

In certain situations, the distributino might be complex, and $F$, $F^-$ are not explicit. For example, $N(\mu,\sigma^2)$ can be solved by looking up in a table, but $\Pi (\cdot \mid X)$ can involve complicated integrals (the density) which make computation of $F_\Pi$ impossible.

There are several ways to tackle this and approximately sample from distributions.

(1) Importance sampling: Let $F$ have density $f$, and random variables i.i.d. $X_i^* \sim h$.

\begin{prop}
$\E_h \left[\frac{g(x)}{h(x)} f(x)\right] = \E_f [g(x)]$.
\begin{proof}
The above is equal to $$ \int_\chi \frac{g(x)}{h(x)} f(x) \cdot h(x) dx = \int_\chi g(x) f(x) dx$$ As a consequence, $$\frac{1}{N} \sum_{i=1}^n \frac{g(X_i^*)}{h(X_i^*)} f(X_i^*) \xrightarrow{a.s.} \E_{X \sim f} [g(X)]$$.
\end{proof}
\end{prop}

(2) Accept/Reject algorithm. As in (1), but $f \leq M\cdot h$ for some constant $M$.\\
Step 1: generate $X \sim h$ and $U \sim U(0,1)$.\\
Step 2: $Y=X$ if $U \leq \frac{f(X)}{M\cdot h(X)}$, otherwise return to step 1. Then $Y \sim f$ (example sheet).

For multivariate problems where conditional distributions are aesy to compute but not joint distributions, we can then use the \emph{Gribbs samples}: In the bivariate case $(X,Y)$, start at the same $X = x_0$, and $Y_1 \sim f_{Y|X}(\cdot |x_0)$, and $X_1 \sim f_{X|Y}(\cdot |y_1)$, etc., $Y_t \sim f_{Y|X}(\cdot | X_{t-1})$, $X_t \sim f_{X|Y}(\cdot | Y_t)$. The sequences $(X_t,Y_t)$,$(X_t)$,$(Y_t)$ are all Markov chains, with invariant distribution $f$, $f_{X|Y}$, $f_{Y|X}$. And we can use the ergodic theorem to approximate expectations $$\frac{1}{N} g(X_t,Y_t) \to \E_{(X,Y) \sim f} [g(X,Y)]$$ This can be used in particular in the case $Q(x,\theta) = f(x,\theta) \pi (\theta)$.

Important: Last lecutre today, no lecutre on Wednesday!!

\subsection{Nonparametric statistics}
Consider observing $X_1,...,X_n \sim P$ i.i.d. with the distribution $P$ having cdf on $\R$: $F(t) = \P(X \leq t)$ for all $t \in \R$. Here we want to estimate directly the function $F$, without a parametric assumption: we cannot "estimate $\theta$ to estimate $F_\theta$".

\begin{rem}
We note that
\begin{equation*}
\begin{aligned}
F(t) = \P(X \leq t) = \E_p [1_{[-\infty,t]}(x)]
\end{aligned}
\end{equation*}
(or $\int_\R 1_{[-\infty,t]} (x) d\P(x)$) if the distribution is continuous.

For all $t \in \R$, the real number $F(t)$ is the expectation of the random variable $1_{[-\infty,t]}(X) \in \{0,1\}$ of which we observe $n$ i.i.d. draws.
\end{rem}

\begin{defi}
The \emph{empirical distribution function} is defined as $$F_n(t) = \frac{1}{n} \sum_{i=1}^n 1_{[-\infty,t]}(X_i)$$
\end{defi}

\begin{rem}
If we are interested in the value of the c.d.f., for some fixed $t$, $F_n(t)$ is a consistent estimator of $F(t)$ by Law of large numbers, and we can control the rate of estimation by limiting distribution of $\sqrt{n} (F_n(t) - F(t))$ (CLT gives $N(0,F(t) (1-F(t)))$). This is just a Bernoulli model.\\
Because we are interested in the overall (all of $\R$) behaviour of $F_n$, and see $F_n$ as the estimator of a function, we have to understand it s dependency structure.
\end{rem}

\begin{thm} (Glivenko-Cantelli theorem)\\
We have, as $n \to \infty$, that $$\sup_{t \in \R} |F_n(t) - F(t)| \xrightarrow{a.s.} 0$$
\begin{proof}
If $f$ is continuous in $t$, writing $q(X_i,t) = 1_{[-\infty,t]]}(X_i)$, we have $\E_p [q(X,t)] = F(t)$ and the uniform law of large numbers applies directly: $$\sup_{t \in \R} \left| \underbrace{\frac{1}{n} \sum_{i=1}^n q(X_i,t)}_{F_n(t)} - \underbrace{\E_p [q(X,t)]}_{F(t)}\right| \xrightarrow{a.s.} 0$$ The case where $F$ is not continuous can be handled as well and the result holds by using that $F$ is non-decreasing and cutting $[0,1]$ into smaller intervals of size $\leq \varepsilon$.
\end{proof}
\end{thm}

\begin{thm} (Donskin-Kolmogorov-Doob theorem)\\
As $n \to \infty$, the random function $\sqrt{n}(F_n-F)$ converges $\sqrt{n} (F_n-F) \xrightarrow{"d"} \mathcal{G}_F$ "in distribution over the space of functions". Here $\mathcal{G}_F$ is a random function from $\R$ to $\R$ such that $\mathcal{G}_F(t)$ is normally distributed $N(0,F(t)(1-F(t)))$, and $Cov(\mathcal{G}_F(s), \mathcal{G}_F(t)) = F(s) (1-F(t))$ for $s \leq t$.

Construction of $\mathcal{G}_F$:\\
Informal definition: A Brownian motion, a Wiener process is defined as\\
$\bullet$ $W_j = 0$ a.s.;\\
$\bullet$ $t \to W_t $ is continuous a.s.;\\
$\bullet$ For $s \leq t$, $W_t - W_s$ is independent of $(W_{s'})_{s' \leq s}$ and has distribution $N(0,t-s)$.

The Brownian bridge is "tied to $0$ at $0$ and $1$", and defined as $B_t = W_t - tW_1$.

The variance of $B_t=t(1-t)$ and $Cov(B_s,B_t) = s(1-t)$ for $s \leq t$. Taking $\mathcal{G}_F (t) = B_{F(t)}$ gives a construction of this process.

\begin{rem}
If $U_1,...,U_n \stackrel{i.i.d.}{\sim} U[0,1]$, then $F(t) = t$ and $\sqrt{n}(F_n-F)$ will converge directly to a Brownian bridge.
\end{rem}
\end{thm}

\begin{thm} (Kolmogorov-Smirnov theorem)\\
$\sqrt{n} ||F_n - F||_\infty \xrightarrow{d} ||B||_\infty$, where $||B||_\infty = \sup_{t \in (0,1)} |B_t|$.
\begin{proof}
$||\mathcal{G}_F||_\infty = \sup_{t \in \R} |B_{F(t)}| = \sup_{t \in (0,1)} |B_t|$.
\end{proof}
\end{thm}

\begin{rem}
$||B||_\infty$ doesn't depend on $F$, there are tables so it can be used in many inference tasks.\\
(1) Non-parametric hypothesis testing: we have $H_0:F = F_0$, or $H_1: F \neq F_0$. Then $\sqrt{n} ||F_n - F_0||_\infty \xrightarrow{d} ||B||_\infty$.\\
(2) Confidence bands for $F$: we can define $C_n(x)$ for all $x$ in $\R$ around $F_n(x)$, and study $\P(F(x) \in C_n(x) \forall x \in \R) \to $\\
Other application of non-parametric statistics:\\
(1) Regression, wher $y_i = f(X_i) + \varepsilon_i$; unknown function $f$.\\
(2) Density estimation: $X_i \sim P$ have density $f$. In many applications $\E|\hat{f}_n - f| \gg \frac{1}{\sqrt{n}}$.
\end{rem}

---end of lecture notes---

\end{document}


\section{Problem Description}
The problem that we have is the utilization of deep reinforcement learning, with the specific model of convolutional neural networks, trained with Deep Q-Learning, whose input is raw pixel values, varying between \( \{0, 1, \dots, 255\} \), of a grayscale version of the image of the current state, and output is the most suitable action from the action space. The question that we would like to address is whether the trained agent can outperform a random agent, which takes random actions at each time step, and if so whether it can outperform a human agent that plays the game in the same settings.


\documentclass{article}
\pagestyle{empty}
\usepackage{amsmath,amssymb,amsfonts}
\usepackage{graphicx}
\usepackage{multicol}
\setlength{\oddsidemargin}{0in} \setlength{\evensidemargin}{0in}
\setlength{\topmargin}{0in} \setlength{\textheight}{8.5in}
\setlength{\textwidth}{6.5in}

\makeatletter
\renewcommand*\env@matrix[1][*\c@MaxMatrixCols c]{%
	\hskip -\arraycolsep
	\let\@ifnextchar\new@ifnextchar
	\array{#1}}
\makeatother

\begin{document}
\begin{flushleft}
	\bfseries{MATH 260, Linear Algebra, Spring `14}\\
	\bfseries{Activity 12:  Null \& Column Spaces}\\
	\bfseries{Honor Code:} \hspace{3.5in}\bfseries{Names:}\\
\end{flushleft}
\begin{flushleft}
\vspace{.75in}
Directions:  Everyone should work on the assignment and should fill out their paper.  You are expected to make corrections based on what is presented on the board.  \\
If you need more explanations after class, you can read Section 5.4 of your textbook.\\
\vspace{0.1in}
\Large
In-Class Learning Goals:\\
\normalsize
\begin{enumerate}
\item Be able to find the null space/kernel of any matrix
\item Be able to give the column space for any matrix
\item Be able to give a basis for the column space of a matrix
\item Be able to describe how the null space and column space relate to each other
%\item Be able to determine if a set of vectors is a basis for a vector space
%\item (stretch) Be able to find the dimension of a set or space (by the end of homework for sure!)

\end{enumerate}

\vspace{0.1in}

\section*{Warm-up:  In the null space?}
%This is the matrix from Sect. 5.4, pg 339, #43
Recall that the null space is all vectors $\vec{x}$ such that $\textbf{A}\vec{x}=\vec{0}$. Given the matrix:\\
\begin{center}
$\textbf{G}=
\begin{bmatrix}
2 & 3 & -1\\
-1 & 4 & 6 \\
1 &  7 & 5
\end{bmatrix}
$\\
\end{center}
1a) Determine which of the following vectors are in the null space of \textbf{G}.\\
\begin{center}
$ \vec{w}_1=\begin{bmatrix} 3 \\ 2 \\ 1 \end{bmatrix} $
\hspace{0.3in}
$ \vec{w}_2=\begin{bmatrix} 8 \\ -4 \\ 4 \end{bmatrix} $
\hspace{0.3in}
$ \vec{w}_3=\begin{bmatrix} -2 \\ 1 \\ -1 \end{bmatrix} $
\end{center}

\vspace{2in}

1b) Find the actual null space of \textbf{G}. Do your answers above make agree?

\newpage

\section*{A useful Null Space}
Given the matrix $\textbf{F} = \begin{bmatrix} 1 & 2 \\ 3 & 0 \end{bmatrix}$ with eigenvalues of -2 and 3.

\vspace{0.1in}

2) Find the null space of the matrix $(\textbf{F} + 2 \textbf{I} )$.

\vspace{1.25in}

3) Find the null space of the matrix $(\textbf{F} - 3 \textbf{I} )$.

\vspace{1.25in}

4) Have you seen these null spaces (or their calculations) before? What do the null spaces you found in (2) and (3) have to do with the matrix \textbf{F}?

\vspace{1in}

\section*{Basis for Column Space}
We know that the column space ( $span( \{ \vec{v}_1 , \vec{v}_2, \ldots \vec{v}_n \} )$ ) is a subspace. However, generally in mathematics we want to describe things in as simple terms as possible. For spaces, this means giving a \textit{basis} only for a space. \\
Recall that a set is a \textbf{basis} of a vector space if it has the properties:\\
(i) The set is linearly independent \hspace{.25in} (ii) The span of the set covers the entire vector space.\\

\vspace{0.1in}

5) The column space of \textbf{G} could be given as \textit{span}$ \left( \left\{ \begin{bmatrix} 2 \\ -1 \\ 1\end{bmatrix}, \begin{bmatrix} 3 \\ 4 \\ 7 \end{bmatrix}, \begin{bmatrix} -1 \\ 6 \\ 5 \end{bmatrix} \right\} \right)$. Does the set $S=\left\{ \begin{bmatrix} 2 \\ -1 \\ 1 \end{bmatrix}, \begin{bmatrix} 3 \\ 4 \\ 7 \end{bmatrix}, \begin{bmatrix} -1 \\ 6 \\ 5 \end{bmatrix} \right\}$ form a basis for the column space? (Show why or why not)

\vspace{1.5in}

We already have all the tools and information to define a valid basis. Let's do so...

\vspace{0.1in}

6) Which columns in the RREF of \textbf{G} are pivot columns?

\vspace{0.75in}

7) Take the columns from the original \textbf{G} that you identified as the pivot columns. Form a new set $S_{B}$ from these columns. These should be $3 \times 1$ vectors with non-zero values in each row.  \textit{Be sure to show work or explain your reasoning below, a yes/no answer for any of the below will NOT receive credit.}

\vspace{0.1in}

7a) Is this new set $S_{B}$ linearly independent?

\vspace{1.5in}

7b) Does it span the entire column space? ( \textit{Hint: Does $span( S ) = span ( S_{B} ) $ } )

\vspace{1.5in}

7c) Is the set $S_{B}$ a basis for the column space? Explain.

\vspace{0.75in}

8) We defined the \textbf{rank} of a matrix as the number of pivot columns in RREF. How does the rank of \textbf{G} relate to the column space of \textbf{G}? (\textit{Hint: What property of a space gives a single value out?} )

\vspace{1.6in}

\Large Note this relationship between rank and column spaces is actually true for any matrix. \normalsize

\newpage

\section*{Rank-Nullity Theorem}
9) For the CPA you found the null space and column-space of $\textbf{A}=\begin{bmatrix} 1 & -2 \\ 1 & -2 \end{bmatrix}$.\\
A basis for the column space is $\begin{bmatrix} 1 \\ 1 \end{bmatrix}$. A basis for the null space is $\begin{bmatrix} 2 \\ 1 \end{bmatrix}$.

\vspace{0.15in}

9a) What is the dimension of the null space and column space for \textbf{A}?

\vspace{0.75in}

9b) How do the dimensions of \textbf{F} relate to the sum: dim( column space of \textbf{A}) + dim( null space of \textbf{A} ) ?

\vspace{1in}

10a) State the dimension of the null space (the \textit{nullity} of \textbf{G} ) and the dimension of the column space for matrix \textbf{G}.

\vspace{0.75in}

10b) How do the dimensions of \textbf{G} relate to the sum: dim( column space of \textbf{G} ) + dim( null space of \textbf{G}) ?

\vspace{1in}

11) The \textbf{Rank-Nullity Theorem} generalizes the above results for the $m\times n$ matrix \textbf{A}. Based on your results, what do you think the \textbf{Rank-Nullity Theorem} is? (Take a guess BEFORE looking this up! You should check with your professor or check your text once you make a guess.)

\vspace{0.75in}

%\section*{Row-Spaces and Left-Nullspace}
%Just as each matrix has a column-space, it also has a row-space. These row spaces are defined analogously. While we could define a whole new set of column operations there's an easier way...
%
%\vspace{0.1in}
%
%7a) Find the transpose of matrix \textbf{G} $\rightarrow \textbf{G}^{T}$.
%
%\vspace{1in}
%
%7b) How are the rows of \textbf{G} related to the columns of $\textbf{G}^T$?
%
%\vspace{1in}
%
%7c) Find a basis for the column-space of $\textbf{G}^T$.
%
%\vspace{1.5in}
%
%8) Explain why the row-space of \textbf{G} and the column-space of $\textbf{G}^T$ are equivalent.
%
%\vspace{1in}
%
%9) Find the null-space of $\textbf{G}^{T}$.
%
%\vspace{1.5in}
%
%This is called the \textit{left nullspace} because it is equivalent to finding the set of vectors $\vec{x}$ which satisfy $\vec{x}^{T} \textbf{A} = \vec{0}$. Notice we have moved $\vec{x}$ to the left-side of \textbf{A}.
%
%\vspace{0.2in}
%
%10a) What is the dimension of the row-space and left-nullspace of matrix \textbf{G}?
%
%\vspace{0.75in}
%
%10b) How do the dimensions of \textbf{G} relate to the sum: dim( Row-Spc ) + dim( Left-Nullspc) ?
%
%\vspace{1in}
%
%\section*{Fundamental Spaces}
%11) Detail the four spaces you found above for \textbf{G}. Give their name, dimension and basis. (A 4 x 3 table with column labels would be perfect).
%
%\vspace{3in}
%
%You have now found the four fundamental spaces of the matrix \textbf{G}! These four spaces can be found for any matrix and are used in a variety of operations. We'll look into this a little next week.
\end{flushleft}
\end{document}

Spring 2014 - Schmitt/Activity Latex/Activity_12_nullcolumnspace.tex