\documentclass{article}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{bm}
\usepackage{enumitem}
\usepackage{float}
\usepackage{fullpage}
\usepackage{graphicx}
\usepackage{hyperref}
\usepackage{tabularx}

\title{On Demand Transportation Scheduling \\ \large{Progress Report}}
\author{Weini Yu\\\texttt{weiniyu}\and Zhouchangwan Yu\\\texttt{zyu21}\and Yutian Li\\\texttt{yutian}}

\begin{document}
\maketitle

\section{Introduction}
Our project aims to develop an intelligent model that provides a solution to the single vehicle pickup and delivery problem (SVPDP). For now we are only going to look at the pickup and delivery problem of a single vehicle and without considering carpool situation.

\section{Model}
We have approached this problem using an MDP model, with the intuition that at a certain location (state), the driver could choose to take one of the requests that come in (action). For each action the driver receives a reward. At the end of the day, the driver has a total reward for all the rides he or she takes. And through solving this MDP, the driver can figure out which request to take at a location in order to maximize the total reward.

\subsection{State}
We originally had states that include the driver's current location and a list of requests at the location. However, this makes the state space very large ($10^{5}$) which is not desired. Also the relative order of requests does not really matter. So we have simplified it such that it only contains the driver's current location, a zone index number, reducing the state space to $10$. For example, \texttt{state=1} means that the driver is at location $1$. Note that we are not losing anything by not including requests. Requests are encoded into actions.

\subsection{Action}
Since state is the current location the driver is at, the action would be the possible request the driver receives, which is a tuple \texttt{(source, destination)} where source is the pick up zone index and destination is the drop off zone index. In our specific model, the driver is given $5$ requests to choose from at the start of the game and the end of each ride. So at any step, there are $5$ legal actions. And the driver must choose one from them.

The requests are generated in the following way. The source location is randomly sampled with lower location indices having a higher weight. The destination location is sampled uniformly randomly. By generating requests in a nonuniform way, we favor locations that have smaller indices and create an imbalance between states. We hope that the agent will figure out some locations are inherently better, and use this knowledge to achieve higher rewards.

\subsection{Reward}
The reward of each action or ride will be $(\text{fare}-\alpha\cdot\text{travel time})$. The intuition here is that gas cost needs to be subtracted and the longer the travel time is the less happy the customer is so there is the negative impact on the reward. The constant $\alpha$ balances fare and travel time, and could be seen as a measure of productivity. In our specific model, $\alpha$ is chosen to be $\frac{1}{150}$.

\section{Algorithm}
\subsection{Baseline and Oracle}
Based on this model, we implemented baseline as a driver that randomly picks one request at each step. And the oracle is a driver that knows the true reward function for all the requests, and picks greedily according to it.

\subsection{Q-learning}
For our agent, we used Q-learning as a first step to solve this problem. For a smaller sized problem we could run exhaustive search, but it is clearly not scalable. We chose Q-learning instead of TD learning, because intuitively states do not matter that much in our model. Though some states are inherently better due to the request generation probability, actions are much more important. Besides, if a driver takes a request and returns to the same location, the state is same but clearly the total reward is not.

For function approximation, we used linear regression to predict the Q value. As a first step, we manually picked a few features.

\subsubsection{Features}
\noindent\textbf{Distance}:
The travel distance is one of the main concern for the drivers. Here we define two features related to the travel distance, the distance from current location to source, and the distance from source to destination.

\noindent\textbf{Zone}: Some zones are more attractive to people than others, so the drives may have preference on where they would like to go. Each zone as current location, source, or destination are all considered as features.

We define indicator features $\bm{1}\{\text{current zone}=z\}$, $\bm{1}\{\text{source zone}=z\}$, and $\bm{1}\{\text{destination zone}=z\}$ for all $z$'s.

\subsubsection{Training}

Using features described above, we learned the respective weights using gradient descent. We updated weights using exponential weighted moving average as $w_{i}\leftarrow w_{i}-\eta(Q(s,a)-r)\phi_{i}(s,a)$, where $\eta=0.1$. Note that we set discount $\gamma=0$. This is because there is no way for the driver to know the legal actions (requests) available at future states. They are randomly generated only when the driver gets to a new state. $\max_{a}Q(s,a)$ is hard to define, hence $\gamma=0$ for simplicity.

\section{Data Preprocessing}

At the end of data preprocessing stage, we generate a \texttt{city.p} pickle file that has everything we need about the environment. It has a list of locations, travel times between locations, fare estimates, and coordinates of the locations.

\noindent\textbf{Locations (a.k.a. zone indices)}: We downloaded from Uber Movement travel times in Washington D.C. during second quarter of year 2017. There are 558 zones in total, and we picked the first 10 zones for downsampling.

\noindent\textbf{Travel times}: We picked the arithmetic mean travel time as the travel time between zones. The distribution of travel times is shown below.

\begin{figure}[H]
\begin{center}
\includegraphics[width=0.6\textwidth]{images/time.png}
\end{center}
\caption{Distribution of traveling time.}
\end{figure}

\noindent\textbf{Coordinates}: We calculated the arithmetic mean longitude and latitude as coordinates of the locations.

\noindent\textbf{Fare}: The fares between these 10 locations are requested form Uber Developers API. Given the longitudes and latitudes of any two locations, the low estimate and high estimate of the fare are generated for various types of vehicles. We use the average of the fare for UberX and keep the data in a $10\times 10$ matrix. The distribution of fare estimates are shown below.

\begin{figure}[H]
\begin{center}
\includegraphics[width=0.6\textwidth]{images/fare.png}
\end{center}
\caption{Distribution of estimated fare.}
\end{figure}

\section{Example}

For a concrete example, we assume the driver is at location $s$. Five requests $(s_{i},d_{i})$ for $1\leq i\leq 5$ are dispatched to the driver. The driver evaluates $Q(s,(s_{i},d_{i}))$ for all five requests using linear estimator $\sum_{i}w_{i}\phi_{i}(s,a)$ and picks the one with the highest estimation. The driver takes this action. Then the reward $r$ is given to the driver, who in turn uses $w_{i}\leftarrow w_{i}-\eta(Q(s,a)-r)\phi_{i}(s,a)$ to perform gradient descent to update the weights $w$.

\section{Results}

We trained our algorithm and compared to baseline and oracle. The resulting reward curve is shown below.

\begin{figure}[H]
\begin{center}
\includegraphics[width=0.6\textwidth]{images/reward.png}
\end{center}
\caption{Reward curve.}
\end{figure}

At first our algorithm is acting randomly because all the weights are just initialized. Then quickly it ramps up and learns the Q function. At iteration 100, it is already receiving twice the reward of baseline, but still about $30\%$ worse than the oracle.

\section{Next Steps}
We will first implement our Q-learning algorithm on a larger sample size (100 zone locations) to see how it performs as the problem is getting more complicated. We will also considering more advanced learning algorithm. We could try to use a neural network instead of hand craft linear features.

\end{document}


\subsection{The Classical Definition of Entropy}
Jumping right into things, entropy, for large systems, can be defined as
\begin{equation}
    dS=\frac{\delta Q_{rev}}{T}
\end{equation}
Where $dS$ is the change in entropy, $\delta Q_{rev}$ is heat reversibly flowing to/from a system, and $T$ is the temperature of the system. Just stating it right here, this equation probably seems to make \textbf{no} sense whatsoever, so let's step back a bit to build up to it.
\input{Entropy/reversableprocesses}
\input{Entropy/secondlaw}
\input{Entropy/thirdlaw}
\input{Entropy/carnotrevisited}


\section{Particle Filter}
\label{sec:particleFilter}

This course introduced two main techniques for localization of the robots current position, these were Kalman Filter and Particle Filter, as described in chapter \ref{chp:kalman} and chapter \ref{chp:partFilter} respectively.
For the localization in this project, particle filters were chosen, since it doesn't require landmarks and landmark recognition.

The implementation of the particle filter was separated into two C\# classes, namely a \emph{Particle} class and a \emph{ParticleFilter} class, as shown in figure \ref{fig:partUML}.

The Particle class serves as a definition of the properties of each individual particle, i.e. the X and Y coordinates, its orientation represented in radians by the variable \emph{theta} and the particle's weight.
The constructor of a Particle object takes the world size as arguments, and generates random X- and Y-coordinates within the given world, along with a random orientation.

The functions \emph{PostionNoise} and \emph{ThetaNoise} are functions for adding a Gaussian noise to the particle, in order to simulate motion uncertainties of the robot.

\myFigure{ParticleFilter/partUML}{UML class diagram of the particle filter}{fig:partUML}{0.4}

\noindent The ParticleFilter class it self, implements the actual functionality of the particle filter.
The constructor calls the function \emph{GenerateParticleSet()}, which generate a particle set of $N$ particles. While doing so, the function utilized the Map function \emph{IsPointInSquare()} to avoid placing particles inside a world object.\\\\
%
\emph{MoveParticles()} takes a distance argument, which corresponds to how much the robot was told to move forward and moves the particles forward the same amount.
This is done by calculating exactly how much in the X- and Y-direction the specific particle should move, according to its orientation.
When the particle have been moved, the noise functions of the Particle class is called, to simulate the uncertainties.
Finally the move function checks if a particle have been moved into a world object, if so, the particle is discarded and replaced by a new particle, with a completely random position and orientation.
This feature was chosen to apply some form of particle redistribution.\\\\
%
The function \emph{Resample()} implements the resampling process described in chapter \ref{chp:partFilter}.
This is done by setting each particles weight to the probability of particle being correct, given the particle's position and orientation compared to the measured distance at the given resampling time.
These weights are then normalized before the actual resampling algorithm is applied.\\\\
%
In order to give the best approximation of the actual robot position the ParticleFilter provides the function \emph{getPosition()} which returns a Particle object, because the Particle object hold information about both X,Y-coordinates and orientation.
The approximation of the robot position is implemented as the average position and orientation of all the particles in the set, not taking into account the particle's weight or any potential clusters.
This was chosen for a fast and easy implementation.

\newpage


\vsssub
\subsubsection{~$S_{uo}$: Unresolved Obstacles Source Term} \label{sec:UOST}
\vsssub

\opthead{UOST}{\ws}{L. Mentaschi}

Unresolved bathymetric and coastal features, such as cliffs, shoals and small islands,
are a major source of local error in spectral wave models.
Their dissipating effects can be accumulated over long distances, and
neglecting them can compromise the simulation skill on large portions of the domain
\citep{tol:Waves01a,tol:WaF02,art:Tuomi2014,art:Mentaschi2015a}.
In \ws\
two approaches are available for subscale modelling
the dissipation due to unresolved obstacles:
a) a propagation-based approach established for regular grids, described in section
\ref{sub:num_obst} \citep{tol:OMOD03a, tol:OMOD08a}; and b) the
Unresolved Obstacles Source Term \citep[UOST,][]{art:Mentaschi2018a,art:Mentaschi2015b}
described here.
In addition to supporting virtually any type of mesh,
UOST takes into account the directional/spatial layout
of the unresolved features, which can improve the model skill
with respect to the established approach \citep{art:HMM00,art:Mentaschi2018b}.

UOST relies on the hypothesis that any mesh can be considered as a set of polygons,
called cells, and that the model estimates the average value of the unknowns
in each cell.
Given a cell (let us call it A, figure \ref{fig:UOST}ab) UOST estimates, for each spectral component,
the effect of a) the unresolved features located in A (Local Dissipation, LD);
b) the unresolved features located upstream of A, and projecting their shadow on A
(Shadow Effect, SE). For the estimation of SE,
an upstream polygon A\textsc{\char13} is defined for each
cell/spectral component, as the intersection between the joint cells neighboring A
(cells B, C and D in figure \ref{fig:UOST}ab), and the flux upstream of A.
For each cell or upstream polygon, and for each spectral component,
two different transparency coefficients are estimated.
1) The overall transparency coefficient $\alpha$; and
2) a layout-dependent transparency $\beta$, defined as the average transparency
of cell sections starting from the cell upstream side.

The source term can be expressed as:

\begin{equation}
S_{uo} = S_{ld} + S_{se}\ ,
\end{equation}

\begin{equation}
S_{ld} = - \psi_{ld}(\mathbf{k}) \frac{1 - \beta_l(\mathbf{k})}{\beta_l(\mathbf{k})} \frac{c_g(\mathbf{k})}{\Delta L} N \ ,
\end{equation}

\begin{equation}
S_{se} = - \psi_{se}(\mathbf{k}) \bigg[ \frac{\beta_u(\mathbf{k})}{\alpha_u(\mathbf{k})} - 1 \bigg] \frac{c_g(\mathbf{k})}{\Delta L} N \ ,
\end{equation}

where $S_{ld}$ and $S_{se}$ are the local dissipation and the shadow effect,
$N$ is the spectral density,
$\mathbf{k}$ is the wave vector,
$c_g$ is the group velocity, $\Delta L$ is the path length of the spectral component
in the cell, and the $\psi$ factors model the reduction of the dissipation in presence of
local wave growth.
The subscripts l and u of $\alpha$ and $\beta$ indicate that these coefficients
can be referred, respectively, to the cell and to the upstream polygon.
For a more detailed explanation on the theoretical framework of UOST,
the reader is referred to
\citep{art:Mentaschi2015b, art:Mentaschi2018a}.

\begin{figure} \begin{center}
\epsfig{file=./eqs/UOST.eps,angle=0,width=4in}
\caption{
a: a square cell (A) and its upstream polygon
(A\textsc{\char13}, delimited by blue line, in light blue color) for a spectral
component propagating with group velocity $c_g$.
The joint BCD polygon represents the neighborhood polygon.
b: same as a, but for a triangular mesh (the hexagons approximate the median dual cells).
c: Computation of $\alpha$ and $\beta$ for a square cell, $N_s=4$,
and a spectral component propagating along the x axis.
d: Like c, but for a hexagonal cell and for a tilted spectral component.
In panel d the gray squares represent unresolved obstacles.
}
\label{fig:UOST} \botline
\end{center}
\end{figure}

\textbf{Automatic generation of mesh parameters.}
An open-source python package (alphaBetaLab, https://github.com/menta78/alphaBetaLab)
was developed
for the automatic estimation, from real-world bathymetry, of the upstream polygons,
of the transparency coefficients
$\alpha_l$, $\beta_l$, $\alpha_u$, $\beta_u$,
and of the other parameters needed by UOST.
alphaBetaLab considers the cells as free polygons, and estimates the transparency coefficients
from the cross section of the unresolved obstacles versus the incident spectral component
(figure \ref{fig:UOST}cd).
This involves, that it can be applied to any type of mesh,
including unstructured triangular and SMC meshes
(as of August 2018 only regular and triangular meshes are handled,
but support for SMC meshes
will be soon added).
We need to mention that while UOST would be able to modulate the energy dissipation
with the spectral frequency, only the direction is currently considered in alphaBetaLab.
For more details on the algorithms implemented in alphaBetaLab,
the user is referred to \cite{art:Mentaschi2018a}.
\cite{art:Mentaschi2018c} provides the documentation of the software and of its architecture,
along with use guidance and illustrative examples.

\textbf{Time step settings.}
In \ws\ the source terms are applied at the end of each global time step.
Therefore, to work properly on a given cell,
UOST needs a global time step lower
or equal to the critical CFL time step of the cell,
i.e., the amount of time needed by the fastest spectral component to entirely cross the cell.
Otherwise, part of the energy will leak through the cell without being blocked \citep{art:Mentaschi2018a}.

In unstructured grids with cells of very different sizes, the application of UOST to
all the cells, including the smallest ones, may come with an excedingly small global time step
that would affect the economy of the model.
To avoid this problem the user can set alphaBetaLab in order to
neglect cells smaller than a user-defined threshold, and then set in \ws\
a global time step equal to the critical CFL time step related with that threshold
\citep{art:Mentaschi2018c}.

\begin{table} \begin{center}
 \footnotesize
\begin{tabular}{|p{4cm}|p{2.5cm}|p{5.5cm}|} \hline \hline
Namelist parameter    &  Description           & default value \\
\hline
  UOSTFILELOCAL &  Local $\alpha$/$\beta$ input file path   &  obstructions\_local.\textit{gridname}.in    \\ \hline
  UOSTFILESHADOW &  Shadow $\alpha$/$\beta$ input file path   & obstructions\_shadow.\textit{gridname}.in   \\ \hline
  UOSTFACTORLOCAL &  Calibration factor for local transparencies  &  1   \\ \hline
  UOSTFACTORSHADOW &  Calibration factor for shadow transparencies  &  1  \\
\hline
\end{tabular} \end{center}
\caption{UOST parameters, their description and default values. } \label{tab:UOST}
\botline
\end{table}


\section{Results}

\subsection{Stability of Hopfield Network}
\begin{figure}[H]
    \centering
    \captionsetup[subfigure]{width=0.9\textwidth, justification=raggedright}
    \begin{subfigure}{0.49\textwidth}
        \includegraphics[width=\textwidth]{figs/stable}
        \caption{By initializing the network to the stored pattern, the network will remain in an equilibrium. The network will never deviate from this state without external interference. If the network is initialized to a noisy state, the network will converge towards the stored pattern.}
    \end{subfigure}
    \begin{subfigure}{0.49\textwidth}
        \includegraphics[width=\textwidth]{figs/stable-energy}
        \caption{With a single stored pattern, the energy will converge towards the lower limit according to \cref{eq:energy-limits}. In this case with 50 neurons, the lower limit is $-\frac{50*\times 49}{2} = -1225$}.
    \end{subfigure}
    \caption{Initializing a Hopfield network with a single stored pattern ($N = 50$) will cause the network to converge towards the stored pattern.}
    \label{fig:stable}
\end{figure}
To test whether the Hopfield network is stable after reaching an equilibrium we simulated the Hopfield network storing a random pattern using $N=50$ neurons. We initialized the network to the stored pattern, once with and once without added noise, as shown in \cref{fig:stable}. The network remained stable at the equilibrium point, and converged towards the equilibrium if not perfectly intialized. With a single pattern stored, the energy converged toward the lower limit expressed in \cref{eq:energy-limits}.

\subsection{Storing multiple patterns}
\begin{figure}[H]
    \centering
    \captionsetup[subfigure]{width=0.9\textwidth, justification=raggedright}
    \begin{subfigure}{0.49\textwidth}
        \includegraphics[width=\textwidth]{figs/multiple-patterns.eps}
        \caption{By initializing the weight matrix $\bf W$ with multiple patterns, the network is able to restore multiple different patterns. The amount of memories and noise the network is able to handle depends on the size and similarity of the stored patterns. If two stored patterns only differ by a few bits, the network may recall the wrong memory if noise is involved. }
        \label{fig:multiple-similarity}
    \end{subfigure}
    \begin{subfigure}{0.49\textwidth}
        \includegraphics[width=\textwidth]{figs/multiple-patterns-energy.eps}
        \caption{The network will always move in the direction of least resistance such that the overall energy decreases, similar to how a ball will always roll downwards into a valley unless there are external forces interfering. Each stored memory corresponds to a local minimum which can be thought of as a valley. As long as the network is initialized to the correct valley, it is able to reconstruct the stored pattern from memory.}
        \label{fig:multiple-energy}
    \end{subfigure}
    \begin{subfigure}{0.49\textwidth}
        \includegraphics[width=\textwidth]{figs/capacity.eps}
        \caption{When storing a large amount of random patterns in the network, the robustness decreases and the network will not be able to reliably restore the correct memories. The figure shows the proportion of correct memories, recalled for different number of stored memories and different noise levels.}
        \label{fig:multiple-capacity}
    \end{subfigure}
    \caption{Simulating the Hopfield network with multiple stored patterns.}
    \label{fig:multiple}
\end{figure}
To test whether the network remains stable even with multiple patterns stored in the weights, we simulated a new network storing two uncorrelated random patterns ($N=50$). We simulated using both patterns as initial conditions as well as simulating with and without noise. In total we ran 4 simulations and the results are shown in \cref{fig:multiple}. The states converged towards the correct pattern in all cases, and remains stable at the equilibrium. The energy in the equilibrium, as shown in \cref{fig:multiple-energy}, was half the lower limit in \cref{eq:energy}.

To test how many pattern we can reliably store, we simulated the Hopfield with different number of memories and noise levels. In \cref{fig:multiple-capacity} we see how the networks ability to recall the correct memory decreaes as both the number of memories and noise level increases. With low noise levels we found that we can store $6$ memories reliably using $N=50$ neurons. For higher noise levels the robustness quickly decreases.

\subsection{Reconstructing partial QR codes from memory} \label{sec:qr-codes}
To better visualize the Hopfield networks ability to store and restore patterns, we constructed a Hopfield network for storing QR-codes. We stored two QR-codes in the network and initialized the network with only partial information. Fig. \ref{fig:qr-codes} shows how the network was able to restore the correct QR code from memory, while \cref{fig:qr-codes-stability} shows how the network remained stable after reaching the equilibrium.
\begin{figure}[H]
    \centering
    \captionsetup[subfigure]{width=0.9\textwidth, justification=raggedright}
    \begin{subfigure}{0.49\textwidth}
        \includegraphics[width=\textwidth]{figs/qr-code}
        \caption{The QR-code contains information encoded within the patterns. A QR decoding app, either online or on a smartphone, can be used to decode the "hidden" message. Using the Hopfield network we are able to reconstruct a destroyed QR-code from memory.}
        \label{fig:qr-codes}
    \end{subfigure}
    \begin{subfigure}{0.49\textwidth}
        \includegraphics[width=\textwidth]{figs/qr-code-sim}
        \caption{The similarity between the current state of the network and the original QR-code we want it to reconstruct.}
        \label{fig:qr-codes-stability}
    \end{subfigure}
    \caption{By applying the Hopfield network to a very noisy QR-code we are able to reconstruct the original. By using any QR-code reader we are able to decode the original and reconstructed QR-codes, while the middle ones contain too much noise. The same Hopfield network was used to reconstruct both patterns.}
\end{figure}

\subsection{Inverting patterns}
According to the weights, \cref{eq:weights}, the Hopfield network only stores patterns and not the actual values in the memories. A pattern $\bf V$ and its inverse $\mathbf{\bar{V}} = -1 \times \bf V$ would yield the exact same weight matrix. For each stored memory there should be two equilibrium states, one for $\bf V$ and one for $\bf \bar{V}$. To test whether this truly is the case, we used the same Hopfield network as in \cref{sec:qr-codes} and initialized the states to the second QR code, but inverting all bits. We ran another simulation using the QR-code with two-thirds of its bits inverted. As can be seen from the results in \cref{fig:inverted-qr} both cases caused the Hopfield network to reconstruct the inverted pattern, and not the ones we originally created. This shows how for each stored pattern there exists two equilibrium points.
\begin{figure}[H]
    \centering
        \includegraphics[width=0.5\textwidth]{figs/qr-inverted}
        \caption{The Hopfield network learns the patterns and the network is not able to distinguish between positive and negative values. The network is not only stable at the initial memories, but also when using the inverted patterns.}
        \label{fig:inverted-qr}
\end{figure}


\documentclass{article}
\usepackage{enumerate}
\usepackage{amsmath, amsthm, amssymb}
\usepackage[margin=1in]{geometry}
\usepackage[parfill]{parskip}
\DeclareMathOperator*{\argmax}{arg\,max}

\title{Econ C103 Problem Set 5}
\author{Sahil Chinoy}
\date{February 28, 2017}

\begin{document}
\maketitle{}

\subsection*{Exercise 1}

\begin{enumerate}[(a)]
	\item

	Each agent $i \in \{1,2\}$ has type indicated by the value of the object that they have at the beginning of the game, $\theta_i \in \Theta = [0,1]$, and can send a message from $M_i = \{0,1\}$ indicating whether they want to exchange objects. If both agents choose $m_i = 1$, then the trade takes place, which we denote with $x = 1$. So the allocation rule is

	\begin{equation*}
	x(m) =
		\begin{cases}
	      0 & \text{if } m_1 = 0 \text{ or } m_2 = 0 \\
	      1 & \text{if } m_1 = m_2 = 1
	   \end{cases}
	\end{equation*}

	and each agent's utility is

	\begin{equation*}
	u_i(x, \theta_i) =
		\begin{cases}
	      \theta_i & \text{if } x = 0 \\
	      \theta_{-i} & \text{if } x  = 1
	   \end{cases}.
	\end{equation*}

	\item

	Trading will never be Pareto efficient, so the only Pareto efficient outcome is the original allocation, $x = 0$. If $\theta_1 \neq \theta_2$, trading would make one agent worse off, since both agents assign the same value to the objects. If $\theta_1 = \theta_2$, then trading won't change either agent's utility and thus won't make either agent strictly better off.

	Either allocation ($x = 0 \text{ or } 1$) maximizes the sum of the agents' utilities, which is always $\theta_1 + \theta_2$.

	\item

	The best response of each agent depends on the message the other agent sends and the other agent's type. If $m_{-i} = 0$, then trade will never take place, so it doesn't matter what message the agent chooses; we will pick $m_i = 0$. Then the best response for agent $i$ is

	\begin{equation*}
	\sigma_i(m_{-i}, \theta) =
		\begin{cases}
	      0 & \text{if } m_{-i} = 0 \text{, or } m_{-i} = 1 \text{ and } \theta_i \geq \theta_{-i} \\
	      1 & \text{if } m_{-i} = 1 \text{ and } \theta_i < \theta_{-i}
	   \end{cases}.
	\end{equation*}

	Thus the agent's best response depends on $m_{-i}$ and $\theta_{-i}$. This means that there is no dominant strategy equilibrium.

	\item

	Given that the types $\theta$ are uniformly distributed on $[0,1]$, each agent initially expects the other agent's type to be $\mathbb{E}[\theta_{-i}] = 0.5$. This means it is rational to trade, i.e. send the message $m_i = 1$, only if $\theta_i  < 0.5$. But then the expected payoff from the trade for the \textit{other} agent is $\mathbb{E}[\theta_{-i} | m_{-i} = 1] = 0.25$, and if the first agent expects the other agent to play optimally, their expected payoff is now $\mathbb{E}[\theta_{-i} | m_{-i} = 1] = 0.125$, and so on. So trading will never be optimal.

	Sending the message $m_i = 0$ is a Bayes-Nash equilibrium, however, because given that every agent expects the other agents to play $\sigma_i(\theta_i) = 0$, a trade can never occur, so it is never in anyone's interest to send the message $m_i = 1$ as it will not change the outcome. Formally, $\forall m_i \;\mathbb{E}[ u_i(x(\sigma_i, \sigma_{-i}), \theta_i) | \theta_i]  = \theta_i = \mathbb{E}[u_i(x(m_i, \sigma_{-i}), \theta_i) | \theta_i]$. So $\sigma_i(\theta_i) = 0$ is a Bayes-Nash equilibrium.

\end{enumerate}

\end{document}