\section{Introduction} % (fold)
\label{sec:introduction}
There has now been over 60 years since Markov decision processes have proven to
be useful in decision making and that associated \enquote{\textit{models have
gained recognition in such diverse fields as ecology, economics,
and communications engineering}}~\cite{puterman2014markov}.
They are therefore an important topic to study and this report presents
our key results applied in the framework of a \emph{Snakes and Ladders} game.

This report will first present in section~\ref{sec:model_and_strategy},
a detailed description of our Markov \emph{model}
as well as the strategy we followed to get the optimal choice of dice.
Then, section~\ref{sec:implementation} will present our concrete \emph{implementation}
of the game and its simulations.
Finally, the most relevant results and strategies comparison as well as
conclusion and further work discussion will be described in sections~\ref{sec:results}
and~\ref{sec:conclusion}.

% section introduction (end)


\chapter{Preliminary work}
\label{chapter:pre_work}
In order to compute the metrics explained in the previous chapter, it is necessary to know the position and orientation (pose) of the contact points located on the phalanges of Vizzy.
\par

It is important to note that through the Denavit Hartenberg parameters and the angles of rotation of each joint it is possible to obtain the position and orientation of each sensor in 3D coordinates.

\par
It is known that the tactile sensors have incorporated sensors with Hall-effect that detects changes in the magnetic field caused, either by the magnet in the elastomer or by the Earth's magnetic field.
In the case where the sensors are not pressed, the influence of the magnetic field is constant, leaving only the Earth's magnetic field responsible for the change of the magnetic field of the sensor.

\par

The idea that came up was that, through the magnetic field readings, be able to discover the position and orientation of the phalanges. To test this idea, an experiment was carried out, in which it consists on obtaining data of the magnetic fields of each sensor placed in each phalanx and the angle of rotation between that same phalanx and the palm of the hand.
With these data, a relationship (through a regression) between each of the 3D components of the magnetic field from the sensor and its angle of rotation relative to the hand will be inferred.
\par

To obtain the ground-truth of the rotation angles, it was used \textit{Aruco} markers that were placed on each of the phalanges. \textbf{(visible in figure xxx).}
It is important to note that the first tests were performed with only one marker on each phalanx, which led to poses with a lot of uncertainties and in some cases the data did not make great sense. To correct these uncertainties were used markers boards ( 4 single markers in a column), which led to results with almost no uncertainty.

\par
These boards give their poses, which with the help of a function of ROS transformations it is obtained the angles of rotation between the board on the phalanx and the board on the anterior phalanx (towards the palm), this is done for each one phalanx of the index finger. These rotation angles will coincide with those of the joints.

\par

In the experiments that took place, the palm of Vizzy's hand was turned in the direction of the ground (i.e. the palm of the hand was parallel to the plane of the ground).
Then, starting from the point where the hand was open, Vizzy began to close his index finger to its final limit, later he opened his finger again until it was fully extended.
Finally, the data of the tactile sensors and the joint angles were analyzed.

In order to understand the graphics of the 3D components of the magnetic field as a function of the angles of rotation, the numbering of the finger sensors in \textbf{Figure xxx}.
\par
The graphics of the magnetic field components as a function of the angles of rotation are shown in Figures \ref{fig:1and2Joint} and \ref{fig:3Joint}. It is noted that there is only a direct relationship between the components of the sensors of each phalanx and the angle of rotation of its previous joint.

It is yet visible that there are components with more noise than others, such is the example of the Y component of the magnetic field of the Figures \ref{fig:2Joint_5sensor} and \ref{fig:3Joint_4sensor}.
It is important to note that in this report the equations of these graphics have not yet been defined because it is still necessary to remove the effect of the earth's magnetic field in relation to the orientation of the pulse.
To take this effect, will be placed an inertial sensor will be placed on the head of Vizzy. Moreover this work will be developed during the thesis.
\begin{figure}
\centering
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=1\linewidth]{images/1Joint_3sensor.png}
  \caption{\nth{1} Joint and the \nth{3} Tactile Sensor}
  \label{fig:fig:1Joint_3sensor}
\end{subfigure}%
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=1\linewidth]{images/2Joint_5sensor.png}
  \caption{\nth{2} Joint and the \nth{5} Tactile Sensor}
  \label{fig:2Joint_5sensor}
\end{subfigure}
\caption{Graphics of the magnetic field components as a function of the \nth{1} and \nth{2} rotation angles }
\label{fig:1and2Joint}
\end{figure}

\begin{figure}
\centering
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=1\linewidth]{images/3Joint_4sensor.png}
  \caption{\nth{3} Joint and the \nth{4} Tactile Sensor}
  \label{fig:3Joint_4sensor}
\end{subfigure}%
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=1\linewidth]{images/3Joint_6sensor.png}
  \caption{\nth{3} Joint and the \nth{6} Tactile Sensor}
  \label{fig:3Joint_6sensor}
\end{subfigure}
\caption{Graphics of the magnetic field components as a function of the \nth{3} rotation angle }
\label{fig:3Joint}
\end{figure}


 \documentclass[../ewet_cwc_report.tex]{subfiles}

\begin{document}

\section{Electrical Design and Controls}
\subsection{Introduction}

\noindent
In the small-scale turbine design, it is critical to identify
the type of the generator that would be used through the
design process because mechanical designs and modeling are
time consuming and constrained by the academic year's available
man hours. It is important to limit complete change of design
course to one or two instances and even that is only affordable
at the initial stage of the design. The team did not have a
ready generator solution from previous project and had no
practical experience in assessing the scale or the needed
parameters in selecting the generator for such a miniature
design. The only notion that was passed down from previous
team was the understanding of the electrical motor KV rating
and importance of minimizing cogging torque at cut in speed.
Thus, these became the starting point parameters in the
generator selection. The team has encountered several
challenges over the power electronics selection process.
Ideas have been tried and discarded, some because component
selection process was not taking into the account the
component's self-power demand, others because the nature of the
component's operation was not fundamentally understood. In some
instances, where a solution was not found by direct approach,
like in voltage regulation attempts, the team has paused on that
front and shifted attention to better understood sections of the
design, allowing for continuous progress. This approach proved
to be very effective, and more elegant and natural solutions
were found as a result.

\subsection{Generator and Load}

\noindent
Faraday's Law indicates that \emph{``Any change in the magnetic
  environment of a coil of wire will cause a voltage (emf) to be
  induced in the coil''}. Formulation \eqref{eq:emf} demonstrates
the physical relationship described by Faraday using
mathematical interpretation,

\begin{equation}
  \text{emf} = -N \frac{\Delta \phi}{\Delta t}
  \text{\,,}
  \label{eq:emf}
\end{equation}

\noindent
where $N$ represents the number of turns in the coil,
$\Delta\phi$ represents a change in magnetic flux over time,
and $\Delta t$ is the change in time.

Magnetic flux is given by formulation \eqref{eq:flux},
\begin{equation}
  \phi = BA \sin\theta
  \text{\,,}
  \label{eq:flux}
\end{equation}

\noindent
where $B$ representing magnetic field produced by opposing
permanent magnets of the rotors, $A$ represents area of the
coiled wire in the stator and $\theta$ represents the angle
between the coil area plane and the magnetic field and since
the coil area and the magnetic field are orthogonal to each
other in the AFPMG.

Formulation \eqref{eq:flux} can be restated as
\begin{equation}
  \phi = BA
  \text{\,.}
  \label{eq:flux-simple}
\end{equation}

According to Lenz's law, the negative sign of
\eqref{eq:emf} indicates that the current produced by the
change in magnetic flux creates its own magnetic field that
opposes the change in flux that produced it. When applying this
notion to a generator this indicates that current induced in the
generator coils will provide an opposing force to the one that
is responsible for the rotation of the generator shaft. Since
the force is rotational it could be restated in terms of opposing
torque $\tau_G$ and compared against aerodynamic torque
$\tau_W$ extracted from the wind,

\begin{equation}
  \tau_W = \frac{P_W}{\omega_{rotor}}
  = \frac{\rho\pi R^3 C_p\paren{\lambda, \theta} V_W^2}{2\lambda}
  \text{\,,}
  \label{eq:tau-wind}
\end{equation}
where $P_W$, in watts, is the power available in the wind and
is given by \eqref{eq:power-wind}, $\omega_{rotor}$ is the
rotational speed in (\unit{rad\per\s}), $\rho$ is the air
density in (kg / m3), $C_p$ is the power coefficient expressed
as a function of the tip speed ratio and pitch angle and is
given by Eqs. \eqref{eq:tau-r} and \eqref{eq:lambda-i}:

\begin{equation}
  P_w = \frac{\rho\pi R^2 C_p\paren{\lambda, \theta} V_W^2}{2}\
  \text{\,,}
  \label{eq:power-wind}
\end{equation}

\begin{alignat}{3}
  \tau_R & = \frac{P_W \times C_p}{\omega_{rotor}}
  C_p\paren{\lambda, \theta}                               \\
         & = 0.22\paren{\frac{116}{\lambda_i} - 0.040 - 5}
  e^\frac{-12.5}{\lambda_i}
  \text{\,,}
  \label{eq:tau-r}
\end{alignat}

\begin{equation}
  \lambda_i = \paren{\frac{1}{\lambda + 0.8} -
    \frac{0.035}{\theta^3 + 1}}^{-1}
  \text{\,.}
  \label{eq:lambda-i}
\end{equation}

In Equation \eqref{eq:power-wind}, $R$ is the wind turbine
rotor radius in meters, $V_W$ is the wind velocity in
\unit{\meter\per\s}, and $\lambda$ is the tip speed ratio (TSR)
given by Equation \eqref{eq:lambda},

\begin{equation}
  \lambda = \frac{R \omega_{rotor}}{V_W}
  \text{\,.}
  \label{eq:lambda}
\end{equation}

It could be further inferred that wind turbine operation could
be imagined as a system of two opposing torques $\tau_W$ and
$\tau_G$ whenever torques are equal the system is in
equilibrium and is generating constant power. Any time
$\tau_W$ changes in response to change in $V_W$, (for the
experimental environment $\rho$ is assumed constant) there
exists a new equilibrium state of $\tau_W$ and $\tau_G$
torques. Simply put, anytime $V_W$ increases it creates
conditions for a new and greater power equilibrium state, this
means the resistance of the load could be decreased, which will
increase the current flow, and thus more power could be
extracted from the wind.

The KV rating of the generator is responsible for predicting
the magnitude of the generated voltage as a function of the
rotor speed. In the wind turbine controls it is sometimes
important to maintain relatively low rotational speed and at
the same time maintain voltage magnitude that is high enough to
keep the microcontroller active. Equations \eqref{eq:KV} and
\eqref{eq:RPM-KV} show this relationship,

\begin{equation}
  \text{KV}_{rating} = \frac{\text{rotor speed}}{\text{voltage}}
  \text{\,,}
  \label{eq:KV}
\end{equation}

\begin{equation}
  V = \frac{\text{RPM}}{\text{KV}_{rating}}
  \text{\,.}
  \label{eq:RPM-KV}
\end{equation}

Cogging torque is a magnetic interaction between the iron teeth
of the stator and magnets of the rotor in a brushless DC motor
for instance. The team did not know how to assess the magnitude
of such torque and its effect on the cut in speed of the future
turbine so naturally attempts were made to find a motor with
little to no cogging torque. This led to the discovery of the
axial flux permanent magnet motor (AFPMM) which could be used
as a three phase DC generator if coupled with a three-phase
full rectifier circuit.

Figure \ref{img:stator_rotor} shows the single rotor and the
stator of the 12 pole 9 coil AFPMG used in the prototype. A set
of two motors were purchased for initial evaluation of the KV
rating. Using dynamometer, it was experimentally determined
that the KV rating of the single motor was 96. This was very
low comparing to all other available options that also had the
appropriate dimensional scale. Evaluating the mechanical
construction of the motor's design, it was observed that it
appeared to be relatively simple to create a stack of these
motors coupled by a single axle. Once the team recognized the
flexibility in the connection schemes, ability to reduce the
KV rating to 24 by stacking motors on the single axle, and the
virtual absence of cogging torque in this design, two more
motors were immediately ordered.
\begin{figure}[th]
  \centering
  \includegraphics[width=0.4\textwidth]{../_images/stator_rotor.png}
  \caption{Stator and rotor}
  \label{img:stator_rotor}
\end{figure}

Figure \ref{img:housing} demonstrates the midterm version of
the generator design. The housing assembly consists of five
main parts: (b and c) front and back bearing supports,
(d) housing block, (e) stabilizer bar, and (f) yaw pivot
assembly.

\begin{figure}[th]
  \centering
  \includegraphics[width=0.4\textwidth]{../_images/housing.png}
  \caption{Housing assembly}
  \label{img:housing}
\end{figure}

The drivetrain assembly is shown in Figure \ref{img:drivetrain}.
It is composed of the central axle (a) that connects four pairs
of inline rotors (b and c) and two pairs of stators (d and e).
The stators are supported by the housing assembly and are
independent of the rotation of the rotor assembly. Connected to
each pair of stators are housing assemblies that contain two
pairs of internally connected full bridge rectifiers potted in
epoxy. Once the generator selection was settled on, mechanical
design and power electronics work began.
\begin{figure}[th]
  \centering
  \includegraphics[width=0.4\textwidth]{../_images/drivetrain.png}
  \caption{Drivetrain assembly}
  \label{img:drivetrain}
\end{figure}

Figure \ref{img:schematic} shows the systems power distribution
of the most recent design review.

The load chosen is constructed of a \qty{200}{\W} variable
resistor with coils and contacts throughout to select the
desired resistance for the appropriate wind speed and
scenario. Supplemental resistance was needed for the lower wind
speed and lower load conditions, so additional resistors are
combined in series to acquire these values. The resistance
values used in the load box vary from \qty{45}{\ohm} up to
\qty{420}{\ohm}. A relay bank controlled by a separate Arduino
from the turbine is used to select the load by closing the
contacts on the desired resistor value. The Arduino is
receiving commands from the turbine through an optically
isolated TX and RX line. This Arduino will send commands to
the turbine if a manual shutdown is required.
\begin{figure}[bh]
  \centering
  \includegraphics[width=\textwidth]{../_images/schematic.png}
  \caption{Pictorial schematic of the turbine's control and load
    power distribution}
  \label{img:schematic}
\end{figure}

\subsection{Control Theory}
The pitch control scheme is made up of a methodology called
Fuzzy Logic, as shown in Figure \ref{img:control}. The
tachometer measures the RPM of the turbine
and then the controller determines how far away that the
current RPM is from the desired RPM of the turbine. The
previous RPM record is also taken into consideration when
determining whether to make a pitch adjustment or not. If the
turbine is rotating too slowly, but is speeding up, the
algorithm will wait another cycle to see if the pitch has
continued to increase the RPM, and if not, it will then make
an appropriate adjustment based on how far away from the
desired RPM it is. If the turbine RPM is within acceptable
bounds to the desired RPM, then no adjustments will be made.

The tachometer utilizes a Hall effect sensor that counts the
number of pulses it receives, which happens once per
revolution, and calculated the period it takes for 20
revolutions to occur and determines the RPM from this math.
There is a timeout scenario where if 20 revolutions have not
been completed in 5 seconds the loop will break and the RPM
will still be calculated, just with less accuracy because the
time period over fewer revolutions will be averaged.

\onecolumn
\begin{figure}[th]
  \centering
  \includegraphics[width=\textwidth]{../_images/control.png}
  \caption{Canonical control model}
  \label{img:control}
\end{figure}
\twocolumn

A current sensor must be used to determine if the load has
been disconnected from the turbine. An ADA260 module
communicates with the Arduino via an I2C connection. When the
load goes above \qty{50}{\mA} the load is detected, and any
drop below that will trigger the pitch to adjust to slow the
blades down to a stop. The minimum load expected at a wind
speed of \qty{5}{\m\per\s} is around \qty{80}{\mA} so this is
not something that could be accidentally tripped.

The wind speed sensor determines the wind speed utilizing the
``hot wire'' method. This method heats up a wire and as the
wind passes by, it cools the wire off changing its resistance
and the voltage across it. The voltage is read by the Arduino
and then the Arduino can determine the wind speed the sensor is
experiencing.

Delays are important for the pitch control system because it
takes some time for the system to stabilize and the RPM to
cease any fluctuations. The problem with using the built-in
delay functions is that the Arduino cannot do anything else
while this is happening. A modified delay function was made so
that crucial measurements can be made in this idle time to
allow action to be taken if something doesn't look right. The
current is monitored so if the load is disconnected, the
turbine can immediately begin slowing down to prevent over
speeding the turbine and damaging both the physical and
electrical components.

Currently, the pitch control mechanism is adjusted by a stepper
motor, but through testing it has been noticed that the stepper
motor has a few flaws, being it is slow, and it draws a lot of
current. The stepper motor takes more than 20 seconds to move
from one extreme pitch setting to the other. This affects the
RPM when the load is disconnected because the elimination of
torque due to the load results in the turbine increasing by
several hundred RPM during disconnects. With faster pitching
speeds, this RPM overshoot can be minimized. Because the
stepper motor draws around \qty{300}{\mA} the added load during
pitch adjustments slows the blades down and results in more
time needed for the RPM to stabilize after pitch adjustments
are made. It is being considered to change to a DC motor with a
gearbox and H-bridge configuration which alleviates both
concerns in preliminary testing, with the current selection
having more speed while also drawing less current.

\subsection{Voltage Regulation Description}
The voltage regulation system consists of four axial flux
motors tied in series after their respective voltages have
been rectified. This allows for a high KV rating which is
important for low RPM operation of the Arduino. Two adjustable
buck converters are each attached to two rectifier outputs in
series, limited to \qty{22.5}{\V}. When these two buck
converters are connected in series the system total voltage is
limited to \qty{45}{\V}. Two buck converters connected in
series are needed because each is rated for \qty{52}{\V},
allowing for a maximum turbine voltage of \qty{104}{\V} which
is achieved at around \qty{2400}{RPM} or twice the
\qty{1200}{RPM} normal operational speed of the turbine. The
initial intent was to achieve a high voltage quickly to run the
Arduino at low RPMs and sustain it for the manual and
emergency stop scenarios so that the turbine could
automatically restart.

Unfortunately, through testing it was discovered that each of
the buck convertors requires a minimum of \qty{4.5}{\V} to
operate, which now brings the minimum operating voltage to
\qty{9}{\V} instead of the \qty{5}{\V} that the Arduino needs
to say alive. Applying the KV rating formula from
\eqref{eq:KV} gives \qty{216}{RPM} as the minimum rotational
speed for turbine's control system to operate. This would
require the normal operational speed of the turbine to go up by
\qtyrange{960}{2160}{RPM}, this is according to CWC 10\% of
maximum speed requirement during load-disconnect shutdown
procedure.

Because during the emergency shutdown it takes time for the
system to respond to the load being disconnected, there exists
a time were RPM continuous to rise. Having only \qty{240}{RPM}
buffer before buck converters' operational limit is surpassed
by overvoltage, brings a self-restart operation of the turbine
to the unpredictable operation conditions, where buck converter
failure is very likely.

From observation, the system can surpass the \qty{240}{RPM}
buffer zone faster than the control system can respond to slow
itself down. At this time, the team was not able to find a
solution for this problem and turbine is designed to be
manually restarted after emergency stop. A faster pitch
control system is being designed, using DC motor, gear box and
limit switches to prevent over travel. More tests are needed
to determine if new pitch control scenario will deliver the
self-restart capability.

\subsection{Software Development}
A lot of adjustments and improvements have been made to the
pitch control algorithm. At first there were five choices the
program could make to adjust the pitch: no adjustment and
small/big adjustment to make the blades go faster/slower.
Later it was observed that if the blades were accelerating
towards the desired RPM, there may not need to be any
adjustment made. The pitch adjustment motor used also requires
power to use, and so with every pitch adjustment, not only is
there a delay required due to the change in aerodynamics, but
the additional load that the motor requires will slow the
turbine down as well, which may need additional time to allow
for RPM stabilization. As the wind speed increases, because
there is more power in the wind, the turbine reacts more
quickly to pitch changes, and so the delay times must be
shortened as wind speed increases. The turbine is also more
sensitive to pitch changes at high wind speeds and RPM, and so
the adjustments made are smaller as the wind speed increase.
If the turbine is trying to supply more power than it can
extract from the wind, which would cause the RPM to drop due to
blade stall. The blades will pitch back to a previous setting
to try and reduce or save the stall condition if too many
``large pitch adjustment faster'' conditions occur
consecutively. While the delay functions built into the
Arduino software work well, the current cannot be measured
during this delay time, and so if the load is disconnected it
could be up to three seconds before it is noticed by the
software, which could cause the blades to overspeed. A
modified delay function had to be created so that sufficient
delays could still be implemented, and the current could be
checked in case of a disconnect, immediately pitching the
blades back and breaking the turbine.

Wind speed measurement has been more difficult than it was
anticipated. The Rev C wind speed sensor is very sensitive at
low wind speeds, but at high wind speeds the voltage it sends
out only varies by a couple hundredths of a volt per meter per
second of wind. This made it challenging to know exactly what
the wind speed was so a lot of measurements are taken and
averaged and then compared with previous wind speed
measurements to try and verify if the wind speed is constant
or it is changing to a higher or lower speed.

\end{document}


\subsection{Task 6: Belief Propagation}

\subsubsection{Experiment on large datasets}
In this experiment, we run our radius algorithm in several large datasets. The statistics of these datasets are presented in Table \ref{t5:table1}

\begin{table}[!htbf]
\caption{Datasets Statistics}
\begin{center}
\begin{tabular}{|c|c|c|}
\hline \hline
dataset & number of vertices & number of edges \\
\hline
DBLP co-authorship network & 317080  & 1049866  \\
Epinions social network & 131828  & 841372  \\
Amazon product co-purchasing network & 334863 & 925872 \\
EU email communication network & 265214 & 420045 \\
Google web graph & 875713 & 5105039 \\
Youtube social network & 1134890 & 2987624 \\
\hline
\end{tabular}
\end{center}
\label{t5:table1}
\end{table}%

Similar to semi-supervised learning, belief propagation algorithm require the graph to be partially labeled.  However, we don't have such information. Therefore, in this experiment, we randomly assign the prior belief for all nodes. Specifically, we randomly assign 5\% of the nodes with positive label, i.e. positive prior belief(0.001), and  5\% of other nodes with negative label, i.e. negative prior belief(-0.001), and the rest with zero belief, meaning that we don't have prior knowledge for these nodes. Then we conduct FABP algorithm on these datasets, the result are shown in Table \ref{t5:table2} .

\begin{table}[!htbf]
\caption{BP Statistics}
\begin{center}
\begin{tabular}{|c|c|c|c|}
\hline \hline
dataset & positively labeled & negatively labeled & unlabeled  \\
\hline
DBLP co-authorship network & 164103  & 144259  & 8718 \\
Epinions social network & 22334  & 22575  & 86919 \\
Amazon product co-purchasing network & 158837 & 160524 & 15502 \\
EU email communication network & 127481 & 109616 & 28117 \\
Google web graph & 385277 & 389975 & 100461 \\
Youtube social network & 584422 & 508628 & 41840 \\
\hline
\end{tabular}
\end{center}
\label{t5:table2}
\end{table}%

\subsubsection{Observation}
1. By applying Belief Propagation algorithm on these graphs, most of the unlabeled nodes are successfully assigned either positive or negative belief.

2. We find that for some graphs, larger proportions of nodes gets labeled than other graph. For example, in DBLP co-authorship network and amazon product co-purchasing network,  about 95\%  of the nodes get labeled using only 10\% labeled nodes. While for like Epinions social network , only about 30\% of the nodes get labeled. Once again, we look back at the connectivity of the graph to find the probable cause. We observed that, graph that is well connected is easier to get more nodes assigned with labels. This observation makes sense in that 'beliefs' can be easier to propagate in well connected graphs than those grape with many disconnected components.

\subsubsection{Proof of Correctness}
As mentioned in the previous section, we don't have any labels for these large datasets, therefore we verify the result according to statistics we got in the last section. We can see that the proportion with positive labels and negative labels are approximately same, which resembles the label distribution with our prior belief. Also, we successfully inferenced the belief of other nodes using only 10\% labeled nodes.
In order to verify the accuracy of our algorithm, we run FABP on small matrix we generate. In essence, the FABP tries to solve a linear system $(I - W)x = prior$, therefore we test our FABP against the linear system solver in MATLAB(function linsolve). And the two get nearly identical result(with error less than 0.01) in solving some toy linear systems consisting of 2 by 2 matrix. This demonstrates the correctness  of our algorithm.


\section*{Problem 1 Solution}

We will create the following simple decay chain graphic, built from the information provided in the problem, to visualize the processes described in the problem.

\begin{center}
\begin{tikzpicture}[node distance=3.25cm]
\node (blank) [placeholder]{};
\node (fission) [isotope, align=center, right of =blank, xshift=2cm, yshift=0.75cm] {\large \textbf{Fission}};
\node (tellurium135) [isotope, align=center, left of =blank] {\large$^{135}$Te};
\node (iodine135) [isotope, align=center, below of =tellurium135, right of =tellurium135] {\large$^{135}$I};
\node (xenon135) [isotope, align=center, below of =iodine135, right of =iodine135] {\large$^{135}$Xe};
\node (xenon136) [placeholder, align=center, left of =xenon135, xshift=-0.5cm] {};
\node (cesium135) [placeholder, align=center, below of =xenon135, right of =xenon135, xshift=-0.9cm, yshift=1cm]{};
\node (yield) [decay, align=center, right of =tellurium135, xshift=3.2cm, yshift=0cm,]{\large \textbf{6\%}};
\node (beta-te-i) [decay, align=center, right of =tellurium135, xshift=-0.5cm, yshift=-1.2cm] {\large$\beta^-$ \\ \footnotesize $t_{\nicefrac{1}{2}} = 19.0$ s};
\node (beta-i-xe) [decay, align=center, right of =iodine135, xshift=-0.5cm, yshift=-1.2cm] {\large$\beta^-$ \\ \footnotesize $t_{\nicefrac{1}{2}} = 6.6$ h};
\node (beta-xe-cs) [decay, align=center, right of =xenon135, xshift=-0.7cm, yshift=-1cm] {\large$\beta^-$ \\ \footnotesize $t_{\nicefrac{1}{2}} = 9.2$ h};
\node (abs-xe-cs) [decay, align=center, below of =xenon136, xshift=1.6cm, yshift=2.7cm] {\large$\sigma_a \phi$};

\draw [farrow] (fission) -- (tellurium135);
\draw [farrow] (fission) -- (iodine135);
\draw [arrow] (tellurium135) -- (iodine135);
\draw [arrow] (iodine135) -- (xenon135);
\draw [arrow] (xenon135) -- (cesium135);
\draw [arrow] (xenon135) -- (xenon136);

\end{tikzpicture}
\end{center}

Starting off, since this is a problem related to decay, we will start from the usual equation for changes in the number density of radionuclides. (This equation may be more familiar in terms of absolute quantity of a material,$N$, in atoms; $n$ is the number density, found by dividing $N$ by the volume containing the $N$ nuclei)
\begin{equation}
\label{general}
\frac{dn}{dt} = \text{production} - \text{loss from decay}
\end{equation}

Nuclide production may occur directly from the fission reaction or from the decay of other fission products into that nuclide. Nuclide losses will be caused by the decay of the radionuclide, or when that nucleus absorbs neutrons in the reactor's high-neutron-flux environment.

For $^{135}$Xe, the fission product of interest, we can decompose this differential equation more explicitly as

\begin{equation}
\label{text-dn-xenon}
\frac{dn_{\text{Xe}}}{dt} = \text{production from }^{135}\text{I} - \text{loss from decay of }^{135}\text{Xe} - \text{loss from absorption} .
\end{equation}

In this equation, the production due to the decay of iodine is just the activity (per volume) of iodine,
$$ \mathcal{A}_{\text{I}} = \lambda_{\text{I}}n_{\text{I}}, $$
and the loss due to the decay of xenon is just the activity (per volume) of xenon,
$$ \mathcal{A}_{\text{Xe}} = \lambda_{\text{Xe}}n_{\text{Xe}}. $$
The loss due to absorption can be calculated by knowing that the absorption rate of neutrons on $^{135}$Xe is $R = \Sigma_a \phi$, where $\phi$ is the neutron flux in the reactor. Since $\Sigma_{a,\text{Xe}}$ is defined as $\sigma_{a,\text{Xe}} n_{\text{Xe}}$, the absorptive losses are
$$ R_{\text{Xe,abs}} \sigma_{a,\text{Xe}} n_{\text{Xe}} \phi .$$
The full equation for xenon can be written,
$$ \frac{dn_{\text{Xe}}}{dt} = \mathcal{A}_{\text{I}} - \mathcal{A}_{\text{Xe}} - R_{\text{Xe,abs}} $$
or
$$ \frac{dn_{\text{Xe}}}{dt} = \lambda_{\text{I}}n_{\text{I}} - \lambda_{\text{Xe}}n_{\text{Xe}} - \sigma_{a,\text{Xe}} n_{\text{Xe}} \phi .$$
We factor out $n_{\text{Xe}}$ from the last two terms to get
\begin{equation}
\label{dn-xenon}
\frac{dn_{\text{Xe}}}{dt} = \lambda_{\text{I}}n_{\text{I}} - (\lambda_{\text{Xe}} + \sigma_a\phi) n_{\text{Xe}}
\end{equation}
In this form, it is easier to see that the number densities are the only two unknown functions in this differential equation. If we could express $n_{\text{I}}$ in terms of $t$, we could solve the equation for $n_{\text{Xe}}$.

To express $n_{\text{I}}$ in terms of $t$, we again use equation (\ref{general}). Proceeding in a similar fashion as we did for xenon, we explicitly write out the equation for iodine akin to equation (\ref{text-dn-xenon})
\begin{equation}
\label{text-dn-iodine}
\frac{dn_{\text{I}}}{dt} = \text{production from fission} + \text{production from }^{135}\text{Te} - \text{loss from decay of }^{135}\text{Xe}.
\end{equation}
We have ignored absorptive losses since the thermal absorption cross section for iodine is essentially zero.

The production of iodine due to fission can be calculated from the iodine-yield of the fission reaction. While this information is not provided, the combined yield, $y$, of iodine and tellurium \textit{is} given. Since the half-life of $^{135}$Te (19.0 s) is practically insignificant in comparison to the multi-hour half-lives of its daughters (to be exact, $t_{\nicefrac{1}{2},\text{Te135}} = 0.0008(t_{\nicefrac{1}{2},\text{I135}})$ and $t_{\nicefrac{1}{2},\text{Te135}} = 0.0006(t_{\nicefrac{1}{2},\text{Xe135}})$), we can treat it as instantaneously decaying into iodine. With this assumption, the production of iodine due to fission is just the fission rate density multiplied by the combined yield of iodine and tellurium. The fission rate density can be found by dividing the power density by the energy per fission, $Q_f$, so
$$ R_{\text{I,fission}} = y\frac{P}{Q_f} .$$
The loss from the decay of iodine is just the activity of iodine,
$$ \mathcal{A}_{\text{I}} = \lambda_{\text{I}}n_{\text{I}}. $$
The full equation for iodine can be written,
$$ \frac{dn_{\text{I}}}{dt} = R_{\text{I,fission}} + \mathcal{A}_{\text{I}} $$
or
\begin{equation}
\label{dn-iodine}
\frac{dn_{\text{I}}}{dt} = y\frac{P}{Q_f} + \lambda_{\text{I}}n_{\text{I}} .
\end{equation}
Considering that the power after the power change is $P=P_1$, we can solve for number density of iodine after the change, $n_{\text{I},1}$.
\begin{equation}
\label{n-iodine1}
n_{\text{I},1}(t) = \frac{y P_1}{\lambda_{\text{I}}Q_f}\left(1-e^{-\lambda_{\text{I}}t}\right) + n_{\text{I},1}(0)e^{-\lambda_{\text{I}}t}.
\end{equation}
\begin{center}(for a detailed derivation of this, see the appendix)\end{center}
Looking closely at this equation, we find that the only unknown quantity here is $n_{\text{I},1}(0)$. While using $t=0$ in this equation gives us the trivial solution that $n_{\text{I},1}(0) = n_{\text{I},1}(0)$, we note that equation (\ref{dn-iodine}) could be solved identically for times before the power change by simply replacing $P_1$ by $P_0$.
\begin{equation}
\label{n-iodine0}
n_{\text{I},0}(t) = \frac{y P_0}{\lambda_{\text{I}}Q_f}\left(1-e^{-\lambda_{\text{I}}t}\right) + n_{\text{I},0}(0)e^{-\lambda_{\text{I}}t}.
\end{equation}
We are told that our reactor has been operating for a long time. If we assume that this time period is sufficiently long that we can approximate the time of the power change as $t=\infty$, then equation (\ref{n-iodine0}) reduces to
$$ n_{\text{I},0}(\infty) = \frac{y P_0}{\lambda_{\text{I}}Q_f} ,$$
and
$$ n_{\text{I},0}(\infty) = n_{\text{I},1}(0) .$$
We replace this value of $n_{\text{I},1}(0)$ in equation (\ref{n-iodine1}) to get
\begin{equation}
\label{n-iodine1-comp}
n_{\text{I},1}(t) = \frac{y P_1}{\lambda_{\text{I}}Q_f}\left(1-e^{-\lambda_{\text{I}}t}\right) + \frac{y P_0}{\lambda_{\text{I}}Q_f}e^{-\lambda_{\text{I}}t}.
\end{equation}
This equation has $n_{\text{I}}$ exclusively as a function of $t$, and so we can now use it back in equation (\ref{dn-xenon}) to find the number density of xenon after the power change.
\begin{align*}
\frac{dn_{\text{Xe},1}}{dt}	&= \lambda_{\text{I}}\left(\frac{y P_1}{\lambda_{\text{I}}Q_f}\left(1-e^{-\lambda_{\text{I}}t}\right) + \frac{y P_0}{\lambda_{\text{I}}Q_f}e^{-\lambda_{\text{I}}t}\right) - (\lambda_{\text{Xe}} + \sigma_{a,\text{Xe}}\phi) n_{\text{Xe},1} \\
							&= \frac{y P_1}{Q_f}\left(1-e^{-\lambda_{\text{I}}t}\right) + \frac{y P_0}{Q_f}e^{-\lambda_{\text{I}}t} - (\lambda_{\text{Xe}} + \sigma_{a,\text{Xe}}\phi) n_{\text{Xe},1}\\
\end{align*}
In this equation we will introduce a parameter called the "effective destruction constant", $\lambda_{\text{Xe}}^{\text{eff}} = \lambda_{\text{Xe}} + \sigma_{a,\text{Xe}} \phi. $ The differential equation above becomes
$$ \frac{dn_{\text{Xe},1}}{dt} = \frac{y P_1}{Q_f}\left(1-e^{-\lambda_{\text{I}}t}\right) + \frac{y P_0}{Q_f}e^{-\lambda_{\text{I}}t} - \lambda_{\text{Xe}}^{\text{eff}} n_{\text{Xe},1} $$

\textbf{Note:} If you've made it this far, you've got the general gist of the problem. Nice work! The remainder get's a little tricky---you need an integrating factor to solve this differential equation---and though the solution looks a bit intimidating, it can begin to reveal some of the complicated behavior of an operating reactor. This is information that will be covered more comprehensively later in the semester, but which is also useful to introduce now.

When we use the integrating factor, we find
\begin{align*}
N_{\text{Xe},1}(t)	&= N_{\text{Xe},1}(0)e^{-\lambda_{\text{Xe}}^{\text{eff}}t} + \frac{yP_1}{\lambda_{\text{Xe}}^{\text{eff}}Q_f}\left(1-e^{-\lambda_{\text{Xe}}^{\text{eff}} t}\right) - \frac{yP_1}{(\lambda_{\text{Xe}}^{\text{eff}}-\lambda_{\text{I}}) Q_f} \left(e^{-\lambda_{\text{I}}t} - e^{-\lambda_{\text{Xe}}^{\text{eff}}t}\right) + \frac{yP_0}{(\lambda_{\text{Xe}}^{\text{eff}}-\lambda_{\text{I}}) Q_f} \left(e^{-\lambda_{\text{I}}t} - e^{-\lambda_{\text{Xe}}^{\text{eff}}t}\right) \\
					&= N_{\text{Xe},1}(0)e^{-\lambda_{\text{Xe}}^{\text{eff}}t} + \frac{yP_1}{\lambda_{\text{Xe}}^{\text{eff}}Q_f}\left(1-e^{-\lambda_{\text{Xe}}^{\text{eff}} t}\right) - \left(\frac{yP_1}{(\lambda_{\text{Xe}}^{\text{eff}}-\lambda_{\text{I}}) Q_f} - \frac{yP_0}{(\lambda_{\text{Xe}}^{\text{eff}}-\lambda_{\text{I}}) Q_f}\right) \left(e^{-\lambda_{\text{I}}t} - e^{-\lambda_{\text{Xe}}^{\text{eff}}t}\right) \\
					&= N_{\text{Xe},1}(0)e^{-\lambda_{\text{Xe}}^{\text{eff}}t} + \frac{yP_1}{\lambda_{\text{Xe}}^{\text{eff}}Q_f}\left(1-e^{-\lambda_{\text{Xe}}^{\text{eff}} t}\right) - \frac{y\left(P_1-P_0\right)}{(\lambda_{\text{Xe}}^{\text{eff}}-\lambda_{\text{I}}) Q_f} \left(e^{-\lambda_{\text{I}}t} - e^{-\lambda_{\text{Xe}}^{\text{eff}}t}\right)
\end{align*}
\begin{center}(for a detailed derivation of this, see the appendix)\end{center}

From this equation, we see that if $P_1 > P_0$, then the sign of the third term is opposite the case when $P_0 > P_1$. Using typical values for all the constants, including flux and energy per fission, this sign change suggests that power increases and decreases will have opposite effects on the xenon population. If the power increases ($P_1 > P_0$) then the xenon population will temporarily decrease. If instead the power decreases ($P_1 < P_0$) then the xenon population will increase for a while, before xenon losses compensate for the production due to decaying iodine. This behavior can lead to situations where an operator is unable to restart a reactor immediately after shutting down (power has decreased, fission product concentrations temporarily jump, and restarting the reactor is unsafe). This was a contributing factor at Chernobyl.


\section{Introduction}
Graphs are a natural way to model many of the modern complex
datasets that typically have interlinked entities connected with various
relationships. Examples include different types of networks, such as
social, biological and technological networks. Tools for rapidly
querying and mining graph data are therefore in high demand. Our focus
is on graph pattern discovery methods that can simultaneously consider
both the structure and content (e.g., node labels).

Whereas frequent graph mining has long been a well studied problem, most
of the prior work has focused on exact pattern discovery.
Graph mining involves two main steps. The first step is to generate
non-duplicate candidate patterns, and the second is to compute the
frequency of each candidate pattern. The former task requires graph
isomorphism testing, whereas the latter requires subgraph isomorphism
checking, since we need to count all the occurrences of a smaller graph
within a much larger graph (or a set of graphs). Many efficient methods
have been proposed for mining exact labeled graph patterns, including
both complete search and sampling based
approaches~\cite{gSpan,HWP03,kuramochi2005ffp,FSG01,IWM03,2009-graphsampling}.
These exact methods require that there be an exact match between the
labels of nodes in the candidate pattern and in the database graph. This
can potentially miss many patterns where nodes may share a high label
similarity, but may not match exactly. This is specially true for more
complex labels (e.g., text data), or in cases where the nodes represent
some real-world objects (e.g., proteins, IT infrastructure nodes), where
it may be possible to easily design a meaningful cost or distance matrix  between node ``labels''. Unfortunately, exact isomorphism
based methods cannot leverage the rich information from the cost matrix.
What is required is a new class of algorithms that can mine
frequent approximate patterns via approximate subgraph isomorphism that
satisfies some bound on the overall cost of the match between a candidate
and the database graph(s). Only a few methods have tackled this
problem~\cite{gapprox,JiaZH11,RAM2008}, but they typically enumerate all
isomorphisms, and are therefore not scalable to large graphs due to the
combinatorial explosion in the number of isomorphisms.

In this paper we present a new approach to mine frequent approximate
patterns in the presence of a cost matrix between the labels. In
particular we make the following contributions:
\begin{itemize}
\item We propose a novel approach to effectively prune the space of
  approximate labeled isomorphisms. Instead of enumerating all the
  possible isomorphisms, we maintain a set of representatives (nodes in
  the database that match a candidate pattern) that is linear in the
  database and pattern size. Pruning is applied on this set to narrow
  down the search to only viable mappings.
\item We propose several iterative label updating methods that yield
  derived cost matrices on the basis of which more effective
  pruning can be achieved. These are based on $k$-hop labels, neighbor
  concatenated labels and a combination of the two.
\item Our method handles both arbitrary as well as binary cost matrices.
\item We place our work within the pattern sampling paradigm,
  thereby avoiding complete search, which can be practically
  infeasible in real-world
  graphs, not to mention the information overload problem.
\end{itemize}
We study the effectiveness of the proposed methods on three real-world
datasets. The first is a configuration management database graph,
where the nodes represents
entities comprising the IT infrastructure and the link represents
relationships between them; approximate mining yields a richer de-facto IT
policies in the company. The second dataset is a graph dataset
representing 3D protein structures; mined patterns represent approximate
motifs. The last dataset comes from a protein interaction network, where
the nodes are proteins and edges indicate whether they interact
physically (i.e., they may bind together or they may be part of the
same protein complex); the mined approximate patterns represent
molecular subnetworks and molecular machines (the protein complexes)
that take part in important cellular processes. We show that our
proposed techniques are indeed scalable and fruitful, allowing us
to mine interesting approximate graph patterns from
large real-world graphs.