\section{Articulated bodies\label{articulatedBodies}}

\subsection{Approaches to constraints\label{approachesToConstraints}}
The methods developed so far allow the simulation of a single rigid body with forces
acting on it. Further challenges arise when we consider multibody systems in which there are
conditions which must not be violated. In an articulated body, for example, each segment
must be modelled as an individual rigid body, under the condition that joints must not be pulled
apart.

There are various strategies for simulating articulated bodies:

\begin{itemize}
\item \emph{Penalty methods} conceptually join bodies together with springs so that when they
    begin to separate, a restoring force causes them to move back together again. These methods
    are simple to implement initially but hard to get right: if the springs are too weak, the
    bodies will separate too far; if they are too stiff, very large forces can arise suddenly,
    causing the simulation to become unstable.

\item Mechanics formulations using \emph{generalized coordinates}
    \cite{Hand:98,Goldstein:80,Featherstone:87,Wilhelms:91} build constraints firmly into
    the system: the number of generalized coordinates equals the actual number of degrees of
    freedom after the constraints have been applied. The state of the system is specified only in
    terms of these generalized coordinates, and they are chosen such that the system will always
    be in a legal state, irrespective of what the values of the coordinates are. These algorithms
    allow efficient and reliable computation. However, the equations are only tractable for
    so-called \emph{holonomic constraints} (definition in~\cite{Hand:98}), which excludes
    many interesting systems.

\item This project makes a compromise between the flexibility of penalty methods and the stability
    of generalized coordinates. As in the penalty method, we initially treat each segment of an
    articulated body separately. The constraints are then formulated in such a way that we can
    not only tell whether the system is currently in an illegal state, but also whether it is
    moving towards one, and even whether it is accelerating towards one. Using the method of
    \emph{Lagrange multipliers} we can determine what forces must be applied to the bodies to
    nullify this unwanted acceleration before it ever occurs. This compromise comes at the
    expense of higher computational cost, because simultaneous linear equations have to be solved
    at run-time.
\end{itemize}

The mathematical formulation of Lagrange multipliers is derived in~\cite{BaraffWitkin:97} and
extended in~\cite{Saunders:PhD} (also see~\cite{Baraff:96} for an optimized algorithm),
and I only state the results here.

\subsection{Lagrange multipliers\label{lagrange}}

Each constraint effectively reduces the number of degrees of freedom by one, and is expressed
as a function $c$ which is zero when the constraint is satisfied. (A ball-and-socket
joint, which allows three modes of rotation but no separation, reduces the number of d.o.f.\ by
three and is therefore expressed as a vector of three scalar constraint functions.) All constraint
functions can then be concatenated into a single constraint vector \ve{c}.

The constraints can be satisfied by solving the equation\footnote{Here the sign convention
of~\cite{BaraffWitkin:97} is adopted, which is the opposite of~\cite{Saunders:PhD}.}
\begin{equation}
\label{lagrangeEquation}
-\m{J}\m{M}^{-1}\m{J}^T\ve{\lambda} = \dot{\m{J}}\dot{\ve{x}} +
    \m{J}\m{M}^{-1}(\ve{\Phi} + \ve{\Phi}_p) + k\ve{c} + d\dot{\ve{c}}.
\end{equation}
The values of all variables except for the vector $\ve{\lambda}$ in this equation are
known, and they are explained in appendix~\ref{constraintPrerequisites}. In brief, they are:

\vspace{10pt}
\begin{tabular}{@{}ll}
\ve{c}, $\dot{\ve{c}}$ & Constraint vector and its first derivative w.r.t.\ time.\\
\m{J}, $\dot{\m{J}}$ & Jacobian matrices derived from \ve{c} (see appendix~\ref{jacCalculation}).\\
\m{M} & Mass-inertia matrix combining the masses and moments of inertia\\
      & of all bodies in the scene. \\
$\dot{\ve{x}}$ & Combined linear and angular velocity vector of all bodies.\\
\ve{\Phi} & Combined forces and torque vectors acting on all bodies.\\
$\ve{\Phi}_p$ & Effects of free precession.\\
$k$, $d$ & Scalar constants regulating error compensation, discussed
    in~\cite{BaraffWitkin:97}.
\end{tabular}
\vspace{10pt}

For us it will suffice to consider this equation to be a `black box' which can be
given to a linear equation solver to obtain $\ve{\lambda}$.

$\ve{\lambda}$ is an $m$-row vector of so-called \emph{Lagrange multipliers} (after which this
algorithm is named), where $m$ is the number of constraints. Once we have solved for it, we can
compute the expression
\begin{equation} \label{lagrangeSolution}
\ve{\Phi}_c = \m{J}^T\,\ve{\lambda}.
\end{equation}
$\ve{\Phi}_c$ is the vector of constraint restoring forces and torques for all bodies.
Speaking in physical terms, these are the reaction forces which balance the action \ve{\Phi}.
All we therefore need to do is to compute $\ve{\Phi} + \ve{\Phi}_c$ and to feed the result into
our ODE solver, and the motion which ensues will satisfy the constraints.
$\ve{\Phi}_p$ must not be added to this term\footnote{provided the moments of inertia are
recalculated based on the bodies' orientation at every time step.}.

\subsection{Modelling the human body}

The human body's selection of joint types is quite limited compared to the range of connectors
found in machines~\cite{Kalra:95}: for example, there are no sliding joints or screws. The simple
types of joint found in human bodies are ball-and-socket joints allowing rotation about three axes
(like shoulders and hips), and revolute joints limited to a single axis (like knees and elbows).
The issue is made more complicated by compound joints like the spine, and motion which occurs
along the length of a segment rather than at a joint (like rotating the wrist about the axis of
the lower arm, which occurs through a relative shift of the ulna and radius
bones~\cite{Anatomy:03}).

To make the model manageable, I chose to be anatomically ignorant and assumed that two adjacent
segments of an articulated body are connected by a single joint, and that rotation occurs only
about this joint. Each bone has a local coordinate system whose origin lies at the joint to the
parent bone. This is illustrated in figure~\ref{jointsFigure}.

\begin{figure}
\centerline{\includegraphics{figures/joint1}}
\caption{Two bones connected by a revolute joint, e.g.\ a knee. Rotation is constrained to be
    possible only about the local x~axis of the knee, but not the local y~axis
    (the lower leg itself) or the local z~axis (sideways).\label{jointsFigure}}
\end{figure}

There are different ways of formulating this model. My approach is to assume initially that every
joint is a ball-and-socket type. For those joints which are not, additional constraints are
added to restrict the permitted axes of rotation.

\subsection{Ball-and-socket joints}

The position of a joint is a particular offset from the centre of mass in one body's frame, and a
different offset from the other body's centre of mass. While the bodies may move around, these
offsets stay constant (as seen in the body's frame).

Figure~\ref{ballAndSocketFigure} gives an example: here, the configuration of bodies satisfies the
ball-and-socket constraint iff $\ve{a}+\ve{s} = \ve{b}+\ve{t}$, i.e.\ if the joint has not
separated. We can rewrite this to give the constraint function
\begin{equation}
\ve{c} = \ve{a} + \ve{s} - \ve{b} - \ve{t}
\end{equation}
which equals \ve{0}, the three-dimensional null vector, iff the constraint is satisfied.

\begin{figure}
\psfrag{frag:o}{\ve{O}}
\psfrag{frag:a}{\ve{a}}
\psfrag{frag:b}{\ve{b}}
\psfrag{frag:s}{\ve{s}}
\psfrag{frag:t}{\ve{t}}
\centerline{\includegraphics{figures/joint2}}
\caption[]{A valid ball-and-socket joint configuration. \ve{s} is constant in body~A's frame,
    while \ve{t} is constant in the frame of body~B. In the world frame, these two vectors
    therefore rotate according to their respective body's orientation.\label{ballAndSocketFigure}}
\end{figure}

\subsection{Rotation constraints\label{rotationConstraints}}

In the case of an elbow or a knee, we also need to prohibit the rotation about particular axes.
After some thought I came up with the constraint function
\begin{equation}\label{rotationConstraintEqn}
c = \Re(\tilde{\ve{n}}\q{p}^{-1}\q{q})
\end{equation}
in which \q{p} is the quaternion of orientation for body~A, \q{q} the quaternion for body~B, and
\ve{n} a vector pointing along the prohibited axis in body~A's frame. Setting $c$ to zero ensures
that no rotation occurs about the axis~\ve{n}. If two different axes are prohibited with two
separate constraints, then rotation is only possible about a single axis orthogonal to the two
prohibited axes. Further details on this constraint are given in appendix~\ref{constrJoystick}.

\subsection{Angle limitation\label{angleLimitation}}

The constraints described so far capture most of the features of the human skeleton, with one
exception: if rotation is permitted about some axis, there is no limit to the amount of rotation
that can occur. For example, permitting a human to look left and right would allow the simulation
to rotate his head by an arbitrary amount. Imposing
limits on the angle of rotation as well as the axis is possible but more complicated, and will
be explained in section~\ref{generalizedCollisions}. It is worth noting that this is our
first example of a non-holonomic constraint, and thus lies beyond the capabilities of
generalized-coordinate approaches.


\definecolor {processblue}{cmyk}{0.96,0,0,0}
\clearpage

\section{DFA Fixed point iteration}
(Prepared by Namrata Priyadarshini, Shivam Bansal)

\textbf{Algorithm(Forward DFA):} \\
Input: control flow graph $CFG = (N, E, Entry, Exit)$ \\

//Boundary condition \\
$OUT[Entry] = Boundary \, Condition \, Value$ \\

//Initialization for iterative algorithm \\
For each basic block B other than Entry \\
\hspace*{0.5cm}  $OUT[B] = Top \, Value $ \\

//Iterate \\
While (changes to any OUT occur) \{ \\
\hspace*{0.5cm}    For each basic block B other than Entry \{ \\
\hspace*{1cm}      $in[B] = $meet over $(out[p])$, for all preds p of B \\
\hspace*{1cm}      $out[B] = f_B(in[B])$ \\
\hspace*{0.5cm}    \} \\
\} \\

So let's look at the DFA fixed point iteration once again. The input to the fixed point iteration algorithm is a control flow graph $CFG = (N, E, Entry, Exit)$. N is the set of nodes, E is the set of edges, Entry is a special node and Exit is also a special node in the control flow graph and then if we look at the forward DFA (we could also have looked at the backward DFA but just for simplicity let's just look at one of them and we arbitrarily pick forward). We initialize out of entry to the boundary condition value so whatever is the boundary condition value for a particular DFA that we're interested in. Then for each other basic block other than entry we just say $OUT[B]=$ top value where top value is again specific to the particular DFA that we are looking at. The top value is based on the meet operator so when we define the meet operator it automatically also defines our top value because top value is something which is greater than equal to everything else (i.e. greater than equal to operator or the ordering operator is also implied by the meet operator).

In the pseudo code we have the the fixed point iteration and so inside the iteration we say while changes to any out occur for each basic block B do this computation. So now the there's an interesting observation. Because initially everything else is top it's only the first node or the entry node that has a value that is other than top i.e. the boundary condition value. So let's say if we pick some basic block which is other than entry then we know the out of the basic block will be top because in[B] will be top because all its predecessors OUT would be top and so on. The only place where it would make sense to pick a basic block at the first step is the basic block that just follows the entry node. So all the nodes that are successors of the entry node are the ones that we really need to pick, everything else we don't need to pick
because if we pick them then they are not going to change their values because their predecessors are top, their current value is top and so even the next value is going to be top because meet over top is just top and then transfer function over top is also top
typically (well transfer function over top is not necessarily top but typically it is top but the point is that typically top represents that something has not been reached or it is the most aggressive value). So we want that things should basically reach that point and only then we should be computing it.

So the question here is do we really need to consider all basic blocks at every iteration or can we omit some basic block at some iteration. For example, in a forward DFA, in the first iteration we only need to pick up the successors of the entry node and then from there on the data will start flowing. If we pick something in the middle or something at the end in the first iteration that is usually not going to be very useful and the other thing is if we have multiple basic blocks from which we could pick then should there be an order in which we should pick them. We're going to see later on that the order doesn't really matter from a correctness or from a point of view of what result we get but the order may matter from an efficiency perspective.

\section{Worklist Algorithm}
\subsection{Intuition for the Algorithm}
There's this famous algorithm called the work list algorithm. It's also commonly called the Kildall's vocalist algorithm, named after the person who first devised this algorithm. The idea here is that for a forward DFA the $out[B]$ value does not change
if none of the $out[p]$ values change where p is the predecessor of B. So the IN value will change only if one of the output values of the predecessor has changed because if none of the OUT values of the predecessor has changed then we are already in a fixed point. Although maybe at other places we are not at the fixed point. Similarly for
backward dfa the $out[B]$ value does not change if none of the $in[s]$ values change where s is a successor of B.

Based on this observation here is the idea that we will maintain a work list which is a list of basic blocks that still need to be processed. So list of basic blocks that we know that they need to be processed because maybe in the forward DFA their predecessor was just changed and so their successors are going to be needing some change as
well. In a forward DFA whenever we remove a basic block from the worklist we compute its out state and if this state has changed the successors are added to the worklist. This basically captures our previous observation. This is going to be hopefully more efficient
than looking at all basic blocks in each iteration and that's the whole idea. For a backward DFA whenever we remove a basic block from the work list we compute its in state if this state has changed the blocks predecessors are added to the worklist.

\subsection{Algorithm}
\textbf{Worklist}: List of basic blocks that still need to be processed.

\textbf{Initialization}: Add basic blocks whose information is known.

\textbf{Termination condition}: Worklist becomes empty.

At the initialization time we add the basic blocks whose information is known so whatever
basic blocks for information is known are added to the vertices i.e. set of basic blocks for which we have boundary condition values. Typically for a forward DFA the boundary condition is known for the entry node and for a backward DFA typically the boundary condition is known for the exit node. Initialization would just involve adding either the entry node to the worklist for a forward DFA or the exit node into the worklist for a backward DFA.

Then the termination condition is that the worklist becomes empty. Things are added to the worklist only if something changed so at a point where the worklist becomes empty it basically indicates that we have reached a fixed point. Note that the worklist is a set
because each block may appear at most once in the worklist at any given time. It's possible that you know a block could have two predecessors and both those predecessors changed and so because of that we tried to add the same block twice into the worklist but if we add the same block twice it's not like we're going to process it twice so we just we just maintain it as a set.

\subsection{Ordering Blocks in Worklist Algorithm}

\begin{figure}[h!]
\caption{Worklist algorithm efficiency}
\begin {center}
\begin {tikzpicture}[-latex ,auto ,node distance =3.5cm and 5cm ,on grid ,
semithick ,
state/.style ={ rectangle ,top color =white , bottom color = processblue!20 ,
draw,processblue , text=blue , scale = 0.7 ,minimum width =3.5 cm, minimum height = 2.5 cm}]
\node[state] (A){} node [label = {[label distance = 0.55cm]90:}, rectangle split,rectangle split parts=1]{%
  b1
  };
\node[state] (B) [below left = of A]{} node [label = {[label distance = 0.55cm]90:},rectangle split,rectangle split parts=1] [below left = of A] {%
  b2
  };
\node[state] (D) [below right =of A]{} node [label = {[label distance = 0.55cm]90:},rectangle split,rectangle split parts=1] [below right = of A] {%
  b3
  };
\path[->] (A) edge node [above = 0.3 cm] {} (D);
\path[->] (A) edge node [above = 0.3 cm] {} (B);
\path[->] (B) edge  (D);

\end{tikzpicture}
\end{center}
\end{figure}

\subsubsection{Intuition with an example}
This basically answers our first question which is - can we omit some basic blocks and if so how do we identify which ones to omit. Now the other question is if there are multiple basic blocks present in the work list at any point and then which one should we pick is it okay if we pick any one. So the answer to this question is yes indeed, it is okay to pick anyone that's just the property of this fixed point iteration algorithm and that's completely independent of the worklist algorithm. We can pick any basic block at any time
and yet we are going to arrive at the same solution and that's going to be the most precise solution for some definition of precision but from an efficiency point of view
does it matter which one we pick. Well it turns out that yes it matters which one to pick and here is an example, so let's say we have an example as shown in the figure. So let's say initially in our work list b1 is present and b1 changes so then we are going to add both b2 and b3 to our worklist. Now b2 and b3 have been added to the
work list now we have two options either we could have picked b3 first and then b2 or we could have picked b2 first and then b3. It feels like it's better to do b2 first because once you have done b2 then the information across (b2, b3) edge will become up to date. On the other hand if we do b3 first then what will happen is that the information on this edge would be stale and so maybe b3 would change or maybe b3 will not change, whatever happens but then we will now do b2 and now because of b2 now maybe b3 has to be added again to the worklist because b2 is pointing to b3. So in this case it would have been better to pick b2 before b3.

\subsubsection{Algorithm}
In general if some block bi has an edge to some other block bj then it seems better to basically pick bi before bj. If the code has no cycle then we can actually order these basic blocks so that nodes that are reachable are considered later so if bi can reach bj then bj will be considered later and bi would be considered before bj. But if there is a cycle in the control flow graph then both bi and bj can reach each other in which case we can pick either of them and we don't really don't have any good way or heuristic to say that which one to pick. So that's basically the idea that we're going to use to order the selection of basic blocks when there are multiple choices in the worklist. So for a forward DFA, it would be fastest if all predecessors of b are processed before b is processed so that when b is being processed we should be able to use the latest information on all the incoming edges and this is for a forward DFA. In the absence of loops it is possible to order the blocks so that the algorithm converges by processing each basic block at most once. We can just use any topological sort for the graph. A topological sort of a graph is a sort of the graph where if a node x can reach a node y then x appears before y. If we process the nodes in the topologically sorted order of the graph then we are going to get a good efficiency, every basic block would need to be processed at most once in the entire execution of the DFA fixed point iteration however. That is not true if there are cycles, let's say in the
previous example if b3 was actually again pointing to b2 and if we pick b2 first then it's possible that we have to do b2 again because we pick b2 then we pick b3 and b3 changes and because of b3 change, b2 needs to be processed again and maybe because of b2 change b3 needs to be processed again and so this can keep going on for some time until  we hit bottom or we hit some fixed point in which case we have to process each basic block potentially more than once. So, in presence of loops in general the reverse postorder is a good idea for forward DFA and postorder is a good idea for backward DFA. There are no theoretical guarantees that it's going to give you the best efficiency but in general it helps because in the presence of cycles if we can just figure out what is the reverse postorder (Reverse post order basically captures the fact that if a node x can reach node y then in the reverse postorder x would appear before y in the absence of cycles and even if there are cycles some reverse postorder will give you some arbitrary ordering because there could be multiple reverse post orders and so it will give you some arbitrary ordering and that would typically work well for a forward DFA because for blocks that are not involved in the cycle it would give you a topological sort.

Isomorphically for the backward DFA, like reverse postorder is ordering things that are reachable later, postorder is ordering things that are reachable earlier because that's what we want in a backward DFA. So reverse postorder and postorder are just inverses of each other and so while one works better for forward the
other works better for backward DFA.

\clearpage
\section{March $12^{th}$ discussion}

\begin{itemize}
    \item \textbf{Arpit} : In the forward DFA, we are only initializing the OUT of every basic block. Does it matter if we were initializing bot IN and OUT or only OUT to TOP value.

        \textbf{A:} NO, it doesn't matter if we initialize both values or just the OUT value as in first iteration all the IN values will be again computed from the predecessors' OUT value. But, conceptually it is better to assume that all the values have been set to TOP.
    \item \textbf{Arpit} : Does the definition of semilattice guarantee unique TOP and BOTTOM values?

        \textbf{A:} No, semilattice is just the set of all values and the less than equal to operator. We can have multiple elements which are not less that any other element because it is a partial order. From DFA point of view, we want to have a unique TOP value. For Example, in constant propagation, we defined a new TOP value just because we wanted a unique TOP value. Also, we don't need a unique BOTTOM value.

    \item \textbf{Anirudh} : Can we prove that the worklist algorithm and the earlier algorithm we had for DFA give the same result?

        \textbf{A:} We can have an example where both algorithms give different answers. For example, we have a disconnected CFG, so the worklist algorithm will never reach the other half but the earlier DFA may change some values in the other half as well and thus produce a different answer(both the answers would be equally good).

        But we define our transfer functions such that if $IN=TOP$ then $OUT=TOP$. Then in this case both algorithms are equivalent. We can use inductive argument to prove the equivalence (by using the fact that in an iteration, a value can potentially change only if its predecessors' value has changed in the previous iteration).

    \item \textbf{Anirudh} : What if in a semilattice there are multiple first common descendants? Is it possible?

        \textbf{A:} ...

    \item \textbf{Namrata} : If the graph is disconnected, doesn't it mean that it is a dead code?

            \textbf{A:} Exactly, both those algorithm will only differ for the graphs where we don't really care about the answer.

    \item \textbf{Arpit} : If the graph has only one entry and one exit then how can the graph be disconnected?

            \textbf{A:} Yes it is possible. There can be $if(0)\{ ...\}$ or some part of the code that never gets accessed. Then we don't want entry to reach there and that's the diconnected part of the program.

    \item \textbf{Anirudh} : Can we say, if we have a level in graph where all the values are set to TOP then the worklist algorithm terminates prematurely?

            \textbf{A:} This confusion arises from the fact that we have defined the Data Flow set of ``equaltions``(using the equal to sign). We could have defined it of the form $OUT[B]\leq meet(p_i)$.

            Very informally we can say that both algorithms are equivalent. But once we go into the mathematics and proofs then we need to have some properties for the input like graph should not be disconnected or if $IN=TOP$ then $OUT=TOP$ etc.

    \item \textbf{Jai} : In reaching definitions we did not remove the definitions in which $x$ was being used but in common subexpression elimination we remove those. In reaching definitions, how is the definition still valid if $x$ has been overwritten?

            \textbf{A:} In one we are interested in the values i.e. the values $y$ can have and in other we are interested in the expressions i.e. what different expressions $y$ can hold(for example for phi nodes, where we only care what different definitions can reach a point and we are not interested in whether $x$ has changed or not).

    \item \textbf{Jai} : In three analysis, reaching definitions, must reach and common subexpression, the first two look quite similar.

            \textbf{A:} It's all about application. In reaching definitions, we are interested in all the definitions that reach this point. In must reach, we want the definitions that ``must`` reach this point. Reaching definitions may be used for phi nodes or specialized program paths. Must reach definitions is useful for dominator analysis or we want to decrease the distance between definition and use. So every analysis has it's value in different context.

    \item \textbf{Sonu} : Given the semilattice, how can we define transfer function?

            \textbf{A:} We have defined the semilattice and we will define the transfer function. Note that these are orthogonal things. It is possible that two different transfer functions have same semilattice. However, there are some properties that transfer function must have with respect to the semmilatice and we will study that.
\end{itemize}
\clearpage


\subsection{Curse of dimensionality}


\subsection{Asymptotic mean and variance}


\section{Compute grid}

\subsection{Processing Element}

\begin{figure}[h]
    \centering
    \includegraphics[scale=0.5]{images/PE.png}
    \caption{Weight Stationary PE}
    \label{fig:fig1}
\end{figure}

Processing Element (PE) is the smallest unit of logic, which is replicated to a multi-dimensional grid. Each PE contains registers to store one value each of input, weight and output, and performs one \textbf{M}ultiply-and-\textbf{AC}cumulate operation each cycle using those values to generate one output value. Depending on the way data flows through the compute module, later explained in Section \ref{}, one data-structure out of inputs, weights and the outputs are kept stationary in the PE and the remaining are streamed across the rows and columns of the systolic grid. A schematic representation of weight stationary PE, where weights are chosen as the data-structure that remain stationary within a PE, is depicted in Figure \ref{fig:fig1}.

Within a PE, the flow is synchronous. In the given figure, the input value will move eastward every cycle. Similarly, for the MAC operation to be complete, the output, once computed, should be able to flow to the next PE, typically downwards.

\subsection{Array of PEs}

\begin{figure}
    \centering
    \includegraphics[scale=0.4]{images/grid.png}
    \caption{Systolic grid}
    \label{fig:fig2}
\end{figure}

The above mentioned PE, is replicated along two dimensions, and interconnected along both the dimensions with immediate PEs, as depicted in Figure \ref{fig:fig2}. In this weight stationary dataflow, input maps flow along the rows from East to West, and output maps flow from North to South.

Each row and column, have a FIFO at the start, from where data is sent into the array. These FIFOs are referred as RowFIFO and ColumnFIFO respectively. They provide the access-execute decoupling. If the FIFOs weren't present, the logic which reads data from memory subsystem and sends it to buffer would stall, if there is any stall in systolic array. With access-execute decoupling, number of stalls can be reduced.

\subsection{Dataflows}
Three different multi-dimensional vectors (input maps, output maps, weight filters) are involved in the convolution operation. One of them is first populated into the systolic grid, with each PE holding one or few values throughout the entire computation. Based on which of the three stays {\em stationary} inside the PE, the dataflow can be classified into input stationary (parts of input is stored in each PE), output stationary (parts of output is accumulated and stored inside each PE) and weight stationary (weight is stored in each PE). Figure \ref{fig:dataflows} gives one possible mapping of a feature map and a weight filter for each possible stationary configuration.

\begin{figure}[h]
    \centering
    \includegraphics[scale=0.4]{images/Dataflow.png}
    \caption{Different Systolic Dataflows (a). Output Stationary (b). Weight Stationary (c). Input Stationary \cite{}}
    \label{fig:dataflows}
\end{figure}

\subsection{Tensor ALU}
The Tensor ALU is similar to the vector ALUs present in modern processors, and supports tensor operations. The tensor ALU performs other operations that correspond to a DNN such as activation functions, batch-normalization, etc. that cannot be performed on the systolic array. In addition the same tensor ALU can also be reused for performing multi-level vector reduction of partial sums across the channels.

%TODO: Write advantages and disadvantages of each
%\subsection{Design space}
%Based on the area and timing targets, different parameters could be varied.
%
%\begin{enumerate}
%    \item Size of systolic grid (Rows, Columns) can be varied according to the area budget, to get proportional performance.
%    \item Similarly, given a number of PEs fit a certain area budget, different spatial orientations (1D, 2D, 3D, etc.) can be explored to find the best configuration.
%    \item In addition, the bit-precision of inputs, weights and outputs can be varied to trade-off accuracy vs (performance/power/area).
%    \item Finally, the dataflow pattern can also be varied for different input sizes and systolic grid dimensions, to find the best performing dataflow. As discussed earlier, certain applications benefit from certain dataflow patterns.
%\end{enumerate}
%
%As a generic implementation, the goal is to have a multi-precision PE, which can simultaneously support multiple dataflows, and find the best dimensions of systolic grid which best fits the area budget.
%


\documentclass[10pt,a4paper]{article}

\renewcommand*\rmdefault{ppl}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[english]{babel}                 % Anpassung f\"ur deutsche Sprache
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{listings}
\usepackage{algorithm,algorithmic}
\usepackage{a4wide}
\usepackage{bm}

\newtheorem{theorem}{Theorem}[section]
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{corollary}[theorem]{Corollary}

\usepackage{amssymb}

\newenvironment{proof}[1][Proof]{\begin{trivlist}
\item[\hskip \labelsep {\bfseries #1}]}{\end{trivlist}}
\newenvironment{definition}[1][Definition]{\begin{trivlist}
\item[\hskip \labelsep {\bfseries #1}]}{\end{trivlist}}
\newenvironment{example}[1][Example]{\begin{trivlist}
\item[\hskip \labelsep {\bfseries #1}]}{\end{trivlist}}
\newenvironment{remark}[1][Remark]{\begin{trivlist}
\item[\hskip \labelsep {\bfseries #1}]}{\end{trivlist}}
\DeclareMathOperator*{\argmin}{arg\,min}

\newcommand{\qed}{\hfill \ensuremath{\Box}}
\setlength{\parindent}{0pt}

\raggedbottom

\begin{document}
\section*{Multilevel S$_N$ for applications in radiation therapy}
The idea of this project is to implement an efficient, but readable S$_N$ code framework. Such a code framework can be used for different applications and we wish to first use it for problems in radiation therapy. Furthermore, we can use the framework to test the idea of multilevel S$_N$, which splits the spatial domain into grids with varying resolution and uses a fine quadrature set on coarse grids as well as coarse quadratures on fine grids\footnote{One could also think about going one step further by extending the multilevel idea to uncertainty quantification for multilevel S$_N$.}. Two main working "products" could be papers on \textit{S$_N$ for radiation therapy}, \textit{Multilevel S$_N$}, \textit{Effects of quadrature sets and triangular discretizations} as well as a functioning and documented software. In the following, we briefly mention the fundamentals of the S$_N$ method and state the idea of multilevel S$_N$:

\section{The discrete ordinates (S$_N$) method}
The radiative transfer (or linear Boltzmann) equation reads
\begin{align}\label{eq:radTransfer}
\partial_t \psi(t,\bm x,\bm\Omega) + \bm\Omega\cdot\nabla \psi(t,\bm x,\bm\Omega) + \sigma_a(\bm x)\psi(t,\bm x,\bm\Omega) = \sigma_s(\bm x) \int_{\mathbb{S}^2} k(\bm\Omega\cdot\bm\Omega')\left(\psi(t,\bm x,\bm\Omega')-\psi(t,\bm x,\bm\Omega)\right)\,d\bm\Omega'.
\end{align}
The equation describes the dynamics of particles traveling through a background material. Particles are subject to absorption (described by the absorption cross section $\sigma_a$) as well as scattering (described by the scattering cross section $\sigma_s$ as well as the collision kernel $k$). Note that for now, we omit the effects of sources. Also, when following the work of Kerstin \cite{kupper2016models}, one often solves the continuous slowing down (CSD) equation for treatment planning, which is closely related to the radiative transfer equation. Hence, a starting point for obtaining a working code base could be \eqref{eq:radTransfer}.

In the field of radiation therapy, the radiative transfer equation is often discretized by Monte-Carlo methods, which commonly yield an increased runtime and stochastic noise. Therefore, we choose a discrete ordinates (S$_N$) discretization (which has its own disadvantages such as ray-effects) instead. The first step to obtain a discretized system is to derive a finite dimensional, nodal representation of the angular variable $\bm\Omega\in\mathbb{S}^2$. Here, different strategies to obtain an accurate discretization exist and we are going to start with a straight forward product quadrature rule on the sphere. To obtain a finite set of directions in $\mathbb{S}^2$, we describe the angular variable
\begin{align*}
\bm\Omega = (\cos\phi \sin \theta, \sin\phi \sin \theta, \cos \theta)^T\;
\end{align*}
at finite positions of $\theta$ and $\phi$. In the following, we choose the nodal positions of $\mu = \cos(\theta) \in[-1,1]$ to be the quadrature points of a Gauss quadrature rule with $N_q$ quadrature points, while choosing an equally spaced grid for $\phi$, i.e.
\begin{align*}
\phi_i = i \Delta\phi \quad \text{for} \quad i=1,\ldots,N_q \quad \text{and}
\quad \Delta\phi = \frac{2\pi}{N_q}\;.
\end{align*}
Now, we have a finite number of nodal directions (also called ordinates) on the sphere, which we denote by $\Omega_k$ for $k = 1,\cdots, Q$, where $Q:=N_q^2$. The solution at these ordinates is defined as $\psi_p(t,\bm x):=\psi(t,\bm x,\bm\Omega_p)$. Furthermore, one can construct a quadrature rule
\begin{align*}
\int_{\mathbb{S}^2} \psi(t,\bm x,\bm\Omega)\,d\bm\Omega \approx \sum_{q=1}^Q w_q \psi_q(t,\bm x).
\end{align*}
Now, evaluating the radiative transfer equation \eqref{eq:radTransfer} at the chosen set of ordinates while making use of the above quadrature rule yields
\begin{align*}
\partial_t \psi_p(t,\bm x) + \bm\Omega_p\cdot\nabla \psi_p(t,\bm x) + \sigma_a(\bm x)\psi_p(t,\bm x) = \sigma_s(\bm x) \sum_{q=1}^Q w_q k(\bm\Omega_p\cdot\bm\Omega_q)\left(\psi_q(t,\bm x)-\psi_p(t,\bm x)\right).
\end{align*}
This is now a system of $Q$ advection equations with a coupling term on the right hand side. Such an equation can be solved in a straight forward manner with a finite volume discretization (we could also think about using DG). For this, we split the (for now two-dimensional) spatial domain into cells $V_{ij} = [x_{i-1/2},x_{i+1/2}]\times[y_{j-1/2},y_{j+1/2}]$ and describe the time evolution of the solution on these cells. Let us denote the solution in cell $V_{ij}$ at time $t_n$ as
\begin{align*}
\psi_{pij}^n := \frac{1}{\vert V_{ij}\vert}\int_{V_{ij}} \psi_p(t_n,\bm x)\,d\bm x.
\end{align*}
At time $t_0 = 0$, the initial particle density is known. To obtain a scheme which time updates the solution, we (when leaving out certain details) replace the time derivative by a finite difference approximation
\begin{align*}
\partial_t \psi_p(t,\bm x_{ij})\Big\vert_{t = t_n} &\approx \frac{\psi_{pij}^n-\psi_{pij}^n}{\Delta t}\;, \\
(\bm\Omega_p)_x\partial_x \psi_p(t^n,\bm x)\Big\vert_{\bm x = \bm x_{ij}} &\approx \frac{1}{|V_{ij}|} \left( g_{p,i+1/2,j}^n - g_{p,i-1/2,j}^n \right)\;, \\
(\bm\Omega_p)_y\partial_y \psi_p(t^n,\bm x)\Big\vert_{\bm x = \bm x_{ij}} &\approx \frac{1}{|V_{ij}|} \left( g_{p,i,j+1/2}^n - g_{p,i,j-1/2}^n \right)\;.
\end{align*}
The function $g_{p,i+1/2,j}$ is called the numerical flux, which approximates the solution at the cell interface at $x_{i+1/2}$. One straight forward choice is the upwind flux
\begin{align*}
g_{p,i+1/2,j} :=
\begin{cases}
(\bm\Omega_p)_x \psi_{pij}^{n} & \text{ if }(\bm\Omega_p)_x>0\\
(\bm\Omega_p)_x \psi_{p,i+1,j}^{n} & \text{ else}
\end{cases}\;.
\end{align*}
When replacing differential operators by these approximations, we obtain the first-order scheme
\begin{align}\label{eq:FV}
\psi_{pij}^{n+1} =& \psi_{pij}^{n} - \frac{\Delta t}{\vert V_{ij} \vert} \left( g_{p,i+1/2,j} - g_{p,i-1/2,j} + g_{p,i,j+1/2}- g_{p,i,j-1/2} \right)\nonumber \\
&+\Delta t  \sigma_{s,ij}\sum_{q=1}^{N_q} w_q k(\bm\Omega_p\cdot\bm\Omega_q) \left(\psi_{q,ij}^{n}-\psi_{p,ij}^{n}\right)-\Delta t\sigma_{a,ij} \psi_{pij}^{n}\;.
\end{align}
This is the equation which needs to be implemented. It can be extended to higher order, but we should start with order one.

\section{Physical model}

\section{Multilevel S$_N$}
In the following, we wish to apply ideas from multilevel Monte-Carlo as well as multilevel stochastic-Collocation to the S$_N$ context. Usually, one is not interested in the angular flux $\psi$, but one rather wishes to determine some quantity of interest, which could for example be the radiation dose received by the patient or the angular flux in a certain region. Let us call this scalar quantity $d$, where the bracket denotes a spherical integration and $d:\mathbb{S}^2\rightarrow\mathbb{R}$. We assume a quantity constant in time and space for sake of simplicity. Assume that we can approximate the quantity $d$ with our code using a certain spatial resolution $h:=\Delta x = \Delta y$ as well as $Q$ quadrature points. Then, when the output for resolution $h$ and $Q$ is denoted as $\tilde d^{h,Q}_k$ for $k = 1,\cdots,Q$. This output can be interpreted as a continuous grid function\footnote{A more elegant interpretation is possible when using a spherical harmonics reconstruction or a grid-based quadrature like the octahedron or icosahedron quadrature.}
\begin{align*}
d_{h,Q}(\Omega) := \tilde d^{h,Q}_k \quad \text{ with $k$ s.t. $\Omega_k$ is the closest ordinate to }\Omega.
\end{align*}
Now the L$^2$ distance to the exact quantity of interest $d$ can be divided into an quadrature as well as a spatial error. Adding and subtracting the term $d_{h}$ inside the brackets, where $d_h:\mathbb{S}^2\rightarrow\mathbb{R}$ uses an exact quadrature with a fixed spatial resolution $h$ (i.e. $Q\rightarrow\infty$) yields
\begin{align*}
\Vert d_{h,Q} - d \Vert_{L^{2}(\mathbb{S}^2)} \leq \Vert d_{h,Q} - d_{h} \Vert_{L^{2}(\mathbb{S}^2)}+\Vert d_{h} - d \Vert_{L^{2}(\mathbb{S}^2)}.
\end{align*}
Then, the first term is the quadrature error and the second term is the spatial discretization error. Assume that the following error estimates hold:
\begin{align*}
\Vert d_{h} - d \Vert_{L^{2}(\mathbb{S}^2)} &\lesssim h^{\alpha} \simeq N_x^{-\alpha} \\
\Vert d_{h,Q} - d_{h} \Vert_{L^{2}(\mathbb{S}^2)} &\lesssim Q^{-\beta}.
\end{align*}
A certain accuracy $\varepsilon$ can be obtained by bounding both error terms by $\varepsilon/2$, i.e. the number of spatial cells must be
\begin{align*}
N_x^{-\alpha} \simeq \varepsilon \quad \Rightarrow \quad N_x \simeq \varepsilon^{-1/\alpha}
\end{align*}
and the number of quadrature points has to be
\begin{align*}
Q^{-\beta} \simeq \varepsilon \quad \Rightarrow \quad Q \simeq \varepsilon^{-1/\beta}
\end{align*}
Assume the cost to run a simulation are $C = N_x^3 \cdot Q$. Then we get
\begin{align}\label{eq:costsMC}
C \simeq \varepsilon^{-\frac{3}{\alpha} - \frac{1}{\beta}}.
\end{align}
Now let us make a multilevel ansatz. For this we define $L$ levels for the spatial grid resolution $h_{\ell}$ with $\ell = 1,\cdots,L$. Then, we can write $d_{h_L,Q}(\Omega)$, i.e. the quantity of interest for the finest resolution $h_L$ with a certain number of quadrature points as
\begin{align}\label{eq:MLSNAnsatz}
d_{h_L,Q}(\Omega) = d_{h_1,Q}(\Omega) + \sum_{\ell = 2}^L \left( d_{h_{\ell},Q}(\Omega) - d_{h_{\ell-1},Q}(\Omega) \right) = \sum_{\ell = 1}^L Y_{\ell,Q}(\Omega)
\end{align}
where we use $Y_{1,Q} = d_{h_1,Q}$ and for $\ell\neq 1$ we use $Y_{\ell,Q} := d_{h_{\ell},Q} - d_{h_{\ell-1},Q}$. Now we must pick an adequate quadrature for the difference terms $Y_{\ell,Q}$. Note that since \eqref{eq:MLSNAnsatz} should converge for $L\rightarrow\infty$, we expect the difference terms on higher levels to fall to zero. This indicates requiring a less accurate quadrature rule on higher levels. Hence, let us define the number of quadrature points on $L$ levels to be $Q_{1} \geq \cdots \geq Q_{L}$, which gives the multilevel S$_N$ (MLS$_N$) estimator
\begin{align}\label{eq:MLSNEstimator}
d_{(h)_{\ell},(Q)_{\ell}}(\Omega) = d_{h_1,Q_1}(\Omega) + \sum_{\ell = 2}^L \left( d_{h_{\ell},Q_{\ell}}(\Omega) - d_{h_{\ell-1},Q_{\ell}}(\Omega) \right) = \sum_{\ell = 1}^L Y_{\ell,Q_{\ell}}(\Omega).
\end{align}
From this starting point, we can again derive the required costs as done in \eqref{eq:costsMC}. An optimal number of levels $L$ as well as the resolution of quadrature and space discretization are then obtained by minimizing these costs for a fixed desired accuracy $\varepsilon$.
\begin{figure}
\centering
\includegraphics[scale=0.25]{test.jpeg}
\end{figure}
\newpage
\bibliographystyle{plain}
\bibliography{references}

\end{document}


% Boris Jeremic (@ucdavis.edu)
\section{Total  Lagrangian Finite  Deformation  (TLFD) Twenty-Node Brick Element
Commands }

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{}
%\label{TLFD20Element}

\begin{verbatim}
element TLFD20nBrick eleTag? nd01? nd02? ... nd20? bf1? bf2? bf3?
\end{verbatim}

The  TLFD20nBrick element is the the three dimensional element implemented based
on  the  tensor  operation.  The  argument to construct the element are its tag,
\texttt{TLFD20nBrick},  element  tag,  \texttt{eleTag},  ordered  twenty  nodes,
\texttt{nd01} to \texttt{nd20}, and the body forces, \texttt{bf1}, \texttt{bf2},
\texttt{bf3}. By default, $3 \times 3 \times 3$ integration points are used. The
valid material tags for this element are hyperelastic material (see Tcl Commands
for  Hyperelastic  Material  Models)  and  hyperelasto-plastic material (see Tcl
Commands for Hyperelasto-Plastic Material Models).


\subsection{The $\pi$-calculus}
\label{sec:PA}

\subsubsection{Syntax and Notations}

The $\pi$-calculus \cite{pi_paper, pi_book} is a name passing calculus, in the sense that computations are represented by passing $\it{names}$ through $\it{channels}$.  The $\pi$-calculus uses lower-case letter to denote a name, which could be either a channel or a variable.

Syntax of the $\pi$-calculus given in Table \ref{pi-syn} also denotes part of its semantics.  The prefix of a process denotes its capability, which could be
\begin{inparaenum}[(i)]
  \item receiving names from a channel;
  \item sending names through a channel; or
  \item performing an internal action that cannot be observed from outside.
\end{inparaenum}
Informally, a process could :
\begin{inparaenum}[(1)]
  \item evolve to another process after performing an action;
  \item evolve as two processes who are running at the same time;
  \item evolve as one of the two processes;
  \item evolve to another process when two names become equal;
  \item a process with a private name;
  \item infinite parallel composition of the same process; or
  \item an inert process that does nothing.
\end{inparaenum}
Finally, two processes are structurally congruent, $\equiv$, if they are equal up to structure rules listed in Table \ref{pi-str-cong}

In the rest of this section, we distinguish reduction relation($\longrightarrow$) from labelled transition relations($\overset{\alpha}{\longrightarrow}$).  Specifically, the assertion $P \longrightarrow P'$ states that process $P$ can evolve to process $P'$ as a result of an action {\it{within}} $P$.  On the other hand, $P \overset{\alpha}{\longrightarrow} Q$ indicates that $P$ can evolve to Q after performing action $\alpha$.  Reduction relations are formally defined in Table \ref{pi-str-red} and transition relations are formally defined in Table \ref{pi-trans}.

\begin{table}[h]
  \begin{center}
  \begin{tabular}{ l c l  l l c l  l }
$\alpha$&:: = &                                                                    	& action           &P & :: =  & & process \\
              & $|$ & $u(\widetilde{x}$)                                       	& input            &   & $|$  & $ \alpha.P$&prefix\\
              & $|$ & $\overline{u} \langle \widetilde{y} \rangle$	& output          &   & $|$ &$P\ |\ P$& composition\\
              & $|$ & $\tau$                                                         	& silent/internal & & $|$ &$P\ +\ P$& summation\\
                                                                                &&&&& $|$ &$ [x = y].P $& match \\
                                                                                &&&&& $|$ & $(\mathit{\nu}x).P$&restriction\\
                                                                                &&&&& $|$ & $!P$ & replication\\
                                                                                &&&&& $|$ & 0 & inert process\\
  \end{tabular}
  \end{center}
  \caption{Syntax of the $\pi$-calculus}
  \label{pi-syn}
\end{table}

\begin{table}[h]
  \begin{center}
  \begin{tabular}{ l r c l }
  SC-MAT					&$[x = x] \alpha.P$			&$\equiv$&$\alpha.P$\\
  SC-SUM-ASSOC		&$P_1\ +\ (P_2\ +\ P_3)$	&$\equiv$&$(P_1\ +\ P_2)\ +\ P_3$\\
  SC-SUM-COMM		&$P_1\ +\ P_2$				&$\equiv$&$P_2\ +\ P_1$\\
  SC-SUM-INACT		&$P\ +\ 0$					&$\equiv$&$P$\\
  SC-COMP-ASSOC	&$P_1\ |\ (P_2\ |\ P_3)$	&$\equiv$&$(P_1\ |\ P_2)\ |\ P_3$\\
  SC-COMP-COMM		&$P_1\ |\ P_2$				&$\equiv$&$P_1\ |\ P_2$\\
  SC-COMP-INACT		&$P\ |\ 0$						&$\equiv$&$P$\\
  SC-RES					&$\nu z\ \nu w\ P$			&$\equiv$&$\nu w\ \nu z\ P$\\
  SC-RES-INACT		&$\nu z\ 0$					&$\equiv$&$0$\\
  SC-RES-COMP		&$\nu z\ (P_1\ |\ P_2)$		&$\equiv$&$P_1\ |\  \nu z\ P_2, if\ z\not{\in} fn(P_1)$\\
  SC-REP					&$!P$							&$\equiv$&$P\ |\ !P$\\
  \end{tabular}
  \end{center}
  \caption{The axioms of structural congruence}
  \label{pi-str-cong}
\end{table}

\vspace{20 mm}

\begin{table} [H]
  \begin{center}

  \begin{tabular}{ c c }
\multicolumn{2}{c}{R-INTER $\frac{}{(\bar{x}\langle y \rangle.P_1\ +\ M_1)\ |\ (x(z).P_2\ +\ M_2)\ \longrightarrow\ P_1\ |\ P_2\{y/z\}}$}\\
\\
R-PAR $\frac{P_1\ \longrightarrow\ P^{'}_{1}}{P_1\ |\ P_2\ \longrightarrow\ P^{'}_{1}\ |\ \ P_{2}}$ &
R-RES $\frac {P\ \longrightarrow\ P^{'}} {\nu z P\ \longrightarrow\ \nu z P^{'}} $\\
\\
R-STRUCT $\frac {P_1\ \equiv\ P_2\ \ P_2\ \longrightarrow\ P_2^{'}\ \ P_2^{'}\ \equiv\ P_1^{'}\ } {P_1\ \longrightarrow\ P^{'}_{1}} $&
R-TAU $\frac {} {\tau  P\ + \ M\ \longrightarrow\ P}$
  \end{tabular}
  \end{center}
  \caption{Reduction rules in the $\pi$-calculus}
  \label{pi-str-red}
\end{table}

\vspace{10 mm}

\begin{table}[H]
  \begin{center}
  \includegraphics[scale=0.4]{pi_trans.png}
  \end{center}
  \caption{Transition rules in the $\pi$-calculus}
  \label{pi-trans}
\end{table}

\subsubsection{Equivalence Relations}

Equivalence relations are important concepts in an algebraic system.  Unfortunately, the variety of equivalence relations in the $\pi$-calculus and other classical process calculi are usually obstacles to learning a calculus.  This section will give a gentle introduction to the most important equivalence relation in the $\pi$-calculus, the strong barbed bisimulation, followed by short definitions cited from \cite{pi_book} in a designed order.  By reading this section, readers will obtain a basic understanding for all equivalences in the Figure \ref{pi-eq} and most other common equivalences in literature.

$\bf{Barbed\ relations}$ study the behaviour of a process via examining its potential interactions with the environment.  Formally,

\begin{defn}
For each name or co-name $\mu$, the observability predicate $\downarrow_\mu$ is defined by:\\
    (1) P $\downarrow_\mu$ if P can perform an input action via channel $\mu$\\
    (2) P $\downarrow_\text{$\overline{\mu}$}$ if P can perform an output action via channel $\mu$
\end{defn}

Strong barbed relations can be defined as follows:

\begin{defn}
A relation S is a $\mathbf{strong\ barbed\ bisimulation}$ if whenever (P,Q) $\in$ S,\\
(1) P $\downarrow_\mu$ implies Q$\downarrow_\mu$\\
(2) P $\overset{\tau}{\longrightarrow}$ P' implies Q $\overset{\tau}{\longrightarrow}$ Q' for some Q' with (P', Q') $\in$ S \\
(3) Q $\downarrow_\mu$ implies P$\downarrow_\mu$\\
(4) Q $\overset{\tau}{\longrightarrow}$ Q' implies P $\overset{\tau}{\longrightarrow}$ P' for some P' with (P', Q') $\in$ S
\end{defn}

\begin{defn}
P and Q are $\mathbf{strong\ barbed\ bisimilar}$ if (P,Q) $\in$ S for some barbed bisimulation S.
\end{defn}

\begin{defn}
P and Q are $\mathbf{strong\ barbed\ congruence}$ if C[P] and C[Q] are strong barbed bisimilar for any context C\footnote{A context, $C[\cdot]$ is a term written using the same syntax of the $\pi$-calculus and an additional constant $\cdot$.  Substituting the $\cdot$ by a process P result in a $\pi$-calculus term $C[P]$ }.
\end{defn}

\begin{defn}
P and Q are $\mathbf{strong\ barbed\ equivalent}$ if P $|$ R and Q $|$ R are strong barbed bisimilar for any process R.
\end{defn}

$\bf{Bisimulation,\ bisimilarity,\ congruence,\ and\ equivalent}$, as revealed in the barbed relations, are four related concepts in process calculi.  For this reason, in the rest of this subsection, where various of equivalent relations will be introduced, only one of the four concepts in each group will be defined explicitly.  The meaning of the rest three relations in each group should be apparent.

\begin{defn}
A relation S is a $\mathbf{reduction\ bisimulation}$ if whenever (P,Q) $\in$ S,\\
(1) P $\overset{\tau}{\longrightarrow}$ P' implies Q $\overset{\tau}{\longrightarrow}$ Q' for some Q' with (P', Q') $\in$ S \\
(2) Q $\overset{\tau}{\longrightarrow}$ Q' implies P $\overset{\tau}{\longrightarrow}$ P' for some P' with (P', Q') $\in$ S
\end{defn}

In terms of evaluation strategy, the $\pi$-calculus could adopt either the early instantiation scheme or the late instantiation scheme.  In the early instantiation scheme, variables are instantiated as soon as an input message is received, more precisely, $\frac{-}{a(x).P\ \overset{av}{\longrightarrow}P\{v/x\}}$.  On the contrary, in the late instantiation scheme, bound variables of input actions are instantiated only when they are involved in an internal communication.

\begin{defn}
A binary relation S on processes $P$ and $Q$ is an $\mathbf{early\ simulation}$, $\sim_e$,  if PSQ implies that\\
1. If P $\overset{\alpha}{\longrightarrow}$ P' and $\alpha$ is a free action\footnote{an internal action or a free output.}, then for some Q', Q $\overset{\alpha}{\longrightarrow}$ Q' and P'SQ'\\
2. If P $\overset{x(y)}{\longrightarrow}$ P' and y$\not{\in}$ n(P,Q), then for all w, there is Q' such that Q$\overset{x(y)}{\longrightarrow}$ Q' and P\{w/y\}SQ\{w/y\}\\
3. If P $\overset{\bar{x}(y)}{\longrightarrow}$ P' and y$\not{\in}$ n(P,Q), then for some Q', Q $\overset{\bar{x}(y)}{\longrightarrow}$ Q' and P'SQ'
\end{defn}

\begin{defn}
A binary relation S on agents is a $\mathbf{late \ simulation}$, $\sim_l$,  if PSQ implies that\\
1. If P $\overset{\alpha}{\longrightarrow}$ P' and $\alpha$ is a free action, then for some Q', Q $\overset{\alpha}{\longrightarrow}$ Q' and P'SQ'\\
2. If P $\overset{x(y)}{\longrightarrow}$ P' and y$\not{\in}$ n(P,Q), then for some Q', Q$\overset{x(y)}{\longrightarrow}$ Q' and for all w, P\{w/y\}SQ\{w/y\}\\
3. If P $\overset{\bar{x}(y)}{\longrightarrow}$ P' and y$\not{\in}$ n(P,Q), then for some Q', Q $\overset{\bar{x}(y)}{\longrightarrow}$ Q' and P'SQ'
\end{defn}

\begin{defn}
P and Q are $\mathbf{full\ bisimilar}$, P $\approx^c$ Q	, if P$\sigma\ \approx$ Q$\sigma$ for every substitution $\sigma$.
\end{defn}

\begin{defn}
A binary relation R over processes is an $\mathbf{open\ bisimulation}$, $\approx_o$, if for every pair of elements (p,q) $\in$ R and for every name substitution $\sigma$ and every action $\alpha$, whenever p$\sigma \overset{\alpha}{\longrightarrow}$ p' then there exists some q' such that q$\sigma\overset{\alpha}{\longrightarrow}$q' and (p', q') $\in$ R.
\end{defn}

\begin{defn}
A relation is $\mathbf{ground\ bisimulation}$, $\approx_g$, iff whenever P $\approx_g$ Q, there is z$\not{\in}$fn(P, Q) such that if P$\overset{\alpha}{\longrightarrow}$Q, where $\alpha$ is an action, then then Q$\overset{\alpha}{\Rightarrow }\approx_g$P.
\end{defn}

To conclude this subsection, Figure \ref{pi-eq}, cited from \cite{pi_book}, presents the hierarchy of equivalences in the $\pi$-calculus.

\begin{figure} [h]
  \begin{center}

\begin{picture}(100, 160)
  \put(90, 160){$weakest$}
  \put(100, 80){\vector(0,1){70}}
  \put(100, 80){\vector(0,-1){70}}
  \put(90, 0){$strongest$}

  \put(50,160){$\approx_g$}
  \put(60,130){\line(0,1){20}}
  \put(50,120){$\approx,\ \cong$}
  \put(45,90){\line(2,3){14}}
  \put(60,90){\line(0,1){20}}
  \put(0,80){$\approx_g^c,\ \approx^c,\ \cong^c$}
  \put(60,80){$\approx_l$}
  \put(45,55){\line(2,3){14}}
  \put(45,55){\line(0,1){20}}
  \put(45,40){$\approx_l^c$}
  \put(45,15){\line(0,1){20}}
  \put(45,0){$\approx_o$}
\end{picture}
  \end{center}

  \begin{center}
  \begin{tabular}{ c l c l c l}
$\approx_g:$&$ground\ bisimilarity$&$\approx:$&$bisimilarity$&$\cong:$&$barbed\ equivalence$\\
$\approx_g^c:$&$ground\ congruence$&$\approx^c:$&$full\ bisimilarity$&$\cong^c:$&$barbed\ congruence$\\
$\approx_l:$&$late\ bisimilarity$&$\approx_l^c:$&$late\ congruence$&$\approx_o:$&$open\ bisimilarity$\\
  \end{tabular}
  \end{center}
  \caption{Hierarchy of equivalences in the $\pi$-calculus}
  \label{pi-eq}
\end{figure}


\paragraph{}
Due to the fact that there are a few error indicators, it could be hard to determine the relationship between them and whether a subdomain needs refinement or not.
As a result, specific technique is required in order to search for the patterns in the data to find this relationship.
After that, the final decision which is a binary classification in this case as each subdomain will be labeled as ``refine'' or ``not refine'' can be made based on the input of the error indicators explained in Sec.~\ref{adap_sec:error_indicator}.
Hence, the discovery of regularities plays a key role in the adaptive analysis and an automatic discovery of regularities is usually associated with pattern recognition by the help of algorithms.

%   ----    %
\subsection{Training Set}
\paragraph{}
The training set is determined by numerical examples in Fig.~\ref{adap_fig:svm_train_chole} and Fig.~\ref{adap_fig:svm_train_cantilever}.
The training data is annotated with whether a subdomain is refined or not and the model can study the data and learn to classify each subdomain based on all four features explained in \ref{adap_sec:error_indicator}.
\begin{figure}[!ht]
    \centering
    \begin{subfigure}[b]{0.5\linewidth}
        \scalebox{0.5}{
            \includegraphics{adaptivity/images/adap_svm_train_chole_0.png}
        }
        \caption{Mesh and displacement field before refinement}
    \end{subfigure}
    \begin{subfigure}[b]{0.5\linewidth}
        \scalebox{0.45}{
            \includegraphics{adaptivity/images/adap_svm_train_chole_1.png}
        }
        \caption{Mesh and displacement field after refinement}
    \end{subfigure}
    \caption{Mesh refinement for square plate with circular hole \citep{Duval2018}}
    \label{adap_fig:svm_train_chole}
\end{figure}
%
\begin{figure}[!ht]
    \centering
    \begin{subfigure}[b]{0.4\linewidth}
        \scalebox{0.6}{
            \includegraphics{adaptivity/images/adap_svm_train_cantilever_0.png}
        }
    \end{subfigure}
    \begin{subfigure}[b]{0.4\linewidth}
        \scalebox{0.6}{
            \includegraphics{adaptivity/images/adap_svm_train_cantilever_1.png}
        }
    \end{subfigure}\\
    \begin{subfigure}[b]{1\linewidth}
        \centering
        \scalebox{0.7}{
            \includegraphics{adaptivity/images/adap_svm_train_cantilever_2.png}
        }
    \end{subfigure}
    \caption{Mesh refinement of a short cantilever beam \citep{Zienkiewicz2005500}}
    \label{adap_fig:svm_train_cantilever}
\end{figure}
%
With the same examples, uniform mesh of quadtree is conducted and the same region are marked as refined as shown in Fig.~\ref{adap_fig:svm_train_my}.
\begin{figure}[!ht]
    \centering
    \begin{subfigure}[b]{1\linewidth}
        \centering
        \scalebox{0.25}{
            \includegraphics{adaptivity/images/adap_svm_train_my_chole.eps}
        }
    \end{subfigure}
    \begin{subfigure}[b]{0.49\linewidth}
        \scalebox{0.25}{
            \includegraphics{adaptivity/images/adap_svm_train_my_cantilever_0.eps}
        }
    \end{subfigure}
    \begin{subfigure}[b]{0.49\linewidth}
        \scalebox{0.25}{
            \includegraphics{adaptivity/images/adap_svm_train_my_cantilever_1.eps}
        }
    \end{subfigure}
    \caption[Training data for SVM]{Training data for SVM: Cells in black is marked as refined.}
    \label{adap_fig:svm_train_my}
\end{figure}
%
Criteria taken into consideration are:
\begin{enumerate}
    \item Ratio of the area of the cell to the total area
    \item Minimal angle formed by the intersecting lines connected by scaling center and adjacent polygon vertexes
    \item Eigenvalue error indicator for displacement
    \item Eigenvalue error indicator for stress
\end{enumerate}
%
\subsection{Regularization for MLP}
\paragraph{Bagging}
\label{adp_sec:ml_bagging}
Bagging is a technique that utilize multiple models in order to increase the accuracy of the prediction \citep{Breiman1996}.
The principle of idea of that is simple: training multiple models separately and let them vote for the prediction.
It is adopted vastly in machine learning.
The idea behind the optimization technique is that different models often make different errors on the same test set.
Take a set of $k$ regression models for an example, error $\epsilon_i$ is made by each model on each example.
The errors are drawn from a zero-mean multivariate normal distribution with variances $E[\epsilon_i^2]$ = $\nu$ and covariances $E[\epsilon_i \epsilon_j]$=$c$.
As a result, the overall error determined by averaging of all models is $\frac{1}{k}\sum_i \epsilon_i$.
The expected squared error of the ensemble predictor is
\begin{equation}
    \begin{aligned}
    E[(\frac{1}{k} \sum_i \epsilon)^2] &=
    \frac{1}{k^2}[
        \sum_i (
            \epsilon_i^2 + \sum_{j\neq i}\epsilon_i \epsilon_j
        )
    ] \\
    &= \frac{1}{k}\nu + \frac{k-1}{k}c
    \end{aligned}
\end{equation}
%
The bagging technique does not help if the errors are highly correlated and $c=\nu$ as the mean squared error can reduce to $\nu$.
On the other hand, the effect of this technique may be significant if the errors are highly uncorrelated and $c=0$, in which situation the overall mean squared error becomes $\frac{1}{k}\nu$.
In other words, the overall mean squared error decreases linearly with the model size.
To conclude, the performance of this technique must not be worse than any of its individual model and a significant improvement can be expected if its members make independent errors.

\paragraph{}
There are several different methods to train ensemble.
For instance, each member of the ensemble can be trained with same training set but different algorithms or different training set with same algorithm.
The reuse of the same kind of models and algorithms is allowed thanks to the bagging.
The bagging technique used in the proposed method is constructing $k$ different training set.
Each training is built by sampling the original data set randomly.
In other words, each individual model may have some dataset which is not appear in any other models.
There is also some dataset are shared by multiple models.

\paragraph{}
Due to the large number of initialization parameter for a neural network, bagging technique can be extremely effective because the random selection of parameters such as initialized weight, mini-batches, hyper-parameters and so on leads to a different outcome even with the same training data.
The method is proved to be reliable and powerful for minimizing the generalization error.
Averaging dozens of models is widely adopted in those who won the machine learning contests and a recent example is Netflix Grand Price \citep{koren2009}.

\paragraph{Dropout}
Dropout \citep{JMLR:v15:srivastava14a} is another optimization approach used in the proposed method.
An effective but computationally cheap regularizing technique is provided.
Roughly speaking, the dropout can be regarded as one method to create different models in bagging described in Sec.~\ref{adp_sec:ml_bagging} by training multiple models.
One problem in bagging is that it could be impractical when each model is so large that the training can be computationally expensive in terms of time and space complexity.
Hence, the number of models used in ensembles tends to be in the range of $5$ and $10$ and the winner of ILSVRC \citep{DBLP:journals/corr/SzegedyLJSRAEVR14} used $6$ models.
As a comparison, dropout provides an cheap algorithm that can train and evaluate a large number of bagged ensemble.

\paragraph{}
In dropout, random hidden units are disabled during the training, which can be easily achieved by manually setting the weight to zero.
By doing so, the contribution of this hidden unit is ignored based on the fact that matrix production is used.
It can also be implemented by removing the unit completely from the network.
In bagging technique, $k$ different models with different training data are trained.
This is shared by the dropout as well, while dropout supports larger number of neural networks.
During the training process using dropout, a minibatch-based learning algorithm introduced in Sec.~\ref{lr_sec:ml_sgd} is used to make insignificant steps.
Whenever a sample is loaded into the minibatch, a random binary mask is applied to all of the inputs and hidden units in the network.
The probability a unit be disabled is independent from others and shall be determined from the user input as one of the hyperparameter.
In the proposed method, an input units has $20\%$ chance of being disabled and the number is $50\%$ for the hidden units.

\paragraph{}
More formally, assume that a mask vector $\mu$ specifies which units to include and $J(\theta,\mu)$ defines the cost of the model defined by parameters $\theta$ and mask $\mu$ \citep{Goodfellow-et-al-2016}.
After that, the training of the dropout becomes minimizing $E_\nu J(\theta,\mu)$.
The expectation may includes great number of terms while it could be possible to determine an unbiased estimation of its gradient by sampling values of $\mu$.

\paragraph{}
The training of the dropout is quite different to that of bagging.
Models use same parameters but different training data in bagging as shared parameters help to represent numerous number of models without occupying too many memory.
During the training using bagging, individual model is trained until convergence while during the training using dropout, most of the models are not explicitly trained.
That is because the number of the possible sub-network in dropout is too significant to finish the training within the lifetime of the universe.
Instead, only minor proportion are trained for a single step.
Parameter sharing guarantee that the remaining sub-networks are able to arrive at good settings.
The bagging algorithms are followed after that.
For instance, the training data used by individual model actually is sampled from the original training set.
Simple majority of the votes from all members are adopted to determine the prediction of the ensemble.
Although bagging and dropout does not require a explicitly probabilistic model, it is assumed that a probability distribution $p^{(i)}(y|x)$ will be the output.
The prediction of the ensemble is given by averaging all these distributions,
\begin{equation}
    \frac{1}{k} \sum_{i=1}^k (y|x)
\end{equation}
%
For dropout, individual model with a mask vector $\mu$ defines a probability distribution $p(y|x, \mu)$.
The arithmetic mean over all masks is given by
\begin{equation}
    \sum_\mu p(\mu) p(y|x, \mu)
    \label{adap_eq:ml_dropout_sum}
\end{equation}
%
where $p(\mu)$ is the probability distribution used to sample $\mu$ during the training.
Since numerous number of terms are involved in the summation, it could be difficult to calculate Eq.~\ref{adap_eq:ml_dropout_sum} without simplification.
However, a deep neural network do not allow such simplification.
Instead, it is possible to approximate the simplification with sampling by calculate the mean value from many masks.
It is suggested that around $15$ masks are able to provide satisfactory performance \citep{Goodfellow-et-al-2016}

\paragraph{}
Yet, the existence of a better approach allows the determination of a satisfactory approximation to the predictions of the ensemble at the cost of one forward propagation.
It can be achieved by adopting geometric mean instead of the arithmetic mean of the ensemble member's predicted distributions \citep{WardeFarley2014SelfinformedNN}.
However, using the geometric mean instead of the arithmetic mean leads to the result which is not guaranteed to be a probability distribution.
In order to enforce a probability distribution as the result, the requirement that all sub-models must assign a non-zero probability to all events is imposed.
After that, the resulted distribution is normalized again.
The unnormalized probability distribution from geometric mean is:
\begin{equation}
    \tilde{p}_{ensemble}(y|x) = 2^d \sqrt{\prod_\mu p(y|x,\mu)}
\end{equation}
%
where $d$ is the number of units that my be dropped.
To simplify the presentation, a uniform distribution over $\mu$ is adopted.
But it should be noted that non-uniform distributions can be treated as well.
Eq.~\ref{adap_eq:ml_renormalize} need to be performed on the ensemble in order to determine the prediction
\begin{equation}
    p_{ensemble}(y|x) = \frac{
        \tilde{p}_{ensemble}(y|x)
    }{
        \sum_{y^\prime} \tilde{p}_{ensemble}(y^\prime|x)
    }
    \label{adap_eq:ml_renormalize}
\end{equation}
%
The key idea behind the dropout is to approximate $p_{ensemble}$ by calculating $p(y|x)$ in one model \citep{hinton2012}.
The aim of this improvement is to capture the correct output from that unit.
It is called weight scaling inference rule which shows outstanding performance empirically.
\paragraph{}
Since an inclusion probability of $\frac{1}{2}$ is widely adopted, the weight scaling rule tends to half the weights after training.
The model is used as usual after that.
It can also be achieved by multiplying the states of the units by $2$ during training.
Both of these two methods are to narrow the difference between the expected total input to a unit at test time and that at the training when approximately half of the hidden units are dropped.
For those models without nonlinear hidden units, the weight scaling inference rule gives exact result.
For instance, a softmax regression classifier with $n$ input variables represented by the vector $v$ is considered:
\begin{equation}
    P(y=y|v)=softmax(W^Tv+b)_y
\end{equation}
%
They can be indexed into the family of sub-models by element-wise multiplication of the input with a binary vector $d$:
\begin{equation}
    P(y=y|v;d) = softmax(W^T(d \odot v)+b)_y
\end{equation}

\subsubsection{Indicator used in adaptivity}
\paragraph{}
All terms used in the performance indicators in machine learning are introduced in Sec.~\ref{lr_ml:indicators}
The balance of the indicators is highly dependent on classifiers' objective.
Take the spam detector for an example, a false positive can be more dangerous than a false negative as an important e-mail being marked as spam can be a disaster.
While in adaptivity, there may be no favour over either of them.
It is because the refinement of a cell with lower error or leaving a cell with higher error unrefined may not produce significant influence on the final result.
As a result, F1 score can be the most important indicator as it takes both recall rate and precision into consideration.

\subsection{Result}
\paragraph{}
All training data are standardized by Eq.~\ref{adap_eq:svm_standardized} where $\overline{x}$ and $\sigma$ is the mean and standard deviation of the data.
It makes features in training data have zero means and unit variance.
Cells that need to be refined are labeled as $1$ and the rest are labeled as $0$.
Radial basis function with $\sigma=0.7624$ is adopted as kernel function in SVM.
    \begin{equation}
        x^\prime = \frac{x-\overline{x}}{\sigma}
        \label{adap_eq:svm_standardized}
    \end{equation}
Half of the training data ($320$ out of $640$) are used to train the model and the reset are used for cross validation.
Different class weights are set for testing different performance in regard to all indicators in Fig.~\ref{adap_fig:svm_performance_0}.
\begin{figure}[h!]
    \centering
    \scalebox{0.3}{
        \includegraphics{adaptivity/images/svm_performance_0.eps}
    }
    \caption{Accuracy, precision, recall rate and F1 score vs different class weight}
    \label{adap_fig:svm_performance_0}
\end{figure}
%
A class weight is a vector that influence the predication directly.
The model will calculate the probability for each classification based on the input and the one with higher probability will be chosen as the result in default situation.
Change class weight to $1:2$ will force the model to choose first class when its probability is higher than $66.67\%$ instead of $50\%$.
A class weight of $3:7$ was chosen from Fig.~\ref{adap_fig:svm_performance_0} to guarantee a balance between precision and recall rate.
The corresponding result is listed in Tab.~\ref{adap_tab:svm_result}.
\begin{table}[h!]
    \centering
    \caption{Result of cross validation}
    \begin{tabular}{cc}
        \toprule
        Accuracy    &   84.38\%    \\
        Precision   &   64.38\%     \\
        Recall rate &   66.20\%     \\
        F1 score    &   65.28\%     \\
        \bottomrule
    \end{tabular}
    \label{adap_tab:svm_result}
\end{table}
%  compare auc