\documentclass[conference]{IEEEtran}

\usepackage{graphicx}
\usepackage{float}
\usepackage[ruled,vlined,linesnumbered,noresetcount]{algorithm2e}
\usepackage{amsmath}
\usepackage{booktabs}
%\usepackage[options ]{algorithm2e}
% correct bad hyphenation here
\hyphenation{op-tical net-works semi-conduc-tor}

\begin{document}

\title{A Novel Heuristic for Evolutionary Clustering}

% author names and affiliations
% use a multiple column layout for up to three different
% affiliations

% make the title area
\maketitle

% As a general rule, do not put math, special symbols or citations
% in the abstract
\begin{abstract}
Clustering is considered a challenging problem of data mining due to its unsupervised nature. The literature is inundated with algorithms and concepts related to determining the most suitable clustering structure in data. These techniques have a mathematical model of a cluster and attempt to obtain a result that shall represent this model as closely as possible. However as the problem of clustering is NP hard such strategies have disadvantages such as converging to local optima or suffering from the curse of dimensionality. In such scenario, meta heuristics could be more suitable strategies. Such techniques utilizes biologically inspired techniques such as swarm intelligence, evolution etc. to traverse the search space. Due to their inherent parallel nature, they are most robust towards converging to a local optima. The objective (cost) function used by such meta heuristics is responsible for guiding the agents of the swarm towards the best solution. Hence it should be designed to achieve trade-off between multiple objectives and constraints and at the same time produce relevant clustering. In this paper, a cost function is proposed (PSO-2) to produce compact well separated clusters by using the concept of intracluster and intercluster distances. Experiments have been performed on artificial benchmark datasets where performance of the particle swarm optimizer using the proposed cost function is evaluated against other evolutionary and non evolutionary algorithms. The clustering structures produced by the  methods have been evaluated using distance based and internal cluster validation metrics to demonstrate that the performance of PSO-2 is comparable to other techniques.  \\
\end{abstract}

% no keywords
\begin{IEEEkeywords}Data clustering, Applications of evolutionary algorithms, Genetic
algorithm.
\end{IEEEkeywords}

% For peer review papers, you can put extra information on the cover
% page as needed:
% \ifCLASSOPTIONpeerreview
% \begin{center} \bfseries EDICS Category: 3-BBND \end{center}
% \fi
%
% For peerreview papers, this IEEEtran command inserts a page break and
% creates the second title. It will be ignored for other modes.
\IEEEpeerreviewmaketitle

\section{Introduction}
% no \IEEEPARstart
Clustering involves creating coarse grained descriptions of data. This is done with the main purpose of highlighting a set of autonomous regions in the data known as clusters. Clustering has found wide applicability in multiple fields ranging from web mining and information retrieval through customer segmentation and bio-informatics \cite{hur}. Regardless of the domain to which it is applied or the algorithm used, the goal of clustering is uniform i.e. to maximize homogeneity within cluster and heterogeneity between clusters \cite{cha}. Clustering algorithms produce partitions of data which may be hard (nodes have unique memberships to clusters), soft (nodes have memberships to multiple clusters) or fuzzy (nodes have varying degrees of memberships to every cluster). Thus, a hard partition of a dataset X = ${x_1, x_2, x_3, x_4 ... x_n}$ where $x_i$ is a data point which stands for a n-dimensional feature vector, is a collection $C = {C_1, C_2, ... , C_k}$ of non null clusters such that $C_i \cap C_j = \phi$ for $ i \neq j$. Relaxing the mutual dis-junction condition to allow $C_i \cap C_j \neq \phi$ for $ i \neq j$ creates overlapping partitions \cite{ma}. \\

Clustering is considered the most challenging task of machine learning due to the unavailability of spatial distribution of the data in terms of its clustering tendency \cite{hur} \cite{cha} \cite{ma}. Furthermore, the requirement of dealing with various types of attributes (binary, categorical, continuous, discrete), their conditions (complete or missing) and scales (nominal, ratio, interval, ordinal) has to be factored in while deciding the approach of clustering.  In addition to this, the lack of information about the orientation of the clusters, their numbers, shapes, densities and volumes makes it difficult in selecting a particular clustering technique and in evaluating the results obtained by it \cite{val1} \cite{val2}. From the optimization perspective, Clustering is considered as a NP-hard grouping problem and the literature consists of a wide range of objective functions, approximation algorithms and heuristics for solving it \cite{hur}.\\

Each approach has its own bias and comes with certain advantages and disadvantages to a given analysis or application scenario. Popular partition based clustering algorithms are K-means, PAM, CLARA, CLARANS. These algorithms have a objective function which is non-convex and hence the results might be locally optimal. Hierarchical algorithms too exist like AGNES, DIANA, BIRCH, ROCK and CHAMALEON etc. but these are computationally expensive. One inherent weakness of hierarchical clustering is that clusters formed in an iteration cannot be undone in subsequent iterations. DBSCAN, OPTICS are approaches that define clusters as dense regions separated by less dense regions. DBSCAN though is popular but the prior specification on $\epsilon$ and $MinPts$ makes it sensitive to parameter tuning. In real life experiments, various types of noise are introduced at different experimental stages during data collection, due to which the performance of these algorithms are less than ideal \cite{hur} \cite{ma}. To effectively handle noisy data, we need an algorithm that is able to overcome noise. These traditional algorithms are based on gradient based methods and are difficulty to extend to multi-objective problems as their basic design doesn't allow the consideration of multiple solutions \cite{zha} \cite{luo}. Population based methods such as evolutionary algorithms have an inherent advantage here \cite{muk} \cite{muktwo}.\\

In the context of solving NP-hard optimization problems, evolutionary algorithms are considered to be particularly effective in obtaining near optimal solutions in reasonable time \cite{lih} \cite{sid}. Evolutionary algorithms are meta-heuristics based on optimization of a fitness function that guides the process of evolutionary search. Computational advantages are also present as the algorithms can be parallelized leading to increased coverage and possibly faster convergence \cite{lin}. However, evolutionary algorithms have their owns drawbacks as different features of these algorithms have to chosen in advance such as encoding scheme for the data points, choice of operators for crossover and mutation and a fitness (cost) function. Fitness functions are basically validity criteria and hence it is better to use multiple criteria in order to avoid the drawbacks of individual ones. One strategy for this is to assign weights to individual criteria and then optimize, but this approach only works when the multiple criteria are commensurable. In this paper, the fitness function proposed computes cost using a variation of intra cluster distance and inter cluster distance. This variation allows computation to be performed in linear time. The conventional definitions of these distances requires $O(n^2)$ computations. Hence a reduction in computations is achieved leading to faster execution.\\

The rest of this paper is organized as follows. In Section II, an overview of the existing clustering algorithms used
for identification of clusters in data is presented, these include both evolutionary and non evolutionary approaches. In Section III, the mathematical model of the proposed evolutionary algorithm and its intuition is described. In Section IV, the algorithm is evaluated and compared with existing clustering algorithms using benchmark artificial datasets. Experimental results are presented to highlight the suitability of the proposed technique. Section V consists of the summary of the work along with the conclusion. \\

\section{Related Work}
Clustering algorithms follow evolutionary and non evolutionary strategies and a selection of the best known algorithms \cite{ma} of both categories are described below.\\
\subsection{Traditional Clustering Algorithms}

Clustering techniques belonging to the k-partitioning family minimize the objective criterion known as variance or sum of squared distances $SSD = \sum_{k=1}^{k} \sum_{x_i \epsilon c_k} \left \| x_i - c_k \right \| ^2$ and hence the detected clustering structures correspond to minimum variance. K-Means is the popular algorithm of this category with a time complexity of $O(kN)$ and hence is scalable. Hierarchical clustering algorithms have a fully unsupervised approach but time complexity is     higher i.e. $O(n^3)$ and $O(n^2 log n)$ for priority queue implementations. This makes scalability an issue on larger datasets. Model based clustering algorithms assume that clusters are generated by probability distributions whose parameters such as mean and covariance matrix have to be estimated using maximum likelihood estimation. Eqn. 1 is the prior probability that denotes the percentage of instances that came from a cluster $c$. Eqn. 2 gives the mean i.e. expected value of attribute $j$ from cluster $c$. Eqn. 3 gives the covariance matrix denoting the covariance of attributes $j,k$ in cluster $c$.

\begin{equation}
P(c)= \frac{1}{n} \sum_{i=1}^{n} P(c\vert\vec{x_i})
\end{equation}
\begin{equation}
\mu_{c,j} = \sum_{i=1}^{n} \Bigl(\frac{P(c\vert\vec{x_i})}{nP(c)}\Bigr)x_{i,j}
\end{equation}
\begin{equation}
\sum_{c}{\mathstrut}_{j,k} = \sum_{i=1}^{n}\Bigl(\frac{P(c\vert\vec{x_i})}{nP(c)}\Bigr)(x_{i,j} - \mu_{c,j})(x_{i,k} - \mu_{c,k})
\end{equation}
\begin{equation}
 P(c\vert\vec{x_i}) = \frac{P(\vec{x_i}\vert c)P(c))}{\sum_{i=1}^{k}P(\vec{x_i}\vert c)P(c)}
 \end{equation}
 \begin{equation}
P(\vec{x_i}|c) = \frac{1}{\sqrt{2\pi\sum_{c}}} \exp\Bigl(- \frac{1}{2}(\vec{x_i} - \vec{\mu_c})^T \sum_{c}^{-1}(\vec{x_i} - \vec{\mu_c})\Bigr)
\end{equation}
\vspace{10px}

\subsection{Neural Network Based Clustering}
Unsupervised learning using neural networks has two main models which are Kohenon's Self Organizing Maps and Grossberg's Adaptive Resonance Theory. ART represents a family of neural networks in which the current input vector is matched to the category prototype vector. If the input vector doesn't match with the prototype then a new prototype is selected. This means that other prototypes are not affected by new input. In ART systems, there are two layers: $L1$ has the comparison field where input patterns and expectations are compared. Layer 2 $L2$ is the competition layer where winner takes all learning strategy is implemented. Connections from Layer 1 to Layer 2 perform clustering operations and each row of weights is a prototype pattern. Layer 2 to Layer 1 connections perform pattern recall (expectation). The vigilance parameter is a user defined gain control mechanism that controls the degree of similarity required for patterns to be assigned to the same cluster. \\

Kohenon's Self Organizing Map are neural networks that map their weights (without any target vector) to conform to the given input data with the goal of representing multidimensional data in a lower dimension. SOM's have a two layered architecture with input nodes connected to every computational nodes to form a lattice pattern. Weight vectors $W_{ij} = W_{ij1}, W_{ij2}, ... , W_{ijn}$ are of the same dimensions as the input vectors $V$. The SOM algorithm has following steps:\\

\begin{itemize}
\item Each nodes weights are initialised randomly.
\item A vector is chosen at random from the set of input vectors and presented to the training data.
\item Every nodes is compared to the input pattern to determine which nodes weights are closest to the input pattern. This winning node is the Best Matching Unit (codebook vector). Closeness measure is euclidean distance.
\item Once the BMU is identified, codebook vectors in the neighbourhood of the BMU are updated. The update rule for the codebook vectors in the neighbourhood is given in Eqn.6 is the BMU, $h_{ci}$ is a non increasing neighbourhood function.
\item The neighbourhood function is given in Eqn. 7\\
\end{itemize}

Where,
\begin{equation}
m_i(t+1) = m_i(t) + h_{ci}(t)[x(t) - m_i(t)]
\end{equation}
\begin{equation}
h_{ci}(t) = \alpha(t) . exp(- \frac{||r_i - r_c||^2}{2 \sigma(t)^2})
\end{equation}

\subsection{Evolutionary Clustering Algorithms}
Particle swarm optimization is an intelligent optimization algorithm which belongs to a class of optimization algorithms called Meta Heuristics. It is based on the paradigm of "swarm intelligence" and is inspired by social behavioral animals like birds or fish proposed by J. Kennedy \textit{et al} in 2007 and modified in 2011 by M. Clerc \textit{et al} \cite{pso}\cite{pso1}\cite{pso2}. Swarm intelligence systems have simple agents that interact locally with each other and with the environment (without any central control as to how individual agents should behave). Local interaction between the agents however lead to emergence of a complex global behavior. In PSO, solutions are assumed as points in an n-dimensional space. The swarm particles $\overrightarrow{x_i}$ are solutions that are randomly initialized in the search space $X$ and these have to traverse it in search of the global optima with a velocity $v_i$. \\

Differential evolution \cite{da} is a stochastic direct search method that minimizes the objective (cost) function  by using the difference of solution vectors to create new candidate solutions. NP D-dimensional parameter vectors $x_{i,G} i = 1, 2, ..., NP$ are used as a population for each generation $G$. DE generates new parameter
vectors by adding the weighted difference between two population vectors to a
third vector. The parameters of this new vector also called the "mutated vector" are mixed with the parameters of another predetermined vector, the target
vector, to obtain the "trial vector". If the
trial vector results in a lower cost than the target vector it
replaces the target vector in the subsequent generation. At every stage of the process each population vector has to serve once as the target vector so that NP
competitions take place in one generation. \\

Genetic algorithms are a class of computational models inspired by biological evolution and selection. The mechanism followed by these models is to encode a potential solution to a problem on a chromosome like data structure and apply recombination operators to preserve critical information. Problem dependent parts are selection of suitable objective function and problem encoding. The optimization problems are often non linear and so it is not possible to treat each parameter as an independent variable which can be solved in isolation from others \cite{ga}.

\section{Mathematical Model}

The mathematical model of a meta heuristic algorithm has structured sections with each section having a logical meaning. The sections in the meta heuristic in this section are Problem definition, Parameters definition, Initialization and the Iterative procedure. The meta heuristic follows the same structure as the standard PSO implemented in \cite{pso1} \cite{pso2}.

\subsection{Problem Definition}

\subsubsection{Cost Function}
The key component of the problem definition is the cost function which is to be optimized using the meta-heuristics. The aim is to achieve clusters well separated and compact. This is done using two distances $WCD$ and $ICD$.

\begin{equation}
WCD = \sum_{i = 1}^{n} \text{dist}(c_i , C)
\end{equation}
\begin{equation}
ICD = \sum_{j = 1}^{k} \sum_{i = 1}^{n}\text{dist}(c_i , C_j)
\end{equation}
Thus $WCD$ is the distance of a point from its centroid summing over all points. $ICD$ is the distance of point from other centroids summing over all points. $WCD$ is to be minimised and $ICD$ maximized, and this is the objective of PSO-2. To achieve efficiency the cost to be minimized is taken as $\frac{WCD}{ICD}$.\\

\begin{algorithm}
\SetAlgoLined
\KwResult{Cost of the cluster: z}
 m = coordinates of centroids\;
 X = coordinates of data points\;
 M = Calculate distance of each point from all centroids\;
 ind = Assign point to centroid closest to it\;
 WCD = Add distances of all points to assigned to a centroid\;
 ICD = M - WCD\;
 z = WCD / ICD\;
 \caption{Cost function: PSO-2}
\end{algorithm}

The dissimilarity matrix isn't required to be calculated and the computations for $WCD$ , $ICD$ are completed in $O(n)$ computations.\\

\subsubsection{Decision Variables}
The decision variables are the $k$ centroids of the data, where each centroid can be represented as a $n$ dimensional vector $\overrightarrow{R^n}$. Thus the total decision variables are $n * k$. The range of the decision variables is limited to be between the maximum $VarMax$ and minimum value $VarMin$ of the points in the data, this also achieves the objective of restricting the search space.

\subsection{Parameters of Meta heuristics}
The parameters of the meta heuristics are maximum iterations allowed for the model, the size of the swarm, coefficients of inertia $w$ and its damping ratio $w_{damp}$ , personal acceleration coefficient $c_1$ (assigned to every particle) and the social acceleration coefficient $c_2$ (assigned to the entire swarm). In addition, limits are set to restrict the velocities of individual particles to the range $VelMin$ to $VelMax$. Constrictions coefficients $phi_1$, $phi_2$, $phi$, $chi$ are also defined as per the Eqn.\ref{eq:1} and Eqn.\ref{eq:2}. The values for these variables are set according to standard settings which may be modified.

\begin{equation}\label{eq:1}
phi = phi_1 + phi_2
\end{equation}\label{eq:2}
\begin{equation}
chi = \frac{2}{phi - 2 + \sqrt{phi^2 - 4*phi}}
\end{equation}
\begin{equation}
w = chi
\end{equation}
\begin{equation}
c_1 = phi_1 * chi
\end{equation}
\begin{equation}
c_2 = phi_2 * chi
\end{equation}

\subsection{Initialization and the Iterative Procedure}

Each particle in the swarm has the components such as position $x_{pos}$, velocity $x_{vel}$, cost $x_{cost}$, best position $x_{pbest}$ and best cost $x_{cbest}$. The Global best $x_{gbest}$ is also defined to be $- \infty$ as the objective function is to be maximized. The positions of the particles are initialized using uniform distribution between $VarMax$ and $VarMin$. The velocities of the particles is initialized to zero. The values for the best position of a particle and the cost at the best position is currently set to the initial position and the cost at the initial position respectively. \\

The iterative procedure updates the values for the best position and best cost obtained so far by each particle in the swarm and also update the variable that records the global best cost and position obtained by the swarm together. The update rule for the particle velocity is given by Eqn. \ref{eq:3}\\

  \begin{equation}\label{eq:3}
   x_{i+1}(vel) = w * x_{i}(vel) + c_1 * (x_{pbest} - x_{pos}) + c_2 * (x_{gbest} - x_{pos})\\
  \end{equation}

The velocity limits are applied as in Eqn: \ref{eq:4} and \ref{eq:5} to prevent the particle velocity from increasing or decreasing beyond the thresholds.

  \begin{equation}\label{eq:4}
  x_{i+1}(vel) = max(x_{i+1}(vel), VelMin)
  \end{equation}
   \begin{equation}\label{eq:5}
  x_{i+1}(vel) = min(x_{i+1}(vel), VelMax)
  \end{equation}

 The position of the particle is updated by the Eqn \ref{eq:6}
   \begin{equation}\label{eq:6}
  x_{i+1}(pos) = x_{i}(pos) * x_{i+1}(vel)
  \end{equation}

  If the position of the particle is outside the thresholds $VarMax, VarMin$ it is updated by the rule in Eqn: \ref{eq:7} and \ref{eq:8}

    \begin{equation}\label{eq:7}
  x_{i+1}(pos) = max(x_{i+1}(pos), VarMin)
  \end{equation}
   \begin{equation}\label{eq:8}
  x_{i+1}(pos) = min(x_{i+1}(pos), VarMax)
  \end{equation}

If the cost of the particle at its new position is better than the cost of the particle at its old position, then the values of personal best cost $x_{cbest}$ and position $x_{pbest}$ so far of the particle are updated to the new values. The values of the global best cost and position of the entire swarm is updated at every iteration to store the best global value at every iteration.

\section{Experiments}

The efficiency of the objective function proposed in this paper is demonstrated vis-a-vis other evolutionary and non evolutionary algorithms on artificial clustering datasets. Standard internal cluster validation metrics such as Separation index $S_i$, Calinski and Harabasz index $CH_i$, Entropy of the distribution of cluster memberships $E$, Pearson Gamma index $P$ and Dunn index $D$ along with distance based statistics such as Within cluster sum of squared distances $M_{wcd}$, Average distance between clusters $M_{avg-CD}$ and Average silhouette width $S_w$  are used to evaluate \cite{val1} \cite{val2}.

\subsection{Dataset}
The artificial benchmarks used for the experiment are given below:

\begin{table}[h!]
\renewcommand{\arraystretch}{1.3}
\caption{Description of the datasets}
\label{table}
\centering
\begin{tabular}{|c|c|c|}
  \hline
\multicolumn{1}{|c|}{\textbf{Sr. No}} & \multicolumn{1}{c|}{\textbf{Name}} & \multicolumn{1}{c|}{\textbf{Size}} \\
  \hline
  1 & Banana &  4811 \\
   \hline
  2 & Cluto &  8000 \\
   \hline
  3 & Cure &  4200 \\
   \hline
  4 & Long &  1261 \\
   \hline
  5 & 2d-4c &  1000 \\
   \hline
  6 & Demo &  300 \\
   \hline
\end{tabular}
\end{table}

\subsection{Results}

\subsubsection{Distance based Statistics}

The within-cluster sum of squares is a measure of the variability of the observations within each cluster. Clusters that have higher values exhibit greater variability of the observations within the cluster. The results of PSO-2 are comparable to other clustering algorithms as seen in Table \ref{mwcd}.

\begin{table}[H]
\caption{Within cluster sum of squares}
\label{mwcd}
\centering
\begin{tabular}{||c|c|c|c|c|c|c||}
  \hline
\multicolumn{1}{|c|}{\textbf{Name}} & \multicolumn{1}{c|}{\textbf{Banana}} & \multicolumn{1}{c|}{\textbf{Cluto}} & \multicolumn{1}{c|}{\textbf{Cure}} & \multicolumn{1}{c|}{\textbf{Long}} & \multicolumn{1}{c|}{\textbf{2d-4c}} & \multicolumn{1}{c|}{\textbf{Demo}}\\
  \hline\hline
  KMeans & 185.20 & 3.42 & 383.25 & 637.02 & 20434.32 & 593.39 \\ [0.5ex]
   \hline
  PAM & 192.88 & 3.42 & 402.30 & 666.56 & 20434.32 & 593.39 \\
   \hline
  ART2 & 190.09 & 37.78 & 1712.27 & 594.87 & 1127417 & 996.77 \\
   \hline
  DE & 185.21 & 3.94 & 385.31 & 637.53
 & 20434.32 & 593.39 \\
   \hline
  PSO & 185.21 & 3.43 & 383.36 & 637.53 & 20434.32 & 593.39 \\
   \hline
  GA & 185.21 & 3.42 & 383.41 & 637.52 & 20434.32 & 593.39 \\
   \hline
  \textbf{PSO-2} & 193.23 & 3.98 & 403.29 & 1051.73 &  94470.4 & \textbf{593.38} \\  [1ex]
  \hline
\end{tabular}
\end{table}

Average distance between clusters should be large as this would mean that the clusters are well separated. Table \ref{wcad} shows the that PSO-2 produces well separated clusters comparable to other algorithms.

\begin{table}[H]
\caption{Average distance between clusters}
\label{wcad}
\centering
\begin{tabular}{||c|c|c|c|c|c|c||}
  \hline
\multicolumn{1}{|c|}{\textbf{Name}} & \multicolumn{1}{c|}{\textbf{Banana}} & \multicolumn{1}{c|}{\textbf{Cluto}} & \multicolumn{1}{c|}{\textbf{Cure}} & \multicolumn{1}{c|}{\textbf{Long}} & \multicolumn{1}{c|}{\textbf{2d-4c}} & \multicolumn{1}{c|}{\textbf{Demo}}\\
 \hline\hline
  KMeans & 0.49 & 300.36 & 1.22 & 1.84 & 62.26 & 5.38 \\ [0.5ex]
   \hline
  PAM & 0.50 & 300.35 & 1.22 & 1.85 & 62.26 & 5.37 \\
   \hline
  ART2 & 0.5 & NaN & 1.18 & 1.72 & \textbf{74.44} & 4.92 \\
   \hline
  DE & 0.49 & 301.83 & 1.22 & 1.85 & 62.26 & 5.37 \\
   \hline
  PSO & 0.49 & 300.36 & 1.22 & 1.85 & 62.26 & 5.37 \\
   \hline
  GA & 0.49 & 300.36 & 1.22 & 1.85 & 62.26 & 5.37 \\
   \hline
  \textbf{PSO-2} & 0.50 & \textbf{303.46} & \textbf{1.22} &  \textbf{2.26} & 66.9 & 5.37 \\  [1ex]
  \hline
\end{tabular}
\end{table}

Silhouette value is a measure of how similar an object is to its own cluster compared to other clusters. The silhouette ranges from −1 to +1, where a high value indicates that the object is well matched to its own cluster and poorly matched to neighboring clusters. Clusters produced by PSO-2 have moderate scores of silhoutte width compared to other algorithms as seen in Table \ref{asw}.

\begin{table}[H]
\caption{Average silhouette width}
\label{asw}
\centering
\begin{tabular}{||c|c|c|c|c|c|c||}
  \hline
\multicolumn{1}{|c|}{\textbf{Name}} & \multicolumn{1}{c|}{\textbf{Banana}} & \multicolumn{1}{c|}{\textbf{Cluto}} & \multicolumn{1}{c|}{\textbf{Cure}} & \multicolumn{1}{c|}{\textbf{Long}} & \multicolumn{1}{c|}{\textbf{2d-4c}} & \multicolumn{1}{c|}{\textbf{Demo}}\\
  \hline\hline
  KMeans & 0.46 & 0.43 & 0.48 & 0.43 & 0.87 & 0.61 \\ [0.5ex]
   \hline
  PAM & 0.47 & 0.43 & 0.45 & 0.42 & 0.87 & 0.61 \\
   \hline
  ART2 & 0.47 & NULL & 0.08 & 0.33 & 0.56 & 0.33 \\
   \hline
  DE & 0.46 & 0.41 & 0.48 & 0.43 & 0.87 & 0.61 \\
   \hline
  PSO & 0.46 & 0.43 & 0.48 & 0.43 & 0.87 & 0.61 \\
   \hline
  GA & 0.46 & 0.43 & 0.48 & 0.43 & 0.87 & 0.61
 \\
   \hline
  \textbf{PSO-2} &  \textbf{0.47} & 0.41 &  0.44 & 0.36 & 0.86 & 0.61 \\  [1ex]
  \hline
\end{tabular}
\end{table}

\subsubsection{Internal Cluster Validation Metrics}

Separation index is computed on the distances for every point to the closest point not in the same cluster. The separation index is then the mean of the smallest proportion of these. Larger value of separation index indicates better clustering and PSO-2 has exceeded baseline SI set by other algorithms on two datasets as seen in Table \ref{si}.

\begin{table}[H]
\caption{Separation Index}
\label{si}
\centering
\begin{tabular}{||c|c|c|c|c|c|c||}
  \hline
\multicolumn{1}{|c|}{\textbf{Name}} & \multicolumn{1}{c|}{\textbf{Banana}} & \multicolumn{1}{c|}{\textbf{Cluto}} & \multicolumn{1}{c|}{\textbf{Cure}} & \multicolumn{1}{c|}{\textbf{Long}} & \multicolumn{1}{c|}{\textbf{2d-4c}} & \multicolumn{1}{c|}{\textbf{Demo}}\\
  \hline\hline
  KMeans & 0.03 & 9.27 & 0.04 & 0.1 & 15.68 & 0.85 \\ [0.5ex]
   \hline
  PAM & 0.03 & 8.79 & 0.04 & 0.10 & 15.68 & 0.85 \\
   \hline
  ART2 & 0.02  & Inf & 0.04 & 0.06 & 50.92 & 0.27 \\
   \hline
  DE & 0.03 & 7.87 & 0.04 & 0.12 & 15.68 & 0.85 \\
   \hline
  PSO & 0.03 & 8.66 & 0.04 & 0.12 & 15.68 & 0.85 \\
   \hline
  GA & 0.03 & 8.59 & 0.04 & 0.12 & 15.68 & 0.85 \\
   \hline
  \textbf{PSO-2} & 0.02 & 8.45 &  0.03 & \textbf{0.21} & 43.43 &  \textbf{0.85}\\  [1ex]
  \hline
\end{tabular}
\end{table}

Although a higher value of CH index as seen in Table \ref{chin} is considered a better result, CH criterion is preferred where clusters are spherical. CH index value obtained by PSO-2 is comparable to other algorithms.

\begin{table}[H]
\caption{Calinski and Harabasz index}
\label{chin}
\centering
\begin{tabular}{||c|c|c|c|c|c|c||}
  \hline
\multicolumn{1}{|c|}{\textbf{Name}} & \multicolumn{1}{c|}{\textbf{Banana}} & \multicolumn{1}{c|}{\textbf{Cluto}} & \multicolumn{1}{c|}{\textbf{Cure}} & \multicolumn{1}{c|}{\textbf{Long}} & \multicolumn{1}{c|}{\textbf{2d-4c}} & \multicolumn{1}{c|}{\textbf{Demo}}\\
  \hline\hline
  KMeans & 5689.28 & 11473.7 & 5683.4 & 1092.8 & 36113.1 & 679.9 \\ [0.5ex]
   \hline
  PAM & 5271.64 & 11447.1 & 5374.5 & 1000.1 & 36113.1 & 679.9 \\
   \hline
  ART2 & 5419.3 & NaN & 517.4 & 619.8 & 730.5 & 229.0 \\
   \hline
  DE & 5688.92 & 9819.6 & 5648.6 & 1091.1 & 36113.1 & 679.9 \\
   \hline
  PSO & 5688.83 & 11450.1 & 5681.6 & 1091.1 & 36113.1 & 679.9 \\
   \hline
  GA & 5688.83 & 11453.2 & 5680.6 & 1091.1 & 36113.1 & 679.9 \\
   \hline
  \textbf{PSO-2} & 5253.3 &  9691.4 & 5359.3 & 268.3 & 11233.4  &  \textbf{679.9}\\  [1ex]
  \hline
\end{tabular}
\end{table}

A higher value of entrophy as seen in Table \ref{ent} indicates higher disorder in the clustering and a lower value is preferred. PSO-2 has produced clustering with lower value for entrophy than other algorithms on three datasets.

\begin{table}[H]
\caption{Entrophy}
\label{ent}
\centering
\begin{tabular}{||c|c|c|c|c|c|c||}
  \hline
\multicolumn{1}{|c|}{\textbf{Name}} & \multicolumn{1}{c|}{\textbf{Banana}} & \multicolumn{1}{c|}{\textbf{Cluto}} & \multicolumn{1}{c|}{\textbf{Cure}} & \multicolumn{1}{c|}{\textbf{Long}} & \multicolumn{1}{c|}{\textbf{2d-4c}} & \multicolumn{1}{c|}{\textbf{Demo}}\\
  \hline\hline
  KMeans & 0.69 & 2.08 & 1.78 & 0.69 & 1.23 & 1.1 \\ [0.5ex]
   \hline
  PAM & 0.65 & 2.08 & 1.77 & 0.68 & 1.23 & 1.1 \\
   \hline
  ART2 & 0.66 & 0 & 1.31 & 0.99 & 0.38 & 1.34 \\
   \hline
  DE & 0.69 & 2.03 & 1.77 & 0.69 & 1.23 & 1.1 \\
   \hline
  PSO & 0.69 & 2.08 & 1.78 & 0.69 & 1.23 & 1.1 \\
   \hline
  GA & 0.69 & 2.08 & 1.78 & 0.69 & 1.23 & 1.1 \\
   \hline
  \textbf{PSO-2} & \textbf{0.65} & \textbf{1.98} & 1.76 & \textbf{0.24} & 0.98 & 1.1 \\  [1ex]
  \hline
\end{tabular}
\end{table}

Higher values of Pearson Gamma Coefficient obtained by PSO-2 indicates better clustering compared to other approaches on three datasets whereas performance on the remaining is in acceptable range as seen in Table \ref{pgc}.

\begin{table}[H]
\caption{Pearson Gamma Coefficient}
\label{pgc}
\centering
\begin{tabular}{||c|c|c|c|c|c|c||}
  \hline
\multicolumn{1}{|c|}{\textbf{Name}} & \multicolumn{1}{c|}{\textbf{Banana}} & \multicolumn{1}{c|}{\textbf{Cluto}} & \multicolumn{1}{c|}{\textbf{Cure}} & \multicolumn{1}{c|}{\textbf{Long}} & \multicolumn{1}{c|}{\textbf{2d-4c}} & \multicolumn{1}{c|}{\textbf{Demo}}\\
  \hline\hline
  KMeans & 0.66 & 0.51 & 0.62 & 0.53 & 0.89 & 0.78 \\ [0.5ex]
   \hline
  PAM & 0.67 & 0.51 & 0.61 & 0.51 & 0.891 & 0.78 \\
   \hline
  ART2 & 0.67 & NA & 0.25 & 0.44 & 0.54 & 0.56 \\
   \hline
  DE & 0.66 & 0.51 & 0.62 & 0.53 & 0.89 & 0.78 \\
   \hline
  PSO & 0.66 & 0.51 & 0.62 & 0.53 & 0.89 & 0.78 \\
   \hline
  GA & 0.66 & 0.51 & 0.62 & 0.53 & 0.89 &  0.78 \\
   \hline
  \textbf{PSO-2} & \textbf{0.67} & \textbf{0.53} & 0.62 & 0.39 & \textbf{0.93} & 0.78 \\  [1ex]
  \hline
\end{tabular}
\end{table}

Higher value of Dunn index as seen in Table \ref{dunn} indicates better clustering results and PSO-2 has obtained clusters with dunn index comparable to those produced by other approaches.

\begin{table}[H]
\caption{Dunn Index}
\label{dunn}
\centering
\begin{tabular}{||c|c|c|c|c|c|c||}
  \hline
\multicolumn{1}{|c|}{\textbf{Name}} & \multicolumn{1}{c|}{\textbf{Banana}} & \multicolumn{1}{c|}{\textbf{Cluto}} & \multicolumn{1}{c|}{\textbf{Cure}} & \multicolumn{1}{c|}{\textbf{Long}} & \multicolumn{1}{c|}{\textbf{2d-4c}} & \multicolumn{1}{c|}{\textbf{Demo}}\\
  \hline\hline
  KMeans & 0.01 & 0.004 & 0.002 & 0.003 & 0.45 & 0.06  \\ [0.5ex]
   \hline
  PAM & 0.003 & 0.003 & 0.005 & 0.01 & 0.45 & 0.06 \\
   \hline
  ART2 & 0.002 & Inf & 0.002  & 0.004 & 0.605 & 0.01 \\
   \hline
  DE & 0.002 & 0.003 & 0.002 & 0.012 & 0.45 & 0.06 \\
   \hline
  PSO & 0.002 & 0.01 & 0.003 & 0.01 & 0.45 & 0.06 \\
   \hline
  GA & 0.002 & 0.001 & 0.003 & 0.01 & 0.45 & 0.06 \\
   \hline
  \textbf{PSO-2} & \textbf{0.004} & 0.003 & \textbf{0.004} &  0.005 &  \textbf{0.94} & 0.05 \\  [1ex]
  \hline
\end{tabular}
\end{table}

\section{Conclusion}
Meta heuristics are considered as efficient alternatives to classical techniques in terms of traversing a search space which is non convex and not solvable in polynomial time. The inherent nature of such algorithms gives them capabilities such as parallelized implementation, fast convergence, ability to consider multiple solutions and the capability to avoid local optima. The objective function for such algorithms has to be chosen to achieve both fast computation as well as achieve a trade-off between multiple objectives. PSO-2 implemented in this paper seeks to uncover clustering structure while respecting constraints of efficiency. The cost function provides modified definitions of intra and inter cluster distances that are calculated in linear time. This achieves an improvement over the existing methods that need $O(n^2)$ to compute both the within and between cluster distances. Clustering structure uncovered by PSO-2 shows compact and well seperated clusters. It however fails to discover non convex clusters in the data. Yet the performance of PSO-2 is comparable to the standard PSO which is based on minimizing within cluster distance only. The experiments have demonstrated that clustering output provided by PSO-2 is comparable to other evolutionary as well as non evolutionary algorithms.

% trigger a \newpage just before the given reference
% number - used to balance the columns on the last page
% adjust value as needed - may need to be readjusted if
% the document is modified later
%\IEEEtriggeratref{8}
% The "triggered" command can be changed if desired:
%\IEEEtriggercmd{\enlargethispage{-5in}}

% references section

% can use a bibliography generated by BibTeX as a .bbl file
% BibTeX documentation can be easily obtained at:
% http://mirror.ctan.org/biblio/bibtex/contrib/doc/
% The IEEEtran BibTeX style support page is at:
% http://www.michaelshell.org/tex/ieeetran/bibtex/
%\bibliographystyle{IEEEtran}
% argument is your BibTeX string definitions and bibliography database(s)
%\bibliography{IEEEabrv,../bib/paper}
%
% <OR> manually copy in the resultant .bbl file
% set second argument of \begin to the number of references
% (used to reserve space for the reference number labels box)
\begin{thebibliography}{}

\bibitem{hur}
Hruschka, E.R., Campello, R.J. and Freitas, A.A., 2009. A survey of evolutionary algorithms for clustering. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 39(2), pp.133-155.

\bibitem{cha}
Chakrabarti, Deepayan, Ravi Kumar, and Andrew Tomkins. "Evolutionary clustering." In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 554-560. ACM, 2006.

\bibitem{ma}
Ma, P.C., Chan, K.C., Yao, X. and Chiu, D.K., 2006. An evolutionary clustering algorithm for gene expression microarray data analysis. IEEE Transactions on Evolutionary Computation, 10(3), pp.296-314.

\bibitem{zha}
Zhang, Xingyi, et al. "A decision variable clustering-based evolutionary algorithm for large-scale many-objective optimization." IEEE Transactions on Evolutionary Computation (2016).

\bibitem{luo}
Luo, J., Jiao, L. and Lozano, J.A., 2016. A sparse spectral clustering framework via multiobjective evolutionary algorithm. IEEE Transactions on Evolutionary Computation, 20(3), pp.418-433.

\bibitem{lin}
Lin, K.P., 2014. A novel evolutionary kernel intuitionistic fuzzy $ c $-means clustering algorithm. IEEE Transactions on Fuzzy Systems, 22(5), pp.1074-1087.

\bibitem{muk}
Mukhopadhyay, A., Maulik, U., Bandyopadhyay, S. and Coello, C.A.C., 2014. A survey of multiobjective evolutionary algorithms for data mining: Part I. IEEE Transactions on Evolutionary Computation, 18(1), pp.4-19.

\bibitem{muktwo}
Mukhopadhyay, A., Maulik, U., Bandyopadhyay, S. and Coello, C.A.C., 2014. Survey of multiobjective evolutionary algorithms for data mining: Part II. IEEE Transactions on Evolutionary Computation, 18(1), pp.20-35.

\bibitem{lih}
Li, H., Zhang, Q. and Deng, J., 2017. Biased multiobjective optimization and decomposition algorithm. IEEE transactions on cybernetics, 47(1), pp.52-66.

\bibitem{sid}
Siddiqi, U.F. and Sait, S.M., 2017. A New Heuristic for the Data Clustering Problem. IEEE Access.

\bibitem{pso}
Eberhart, R. and Kennedy, J., 1995, October. A new optimizer using particle swarm theory. In Micro Machine and Human Science, 1995. MHS'95., Proceedings of the Sixth International Symposium on (pp. 39-43). IEEE.
\bibitem{pso1}
Bratton, D. and Kennedy, J., 2007, April. Defining a standard for particle swarm optimization. In Swarm Intelligence Symposium, 2007. SIS 2007. IEEE (pp. 120-127). IEEE.
\bibitem{pso2}
Clerc, M., 2010. Particle swarm optimization (Vol. 93). John Wiley \& Sons.

\bibitem{da}
Storn, R. and Price, K., 1997. Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. Journal of global optimization, 11(4), pp.341-359.

\bibitem{ga}
Whitley, D., 1994. A genetic algorithm tutorial. Statistics and computing, 4(2), pp.65-85.
 \bibitem{val1}
Hennig, C. and Liao, T. (2013) How to find an appropriate clustering for mixed-type variables with application to socio-economic stratification, Journal of the Royal Statistical Society, Series C Applied Statistics, 62, 309-369.
 \bibitem{val2}
Hennig, C. (2013) How many bee species? A case study in determining the number of clusters. In: Spiliopoulou, L. Schmidt-Thieme, R. Janning (eds.): "Data Analysis, Machine Learning and Knowledge Discovery", Springer, Berlin, 41-49.

\end{thebibliography}

% that's all folks
\end{document}


\section{Generating and Working with IL}

One of BAP's central features is the ability to represent the
semantics of binary code in a simple language called BIL (BAP
Intermediate Language).  This example demonstrates how to lift binary
code to the IL and manipulate it in BAP.  Let's start with the
following assembly file:

\lstinputlisting[basicstyle=\ttfamily,numbers=left]{chap-examples/basic.S}

\subsection{Lifting to the IL}

In this form, the program looks very simple.  Let's see what the BAP
representation of the program shows us.  Compile this file (basic.S)
to an object file with \cmdline{gcc -c basic.S -o basic.o}.  We can
lift this object file with BAP's \cmdline{toil} command: \cmdline{toil
  -bin basic.o -o basic.il}. If you inspect \cmdline{basic.il}, you
should see something like:

\verbatiminput{chap-examples/basic.il}

You can see from this example that BAP IL is much more verbose than
assembly; this is intentional.  It might make BAP IL a little more
tedious to read, but it makes writing analyses much simpler.  Let's go
through some of the IL line by line to explain what is happening.

\begin{verbatim}
addr 0x0 @asm "add    %eax,%ebx"
label pc_0x0
\end{verbatim}
These two statements mark the beginning of a new assembly instruction.
Note the label for pc\_0x0; any jump to address zero will go to to
this label.

\begin{verbatim}
T_t1:u32 = R_EBX_32:u32
\end{verbatim}
Here we are saving the original value of \%ebx before it is modified.
This original value will be used when computing flags.

\begin{verbatim}
T_t2:u32 = R_EAX_32:u32
\end{verbatim}
Here we are saving the original value of \%eax.  \%eax is not modified
in this struction, but a temporary is used to avoid duplicating the
operand in the IL.  Dupliation is a more serious problem for memory
operands.

\begin{verbatim}
R_EBX_32:u32 = R_EBX_32:u32 + T_t2:u32
\end{verbatim}
This is the primary computation of the add instruction: adding \%eax
to \%ebx.

\begin{verbatim}
R_CF:bool = R_EBX_32:u32 < T_t1:u32
\end{verbatim}
Here we explicitly compute the value of the carry flag.  After
executing the add instruction, the carry flag is set if the resulting
value of \%ebx is less than the original value of \%ebx (which is
saved in t).

\begin{verbatim}
R_OF:bool =
  if T_origCOUNT:u32 == 0:u32 then R_OF:bool else
  if T_origCOUNT:u32 == 1:u32 then high:bool(R_EBX_32:u32)
    ^ R_CF:bool else
  unknown "OF undefined after shift":bool
\end{verbatim}
Here we are explicitly computing the overflow flag after the shl
instruction.  Note that the IL uses if then else expressions.  Also
note that the the overflow flag can be set to an unknown expression
when the Intel semantics says the flag should be undefined.

\begin{verbatim}
cjmp R_CF:bool, 8:u32, "nocjmp0"
\end{verbatim}
This is a conditional jump.  If the condition (in this case, the carry
flag) evalutes to true, control will transfer to label pc\_0x8.
Otherwise, it will go to the label nocjmp0.

\subsection{Built-in Graphs and Analyses}

We might want to visualize what's going on.  There are several ways to
do this in BAP, including control flow graphs (CFG), control
dependence graphs (CDG), and data dependence graphs (DDG).  These can
be generated with the iltrans tool.  iltrans takes a program as input,
and then applies transformations in a pipeline.  For instance, we can
print an overview control flow graph by coalescing and then printing
the graph with \cmdline{iltrans -il basic.il -to-cfg -prune-cfg
  -coalesce-ast -pp-ast-asms out.dot}. The resulting file out.dot is a
file that can be processed with GraphViz's dot command: \cmdline{dot
  -Tpdf out.dot -o out.pdf}.  The output should look similar to what
is shown in Figure~\ref{fig:basiccfg}. The next visualizations will
require the program to be in Single Static Assignment (SSA) form.  The
command
\begin{verbatim}
iltrans -il basic.il -to-ssa
\end{verbatim}
will do this.  If we want to print a detailed control flow graph from
SSA form, we can add \cmdline{-pp-ssa out.dot} to the end, for
instance:
\begin{verbatim}
iltrans -il basic.il -to-ssa -pp-ssa out.dot
\end{verbatim}
The output should look similar to what is shown in
Figure~\ref{fig:ssacfg}. Alternatively, we can also convert out of SSA
form and print that CFG by appending \cmdline{-to-cfg -pp-ast-cfg
  out.dot} to the command line, such as in:
\begin{verbatim}
iltrans -il basic.il -to-ssa -to-cfg -pp-ast-cfg out.dot
\end{verbatim}
As can be seen in Figure~\ref{fig:ssaconvcfg}, sometimes CFGs that
have been converted out of SSA can be easier to read. The CDG and DDG
can be generated by using \cmdline{-pp-ssa-cdg} and
\cmdline{-pp-ssa-ddg} respectively. Examples are shown in
Figures~\ref{fig:cdg} and \ref{fig:ddg}.

BAP also has built-in optimizations.  To apply them, simply use the
\cmdline{-simp-ssa} iltrans flag.  If we apply this before producing
the CFG, we can see the IL is greatly simplified: \cmdline{iltrans -il
  basic.il -to-ssa -simp-ssa -to-cfg -pp-ast-cfg out.dot}. This is shown in
Figure~\ref{fig:cfgsimp}.

\begin{figure}[!p]
  \begin{center}
    \includegraphics[height=3in]{chap-examples/basiccfg.pdf}
  \end{center}
  \caption{Example Basic CFG}
  \label{fig:basiccfg}
\end{figure}

\begin{figure}[!p]
  \begin{center}
    \includegraphics[height=.9\textheight]{chap-examples/ssacfg.pdf}
  \end{center}
  \caption{Example Detailed SSA CFG}
  \label{fig:ssacfg}
\end{figure}

\begin{figure}[!p]
  \begin{center}
    \includegraphics[height=.9\textheight]{chap-examples/ssaconvcfg.pdf}
  \end{center}
  \caption{Example SSA CFG converted out of SSA}
  \label{fig:ssaconvcfg}
\end{figure}

\begin{figure}[p]
  \begin{center}
    \includegraphics{chap-examples/cdg.pdf}
  \end{center}
  \caption{Example CDG}
  \label{fig:cdg}
\end{figure}

\begin{figure}[p]
  \centerbox{
  \includegraphics[height=.9\textheight]{chap-examples/ddg.pdf}
}
  \caption{Example DDG}
  \label{fig:ddg}
\end{figure}

\begin{figure}[p]
  \begin{center}
    \includegraphics[height=.9\textheight]{chap-examples/ssasimpcfg.pdf}
  \end{center}
  \caption{Example Simplified CFG}
  \label{fig:cfgsimp}
\end{figure}
\FloatBarrier

\subsection{Verification Conditions}

Let's say we want to know if we can take the jump at address four.  We
can test this in BAP by using verification conditions (VCs).  To do
this, we need to add a helper variable to the IL that represents
success.  We can do this by setting the goal variable to \texttt{true}
at the beginning of the program, and setting it to false if the
desired branch is not taken.  Below is the modified IL that does this.

\begin{centering}
\begin{scriptsize}
\verbatiminput{chap-examples/basic_mod.il}
\end{scriptsize}
\end{centering}

To see if we take the jump, we can use topredicate with the post
condition \cmdline{goal}: \cmdline{topredicate -q -il basic\_mod.il
  -stp-out /tmp/f -post goal -solve}.  This will create a VC, and then
solve it using the \cmdline{stp} solver.  \cmdline{stp} returned
\begin{verbatim}
R_EAX_32_11 -> 20000000
R_EBX_32_13 -> 0
R_ECX_32_15 -> 3
\end{verbatim}
as a satisfying answer. Sure enough, $0x0 + 0x20000000 = 0x20000000$
and $0x20000000 << 0x3$ sets the carry flag, so the program would take
the jump.

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "../main"
%%% End:


\section{Conclusion and Outlook}
The understanding of the thermalization process of gluons plays a central role in finding an appropriate theoretical description of the complex interplay of physical processes during relativistic heavy-ion collisions. \\
\noindent
Using kinetic theory and statistical transport equations we were able to estimate important quantities such as the equilibration time $\tau_{\mathrm{eq}}$ by understanding thermalization as a dynamical interplay of elastic and inelastic scatterings. \\
\noindent
One explanation for the apparent excess of particles in the thermal spectrum compared to the expected equilibrium distribution is the formation of a Bose-Einstein condensate which may survive during most of the thermalization process. This option seems however not to be the favored interpretation amongst the researchers nowadays as the understanding of the various inelastic contributions to the gluon scattering is understood better than back when the idea of the condensate came up first.\\
\noindent
In the second part we focused on the derivation and solution of a nonlinear boson diffusion equation providing further insights into the thermalization process and an analytically accessible model to study different aspects of thermalization in more detail.\\
\noindent
In the future one may elaborate on the different approximation schemes and solution techniques for example by considering the time- and energy-dependence of the values of the transport coefficients or by extending the model in more than $1+1$ spacetime dimensions to be able to account for example for possible anisotropies.


\section{Introduction}
\label{sec:introduction}

Type errors are a common stumbling block for students
trying to learn typed functional languages like \ocaml\
and \haskell.
%
Consider the ill-typed @fac@ function on the left in
Figure~\ref{fig:factorial}.
%
The function returns @true@ in the base case (instead of @1@),
and so \ocaml responds with the error message:
%
\begin{verbatim}
  This expression has type
    bool
  but an expression was expected of type
    int.
\end{verbatim}
%
This message makes perfect sense to an expert who is familiar
with the language and has a good mental model of how the type
system works.
%
However, it may perplex a novice who has yet to develop such a
mental model.
%
To make matters worse, unification-based type inference algorithms
often report errors far removed from their source.
%
This further increases the novice's confusion and can actively mislead
them to focus their investigation on an irrelevant piece of code.
%
Much recent work has focused on analyzing unification constraints
to properly \emph{localize} a type error~\cite{Lerner2007-dt,Chen2014-gd,Zhang2014-lv,Pavlinovic2014-mr},
but an accurate source location does not explain \emph{why} the
program is wrong.

\begin{figure}[t]
\centering
\begin{minipage}{.49\linewidth}
\centering
\begin{ecode}
  let rec fac n =
    if n <= 0 then
      true
    else
      n * (*@\hlOcaml{fac (n-1)}@*)
\end{ecode}
\vspace{2em}
\includegraphics[height=1.5in]{fac-overview.png}
\end{minipage}
\begin{minipage}{.49\linewidth}
\centering
\includegraphics[height=3in]{fac-long.png}
\end{minipage}
\vspace{1em}
\caption{(top-left) An ill-typed \texttt{fac} function \hlOcaml{highlighting} the error location reported by \ocaml. (bottom-left) Dynamically witnessing the type error in \texttt{fac}, showing only function call-return pairs. (right) The same trace, fully expanded to show each small-step reduction in the computation.}
\label{fig:factorial}
\end{figure}

In this paper we propose a new approach that explains
static type errors by \emph{dynamically} witnessing
how an ill-typed program goes wrong.
%
We have developed \toolname, an interactive tool that uses
the source of the ill-typed function to automatically synthesize
the result on the bottom-left in Figure~\ref{fig:factorial}, which
shows how the recursive calls reduce to a configuration where
the program ``goes wrong'' --- \ie\ the @int@ value @1@ is to be
multiplied with the @bool@ value @true@.
We achieve this via three concrete contributions.

\paragraph{1. Finding Witnesses}
Our first contribution is an algorithm for searching for
\emph{witnesses} to type errors, \ie\ inputs that cause a
program to go wrong~(\S~\ref{sec:searching-witness}).
%
This problem is tricky when we cannot rely on
static type information, as we must avoid the
trap of \emph{spurious} inputs that cause
irrelevant problems that would be avoided
by picking values of a different, relevant type.
%
We solve this problem by developing a novel
operational semantics that combines evaluation
and type inference.
%
We execute the program with \emph{holes} --- values whose type is
unknown --- as the inputs.
%
A hole remains abstract until the evaluation
context tells us what type it must have, for
example the parameters to an addition operation
must both be integers.
%
Our semantics conservatively instantiates holes
with concrete values, dynamically inferring the
type of the input until the program goes wrong.
%
We prove that our procedure synthesizes \emph{general}
witnesses, which means, intuitively, that if a witness
is found for a given ill-typed function, then, \emph{for all}
(inhabited) input types, there exist values that can make
the function go wrong.

Given a witness to a type error, the novice may still be at a loss.
%
The standard \ocaml\ interpreter and debugging infrastructure expect
well-typed programs, so they cannot be used to investigate \emph{how}
the witness causes the program to crash.
%
More importantly, the execution itself may be quite long and may contain
details not relevant to the actual error.

\paragraph{2. Visualizing Witnesses}
Our second contribution is an interactive visualization of the
execution of purely functional \ocaml\ programs, well-typed or not~(\S~\ref{sec:interactive}).
%
We extend the semantics to also build a \emph{reduction graph}
which records all of the small-step reductions and the context
in which they occur.
%
The graph lets us visualize the sequence of
steps from the source witness to the stuck term. The user can
interactively expand the computation to expose intermediate steps
by selecting an expression and choosing a traversal strategy.
%
The strategies include many of the standard debugging moves, \eg\
stepping \emph{forward} or \emph{into} or \emph{over} calls, as well
stepping or jumping \emph{backward} to understand how a particular
value was created, while preserving a context of the intermediate
steps that allow the user to keep track of a term's provenance.

We introduce a notion of \emph{jump-compressed} traces to abstract away
the irrelevant details of a computation.
%
A jump-compressed trace includes only function
calls and returns. For example, the trace in the bottom-left of
Figure~\ref{fig:factorial} is jump-compressed.
%
Jump-compressed traces are similar to stack traces in that both show a
sequence of function calls that lead to a crash. However, jump-compressed
traces also show the return values of successful calls, which can be
useful in understanding why a particular path was taken.

\paragraph{3. Evaluating Witnesses}
%
Of course, the problem of finding witnesses is
undecidable in general. In fact, due to the necessarily
conservative nature of static typing, there
may not even exist any witnesses for a given
ill-typed program.
%
Thus, our approach is a heuristic that is only useful
if it can find \emph{compact} witnesses for
\emph{real-world} programs.
%
Our third contribution is an extensive evaluation of our approach
on two different sets of ill-typed programs obtained by instrumenting
compilers used in beginner's classes~(\S~\ref{sec:evaluation}).
%
The first is the \uwbench\ data set~\cite{Lerner2007-dt}
comprising \uwsize\ ill-typed programs.
%
The second is a new \ucsdbench\ data set, comprising \ucsdsize\
ill-typed programs.
%
We show that for both data sets, our technique is able to generate
witnesses for around 85\% of the programs, in under a second in the
vast majority of cases.
%
Furthermore, we show that a simple interactive strategy yields
compact counterexample traces with at most 5 steps for 60\%
of the programs, and at most 10 steps for over 80\% of the programs.
%
We can even use witnesses to \emph{localize} type errors with a simple
heuristic that treats the values in a ``stuck'' term as \emph{sources}
of typing constraints and the term itself as a \emph{sink},
achieving around 70\% accuracy in locating the source of the error.

The ultimate purpose of an error report is to help the programmer
\emph{comprehend} and \emph{fix} problematic code.
%
Thus, our final contribution is a user study that compares \toolname's
dynamic witnesses against \ocaml's type errors along the dimension of
comprehensibility~(\S~\ref{sec:user-study}).
%
Our study finds that students given one of our witnesses are
consistently more likely to correctly explain and fix a type
error than those given the standard error message produced by
the \ocaml compiler.

%
% \subparagraph{Witness Utility}
%
% Even if we can find small witnesses for the majority of type errors, it
% may be that the witnesses do not actually help developers
% \emph{understand} the errors.
%
% In other words, perhaps the static error message is sufficient to
% diagnose and fix the error, or perhaps the witness simply does not add
% enough information to make a difference.
%
%
% Thus, our final contribution is a user study that compares the utility
% of our witnesses with that of the error messages provided by the \ocaml
% compiler~(\S~\ref{sec:user-study}).
%

\smallskip
All together, our results show that in the vast majority of cases, (novices') ill-typed
programs \emph{do} go wrong, and that the witnesses to these errors can be
helpful in understanding the source of the error. This, in turn, opens the
door to a novel dynamic way to explain, understand, and appreciate the
benefits of static typing.

\paragraph{Contributions Relative to Prior Publications}
This paper extends our ICFP '16 paper of the same
name~\cite{Seidel2016-ul}, focusing on the experimental evaluation.
%
First, in \S~\ref{sec:how-safe} we investigate the student programs for
which we were unable to synthesize a witness. We group the failures into
five categories, give representative examples, and suggest ways to
improve our feedback in these cases. Interestingly, we find that in the
majority of these failed cases, the programs do not actually admit a
witness in our semantics.
%
Second, in \S~\ref{sec:locating} we attempt to use our witnesses to
localize type errors with a simple heuristic. We treat the stuck term as
a sink for typing constraints, and the values it contains as sources of
constraints. We can then predict that either the stuck term or one of
the terms that \emph{produced} a value it contains is likely at fault
for the error. We compare our localizations to \ocaml and two
state-of-the-art type error localization tools, and find that we are
competitive with the state of the art.
%
Finally, we have also extended \S~\ref{sec:user-study} with an analysis
of the statistical significance of our user study results.

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "main"
%%% End:


\section{Dense gas dispersion in obstructed terrain}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Falcon series}
The Falcon series were conducted by LLNL in 1987. These comprises 5 large-scale LNG spill tests aiming at evaluating the effectiveness of impoundment walls as a mitigation technique for accidental releases of LNG. LNG was spilled onto a rectangular water pond (60m x 40m). The evaporation rate could be roughly equivalent to the spill flow rate as the designed recirculation system was involved to maximize the evaporation process \cite{Gavelli2008}. LNG was supplied to the pond through 4 pipes, fitted with 0.11m diameter orifices and spaced at 900 intervals. The vapour fence, about 8.7m high, surrounded the water pond of a total area of 44m x 88m. The billboard of 13.3m tall, 17.1m wide was used to simulate the effect of a storage tank or other obstruction. The terrain was flat and the atmospheric condition was stable or neutrally stable.

Three Falcon tests included in MEP, Falcon 1, Falcon 3 and Falcon 4 are simulated in OpenFOAM to examine the effective of proposed models in simulating field LNG dispersion in present of obstacles.

\subsection{Boundary conditions}
\paragraph{Atmospheric inlet boundary}
Monin-Obukhov similarity theory is used to specify the wind velocity and temperature profile at the inlet. The velocity profile is calculated from Equation~\ref{eq:MO-velocity}. All required meteorological parameters:
\begin{table}[h!]
	\caption{Falcon tests meteorological parameters} \label{tab:falcon_params}
	\centering
	\begin{tabular}{lrrr}
		\toprule
		& Falcon 1	& Falcon 3	& Falcon 4		\\
		\midrule
		Stability & G & D & D-E \\
		$L$		& 4.96 & -422 & 69.4	\\
		$u_*$	& 0.061 & 0.305 & 0.369	\\
		$T_*$	& 0.058 & -0.018 & 0.152	\\
		$z_0$ 	& 0.008 & 0.008 & 0.008 \\
		\bottomrule
	\end{tabular}
\end{table}

\paragraph{Vapour gas inlet}
Vapour gas inlet condition is usually obtained from separate source term modelling. There is not much information about the vaporisation of LNG from the experimental data. Therefore, uncertainty arises at the setting of this condition.

Mass flux of LNG or the LNG vaporization rate is used to derive source term of LNG spill. \textcite{Luketa-Hanlin2007} reviewed a number of experiments conducting to estimate the LNG vaporization rate of the spill on water, the range of this value varied between approximately \SIrange[range-units=single]{0.029}{0.195}{\kilogram\per\square\meter\per\second}. In the case of Burro test, the simulated vaporisation rate is assumed to be $\dot{m} = $\SI{0.167}{\kilogram\per\square\meter\per\second}. The spill diameter is derived from this vaporization rate, reported spill mass $m$ and duration $\delta t$:
\begin{equation}
D = \sqrt{\frac{4m}{\pi \dot{m} \delta t}}
\end{equation}
LNG spill variables used in simulation are tabulated in Table~\ref{tab:burro_releaseVar}.
\begin{table}[h!]
	\caption{Falcon test spill condition} \label{tab:falcon_releaseVar}
	\centering
	\begin{tabular}{rrrr}
		\toprule
		& Falcon 1	& Falcon 3	& Falcon 4		\\
		\midrule
		Vaporization rate (\si{\kilogram\per\square\meter\per\second}) & 0.167 & 0.167 & 0.167 \\
		Spill mass (\si{\kilogram}) 		& 28074 	& 21435 	& 18984  	\\
		Spill duration (\si{\second}) 		& 131 	& 154 	& 301  	\\
		Spill pool diameter (\si{\meter}) 		& 19.5 	& 16.0 	& 10.8 	\\
		\bottomrule
	\end{tabular}
\end{table}

\subsection{Results and Discussion}
\subsubsection{Arcwise prediction}
Maximum concentration at four arcwise sensor arrays at $50 m$, $150 m$ and $250 m$ of three Falcon tests are compared with experimental data. Prediction of maximum concentration are best for Falcon 4 test.

\begin{figure}[htbp]
	\centering
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon1ConMax}
		\caption{}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon3ConMax}
		\caption{}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon4ConMax}
		\caption{}
	\end{subfigure}

	\caption{Maximum arc-wise concentration (a) Falcon 1 (b) Falcon 3 (c) Falcon 4}
	\label{fig:FalconConMax}
\end{figure}

\subsubsection{SPMs}
Statistical performance of \bera{OpenFOAM} results are compared with \bera{FLACS} which data extracted from \cite{Hansen2010a} in Table~\ref{tab:Falcon_SPMs}:

\begin{table}[h!]
	\caption{Statistical performance measure values of Falcon} \label{tab:Falcon_SPMs}
	\centering
	\begin{tabular}{lrrrrr}
		\toprule
		& MRB & MG & RMSE & VG  &FAC2 \\
		\midrule
		$FLACS$	&1.35 &5.56 &1.88 &23.65 &0	\\
		\bottomrule
	\end{tabular}
\end{table}


%%%%%%%%%%%%%%%%
% LaTeX
%%%%%%%%%%%%%%%%%%
\documentclass[12pt]{article}
\oddsidemargin 0in \evensidemargin 0in \textheight 9in \textwidth
6.5in \topmargin 0in \headheight 0in
\parindent 20 pt
\headsep 0 in

\usepackage{amsthm,amssymb}

\usepackage{gb4e} % by me

\newtheorem*{thm}{Theorem}
%\newtheorem{cor}[thm]{Corollary}
\newtheorem{theorem}{Theorem}%[section]
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{hypothesis}[theorem]{Hypothesis}
\newtheorem{example}[theorem]{Example}

%\newenvironment{proof}[1][Proof]%
%{\par\addvspace{6pt}\noindent{\bf #1.}\hskip\labelsep\ignorespaces}%
%{{\hfill $\square$}\par\addvspace{6pt}}

\def\bull{\vrule height .9ex width .8ex depth -.1ex }
%\magnification=\magstep1
\raggedbottom
\def\card#1{\vert #1 \vert}
\def\gpindex#1#2{\card {#1\colon #2}}
\def\irr#1{{\textrm  Irr}(#1)}
\def\ibr#1{{\textrm IBr} (#1)}
\def\irri#1#2{{\textrm Irr}_{#1} (#2)}
\def\cd#1{{\textrm  cd}(#1)}
\def\cent#1#2{{\bf C}_{#1}(#2)}
\def\gpcen#1{{\bf Z} (#1)}
\def\ker#1{{\textrm ker} (#1)}
%\def\ref#1{{\bf [#1]}}
\def\form#1#2#3{\langle\langle #1, #2 \rangle\rangle_{#3}}
\def\empform#1{\form {\cdot}{\cdot}{#1}}
\def\norm#1#2{{\textrm N}_{#1} (#2)}
\def\NN{{\cal N}}
\def\SS{{\cal S}}
\def\UU{{\cal U}}
\def\TT{{\cal T}}
\def\B#1#2{{\textrm B}_{#1} (#2)}
\def\Bpi#1{\B {\pi}{#1}}
\def\D#1#2{{\textrm D}_{#1} (#2)}
\def\Dpi#1{\D {\pi} {#1}}
\def\I#1#2{{\textrm I}_{#1} (#2)}
\def\Ipi#1{\I {\pi}{#1}}
\def\phi{\varphi}

\newcommand \IIpi[3] {{\textrm I}_{#1} (#2 \mid #3)}

\begin{document}

\title{Inducing $\pi$-partial characters with a given vertex}

\author {
       Mark L.\ Lewis
    \\ {\it Department of Mathematical Sciences, Kent State University}
    \\ {\it Kent, Ohio 44242}
    \\ E-mail: lewis@math.kent.edu
       }
%\date{July 17, 2010}

\maketitle

\begin{abstract}
Let $G$ be a solvable group.  Let $p$ be a prime and let $Q$ be a
$p$-subgroup of a subgroup $V$.  Suppose $\phi \in \ibr G$.  If
either $|G|$ is odd or $p = 2$, we prove that the number of Brauer
characters of $H$ inducing $\phi$ with vertex $Q$ is at most $|\norm
GQ: \norm VQ|$.

MSC Primary: 20C20, MSC Secondary: 20C15

Keywords: Brauer characters, partial characters, vertices
\end{abstract}

%Inducing partial characters with a given vertex

%These were originally notes written in December 2010.  This result
%arose in some work that I doing with JP.

\section{Introduction}

Throughout this note, $G$ is a finite group, and $\irr G$ is the set
of irreducible characters of $G$.  Suppose $\chi \in \irr G$ and $H$
is a subgroup of $G$.  It is easy to obtain an upper bound on the
number of characters in $\irr H$ that induce $\chi$.  Let $\phi_1,
\dots, \phi_n \in \irr H$ be the characters so that $\phi_i^G =
\chi$.  Evaluating at $1$, we obtain $\phi_i (1) = \chi (1)/|G:H|$
for each $i$.  By Frobenius reciprocity (Lemma 5.2 of \cite{text}),
each $\phi_i$ is a constituent of $\chi_H$ with multiplicity $1$.
Since there are $n$ such characters occurring as constituents of
$\chi_H$, it follows that $n (\chi (1)/|G:H|) \le \chi (1)$.  We
deduce that $n \le |G:H|$, and we have an upper bound.  If $H$ is
normal in $G$, this bound is obtained, and it is not particularly
difficult to find nonnormal subgroups where this bound is obtained.

We now turn our attention to Brauer characters.  Fix a prime $p$. We
will write $\ibr G$ for the irreducible $p$-Brauer characters of
$G$.  If $\phi \in \ibr G$, then it is easy to adapt the above proof
to show that $\phi$ is induced by at most $|G:H|$ Brauer characters
of $H$.  However, associated with $\phi$ are certain $p$-subgroups
of $G$ called the vertex subgroups.  When $G$ is a $p$-solvable
group, a $p$-subgroup $Q$ of $G$ is defined to be a vertex for
$\phi$ if there is a subgroup $U$ of $G$ so that $\phi$ is induced
by a Brauer character of $U$ with $p'$-degree and $Q$ is a Sylow
subgroup of $U$.  It is known that all the vertex subgroups of
$\phi$ are conjugate in $G$. If $\phi$ is induced from $\tau \in
\ibr H$, it is easy to see that a vertex for $\tau$ is a vertex for
$\phi$.  Thus, $H$ contains some vertex $Q$ for $\phi$.  Now,
different Brauer characters of $H$ that induce $\phi$ may have
vertex subgroups that are not conjugate in $H$ but are necessarily
conjugate in $G$. Hence, one can ask the following question: Suppose
$\phi \in \ibr G$ has vertex $Q$, and $Q \le H$, how many characters
in $\ibr H$ with vertex $Q$ induce $\phi$?  When either $|G|$ is odd
or $G$ is solvable and $p = 2$, we can obtain an upper bound for
this question.

\begin{theorem} \label{main}
Let $G$ be a solvable group and $p$ a prime.  Assume either $|G|$ is
odd or $p = 2$. Let $Q$ be a $p$-subgroup of $H$. If $\phi \in \ibr
G$, then the number of Brauer characters of $H$ with vertex $Q$ that
induce $\phi$ is at most $|\norm GQ:\norm HQ|$.
\end{theorem}

At this time, we are not able to determine whether or not this
theorem is true if we loosen the hypothesis that either $|G|$ is odd
or $p = 2$.  In other words, is the conclusion still true if $G$ is
a solvable group of even order and $p$ is an odd prime.

%%%%%%%%%%%%%%%%%%INSERT HERE

%%%%%%%%%%%END INSERT

This result
was motivated by our work with J. P. Cossey. If we could prove the
conclusion of Theorem \ref{main} when $p$ is odd, then we would be
able to prove J. P. Cossey's conjecture that the number of lifts of
a Brauer character is bounded by the index of a vertex subgroup in
the vertex subgroup when $p$ is odd. Our argument can be found in
the preprint \cite{preprint}.

We would like to thank J. P. Cossey and I. M. Isaacs for several
helpful discussions while we were preparing this note.

\section{Results}

We will in the more general setting of irreducible $\pi$-partial
characters of a $\pi$-separable group $G$.  We here briefly mention
that if $\pi$ is a set of primes and $G$ is a $\pi$-separable group,
one can define (see \cite{pipart} for more details) a set of class
functions $\Ipi G$ from the set $G^o$ (which consists of the
elements of $G$ whose order is divisible by only the primes in
$\pi$) to ${\bf C}$ that plays the role of $\ibr G$, and in fact
$\Ipi G = \ibr G$ if $\pi = \{ p' \}$, the complement of the prime
$p$.

%In this note, I fix a $\pi$-partial character $\phi$ and a subgroup
%$V$ that contains a $\pi'$-subgroup $Q$.  I obtain the upper bound
%of $|\norm GQ:\norm VQ|$ on the number of $\pi$-partial characters
%of $V$ with vertex $Q$ that induce $\phi$ under the assumption that
%$G$ is solvable and either $|G|$ is odd or $2 \not\in \pi$.
%
%Of course, the case I need for the work with JP is the case when $2
%\in \pi$.  :-(

We start by considering vertices in Clifford correspondence (see
Proposition 3.2 of \cite{Fong}).  Let $G$ be a $\pi$-separable
group. Let $N$ be a normal subgroup of $G$.   Fix $\phi \in \Ipi G$.
If $\alpha \in \Ipi N$ is a constituent of $\phi_N$, then we write
$G_\alpha$ for the stabilizer of $\alpha$ in $G$, and we write
$\phi_\alpha$ for the Clifford correspondent of $\phi$ with respect
to $\alpha$. In particular, the vertices of the Clifford
correspondent form an orbit under the action of the normalizer of a
particular vertex.

\begin{lemma}\label{cliff}
Let $G$ be a $\pi$-separable group.  Let $N$ be a normal subgroup of
$G$.  Suppose that $\alpha \in \Ipi N$.  Let $\phi \in \Ipi G$ and
$\hat\phi \in \Ipi {G_\alpha}$ so that $\hat\phi^G = \phi$.  Suppose
that $Q$ is a vertex for $\hat\phi$.  Then $Q$ is a vertex is
$\hat\phi^g$ if and only if there exists $n \in \norm GQ$ so that
$G_\alpha g = G_\alpha n$.
\end{lemma}

\begin{proof}
We first suppose that there exists $n \in \norm GQ$ so that
$G_\alpha g = G_\alpha n$.  Thus, $g = tn$ for some $t \in
G_\alpha$.  We see that $\hat\phi^g = \hat\phi^{tn} = \hat\phi^n$.
We see that $Q = Q^n$ is a vertex for $\hat\phi^n = \hat\phi^g$.

Conversely, suppose that $Q$ is a vertex for $\hat\phi^g$.  Then
$Q^{g^{-1}}$ is a vertex for $\hat\phi$.  Since $Q$ is also a vertex
for $\hat\phi$, we have $Q^{g^{-1}} = Q^t$ for some $t \in
G_\alpha$.  It follows that $Q = Q^{tg}$, and so, $tg \in \norm GQ$.
This implies that $tg = n$ for some $n \in \norm GQ$.  This implies
that $n \in G_\alpha g$, and we conclude that $G_\alpha n = G_\alpha
g$.
\end{proof}

We continue to work in the context of the Clifford correspondence.
In this case, we can get an exact count of the number of partial
characters in $N$ whose Clifford correspondent has vertex $Q$.

\begin{corollary}\label{cliff count}
Let $G$ be a $\pi$-separable group.  Let $N$ be a normal subgroup of
$G$, let $\phi \in \Ipi G$ have vertex $Q$, and suppose that $\beta$
is an irreducible constituent of $\phi_N$ so that $\phi_\beta$ has
vertex $Q$. Then $|\{ \alpha \in \Ipi N \mid \phi_\alpha {\textrm ~
has~vertex~} Q \}| = |\norm GQ : \norm {G_\beta}Q|$.
\end{corollary}

\begin{proof}
%We know that $\phi_\beta$ has vertex $Q^*$. Observe that $Q^*$ is a
%vertex for $\phi$ so $(Q^*)^g = Q$ for some $g \in G$.  Replacing
%$\beta$ by $\beta^g$, we may assume that $Q^* = Q$, so $Q$ is a
%vertex for $\phi_\beta$.
By Lemma \ref{cliff}, we see that $\phi_\alpha$ has $Q$ as a vertex
if and only if $\alpha = \beta^g$ where $g \in G$ and $g \in G_\beta
n$ for some $n \in \norm GQ$. Finally, we observe that $G_\beta n_1
= G_\beta n_2$ if and only if $\norm {G_\beta}Q n_1 = \norm
{G_\beta}Q n_2$ for $n_1, n_2 \in \norm GQ$.  We have $|\{ \alpha
\in \Ipi N \mid \phi_\alpha {\textrm ~ has~vertex~} Q \}| = | \{ G_\beta
n \mid n \in \norm GQ \} | = |\norm GQ :\norm {G_\beta}Q|$.
\end{proof}

We now look at the conditions of a minimal counterexample.  For this
we need to review and develop more notation.  We make use of the
canonical set of $\pi$-lifts, $\Bpi G$, that was defined in
\cite{pisep} by Isaacs.  In other words, $\Bpi G \subseteq \irr G$
and the map $\chi \mapsto \chi^o$ is a bijection from $\Bpi G$ to
$\Ipi G$.  Closely related to this set is the subnormal nucleus
which also was defined in \cite{pisep}.  To define the subnormal
nucleus, we need the $\pi$-special characters.  Let $G$ be a
$\pi$-separable group.  A character $\chi \in \irr G$ is
$\pi$-special if $\chi (1)$ is a $\pi$-number and for every
subnormal group $M$ of $G$, the irreducible constituents of $\chi_M$
have determinants that have $\pi$-order.  Many of the basic results
of $\pi$-special characters can be found in Section 40 of
\cite{hupte} and Chapter VI of \cite{Mawo}. One result that is
proved is that if $\alpha$ is $\pi$-special and $\beta$ is
$\pi'$-special, then $\alpha \beta$ is necessarily irreducible. We
say that $\chi$ is {\bf factored} if $\chi = \alpha \beta$ where
$\alpha$ is $\pi$-special and $\beta$ is $\pi'$-special.  We also
note that if $\chi \in \Bpi G$ and $N$ is normal in $G$, then the
irreducible constituents of $\chi_N$ lie in $\Bpi N$.

%We need to define Dpi and then Cpi.  We also need to talk about the
%magic character automorphism.

%Next, we prove a simple lemma regarding $\pi$-special characters
%that essentially follows from \cite{gajen} or Lemma 21.4 of
%\cite{Mawo}.  We should note that this lemma is not particularly
%new.

%\begin{lemma} \label{pispec}
%Let $N \le K$ be normal subgroups of a $\pi$-separable group $G$.
%Suppose that $K/N$ is a $p$-group for some prime $p$.  If $\alpha
%\in \irr N$ is $\pi$-special, then all irreducible constituents of
%$\alpha^K$ lying in $\Cpi K$ are $\pi$-special.
%\end{lemma}

%The proof needs to be changed for Cpi.
%\begin{proof}
%If $p \in \pi$, then by Lemma 21.4 of \cite{Mawo} all the
%irreducible constituents of $\alpha^K$ are $\pi$-special, and the
%result follows. Thus, we may assume that $p$ is not in $\pi$. Thus,
%$\alpha (1) o(\alpha)$ is coprime to $|K:N|$. By Corollary 6.28 of
%\cite{text}, $\alpha$ extends to $K$, and $\alpha$ has a unique
%extension $\beta$ with $o(\beta) = o (\alpha)$.  By Lemma 21.4 of
%\cite{Mawo}, $\beta$ is $\pi$-special.  On the other hand, we have
%by Gallagher's theorem (Corollary 6.17 of \cite{text}), the
%irreducible constituents of $\alpha^K$ have the form $\beta \nu$
%where $\nu \in \irr {K/N}$.  Since $K/N$ is a $\pi'$-group, $\nu$ is
%$\pi'$-special.  We now apply Lemma 5.4 of \cite{pisep} to see that
%a character in $\Bpi K$ is factored if and only if it is
%$\pi$-special.  Since all the irreducible constituents of $\alpha^K$
%are $\pi$-factored, the irreducible constituents of $\alpha^K$ lying
%in $\Bpi K$ are $\pi$-special.
%\end{proof}

If $\chi \in \irr G$, Isaacs constructs the subnormal vertex as
follows.  Let $M$ be maximal so that $M$ is subnormal in $G$ and the
irreducible constituents of $\chi_M$ are factored.  Let $\mu$ be an
irreducible constituent of $\chi_M$ and let $T$ be the stabilizer of
$(M,\mu)$ in $G$.  Isaacs proved in \cite{pisep} that there is a
Clifford theorem for $T$.  In other words, there is a unique
character $\tau \in \irr {T \mid \mu}$ so that $\tau^G = \chi$.  He
also proved that $(M,\mu)$ is unique up to conjugacy, and so,
$(T,\tau)$ is unique up to conjugacy.  If $T = G$, then $\chi$ is
$\pi$-factored and we take $(G,\chi)$ to be the subnormal nucleus of
$\chi$.  If $T < G$, then inductively, the subnormal nucleus for
$\tau$ is the subnormal nucleus for $\chi$.  We write $(W,\gamma)$
for the subnormal nucleus of $\chi$, and Isaacs showed that
$\gamma^G = \chi$, $\gamma$ is factored, and $(W,\gamma)$ is unique
up to conjugacy.  A character $\chi \in \irr G$ is in $\Bpi G$ if
and only if the character of its nucleus is $\pi$-special.

If $Q$ is a $\pi'$-subgroup of $G$, then we use $\Ipi {G \mid Q}$ to
denote the $\pi$-partial characters in $\Ipi G$ that have vertex
$Q$.  If $\phi \in \Ipi G$ and $V \le G$, then we write $\IIpi
{\phi}VQ = \{ \eta \in \Ipi {V \mid Q} \mid \eta^G = \phi \}$.  We
now find details about properties of a minimal counterexample.  We
will see that a counterexample cannot occur when either $|G|$ is odd
or $2$ is not in $\pi$.  Our goal is find enough information so that
we can either find a contradiction or build an example when $|G|$ is
even and $2 \in \pi$.

\begin{theorem} \label{min counter}
Let $G$ be a solvable group.  Assume $\phi \in \Ipi G$ has vertex
$Q$, let $V$ be a subgroup of $G$, and let $N$ be the core of $V$ in
$G$. If $G$ and $V$ are chosen so that $|G| + |G:V|$ is minimal
subject to the condition that $|\IIpi {\phi}VQ| > |\norm GQ:\norm
VQ|$, then the following are true:

\begin{enumerate}
\item $V$ is a nonnormal maximal subgroup of $G$,
\item $|G:V|$ is a power of $2$,
\item $2 \in \pi$,
\item $Q \le V$,
\item $\phi_N = a \alpha$ for some $\alpha \in \ibr N$,
\item $\alpha (1)$ is a $\pi$-number,
\item if $K$ is normal in $G$ so that $K/N$ is a chief factor for
$G$, then $\alpha$ is fully ramified with respect to $K/N$.
\end{enumerate}
\end{theorem}

\begin{proof}
If either $V = G$ or $\IIpi {\phi}VQ$ is empty, then $|\IIpi
{\phi}VQ| \le |\norm GQ:\norm VQ|$ contradicting the hypotheses.
Thus, $V < G$ and $\IIpi {\phi}VQ$ is not empty, and so, $Q \le V$
and there exist characters in $\Ipi V$ that induce $\phi$ and have
vertex $Q$.

We begin by showing that $V$ is a maximal subgroup.  Suppose that $V
< M < G$ for some subgroup $M$.  Let $\IIpi {\phi}MQ = \{ \eta_1,
\dots, \eta_m \}$.  Using minimality, we have $m = |\IIpi {\phi}MQ|
\le |\norm GQ:\norm MQ|$. Suppose that $\zeta \in \IIpi {\phi}VQ$,
then $\zeta^M \in \Ipi M$ and $\zeta^M$ has $Q$ as a vertex.  Since
$(\zeta^M)^G = \zeta^G = \phi$, we see that $\zeta^M \in \IIpi
{\phi}MQ$.  It follows that $\zeta^M = \eta_i$ for some $i$.  We
conclude that $|\IIpi {\phi}VQ| \le \sum_{i=1}^m |\IIpi
{\eta_i}VQ|$.  Since this contradicts our hypothesis, we obtain
$|\IIpi {\eta_i}VQ| \le |\norm MQ:\norm VQ|$.  We deduce that
$$
|\IIpi {\phi}VQ| \le m|\norm MQ:\norm VQ| \le
|\norm GQ:\norm MQ||\norm MQ: \norm VQ| = |\norm GQ:\norm VQ|.
$$
Since this violates the hypotheses, $V$ is maximal in $G$.

If $V$ is normal in $G$, then either $\phi$ is induced from $V$ or
$\phi$ restricts irreducibly to $V$.  If $\phi$ is induced from $V$,
then we can apply Corollary \ref{cliff count} to see that $|\IIpi
{\phi}VQ| \le |\norm GQ:\norm VQ|$ in violation of the hypotheses.
If $\phi$ restricts irreducibly, then it cannot be induced from $V$,
and we have seen that this is also a contradiction.  We conclude
that $V$ is not normal in $G$.

Suppose $\alpha \in \Ipi N$ is a constituent of $\phi_N$.  We use
$\phi_\alpha \in \Ipi {G_\alpha \mid \alpha}$ to denote the Clifford
correspondent for $\phi$ with respect to $\alpha$ (see Proposition
3.2 of \cite{Fong} again).   Write $\{ \alpha \in \Ipi N \mid
\phi_\alpha {\textrm ~has~vertex~} Q \} = \{ \alpha_1, \dots, \alpha_k
\}$, and let $\phi_i = \phi_{\alpha_i}$ and $G_i = G_{\alpha_i}$. By
Lemma \ref{cliff count}, we know that $k = |\norm GQ:\norm {G_i}Q|$.

Suppose $\eta \in \IIpi {\phi}VQ$.  Denote $\{ \beta \in \Ipi N \mid
\eta_\beta {\textrm ~has~vertex~} Q \} = \{ \beta_1, \dots, \beta_l \}$,
and let $\eta_j = \eta_{\beta_j}$ and $V_j = V_{\beta_j}$.  By Lemma
\ref{cliff count}, $l = |\norm VQ: \norm {V_i}Q|$.

We see that $(\eta_j)^G = ((\eta_j)^V)^G = \eta^G = \phi$.  This
implies that $(\eta_j)^{G_{\beta_j}}$ is irreducible and has vertex
$Q$.  It follows that $\beta_j = \alpha_{i_j}$ for some $i_j$.  We
obtain $G_{\beta_j} = G_{i_j}$ and $(\beta_j)^{G_{i_j}} =
\alpha_{i_j}$. Observe that $V_j = G_{i_j} \cap V$, and we denote
this subgroup by $V^*_{i_j}$.

Now, we assume that $k > 1$, and we start to count.  We see that
$\eta \in \IIpi {\phi}GQ$ is induced by $|\norm VQ:\norm
{V^*_{i_j}}Q|$ partial characters in $\bigcup \IIpi
{\phi_i}{V^*_i}Q$.  Because $G_i < G$, we may use minimality of $|G|
+ |G:V|$ to deduce $|\IIpi {\phi_i}{V^*_i}Q| \le |\norm {G_i}Q:\norm
{V^*_i}Q|$. We compute
$$
|\IIpi {\phi}VQ| = \sum_{i=1}^k \frac 1{|\norm VQ:\norm {V^*_i}Q|}
|\IIpi {\phi_i}{V^*_i}Q| \le \sum_{i=1}^k \frac 1{|\norm VQ:\norm
{V^*_i}Q|} |\norm {G_i}Q:\norm {V^*_i}Q|.
$$
We determine that
$$
\frac 1{|\norm VQ:\norm {V^*_i}Q|} |\norm {G_i}Q:\norm {V^*_i}Q| =
\frac {|\norm {G_i}Q|}{|\norm VQ|},
$$
for each $i$.   Notice that
$|\norm {G_i}Q| = |\norm {G_1}Q$ for all $i$ and $k = |\norm GQ:
\norm {G_1} Q$. This yields
$$
|\IIpi {\phi}VQ| \le \sum_{i=1}k \frac {|\norm {G_1}Q}{\norm VQ} =
\frac {|\norm GQ:\norm {G_1}Q| |\norm {G_1}Q|}{|\norm VQ|} = |\norm
GQ:\norm VQ|.
$$
This contradicts the hypothesis.  We deduce that $k = 1$, and
$\alpha$ is invariant in $G$.

Set $\alpha = \alpha_1$, and let $\alpha^*$ be the character in
$\Bpi N$ satisfying $(\alpha^*)^o = \alpha$.  Write $(W,\hat\alpha)$
for the nucleus of $\alpha^*$, and take $T$ to be the stabilizer of
$(W,\hat\alpha)$ in $G$. By Lemma 2.3 of \cite{Laradji}, there is a
unique character $\hat\phi \in \IIpi {}T{\hat\alpha}$ so that
$\hat\phi^G = \phi$ and $Q$ is a vertex for $\hat\phi$.  Similarly,
if $\eta \in \IIpi {\phi}VQ$, then there is a unique character
$\hat\eta \in \IIpi {}{T \cap V}{\hat\alpha}$ so that $\hat\eta^V =
\eta$ and $Q$ is a vertex for $\hat\eta$. Observe that $\hat\eta^T
\in \IIpi {}T{\hat\alpha}$ and induces $\phi$, so $\hat\eta^T =
\hat\phi$.  It follows that $|\IIpi {\phi}VQ| = |\IIpi {\hat\phi}{T
\cap V}Q|$.  If $T < G$, then we can use the minimality of $|G| +
|G:V|$ to see that $|\IIpi {\hat\phi}{T \cap V}Q| \le |\norm TQ:
\norm {V \cap T}Q|$.  By the diamond lemma, we have $|\norm TQ:
\norm {V \cap T}Q| = |\norm TQ: V \cap \norm TQ| \le |\norm GQ :
\norm VQ|$.  This contradicts the hypotheses, and so $T = G$.

We now have that $(W,\hat\alpha)$ is $G$-invariant.  By the
construction of the subnormal, this implies that $W = N$.  Since
$\alpha^* \in \Bpi N$, the nucleus for $\alpha^*$ has a character
that is $\pi$-special.  Thus, $\hat\alpha$ is $\pi$-special, and
since $W = N$, we see that $\hat\alpha = \alpha^*$.  In particular,
$\hat\alpha$ is $\pi$-special.  We deduce that $\alpha (1)$ is a
$\pi$-number.

Take $K$ normal in $G$ so that $K/N$ is a chief factor for $G$. This
is the point where we use the fact that $G$ is solvable to see that
$G = VK$ and $V \cap K = N$ where $K/N$ is an elementary abelian
$p$-group for some prime $p$.  (This is the only place we use the
hypothesis that $G$ is solvable in place of $G$ being
$\pi$-separable.)
%By Lemma \ref{pispec}, the constituents of
%$(\alpha^*)^K$ that lie in $\Cpi K$ are $\pi$-special.
%, then the irreducible constituents of  ${\phi^*}_K$ lie in $\Cpi K$.  Notice
%that some irreducible constituent $\delta$ of ${\phi^*}_K$ is a
%constituent of $(\alpha^*)^K$.  Since $\delta$ is a constituent of
%$(\alpha^*)^K$ and $\delta \in \Bpi K$, we see that $\delta$ must be
%$\pi$-special, and so, all the irreducible constituents of
%$(\phi^*)_K$ are $\pi$-special.
%
%This needs more explanation.
%It follows that $Q \cap K$ is a Hall $\pi$-complement of $L$.  Since
%$Q \cap K \le N$, it follows that $K/N$ is a $\pi$-group and $p \in
%\pi$.
%
Let $L/K$ be a chief factor for $G$.  We know that $(|L:K|,|K:N|) =
1$ and $\cent {L \cap V/N}{K/N}$.  (See Lemma 5.1 of \cite{max} for
a proof of this.)  By Problem 6.12 of \cite{text}, either $\alpha^*$
extends to $K$ or $\alpha^*$ is fully-ramified with respect to
$K/N$.  .

Suppose first that $\alpha^*$ extends to $K$. Notice that
multiplication by $\irr {K/L}$ is a transitive action on the
irreducible constituents of $(\alpha^*)^K$.  Also, $(V \cap K)/L$
acts on compatibly on the irreducible constituents of $(\alpha^*)^K$
and on $\irr {K/L}$ where the action on $\irr {K/L}$ is coprime.  We
can use Glauberman's lemma (Lemma 13.8 of \cite{text}) to see that
$\alpha^*$ has a $V \cap L$-invariant extension.  The corollary to
Glauberman's lemma (Corollary 13.9 of \cite{text}) can be applied to
see that $\alpha^*$ has a unique $V \cap L$-invariant extension
$\delta$. Since $V$ permutes the $V \cap L$-extensions of
$\alpha^*$, it follows that $\delta$ is $V$-invariant.  We now use
Corollary 4.2 of \cite{pisep} to see that restriction is a bijection
from $\irr {G \mid \beta}$ to $\irr {V \mid \alpha^*}$.

Let $\eta \in \IIpi {\phi}VQ$ so that $\eta^G = \phi$.  We can find
$\eta^* \in \Bpi V$ so that $(\eta^*)^o = \eta$. Since
$({\eta^*}^G)^o = ({\eta^*}^o)^G = \eta^G = \phi \in \ibr G$, we see
that $\eta^G$ is irreducible.  On the other hand, $({\eta^*}^o)_N =
(\eta_N)^o = b \alpha$ for some integer $b$.  Since the irreducible
constituents of ${\eta^*}_N$ lie in $\Bpi N$, we deduce that $\eta^*
\in \irr {V \mid \alpha^*}$.  But we saw that this implies that
$\eta^*$ extends to $G$.  Since $V < G$, it is not possible for
$\eta^*$ to both extend to $G$ and induce irreducibly.  Therefore,
we have a contradiction.  We see that $\alpha^*$ (and hence,
$\alpha$) is fully ramified with respect to $K/N$.  Notice that if
$p$ is not in $\pi$, then Corollary 6.28 of \cite{text} applies and
$\alpha^*$ extends to $K$. Therefore, $p \in \pi$.

%Suppose $|G|$ is odd or $2 \in \pi$, and let $\eta^* \in \Bpi V$. We
%know that $({\eta^*}^G)^\sigma = ({\eta^*}^\sigma)^G = (\eta^*)^G$
%and .  It follows that $\eta^* \in \Bpi G$. We conclude that
%$(\eta^*)^G = \phi^*$.
%We need more details here
%We now suppose $2 \not\in \pi$, and we let $\eta^* \in \Dpi V$ so
%that $(\eta^*)^o$. We see that $(s_{(G:V)} \eta^*)^G = \phi^*$.  In
%either case, $\phi^*$ is induced from $V$ and this is a
%contradiction.
We suppose that $p$ is odd, and we work for a contradiction.  Since
$\alpha^*$ is fully-ramified with respect to $K/N$ and $|K:N|$ has
odd order, main theorem of \cite{fram} implies that no character in
$\irr {V \mid \alpha}$ induces irreducibly to $G$.  (A stronger
theorem is proved in \cite{brown}.) As in the previous paragraph,
this implies that $\phi$ is not induced from $V$ which contradicts
the assumption that $\IIpi {\phi}VQ$ is not empty. (This strongly
uses the fact that $p$ is odd. When $p = 2$, it is tempting to try
use the correspondence in \cite{strong}, but that correspondence
does not preclude inducing characters in $\irr {G \mid \alpha}$ from
$V$. In fact, ${\textrm GL}_2 (3)$ is an example where this occurs.) We
conclude that $p = 2$. Since $|G:V| = |K:N|$, we see that $|G:V|$ is
a power of $2$. This proves the theorem.
\end{proof}

As a corollary, we obtain Theorem \ref{main} stated for
$\pi$-partial characters.

\begin{corollary}
Let $G$ be a solvable group.  Assume either $|G|$ is odd or $2
\not\in \pi$.  Let $Q$ be a $\pi'$-subgroup of $G$ and suppose that
$Q \le V$.  If $\phi \in \Ipi G$, then $|\IIpi {\phi}VQ| \le |\norm
GQ:\norm VQ|$.
\end{corollary}

\begin{proof}
We suppose the result is not true.  Let $G$ be a counterexample with
$|G| + |G:V|$ as in Theorem \ref{min counter}.  By that result, we
have that $|G:V|$ is a nontrivial power of $2$ which is a
contradiction if $|G|$ is odd.  We also have $2 \in \pi$ which is a
contradiction to $2 \not\in \pi$.  This proves the corollary.
\end{proof}

\begin{thebibliography}{99}

\bibitem{preprint} J.~P.~Cossey and M.~L.~Lewis, Counting lifts of
        Brauer characters, $\langle$arXiv:1007.3011v1$\rangle$

\bibitem{hupte} B. Huppert, ``Character Theory of Finite Groups,''
         Walter de Gruyter, Berlin, 1998.

\bibitem{gajen} D. Gajendragadkar, A characteristic class of
    characters of finite $\pi$-separable groups,
    {\it J. Algebra} {\bf 59} (1979), 237-259.

\bibitem{brown}   I.~M.~Isaacs, Characters of solvable and symplectic groups,
                  {\it Amer. J. Math.} {\bf 95} (1973), 594-635.

\bibitem{text}    I.~M.~Isaacs, ``Character Theory of Finite
        Groups,'' Academic Press, San Diego, California, 1976.

\bibitem{strong}  I.~M.~Isaacs, Character correspondences in
                  solvable groups, {\it Adv. Math.} {\bf 43} (1982),
                  284-306.

\bibitem{fram}    I.~M.~Isaacs, On the character theory of fully ramified sections,
                  {\it Rocky Mountain J. Math.} {\bf 13} (1983),
                  689-698.

\bibitem{pisep}   I.~M.~Isaacs, Characters of $\pi$-separable groups,
                  {\it J. Algebra} {\bf 86} (1984), 98-128.

\bibitem{Fong}    I.~M.~Isaacs, Fong characters in $\pi$-separable
        groups, {\it J. Algebra} {\bf 99} (1986), 89-107.

\bibitem{pipart}  I.~M.~Issacs, Partial characters of $\pi$-separable
        groups.  In: Representation Theory of Finite Groups
        and Finite Dimensional Algebras (Bielefield, 1991) in Progr. Math,
        vol. 95, Birkhauser, Basel, 1991, 273-287.

\bibitem{Laradji} A.~Laradji, On normal subgroups and simple modules
        with a given vertex in a $p$-solvable group, {\it J. Algebra}
        {\bf 308} (2007), 484-492.

\bibitem{max}    M.~L.~Lewis, Characters of maximal subgroups of
       $M$-groups, {\it J. Algebra} {\bf 183} (1996), 864-897.

\bibitem{Mawo} O. Manz and T. R. Wolf, ``Representation of Solvable
        Groups,'' Cambridge University Press, Cambridge, 1993.

\end{thebibliography}
\end{document}


\clearpage
\phantomsection
\addcontentsline{toc}{subsection}{SLL}
\label{insn:sll}
\subsection*{SLL: Shift Left Logical}

\subsubsection*{Format}

\textrm{SLL \%rd, \%r1, \%r2}

\begin{center}
\begin{bytefield}[endianness=big,bitformatting=\scriptsize]{32}
\bitheader{0,7,8,15,16,23,24,31} \\
\bitbox{8}{0x04}
\bitbox{8}{r1}
\bitbox{8}{r2}
\bitbox{8}{rd}
\end{bytefield}
\end{center}

\subsubsection*{Description}

This instruction shifts the value found in register \register{r1}
\emph{left} by the number of bits found in register \register{r2},
placing the results in register \register{rd}.

\subsubsection*{Pseudocode}

\begin{verbatim}
%rd = %r1 << %r2
\end{verbatim}

\subsubsection*{Load-time constraints}
The registers \registerop{r1}, \registerop{r2} and \registerop{rd} must be
valid registers and \registerop{rd} must not be \registerop{r0}.

\subsubsection*{Failure modes}

This instruction has no run-time failure modes beyond its constraints.