% small.tex
\documentclass[aspectratio=169]{beamer}

\usetheme{Boadilla}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{wrapfig}
%algorithms and pseudo code
\usepackage{algorithm}
\usepackage[noend]{algpseudocode}
\usepackage{numprint}
\usepackage{subcaption}
\usepackage{media9}
\usepackage{bibentry}
\usepackage[autoplay,loop]{animate}
\usepackage[justification=centering]{caption}
\nobibliography*

\setlength{\abovecaptionskip}{1pt}
\setlength{\belowcaptionskip}{-10pt}
\setlength{\leftmargini}{5pt}
\setlength{\leftmarginii}{10pt}
\setlength{\leftmarginiii}{15pt}

\usepackage{graphicx}

\setbeamertemplate{bibliography item}[text]
\setbeamertemplate{author in head/foot}{\insertshortauthor}
\setbeamertemplate{navigation symbols}{}

\newcommand{\lenitem}[2][.6\linewidth]{\parbox[t]{#1}{\strut #2\strut}}
\newcommand{\outline}{
  \begin{frame}<beamer>
    \frametitle{Outline}
    \tableofcontents[currentsection]
  \end{frame}
}

\newcommand{\outlinefull}{
  \begin{frame}<beamer>
    \frametitle{Outline}
    \tableofcontents
  \end{frame}
}

\begin{document}
\title[Partition Improvement for PIC]{

  MS60 Partitioning and Process Mapping for Emerging Architectures - Part II of II

  \textbf{Partition Improvement for High-Order Unstructured Mesh and Particle-In-Cell Simulations}
}

\author[G. Diamond]{\underline{Gerrett Diamond}, Cameron W. Smith, Mark S. Shephard}

\institute[SCOREC]{
  Scientific Computation Research Center \\
  Rensselaer Polytechnic Institute
}

\date{February 25, 2022}

%----------- titlepage ----------------------------------------------%
\begin{frame}[plain]
  \titlepage
\end{frame}

\outlinefull

\section{EnGPar: Diffusive Partition Improvement Tool}

\begin{frame}
  \frametitle{What is EnGPar?}
  \begin{itemize}
    \setlength\itemsep{1em}
  \item A partitioning tool to complement existing partitioning methods.
  \item Provides a diffusive load balancing algorithm for partition improvement with support for multi-criteria partitioning.
  \item Fast iterative algorithm permits usage during simulation runtime for dynamic load balancing.
  \item Utilizes a specialized multigraph structure to represent relation based data.
  \item Original methods designed for unstructured mesh simulations, but the usage of this graph structure allows for a broader range of applications.
  \item EnGPar's source can be found at \url{scorec.github.io/EnGPar}
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{N-graph}
  \begin{columns}
    \begin{column}{.75\textwidth}
      \begin{itemize}
      \item Uses multi-hypergraph, N-graph, to represent user data with multiple types of relational connections:
        \begin{itemize}
        \item Vertices - Uniquely owned, represent partitioned data
        \item Hyperedges - Relational connections between sets of vertices
        \item $N$ sets of hyperedges represent different types of relations for multi-criteria
        \end{itemize}
      \item To map simulation data to the N-graph simulations:
        \begin{itemize}
        \item Define the partitioned units of work as the graph vertices.
        \item Define relationships between the units of work as hyperedges between the graph vertices.
        \item Additional types of relationships or criteria that require balancing are added as additional sets of hyperedges.
        \end{itemize}
      \end{itemize}
        \end{column}
    \begin{column}{.25\textwidth}
      \begin{figure}
        \includegraphics[height=.6\textheight]{examplegraphs.png}
        \caption*{Example N-graphs with one (top) and two (bottom) hyperedge types.}
      \end{figure}
    \end{column}
    \end{columns}
\end{frame}

\begin{frame}
  \frametitle{EnGPar - Diffusive Load Balancing Method}
  \begin{itemize}
  \item EnGPar takes as input a list of criteria (graph vertices or hyperedge types) and target imbalances for each criteria.
  \item The algorithm iteratively performs steps to improve the balance of one criteria at a time.
  \item In each iteration weight is diffused across part boundaries from highly loaded parts to lighter neighbors.
  \item The iterations are repeated until the criteria is satisfied or stagnation is detected.
  \item The next criteria is balanced with a new set of iterations maintaining the balance of previous criteria.
  \end{itemize}
\end{frame}

\section{Load Balancing High-Order Unstructured Mesh Simulations}
\outline

\begin{frame}
  \frametitle{High-Order Unstructured Mesh Case}
  \begin{itemize}
  \item Unstructured meshes coming from finite element/volume simulations are typically partitioned by:
    \begin{itemize}
    \item Elements - Element-Partitioned Mesh
    \item Vertices - Vertex-Partitioned Mesh
    \end{itemize}
  \item Costs associated with mesh entities are classified in two categories
    \begin{itemize}
    \item Computational - proportional to the number of mesh entities involved in the simulation computations.
    \item Communication - proportional to the number of shared mesh entities on the partition model boundary.
    \end{itemize}
  \item For high-order simulations, computational and communication costs are associated with multiple dimensions of mesh entity.
    \item Common mesh partitioning techniques (multilevel-graph/hypergraph and geometric) do not properly partition all mesh entities.
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{N-graph for High-Order Unstructured Mesh}
  \begin{itemize}
  \item High-order experiments are run on an 3D element-partitioned mesh.
  \item Computation costs associated to all lower mesh-entity dimensions are represented.
  \item The N-graph is constructed with:
    \begin{itemize}
    \item Graph vertices constructed for each mesh element.
    \item Hyperedges constructed for each lower mesh entity.
    \item Two approaches to the sets of hyperedges are explored:
      \begin{enumerate}
      \item Hyperedge types for each lower mesh dimension (mesh vertices, mesh edges, mesh faces)
      \item One hyperedges type for all lower mesh entities.
      \end{enumerate}
    \end{itemize}
  \end{itemize}
  \begin{figure}
    \centering
    \includegraphics[height=.25\textheight]{meshToNgraphs.eps}
    \caption*{A 2D mesh (left) and N-graph (right) for element-partition with 2 hyperedge types for the mesh vertices (white squares) and mesh edges (black squares).}
  \end{figure}

\end{frame}

\begin{frame}
  \frametitle{High-Order Mesh Experiment}
  \begin{itemize}
    \item Problem Setup:
    \begin{itemize}
    \item 60 million mixed element mesh
    \item Global ParMETIS part k-way to 1Ki, 2Ki, 4Ki, and 8Ki parts
    \end{itemize}
    \item Hyperedges are weighted as follows:
      \begin{itemize}
      \item Mesh vertices have weight of 1
      \item Mesh edges have weight of 2
      \item Mesh triangles have weight of 1, Mesh quadrilaterals have weight of 2
      \end{itemize}
    \item Target reducing the total imbalance of the sum of hyperedge weights followed by the graph vertices.
    \item Experiments run on the Theta Supercomputer at Argonne
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{High-Order Mesh - Imbalance}
  \begin{figure}[!ht]
    \centering
    \includegraphics[width=.47\linewidth]{dof_v_cores.eps}
    \includegraphics[width=.47\linewidth]{elm_v_cores.eps}
    \caption*{Imbalances of hyperedges (left) and graph vertices (right). \bf Lower is better}
  \end{figure}
  \vspace{.3cm}
  \begin{itemize}
  \item Single hyperedge type achieves a much lower imbalance of the hyperedges and slightly better for graph vertices.
  \item Single hyperedge iterations are more expensive than the multiple hyperedge case because the number of hyperedges traversed in each iteration is greater.
  \end{itemize}
\end{frame}

\section{Load Balancing for Particle-in-Cell Simulations}
\outline

\begin{frame}
  \frametitle{Unstructured Mesh Particle-In-Cell Simulations}
  \begin{itemize}
  \item Particle-In-Cell (PIC) simulations are time-advancing procedures that track particles as they move in a domain and interact with the governing fields.
  \item The domain is discretized using an unstructured mesh.
  \end{itemize}
  \begin{columns}
    \begin{column}{.6\linewidth}
      \begin{itemize}
      \item The general particle loop for a PIC application iterates over four steps:
        \begin{itemize}
        \item Particle Push - particle positions are updated based on mesh fields.
        \item Charge Deposition - based on the new particle positions, mesh fields are updated.
        \item Field Solve - domain level PDEs to update global mesh fields.
        \item Field-to-Particle - particle information is updated for the next push operation.
        \end{itemize}
      \end{itemize}
    \end{column}
    \begin{column}{.4\linewidth}
      \begin{figure}
        \centering
        \includegraphics[width=.75\textwidth]{pic_loop.jpg}
        \caption*{Operations of PIC simulation's iteration loop}
      \end{figure}
    \end{column}
  \end{columns}
\end{frame}

%Mesh-based PIC + PICparts
\begin{frame}
  \frametitle{Approaches to PIC}
  \begin{itemize}
  \item Traditional approach to PIC is to primarily store particles
    \begin{itemize}
    \item Particles maintain knowledge of the mesh element it is within (parent element).
    \item A full copy of the mesh is maintained on all processes.
    \item Particles are distributed across processes
    \item Scalable wrt number of particles but not wrt mesh.
    \end{itemize}

  \item Mesh-based approach to PIC simulation store particles based on the parent mesh element:
    \begin{itemize}
    \item Particles are stored in memory based on their parent element.
      \begin{itemize}
      \item Easier to maintain a distributed mesh
      \end{itemize}
    \item Both particles and the mesh are distributed across processes
    \item Scalabale wrt number of particles and mesh entities.
    \end{itemize}
  \end{itemize}
\end{frame}

%Describe N-graph for PICparts and how to balance
\begin{frame}{Dynamic Load Balancing of Particles in Mesh-Based PIC}
  \begin{columns}
    \begin{column}{0.7\linewidth}
      \begin{itemize}
      \item To reduce communications the mesh partition is supplemented with sufficient buffering of nearby part's mesh entities.
      \item Each processes owned mesh entities plus the buffered entities make up a PICpart.
      \item A subset of the elements in a PICpart are denoted as safe for particles to be stored and operated on.
        \begin{itemize}
        \item If a particles moves to an unsafe element, then it must be migrated to a different process.
        \item Since there is overlap of the safe elements across processes there is an opportunity to migrate additional particles from a safe process to another safe process.
        \end{itemize}
      \end{itemize}
    \end{column}
    \begin{column}{0.3\linewidth}
      \begin{figure}
        \centering
        \includegraphics[height=.35\textheight]{xgcm_partition.png}
        \caption*{Partition of 2D mesh}
      \end{figure}
      \vspace{-.25cm}
      \begin{figure}
        \centering
        \includegraphics[width=\textwidth]{picpart.png}
        \caption*{PICpart for part A (left) and its safe zone (right)}
      \end{figure}
    \end{column}
  \end{columns}

\end{frame}

\begin{frame}{Overlapping Safe Zones}
  \begin{columns}
    \begin{column}{.5\textwidth}
      \begin{itemize}
      \item To determine where particles can migrate to, we introduce the notion of overlapping safe zones.
      \item For a set of PICparts $P$, the overlapping safe zone, $\bar{S}_P$, is the set of mesh elements such that each element is safe on each PICpart in $P$.
      \item The overlapping safe zones are constructed such that every mesh element belongs to exactly one $\bar{S}_P$.
        \begin{itemize}
        \item Some overlapping safe zones may be defined on one PICpart.
        \end{itemize}
      \item Any particle in an element in $\bar{S}_P$ can be migrated to any PICpart in $P$.
      \end{itemize}
    \end{column}
    \begin{column}{.5\textwidth}
      \begin{figure}
        \includegraphics[width=.95\textwidth]{picpart_lb_new.eps}
        \caption*{Four cores (left) with three $\bar{S}_P$(right) on the boundary between cores A and B.}
      \end{figure}
    \end{column}
  \end{columns}
\end{frame}

\begin{frame}{Overlaping Safe Zones}
  \begin{columns}
    \begin{column}{.5\textwidth}
      \begin{itemize}
      \item We construct a hypergraph based on the overlapping safe zones as follows:
        \begin{itemize}
        \item For each $\bar{S}_P$ a subhypergraph is constructed.
        \item In the subhypergraph, one graph vertex is made for each PICpart in $P$.
        \item Each graph vertex in the subhypergraph is connected by one hyperedge for the $\bar{S}_P$.
        \end{itemize}
      \item The size of the hypergraph is significantly smaller than the original mesh as seen in the example table below.
      \end{itemize}
    \end{column}
    \begin{column}{.5\textwidth}
      \begin{figure}
        \includegraphics[width=.95\textwidth]{picpart_lb_new.eps}
        \caption*{Three $\bar{S}_P$(middle) on the boundary of cores A and B and corresponding subhypergraphs(right).}
      \end{figure}
    \end{column}
  \end{columns}
  \begin{table}
    \caption*{PICpart vs Hypergraph sizes for 11.4 million element mesh}
    \vspace{.1in}
    \begin{tabular}{|c|c|c|c|c|}
      \hline
      Number of PICparts & 6 & 12 & 24 & 48 \\
      \hline
      Average Elements per PICpart & 8.8M & 5.3M & 3.7M & 2.3M\\
      \hline
      Total Graph Vertices & 20 & 102 & 450 & 1931 \\
      \hline
      Total Graph Hyperedges & 3 & 19 & 69 & 266 \\
      \hline
    \end{tabular}
  \end{table}
\end{frame}

\begin{frame}{Diffusive Load Balancing with EnGPar}
  %Introduce EnGPar
  \begin{itemize}
    %Discuss how diffusive load balancing easily maps to this problem
  \item Applying EnGPar to our graph of overlapping safe zones
    \begin{itemize}
    \item Treat particles as contributions to the weight on each graph vertex.
    \item Weight is diffused between vertices across hyperedges.
    \item EnGPar creates a plan defining how much weight to send from process to process per hyperedge.
    \end{itemize}
  \end{itemize}
  Steps to perform load balancing with EnGPar
  \begin{enumerate}
    %Discuss the weighting of the graph
  \item Apply the number of particles in each $\bar{S}$ on each PICpart as the weight of the corresponding graph vertex.
  \item Run EnGPar's weight load balancer on the hypergraph to produce migration plan.
  \item Select particles to satisfy the plan from EnGPar.
  \end{enumerate}
\end{frame}

%Introduce GITRm example
\begin{frame}
  \frametitle{GITRm Experiments}
  \begin{itemize}
  \item Experiments performed on RPI's AiMOS supercomputer.
    \begin{itemize}
    \item Each Node has six NVIDIA Tesla V100 with 32GiB of memory.
    \item One MPI process per GPU.
    \end{itemize}
  \item Performance studies performed on GITRm
    \begin{itemize}
    \item Particles start at the bottom region of the domain.
    \item Particles curve around the domain moving upwards.
    \end{itemize}
  \item 11.4 million element mesh distributed up to 48 ranks.
  \item 25 million to 400 million particles simulated.
  \item 10,000 iterations performed with load balancing every 100 iterations.
  \end{itemize}
  \begin{figure}
    \includegraphics[height=.3\textheight]{gitrm_part_4.png}
    \includegraphics[height=.3\textheight]{gitrm_ptcl_init.png}
    \includegraphics[height=.3\textheight]{gitrm_ptcl_movement.png}
    \caption*{Partitioned mesh, elements with particles at iteration 0, trajectories of some particles}
  \end{figure}
\end{frame}

% Show results + summary
\begin{frame}
  \frametitle{GITRm Particle Imbalance Results}
  %Report no lb vs lb timing of full simulation
  %Show 3-6 plots of lb for different cases
  \begin{figure}
    \centering
    \includegraphics[width=.24\textwidth]{gitrm_imb_50m_6p.png}
    \includegraphics[width=.24\textwidth]{gitrm_imb_50m_12p.png}
    \includegraphics[width=.24\textwidth]{gitrm_imb_50m_24p.png}
    \includegraphics[width=.24\textwidth]{gitrm_imb_50m_48p.png}
  \end{figure}
  \vspace{-.25in}
  \begin{figure}
    \centering
    \includegraphics[width=.24\textwidth]{gitrm_imb_150m_6p.png}
    \includegraphics[width=.24\textwidth]{gitrm_imb_250m_12p.png}
    \includegraphics[width=.24\textwidth]{gitrm_imb_250m_24p.png}
    \includegraphics[width=.24\textwidth]{gitrm_imb_400m_48p.png}
    \caption*{Various GITRm setups executed with and without load balancing. \bf Lower is better}
  \end{figure}
  \begin{itemize}
  \item Particles are balanced below 5\% within the first few load balance exectutions.
  \item Particle imbalance is maintained after initial dispersion of particles
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{GITRm Timing Summary}
  \begin{itemize}
  \item Performing load balancing took on average .04\% of the particle loop time.
  \item At most load balancing took .07\% of the particle loop time with 25 million particles on  48 processes.
  \item Speedups of up to 20\% were achieved when running load balancing.
  \item When the number of particles per GPU is low, using load balancing results in a slowdown due to increase communications without any speedup in computation.
  \item Ongoing research into which operations are speeding up or slowing down as a result of load balancing the particles.
  \end{itemize}
\end{frame}

%Introduce XGC partitioning requirements
\begin{frame}
  \frametitle{Traditional PIC simulation: XGC}
  \begin{columns}
    \begin{column}{.7\textwidth}
      \begin{itemize}
      \item In XGC, a copy of the 2D unstructured mesh is maintained on each process.
      \item The third spatial dimension is implicitly simulated by copies of the plane around the toroidal domain.
      \item Particles and operations performed on the mesh for each plane are partitioned
        across processes.
      \item The mesh is partitioned by a 1D partition of the mesh vertices
        \begin{itemize}
        \item $[0, N_0) \rightarrow$ process 0, $[N_0, N_1) \rightarrow$ process 1, etc.
        \end{itemize}
      \item The current approach to load balancing specifically targets three major components of the simualtion:
        \begin{itemize}
        \item Electron particles
        \item Ion particles
        \item The most expensive mesh operation: the collision operator.
        \end{itemize}
      \item Current load balancing algorithm in XGC is specificly targeting these three criteria.
      \end{itemize}
    \end{column}

    \begin{column}{.3\textwidth}
      %TODO: Generate better image of mesh
      \begin{figure}
        \centering
        \includegraphics[height=.35\textheight]{itg24k.png}
        \caption*{2D mesh of the plane}
      \end{figure}
      \vspace{-.25cm}
      \begin{figure}
        \centering
        \includegraphics[height=.3\textheight]{cyclone_590k_30kptcls.png}
        \caption*{Eight planes depicted around the toroidal dimension}
      \end{figure}
    \end{column}
  \end{columns}
\end{frame}

%How EnGPar can be applied
\begin{frame}
  \frametitle{Applying EnGPar in XGC}
  \begin{itemize}
  \item Goal:
    \begin{itemize}
    \item Improve the load balancing in XGC using EnGPar.
    \item Provide a general approach to load balancing for future improvements.
    \end{itemize}
  \item Given the 1D partition of the mesh, the N-graph is defined as:
    \begin{itemize}
    \item One graph vertex for each mesh vertex.
    \item Graph edges connect mesh vertices with adjacent indices.
    \end{itemize}
  \item The weights computed by XGC for the ions, electrons and the collision operator are used as three different criteria to be balanced in EnGPar.
  \end{itemize}
\end{frame}

%Summary Slide
\begin{frame}
  \frametitle{Closing Remarks}
  \begin{itemize}
  \item EnGPar provides dynamic diffusive load balancing for a range of applications that can be represented by a hypergraph.
  \item We applied EnGPar to  improve imbalances of lower dimensional mesh entities of $>50\%$ for high-order unstructured mesh examples.
  \item EnGPar can reduce and maintain particle imbalances in mesh-based particle-in-cell simulations.
  \end{itemize}
  \textbf{Future Work}
  \begin{itemize}
  \item Compare EnGPar to the current load balancing method in XGC
    \begin{itemize}
    \item Explore further options in improving XGC's partition.
    \end{itemize}
  \item Increase the size and scale of current examples to see how these approaches perform.
  \item Apply EnGPar to other simulations and domains.
  \end{itemize}
\end{frame}
%Thank you slide
\begin{frame}
  \begin{columns}
    \begin{column}{.33\linewidth}
      \begin{figure}
        \centering
        \includegraphics[width=\linewidth]{fastmath_logo.png}
      \end{figure}
    \end{column}
    \begin{column}{.33\linewidth}
      \begin{center}
        {\huge
          Thank You\\
          \bigskip
          \bigskip
          Questions?\\
          \bigskip
        }
      \end{center}
    \end{column}
    \begin{column}{.33\linewidth}
      \begin{figure}
        \centering
        \includegraphics[width=\linewidth]{hbps_logo.png}
      \end{figure}
    \end{column}
  \end{columns}
  \vspace{.3cm}
  Part of the SciDAC supported project, ``Unstructured Mesh Technologies for Fusion Simulation Codes'' \\
  In collaboration with:
  \begin{itemize}
  \item FASTMath SciDAC Institute
  \item High-Fidelity Boundary Plasma Simulation SciDAC Partnership
  \item Plasma Surface Interactions SciDAC Partnership
  \item COPA: ECP Co-Design Center for Particle Applications
  \end{itemize}
\end{frame}

\end{document}


% !TEX root = ../main.tex

\section{Contents}
\begin{frame}
\frametitle{Holistic approach}
\begin{figure}[H]
\centering
\begin{tikzpicture}
    %\draw [thin] (0,0) circle [radius=2];
    \draw (0,2) node{Synchronisation};
    \draw (1.732,-1) node{Topology};
    \draw (-1.732,-1) node{Plasticity};
     \draw[black, <->, thick] (-1,1.73)to[out=180+25,in=120-7.5](-1.85,-0.77);
     \draw[black, <->, thick] (1,1.73)to[out=-25,in=180-120+7.5](1.85,-0.77);
     \draw[black, <->, thick] (-1.59,-1.22)to[out=-60+7.5,in=-120-7.5](1.59,-1.22);
    \end{tikzpicture}
\end{figure}
\end{frame}

\section{Introduction}
\begin{frame}
\frametitle{Neuron dynamics}
How do neurons communicate?
\begin{itemize}
\item Neurotransmitters \\
\item Action potential = explosion of electrical activity \\ [0.5cm]
\end{itemize}
%How can we capture this behaviour?
%\begin{itemize}
%\item Human brain $\sim$ 100 billion neurons \\
%\item \MFR: average dynamics of the network \\ [0.5cm]
%\end{itemize}
How do neurons learn?
\begin{itemize}
\item \textsl{Fire and wire}: Correlation of neuronal activity \\
\end{itemize}
\end{frame}

\section{\theory The Theta Neuron Model}
\begin{frame}
\frametitle{Model Description}
%\tabitem Formulation
%\begin{align*}
%\dot{\theta} = (1-\cos \theta)+(1+\cos \theta) \cdot I \qquad \theta \in \T
%\end{align*}

\tabitem Normal form of SNIC bifurcation
\begin{figure}[H]
\minipage{0.33\linewidth}
\centering
\begin{tikzpicture}
    \draw[thick] (0,0) circle [radius=1];
    \draw (0,-1.2) node[below]{Excitable regime: $I < 0$};
    \draw (-1,0) node[left]{$\pi$};
    \draw[fill=black, black] (1,0) circle [radius=0.025];
    \draw (1,0) node[right]{0};
    \draw[fill=red, red] (-1,0) circle [radius=0.05];
    \draw (-1,0) node[right]{spike};

    \draw[black, ->, thick] (0.866, 0.5)to[out=-60,in=90](1,0);
    \draw[fill=white, draw=black] (0.866,0.5) circle [radius=0.1];
    \draw (0.866,0.5) node[left]{\small{threshold}};

    \draw[fill=black, draw=black] (0.866,-0.5) circle [radius=0.1];
    \draw (0.866,-0.5) node[left]{\small{rest}};

    \draw[black, ->, thick] (0.5,0.866)to[out=150,in=0](0,1);
    \draw[black, ->, thick] (-0.5,-0.866)to[out=-30,in=180](0,-1);
\end{tikzpicture}
\endminipage
\minipage{0.33\linewidth}
\centering
\begin{tikzpicture}
    \draw[thick] (0,0) circle [radius=1];
    \draw (0,-1.2) node[below]{Bifurcation: $I = 0$};
    \draw (1.1,0) node[right]{0};
    \draw[fill=red, red] (-1,0) circle [radius=0.05];
    \draw (-1,0) node[right]{spike};
    \draw (-1,0) node[left]{$\pi$};

    \draw[fill=gray, draw=black] (1,0) circle [radius=0.1];

    \draw[black, ->, thick] (0.5,0.866)to[out=150,in=0](0,1);
    \draw[black, ->, thick] (-0.5,-0.866)to[out=-30,in=180](0,-1);
\end{tikzpicture}
\endminipage
\minipage{0.33\linewidth}
\centering
\begin{tikzpicture}
    \draw[thick] (0,0) circle [radius=1];
    \draw (0,-1.2) node[below]{Periodic regime: $I > 0$};
    \draw (-1,0) node[left]{$\pi$};
    \draw (1,0) node[right]{0};
    \draw[fill=black, black] (1,0) circle [radius=0.025];
    \draw[fill=red, red] (-1,0) circle [radius=0.05];
    \draw (-1,0) node[right]{spike};

    \draw[black, dotted] (0,0)to(1,0);
    \draw(0,0) node[above]{$\theta$};
    \draw[black, dotted] (0,0)to(0.866,0.5);

    \draw[black, ->, thick] (0.5,0.866)to[out=150,in=0](0,1);
    \draw[black, ->, thick] (-0.5,-0.866)to[out=-30,in=180](0,-1);
\end{tikzpicture}
\endminipage
\label{fig:thetaneuronbifurcationtikz}
\end{figure}
\end{frame}

%\begin{frame}
%\frametitle{Response}
%\tabitem Formulate bifurcations in terms of spiking frequency or phase angle
%\begin{figure}[H]
%\centering
%\includegraphics[width = \textwidth]{../Figures/ThetaNeuronfIandPRC.pdf}
%\label{fig:ThetaNeuronfIandPRC}
%\end{figure}
%\end{frame}

\section{\theory Network Topologies}
\begin{frame}
\frametitle{Three basic networks}
\begin{figure}[H]
\centering
\includegraphics[width = \textwidth]{../Figures/Distributions/1D.pdf}
\label{fig:1Dpdfs}
\end{figure}
\end{frame}

\begin{frame}
\frametitle{Networks of Theta neurons}
\tabitem For arbitrary network topology:
\begin{align*}
\dot{\theta}_{i} &=\left(1-\cos \theta_{i}\right)+\left(1+\cos \theta_{i}\right) \cdot \left[\eta_{i} + I_{i}(t)\right] \qquad \theta_i \in \T^N \\
I_{i}(t) &=\frac{\kappa}{\kmean} \sum_{j=1}^{N} A_{i j} \cdot \mathcal{P}_{n}(\theta_{j})
\end{align*}

\tabitem Capture average/mean synchronisation
\begin{align*}
Z(t) = \frac{1}{N} \sum_{j=1}^N e^{\ic\theta_j}  \qquad Z \in \C
\end{align*}
\end{frame}

\section{\theory Mean Field Reduction}
\begin{frame}
\frametitle{Predict synchronisation dynamics}
The \MFR = solution for $Z(t)$\\
\tabitem Simple for fixed-degree networks: one equation\\[0.5cm]

\textsl{Solution?} Formulate $Z(t)$ per degree $z(\k,t)$!\\
\tabitem Less unique degrees than neurons \\
\tabitem = less equations! \\
%\tabitem $M_{\k} \ll N$ unique node degrees \\
%\tabitem Only $M_{\k}$ equations left \\
%\tabitem Weighed by $P(\k)$
\begin{align*}
\bar{Z}(t) &= \frac{1}{N} \sum_{\k} P(\k) z(\k, t) \qquad \bar{Z} \in \C \label{eq:OttAntonsenMeanField}
\end{align*}
%Problem: $M_{\k}$ still too large when $P(\k)$ is bivariate
\end{frame}

\begin{frame}
\frametitle{Fixed-degree networks}
\begin{figure}[H]
\centering
\begin{subfigure}[b]{0.32\linewidth}
   \centering
  \includegraphics[width=\linewidth]{../Figures/PhaseSpace/MFRPSR.pdf}
  \caption{PSR}
   \label{fig:MFRPSR}
\end{subfigure} \hfill
\begin{subfigure}[b]{0.32\linewidth}
   \centering
  \includegraphics[width=\linewidth]{../Figures/PhaseSpace/MFRPSS.pdf}
  \caption{PSS}
   \label{fig:MFRPSS}
\end{subfigure} \hfill
\begin{subfigure}[b]{0.32\linewidth}
   \centering
  \includegraphics[width=\linewidth]{../Figures/PhaseSpace/MFRCPW.pdf}
    \caption{CPW}
   \label{fig:MFRCPW}
\end{subfigure}
   \label{fig:macroscopicstatesfixeddegree}
\end{figure}
\end{frame}

\section{\mywork Mean Field Reductions for undirected graphs}
\begin{frame}
\frametitle{Goals}
$Z(t)$ can be measured and predicted: are they the same? \\
\begin{itemize}
\item Formulate directed networks \\
\item Construct adjacency matrix from degree distribution \\
\item Initial conditions
\item Compare directed and undirected networks
\end{itemize}
\end{frame}

\begin{frame}
\frametitle{Directed networks}
%Directed networks = bivariate degree distribution = asymmetric adjacency matrix \\
Sample degree vectors from bivariate distribution? Difficult!\\
\tabitem Use independant univariate distributions as marginal \\
\tabitem In- and out degree vectors are a permutation
\end{frame}

\begin{frame}
\frametitle{Directed networks}
\begin{figure}[ht]
\centering
\includegraphics[width = 0.95\textwidth]{../Figures/Distributions/2D.pdf}
\label{fig:2Ddistributions}
\end{figure}
\end{frame}

\begin{frame}
\frametitle{Adjacency matrix}
Find a probable solution by sampling from the in- and outdegrees
\begin{figure}[H]
\centering
\includegraphics[width = \textwidth]{../Figures/Adjacency_matrices.pdf}
\label{fig:adjacencymatrices}
\end{figure}
\end{frame}

\begin{frame}
\frametitle{Initial conditions}
\begin{figure}[H]
\centering
\includegraphics[trim=0cm 8.5cm 0cm 0cm, clip=true, width = 0.75\textwidth]{../Figures/PhaseSpace/Mappings.pdf}
   \label{fig:mappings}
\end{figure}
\end{frame}

%\begin{frame}
%\frametitle{Final conditions}
%\begin{figure}[H]
%\centering
%\includegraphics[width = \textwidth]{../Figures/Distributions/FinalConditions.pdf}
%\label{fig:FinalConditions}
%\end{figure}
%\end{frame}

\begin{frame}
\frametitle{Results}
\begin{figure}[H]
\centering
\includegraphics[width = \textwidth, trim={0 3mm 0 3mm},clip]{../Figures/InspectMeanFieldFixedDegree.pdf}
\label{fig:InspectMeanFieldFixedDegree}
\end{figure}
\end{frame}

\begin{frame}
\frametitle{Results}
\begin{figure}[H]
\centering
\includegraphics[width = 0.45\textwidth]{../Figures/PhaseSpace/ScalefreeLimCycles.pdf}
\label{fig:InspectMeanFieldScaleFreePhaseSpace}
\end{figure}
\end{frame}

\section{\theory Hebbian Learning and Synaptic Plasticity}
\begin{frame}
\frametitle{Temporal Interpretation: \STDP}
\tabitem \textsl{Fire and wire}: correlate successive action potentials
\begin{align*}
\Delta \text{Synaptic strength  } \sim \sum_{t_{j}^{f}, t_{i}^{n} \in \mathcal{T}} W\left(t_{j}^{f}-t_{i}^{n}\right)
\end{align*}
\tabitem \IP: adjust neuron sensitivity to incoming action potentials
\begin{figure}[H]
\centering
\includegraphics[width = \textwidth]{../Figures/Learning/LearningWindows.pdf}
\label{fig:LearningWindows}
\end{figure}
\end{frame}

\section{\mywork Emerging Network Topologies}
\begin{frame}
\frametitle{\STDP}
\begin{figure}[H]
\centering
\includegraphics[trim=0cm 24.7cm 0cm 8.4cm, clip=true, height = 0.455\textheight]{../Figures/Learning/STDP.pdf}
\label{fig:STDP}
\end{figure}
\begin{figure}[H]
\centering
\includegraphics[trim=0cm 7.2cm 0cm 26cm, clip=true, height = 0.45\textheight]{../Figures/Learning/STDP.pdf}
\label{fig:STDP}
\end{figure}
\end{frame}

\begin{frame}
\frametitle{\STDP + \IP}
\begin{figure}[H]
\centering
\includegraphics[trim=0cm 28.5cm 0cm 7.1cm, clip=true, height = 0.475\textheight]{../Figures/Learning/STDPandIP.pdf}
\label{fig:STDP}
\end{figure}
\begin{figure}[H]
\centering
\includegraphics[trim=0cm 13.3cm 0cm 22.7cm, clip=true, height = 0.45\textheight]{../Figures/Learning/STDPandIP.pdf}
\label{fig:STDP}
\end{figure}
\end{frame}

\section{Conclusion}
\begin{frame}
\frametitle{Accomplishments}
\tabitem Built and simulated directed networks \\
\tabitem Compared simulation and prediction at every timepoint \\
\tabitem Network structure emerges from learning strategy \\
\tabitem Unification of dynamics \textsl{on} and \textsl{of} networks
\end{frame}

\begin{frame}
\frametitle{Future Work}
\tabitem Further investigation of initial and final conditions \\
\tabitem Computational challenges
\end{frame}


% This percent indicates a comment.
% This is a very simple latex article that introduces the way
% equations are typeset. Do this in GNU/Linux:
%
% latex first.tex
% latex first.tex
% xdvi first.dvi
% dvips -o first.ps first.dvi
% gv first.ps
% lpr first.ps
% pdflatex first.tex
% okular first.pdf
\documentclass[12pt]{article}
\title{First \LaTeX}
\author{Joe Student}
\date{\today}

\begin{document}

\maketitle

\abstract{This is a very simple example of using \LaTeX\ for typesetting.
The procedure for typesetting equations is introduced.}
\begin{sdsd}
\end{bar}

\section{A few equations}
Equations can be typeset inline like so: $\vec{F}=m\vec{a}$ .
Equations can also be separated form the text: $$ \vec{F}=m\vec{a} \ . $$
Notice the punctuation, the ``.'', had to be included with the equation.
Equations can also have a number assigned and a label attached,
as in the following:
\begin{equation}
\frac{D\vec{\omega}}{Dt} =
( \vec{\omega} + \vec{\Omega}) \cdot \nabla  \vec{U}
+ \frac{1}{\rho^2} \nabla \rho \times \nabla p
+ \nu \nabla^2  \vec{\omega}
\label{vorteq}
\end{equation}
The vorticity equation (\ref{vorteq}) is referenced by label,
not by number.  The numbers may change as the document grows and
more equations are added.

New paragraphs are indicated with a blank line in the source code.

\section{The End}

Further examples of \LaTeX\ can be found at {\tt it.metr.ou.edu}

\end{document}


\documentclass[paper=a4, fontsize=11pt]{scrartcl} % A4 paper and 11pt font size

\usepackage[T1]{fontenc} % Use 8-bit encoding that has 256 glyphs
\usepackage{fourier} % Use the Adobe Utopia font for the document - comment this line to return to the LaTeX default
\usepackage[english]{babel} % English language/hyphenation
\usepackage{amsmath,amsfonts,amsthm} % Math packages
\usepackage{wrapfig}
\usepackage{lipsum} % Used for inserting dummy 'Lorem ipsum' text into the template
\usepackage{tikz}
\usepackage{amsmath}
\usepackage{sectsty} % Allows customizing section commands
\allsectionsfont{\centering \normalfont\scshape} % Make all sections centered, the default font and small caps

\usepackage{fancyhdr} % Custom headers and footers
\pagestyle{fancyplain} % Makes all pages in the document conform to the custom headers and footers
\fancyhead{} % No page header - if you want one, create it in the same way as the footers below
\fancyfoot[L]{} % Empty left footer
\fancyfoot[C]{} % Empty center footer
\fancyfoot[R]{\thepage} % Page numbering for right footer
\renewcommand{\headrulewidth}{0pt} % Remove header underlines
\renewcommand{\footrulewidth}{0pt} % Remove footer underlines
\setlength{\headheight}{4pt} % Customize the height of the header
\usepackage{float}
\numberwithin{equation}{section} % Number equations within sections (i.e. 1.1, 1.2, 2.1, 2.2 instead of 1, 2, 3, 4)
\numberwithin{figure}{section} % Number figures within sections (i.e. 1.1, 1.2, 2.1, 2.2 instead of 1, 2, 3, 4)
\numberwithin{table}{section} % Number tables within sections (i.e. 1.1, 1.2, 2.1, 2.2 instead of 1, 2, 3, 4)

\setlength\parindent{0pt} % Removes all indentation from paragraphs - comment this line for an assignment with lots of text

%----------------------------------------------------------------------------------------
%	TITLE SECTION
%----------------------------------------------------------------------------------------

\newcommand{\horrule}[1]{\rule{\linewidth}{#1}} % Create horizontal rule command with 1 argument of height

\title{
\normalfont \normalsize  % Your university, school and/or department name(s)
\horrule{0.5pt} \\[0.4cm] % Thin top horizontal rule
\huge Physics 422 H.W. 1\\ % The assignment title
\horrule{2pt} \\[0.5cm] % Thick bottom horizontal rule
}

\author{Josh Lucas} % Your name

\date{\normalsize\today} % Today's date or a custom date

\begin{document}

\maketitle % Print the title

%----------------------------------------------------------------------------------------
%	PROBLEM 1
%----------------------------------------------------------------------------------------

\section{Lattice Cells and Basis}
\textbf{Consider the pattern and indicate the following: A \textbf{Rectangular unit cell}, a \textbf{Primitive unit cell}, and the \textbf{Basis} of this "crystal."}\\
\begin{wrapfigure}[20]{l}{0.55\textwidth}
\includegraphics[width=8cm]{Lattice}
\caption{2-D Lattice from bottom to top: Primitive cell, Retangular cell}
\end{wrapfigure}\\
We can place the \textbf{rectangular cell} around the top two rows, if stacked on top of each other they will form the pattern of atoms repeatedly.\\ The rectangular cell is not the smallest volume comprising all of the atoms, the \textbf{primitive unit cell} is the smallest volume containing all points and can be formed by diagonal lines at a relative angle of $\frac{\pi}{2}$ to each other. With this primitive unit cell we can repeat the pattern with only one instance of each particle.\\
The \textbf{Basis} consists of four particles. If considering the origin to be the bottom left particle and disregarding the tilt of the axes we can say the basis is
\begin{equation*}
basis = \begin{cases} \odot\  & @ (0,0) \\ \bigcirc , & @ (\frac{1}{8},0) \\ \diamond & @ (0,\frac{1}{2}) \\ \oplus &  @(\frac{1}{8},\frac{1}{2}) \end{cases}
\end{equation*}

If the axes are taken into account with the corner at the origin the basis becomes,
\begin{equation*}
 Basis= \begin{cases} \odot\  & @ (\frac{1}{8},\frac{1}{4}) \\ \bigcirc  & @ (\frac{1}{4},\frac{1}{8}) \\ \diamond & @ (\frac{3}{4},\frac{7}{8}) \\ \oplus &  @(\frac{7}{8},\frac{3}{4}) \end{cases}
\end{equation*}

%\begin{tikzpicture}
%  %\draw[thin,gray!40] (-2,-2) grid (2,2);
%  %\draw[<->] (-2,0)--(2,0) node[right]{$x$};
% % \draw[<->] (0,-2)--(0,2) node[above]{$Z$};
%  \draw[line width=2pt,red,-stealth](0,0)--(6,0) node[anchor=south west]{$\boldsymbol{\vec{a_1}}$};
%  \draw[line width=2pt,red,-stealth](6,0)--(6,2) node[anchor=north east]{$\boldsymbol{\vec{a_2}}$};
%    \draw[line width=2pt,black,-stealth](0,2)--(6,2) node[anchor=south west]{};
%  \draw[line width=2pt,black,-stealth](0,0)--(0,2) node[anchor=north east]{};
%\end{tikzpicture}
%\hspace{25mm}
%\begin{tikzpicture}
%  %\draw[thin,gray!40] (-2,-2) grid (2,2);
%  %\draw[<->] (-2,0)--(2,0) node[right]{$x$};
% % \draw[<->] (0,-2)--(0,2) node[above]{$Z$};
%  \draw[line width=2pt,red,-stealth](0,0)--(1,1) node[anchor=north west]{$\boldsymbol{\vec{a_1}}$};
%  \draw[line width=2pt,red,-stealth](1,1)--(0,2) node[anchor=north east]{};
%   \draw[line width=2pt,black,-stealth](0,2)--(1,3) node[anchor= east]{};
%  \draw[line width=2pt,black,-stealth](1,1)--(2,2) node[anchor=north west]{};
%   \draw[line width=2pt,black,-stealth](2,2)--(1,3) node[anchor= west]{};
%\end{tikzpicture}

\section{Nearest Neighbors}
%------------------------------------------------
\textbf{a) What are the number of nearest neighbors to each lattice point in the simple cubic, BCC, and FCC latices?}
\\
\\
\textbf{Simple Cubic }$\bullet$ In the simple cubic lattice the atom has four neighbors in the horizontal plane and a neighbor above and below giving it a \textbf{coordination number of 6}.
\newline
\textbf{Body-Centered Cubic}$\bullet$ In the body-centered cubic, BCC, the atom at the center has four  neighbors above and four below giving it a \textbf{coordination number of 8.}\\
\textbf{Face-Centered Cubic}$\bullet$  The face centered cubic has an atom centered on each face with nearests neighbors located ad the 8 other adjacent faces as well as the four corners of the atom's face giving it a \textbf{coordination number of 12.}
\newline \\
\textbf{b)  If the conventional unit cell has sides of length $a_1$, what are the distances between nearest neighbors in each case?}
\\
\textbf{Simple Cubic }$\bullet$  The distance to the nearest neighbor in the simple cubic is the length of the primitive vector $a_1$ with a magnitude of a.\\
\textbf{Body-Centered Cubic}$\bullet$  The body-centered cubic has an atom at the center of the cube with equal distance to the eight corners. The vector from a corner at the origin to the center of the cube is $$\vec{r} = \frac{1}{2} (a_1+a_2+a_3)$$  making the nearest atom's distance the magnitude of the vector,
\begin{equation*}
\bigg|\frac{1}{2} (a_1+a_2+a_3)\bigg| = \sqrt{\frac{1}{2}a_1^2+\frac{1}{2}a_2^2+\frac{1}{2}a_3^2 } = \sqrt{\frac{3}{4}a^2} = \frac{a}{2}\sqrt{3}
\end{equation*}
The distance to the nearest neighbor in the body centered cubic is $\frac{a}{2}\sqrt{3}$ \\

\textbf{Face-Centered Cubic}$\bullet$  The face-centered cubic is more densely packed than the BCC. the nearest neighbor will always be $\frac{1}{2}a$ in two orthogonal directions and in the same plane as another, such as,

$$\vec{r} =  \frac{1}{2}\vec{a_x} +\frac{1}{2}\vec{a_y} + (0)\vec{a_z}$$
or
$$\vec{r} =  (0)\vec{a_x} +\frac{1}{2}\vec{a_y} + \frac{1}{2}\vec{a_z}$$
 making the nearest atoms distance the magnitude
\begin{equation*}
\bigg| \frac{1}{2}\vec{a_x}+\frac{1}{2}\vec{a_y}\bigg| = \sqrt{(\frac{1}{2}a_x)^2+(\frac{1}{2}a_y^)2 } = \sqrt{\frac{1}{2}a^2} = \frac{a}{2}\sqrt{2}
\end{equation*}
The distance to the nearest neighbor in the body centered cubic is $\frac{a}{2}\sqrt{2}$ \\

\section{Zincblende}
\textbf{Considering the figure:} \\
\textbf{a) What is the Bravais Lattice type?}\\
The lattice type is a \textbf{face center cubic} with four lattice points per unit cell. The individual lattice points form a tetrahedral structure consisting of four Zn atoms and one S atom.

\textbf{b) Describe the basis.}\\
The basis consists of four Zn atoms and  a single S atom connected in a tetrahedral structure with four lattice points per unit cell. Starting with origin at the bottom left corner of the unit cell the basis's additional Zinc atoms are the front face, bottom face and left face with the S atom at the center.
\begin{figure}[h]
\includegraphics[width=7cm]{ZnS}
\caption{Basis of Zincblende showing tetrahedral structure}
\end{figure}

\begin{figure}[h]
\includegraphics[width=7cm]{ZnSUnit}
\caption{Expanded structure of unit cell}
\end{figure}
\begin{equation*}
Basis = \begin{cases} Zn_1  & @ (0,0,0) \\
 Zn_2  & @ (\frac{1}{2},0,\frac{1}{2})\\
 Zn_3 & @ (\frac{1}{2},\frac{1}{2},0) \\
 Zn_4 & @ (0,\frac{1}{2},\frac{1}{2})\\
 S & @ (\frac{1}{4}, \frac{1}{4}, \frac{1}{4}) \end{cases}
\end{equation*}

\vspace{5mm}

\textbf{c)  Given that a = 0.541 nm, calculate the nearest neighbor Zn-Zn, Zn-S, and S-S distances.}

\textbf{Zn - Zn }$\bullet$ The vector from a Zn atom to another is the face-centered vector with a magnitude of,
$$a \frac{\sqrt{2}}{2} \rightarrow (0.541nm)\frac{\sqrt{2}}{2} = 3.82\AA$$
The distance from a Zn atom to another Zn atom is $3.82\AA$\\

\textbf{Zn - S} $\bullet$ From a Zn atom located at the origin on a face to a S atom the vector would be a quarter of a in each direction,
\begin{equation*}
\text{Zn-S} = \vec{r} = \frac{1}{4}(\vec{a_x}+\vec{a_y}+\vec{a_z})
\end{equation*}
Where the magnitude is,
\begin{equation*}
\sqrt{(\frac{1}{4}a)^2}= \sqrt{\frac{3}{16}a^2} = \frac{\sqrt{3}}{4} a \rightarrow \frac{\sqrt{3}}{4}\ (.541 nm) = 2.34\AA
\end{equation*}
 The distance from a Zn atom to a S atom is 2.34\AA \\

\textbf{S - S} $ \bullet$ The distance from the center of an S atom to another S atom is the same as the Zn to Zn which is,  $a\frac{\sqrt{2}}{2} = (0.541nm)\frac{\sqrt{2}}{2} = 3.82\AA$

\section{Packing Fraction}
\textbf{Imagine a FCC lattice with a one atom basis. Determine the packing fraction of this crystal by modeling the atoms as hard spheres that touch their nearest neighbors.}\\

The packing efficiency is equivalent to the volume of the atoms in the unit cell per total volume of the unit cell multiplied by 100.
\begin{equation*}
\text{Packing Efficiency} = \frac{\text{Volume}_{\text{Atoms  in  unit cell}}}{\text{Volume}_{\text{unit cell}}} \times 100
\end{equation*}

\begin{figure}[h]
\includegraphics[width=10cm]{bcc}
\centering
\caption{Face Centered Cubic showing the truncated atoms in the unit cell}
\end{figure}

 To find the volume of the cell we need to find the lengths of the sides. The four corner atoms are themselves only a quarter inside the cell, with the face atom half in the cell, but along the full diameter. The bottom corner to top corner angle is $\frac{\pi}{4}$ as seen in figure 4.1, so we can then solve for the length of sides using the Pythagorean theorem or trigonometric relations,
 \begin{equation*}
 b = \frac{1}{2}r + 2r + \frac{1}{2}r = 4r
 \end{equation*}
\begin{equation*}
\begin{split}
b \cos{\theta} & = l \\
 4r \cos{\frac{\pi}{4}}& = l \\
 4r(\frac{\sqrt{2}}{2}) & = l \\
 2\sqrt{2} r & = l \\
\end{split}
\end{equation*}
The sides extend in to three dimensions making the volume of the unit cell $l^3$,
\begin{equation*}
l^3 = (2\sqrt{2}r)^3 = 16 \sqrt{2} r^3
\end{equation*}

We know that there is a total of four atoms contained in a FCC unit cell, two from the half atoms at the faces and 2 from the quartered corners. The volume of the sphere is $\frac{4 \pi r^3}{3}$, multiplying the volume by four atoms,
\begin{equation*}
Volume_{Atoms} = 4\ atoms\ (\frac{4 \pi r^3}{3}) = \frac{16 \pi r^3}{3}
\end{equation*}

When we divide by the total volume of the unit cell and multiply by 100 we have,
\begin{equation*}
\text{Packing Efficiency} = \frac{\text{Volume}_{\text{Atoms  in  unit cell}}}{\text{Volume}_{\text{unit cell}}} \times 100 = \frac{\frac{16 \pi r^3}{3} }{16\sqrt{2}\ r^3 } = \frac{\pi}{3\sqrt{2}} \times 100 = 74 \%
\end{equation*}

The \textbf{Face Centered Cubic} when confined to only one type of atom has a packing efficiency of approximately $74\%$.
%----------------------------------------------------------------------------------------

\end{document}


\documentclass{llncs}
\usepackage{url}
\usepackage{proof}
\usepackage{amssymb}
\usepackage{stmaryrd}
\usepackage{listings}
\usepackage{graphicx}
\usepackage{comment}

\newcommand{\dgm}[2][1.5]{
\begin{center}
\scalebox{#1}{
\includegraphics{diagrams/#2.pdf}
}
\end{center}
}

\newcommand{\todo}[1]{\textbf{TODO:} #1}
\newcommand{\jacques}[1]{\textsc{Jacques says:} #1}
\newcommand{\amr}[1]{\textsc{Amr says:} #1}

\hyphenation{a-reas}

%subcode-inline{bnf-inline} name Pi
%! swap+ = \mathit{swap}^+
%! swap* = \mathit{swap}^*
%! dagger =  ^{\dagger}
%! assocl+ = \mathit{assocl}^+
%! assocr+ = \mathit{assocr}^+
%! assocl* = \mathit{assocl}^*
%! assocr* = \mathit{assocr}^*
%! identr* = \mathit{uniti}
%! identl* = \mathit{unite}
%! dist = \mathit{distrib}
%! factor = \mathit{factor}
%! (o) = \fatsemi
%! (;) = \fatsemi
%! (*) = \times
%! (+) = +
%! foldB = fold_B
%! unfoldB = unfold_B
%! foldN = fold_N
%! unfoldN = unfold_N
%! trace+ = \mathit{trace}^{+}
%! trace* = \mathit{trace}^{\times}
%! :-* = \multimap
%! :-+ = \multimap^{+}
%! emptyset = \emptyset

%subcode-inline{bnf-inline} regex \{\{(((\}[^\}])|[^\}])*)\}\} name main include Pi
%! [^ = \ulcorner
%! ^] = \urcorner
%! [v = \llcorner
%! v] = \lrcorner
%! [[ = \llbracket
%! ]] = \rrbracket
%! ^^^ = ^{\dagger}
%! eta* = \eta
%! eps* = \epsilon
%! Union = \bigcup
%! in = \in
%! |-->* = \mapsto^{*}
%! |-->> = \mapsto_{\ggg}
%! |-->let = \mapsto_{let}
%! |--> = \mapsto
%! <--| = \mapsfrom
%! |- = \vdash
%! <=> = \Longleftrightarrow
%! <-> = \leftrightarrow
%! -> = \rightarrow
%! ~> = \leadsto
%! ::= = ::=
%! /= = \neq
%! vi = v_i
%! di = d_i
%! si = s_i
%! sj = s_j
%! F = \texttt{F}
%! T = \texttt{T}
%! forall = \forall
%! exists = \exists
%! empty = \emptyset
%! Sigma = \Sigma
%! eta = \eta
%! where = \textbf{where}
%! epsilon = \varepsilon
%! least = \phi
%! loop+ = loop_{+}
%! loop* = loop_{\times}
%! CatC = {\mathcal C}
%! CatA = {\mathcal A}
%! gamma = \gamma
%! {[ = \{
%! ]} = \}
%! elem = \in
%! dagger = ^\dagger
%! alpha = \alpha
%! beta = \beta
%! rho = \rho
%! @@ = \mu
%! @ = \,@\,
%! Pow = \mathcal{P}
%! Pi = \Pi
%! PiT = \Pi^{o}
%! PiEE = \Pi^{\eta\epsilon}_{+}
%! PiT = \Pi^{o}
%! PiTF = \Pi^{/}
%! bullet = \bullet
%! * = \times

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

\section{Higher-Order Reversible Functions}

To get from {{Pi}} types and combinators to a higher order setting, we hope to
add \emph{fractional types} to the language. Recall that to construct the
(positive) rational numbers {{Q}}, we can take pairs of positive integers
{{(m, n)}} and quotient by the equivalence relation that says {{(m1, n1)}} is
equal to {{(m2, n2)}} if {{m1 * n2 = m2 * n1}}. We can write this as an
inference rule as follows:

%subcode{proof} include main
%@ m1 * n2 = m2 * n1
%@@ (m1, n1) = (m2, n2)

\noindent
Adding in computational content, pi-style, suggests that we want something like:

%subcode{proof} include main
%@ c : A1 * B2 <-> B1 * A2
%@@ \hat{c} : (B1 |--> A1) <=> (B2 |--> A2)

\noindent
However, this does not quite work. To see why, recall that we want to treat the
pair {{(m, n)}} (written here as {{A |--> B}}) as an internalization of the
reversible function space. Thus, the type {{(B1 |--> A1) <=> (B2 |--> A2)}}
should be a transformation of a {{Pi}} combinator of type {{B1 <-> A1}} into a
{{Pi}} combinator of type {{B2 <-> A2}}. In order to do this, we need a trace
combinator (TODO: draw a diagram that demonstrates this) with the type

%subcode{proof} include main
%@ c : A * C <-> B * C
%@@ trace c : A <-> B

\noindent
This is an issue, since prior attempts at defining an adequate trace operator
failed: we got a language that could express all relations, but was no longer
reversible in a meaningful sense.

\subsection{Defining trace}

The trace operator had two problems: it allowed arbitrary function definition,
and it would not work in a language that contained the empty type {{0}}. The
former problem arose due to the logic programming-style interpretation we
assigned to trace: it would create *all* possible values of type {{C}}, then run
the given combinator on them and throw away the branches that did not return the
same {{C}} value.

We can solve this problem by parameterizing the trace operator with a value {{v
: C}}. This results in the following new typing rule for trace:

%subcode{proof} include main
%@ c : A * C <-> B * C
%@ v : C
%@@ trace_v c : A <-> B

\noindent
This ends up solving both problems: we give the trace a default value, and
prevent it from running on the empty type (since {{C}} now must be inhabited).
However, we still must assign dynamic semantics to trace. The na\"ive solution
of running {{c}} on the pair of the input value and {{v}} does not work, as this
is trivially irreversible---consider, for example, {{trace_v swap*}}, which
takes any value of the same type as {{v}} and returns {{v}}. Thus, the semantics
for {{trace}} must be slightly more complicated.

Recall that in {{Pi}}, combinators can be viewed as permutations on finite sets.
A combinator of type {{A * C <-> B * C}} can be seen as two separate
permutations: one on {{A}} (or equivalently, B, since they must have the same
size and thus the same denotation as a finite set) and one on {{C}}. If we
apply the fragment of the combinator that acts on {{C}} enough times, we will
eventually get out the same value that we started with\footnote{I want to prove
that this is actually true; having the extra {{A}} there seems like it might
complicate things.}. Thus, the semantics that we want is for {{(trace_v c) v'}}
to run {{c}} on the pair of {{v'}} and the last {{C}} value (starting with
{{v}}) until the {{C}} value matches {{v}}, at which point it returns the {{B}}
value returned by the combinator. In pseudocode:

\begin{verbatim}
TODO: figure out how to do the code environment in subcode

trace v c vin =
  v0 = v
  <vout, v'> = c <vin, v>

  while (v' != v)
    <vout, v'> := c <vin, v'>

  return vout
\end{verbatim}

A more concise formulation as an inference rule is as follows:

%subcode{proof} include main
%@ \exists v0 : C. c <v_{in}, v> = <v_{out}, v>
%@@ (trace_v c) v_{in} = v_{out}

This is easily reversible: {{(trace_v c)dagger = trace_v c
dagger}}\footnote{Also need to prove this.}.

\subsection{Higher-Order Combinators}

Now that we have the trace combinator defined, we can proceed with defining
higher-order combinators. Note that this parameterizes our definition of
{{\hat{c}}} by {{v}}, the value of type {{A}} (or equivalently, type {{B}}) used
in the trace:

%subcode{proof} include main
%@ c : A1 * B2 <-> B1 * A2
%@@ \hat{c}_v : (B1 |--> A1) <=> (B2 |--> A2)

\noindent
We want this to be a transformation on combinators, taking a combinator of type
{{B1 <-> A1}} to a combinator of type {{B2 <-> A2}}. This can be achieved by
letting the combinator {{\hat{c}_v c'}} be {{trace_v (c ; c' * id)}}, where {{c
: B1 <-> A1}} and {{v : A1}}.

From here, we want to define bifunctors {{/}}, {{*}}, and {{+}}, to make sure
that our definition of functions is truly higher-order.

\begin{itemize}

\item {{(B |--> A) / (D |--> C) = (A * D) |--> (B * C)}}, with {{{[c1]}_{v1} :
((B |--> A) <=> (D |--> C)) / {[c2]}_{v_2} : ((B' |--> A') <=> (D' |--> C')) =
{[shuffle ; c1 * c2 dagger ; shuffle]}_{<v1, c2 v2>} : B1 * A2 |--> A1 * B2 <=>
D1 * C2 |--> C1 * D2}}, where {{shuffle : (A * B) * (C * D) <-> (A * C) * (B *
D)}}.

\item {{(B |--> A) * (D |--> C) = (B * D) |--> (A * C)}}, with

\end{itemize}

\end{document}


% To compile single chapters put a % symbol in front of "\comment" and "}%end of comment" below
%    and take off the % symbol from "\end{document" at the bottom line. Undo before compiling
%    the complete thesis
%%Note: You can only use \section command, you are not allowed, per TTU Graduate School, use
%%\subsection command for ghigher level subheadings. At most level 2 subheadings are allowed.

\chapter{Estimating Normal}
\label{Estimating Normal Chapter}

\section[DASP Overview]{DASP Overview}

The processing of conducted URE for the purposes of device classification often requires intimate knowledge of the underlying signal characteristics which are determined by the physical implementation of the device's electronic circuitry.  DASP provides a method for aligning those signal dimensions inherent to the majority of electronic circuits without prior knowledge of the device's physical or conducted noise characteristics.  The following DASP algorithms are presented to align conducted URE dimensions of interest for device classification: Harmonically Aligned Signal Projection (HASP), Modulation Aligned Signal Projection (MASP), Cross-Modulation Aligned Signal Projection (CMASP), and Spectral Correlation Aligned Projections (SCAP).   In addition, two different methods for implementing the HASP algorithm will be presented, the fixed and decimating types.

An additional DASP algorithm widely used within the signal processing community is the Short-Time Fourier Transform (STFT, sometimes referred to as a spectrogram) \cite{Welch1967, Allen1977, Griffin1984}.  The STFT computes the Fast Fourier Transforms (FFT) over short time segments and \textit{aligns} the frequency bins of these FFT segments over time.  For consistency, I will use the term Frequency Aligned Signal Projection (FASP) for the STFT when used in the context of a DASP transform.

Techniques for processing and extracting features from DASP output arrays are discussed in Chapter \ref{DASP Feature Extraction Chapter}.  Chapter \ref{DASP Device Classification Chapter} provides results from the application of supervised and unsupervised machine learning techniques to DASP generated arrays and features to determine the applicability of dimensional alignments to classify devices based upon their URE.

\section[Harmonically Aligned Signal Projection (HASP)]{Harmonically Aligned Signal Projection}
\label{Harmonically Aligned Signal Projection}

The HASP algorithm provides a method for aligning a fundamental frequency with its respective harmonics in the column of a 2D matrix.   Harmonic content analysis can provide information related to a signal's stability, shape, and the underlying physical circuitry.  All non-pure sinusoidal periodic functions, such as square or triangle waves, are composed of harmonics with varying magnitudes and phases which define the shape of the time domain signal.  Aligning the harmonic structure of a signal is complicated by frequency uncertainties and offsets of the underlying fundamental.  For instance, the $40$th harmonic of a $50$Hz signal should be at $2000$Hz exactly, however a $0.5$Hz offset in the fundamental will result in the $40$th harmonic appearing at $2020$Hz.  Assuming a $1$Hz frequency resolution in the FFT, the harmonic will appear $20$ bins from where expected. To demonstrate, Figure \ref{fig:harm_aligned_example} shows the alignment of frequency bins through the $40$th harmonic for an exact $50$Hz signal and one with a $0.5$Hz offset.

\begin{figure}[tb]
	\includegraphics[width=\textwidth]{./dasp_algorithm_results/harmonic_aligned_example.eps}
	\centering
	\caption{Image showing the alignment of the harmonics of $50$Hz and $50.5$Hz square waves from a synthetic test signal generated from the addition of two square waves with $50$Hz and $50.5$Hz fundamental frequencies.  The frequency of each row is multiplied by the harmonic number, such that the center bin of the second row is $2 \times 50$Hz$ = 100$Hz.  The last row shows the $40$th harmonic of the $50$Hz signal at $2000$Hz and the $40$th harmonic of the $50.5$Hz signal at $2020$Hz.}
	\label{fig:harm_aligned_example}
\end{figure}

In addition to the misalignment of the $50.5$Hz fundamental with its respective harmonics, the harmonic structure of a square wave is evidenced by the alternating intensities of harmonics.  Every odd harmonic row (i.e. $3$rd, $5$th, $7$th, etc.) shows a harmonic signal, whereas the even harmonic rows ($2$nd, $4$th, $6$th, etc.) do not appear to show any harmonic structure.

\subsection[Harmonically Aligned Signal Projection - Fixed Type (HASP-F)]{Harmonically Aligned Signal Projection - Fixed Type (HASP-F)}
\label{Harmonically Aligned Signal Projection - Fixed Type}

Figure \ref{fig:harm_aligned_example} provides an example of the Fixed Type HASP algorithm (HASP-F), where a fixed number of frequency bins are selected around a specific fundamental and its respective harmonics and are aligned in to rows to form a 2-D image.  Assuming the fundamental frequency is known perfectly, the harmonics will align within a single column and provide a strong feature that could be used for classification purposes, as shown in the diagram of the HASP-F process in Figure \ref{fig:haspf_diagram_independent}.  In addition, modulation sidebands centered around the fundamental frequency and its harmonics also align within the HASP-F image, as shown in Figure \ref{fig:haspf_diagram_modulation}, and provide an additional dimension alignment and further insights in to the underlying signal structure.

\begin{figure}[tb]
	\includegraphics[width=\textwidth]{./misc_graphics/haspf_diagram_independent.jpg}
	\centering
	\caption{Diagram of the HASP-F process applied to two independent carriers at $10$Hz and $11$Hz with harmonic content, as represented by the red and blue blocks, respectively, and each frequency bin represents a $1$Hz resolution.  As a fixed number of bins are selected to align the $10$Hz harmonics, the $11$Hz harmonics slant across the image.}
	\label{fig:haspf_diagram_independent}
\end{figure}

\begin{figure}[tb]
	\includegraphics[width=\textwidth]{./misc_graphics/haspf_diagram_modulation.jpg}
	\centering
	\caption{Diagram of the HASP-F process applied to a carrier at $10$Hz with $2$Hz modulation sidebands, as represented by the red and green blocks, respectively.  As a fixed number of bins are selected to align the $10$Hz harmonics, the $2$Hz sidebands also align accordingly.}
	\label{fig:haspf_diagram_modulation}
\end{figure}

As demonstrated in Figure \ref{fig:haspf_diagram_independent}, the $10$Hz fundamental frequency aligns with its respective harmonics when a fixed number of frequency bins are selected around each harmonic, whereas, the $11$Hz independent carrier slants across the 2-D array because each progressive $11$Hz harmonic is $1$Hz, or $1$ frequency bin, further away from the corresponding $10$Hz harmonic.  Because modulation sidebands are always at fixed frequencies around both the fundamental and its harmonics, Figure \ref{fig:haspf_diagram_modulation} shows that modulation sidebands also align when performing fixed harmonic alignment.

As evidenced from Figure \ref{fig:haspf_diagram_independent} and Figure \ref{fig:haspf_diagram_modulation}, the fixed harmonic alignment requires a specific fundamental frequency to be known and selected for perfect vertical alignment to occur, otherwise any frequency offset will result in a slanted line in the resulting 2-D image.  The HASP-F algorithm is described in more detail in Algorithm \ref{alg:haspfalg}. The algorithm operates on a frequency domain input, $\hat{r}$, along with the input parameters $f_c$ and $B$, center frequency and bandwidth, respectively. The final 2D image will be a function of $f_c$, the center frequency of the image, and $B$, the bandwidth around this center frequency (or width of the image). The algorithm first derives the maximum harmonic, $K$, of $f_c$, based on the sampling frequency and the bandwidth around $f_c$.  The maximum harmonic, $K$, corresponds to the height of the 2-D HASP image, where each row of the image corresponds to the $k$th harmonic of $f_c$. The image is formed by iterating through all harmonics from the fundamental frequency to the maximum harmonic $K$.  For each harmonic, the magnitude of the input PSD, $\hat{r}$, is indexed by the corresponding $(k \times f_c) \pm B$ frequency bins, where $k$ is the harmonic number $\left[1,2,3 \ldots{} K \right]$.  Each row is comprised of a fixed number of frequency bins as set by the bandwidth, $B$, around the fundamental frequency.  The resulting HASP-F image dimensions are $K$ by the number of discrete frequency bins comprising the continuous frequencies $f_c \pm B$.

\begin{algorithm}
	\caption{Harmonic Aligned Signal Projection Algorithm - Fixed Type (HASP-F)} \label{alg:haspfalg}
	\scriptsize
	\centering
	\begin{algorithmic}[1]
		\Require~~
		\Statex $\hat{r}$ - Input Power Spectrum
		\Statex $f_c$ - Center Frequency
		\Statex $B$ - Bandwidth
		\Ensure~~
		\Statex $\bf{H}$ - HASP Output Array
		\Statex
		\For  {$k = 1, 2, \ldots K$}
		\State    $\mathbf{f}_k \gets \left[ \hat{r}(x_i) \right], (k \times f_c) - B \leq x_i \leq (k \times f_c) + B$
		\State		$k$-th row of $\mathbf{H} \gets \mathbf{f}_k$
		\EndFor
	\end{algorithmic}
\end{algorithm}

To illustrate the HASP-F algorithm images, conducted URE was collected from fluorescent lights in an RF shielded enclosure using a USRP N210 collection platform with an LFRX analog to digital processing board operating at $2$MS/s, as described in Chapter \ref{URE Data Collection Chapter}.  Figure \ref{fig:hasp_fft_example} is a plot of the FFT of the conducted URE collected from two fluorescent light fixtures, showing a fundamental frequency at approximately $45$kHz along with $21$ harmonics up to $1$MHz.  Figure \ref{fig:haspf_example} is the resulting image after application of the HASP-F algorithm to the frequency spectrum of the same time-domain capture.  The blurred vertical line and additional blurred slanting line represent the harmonic structure of the two different fluorescent lights with a very minute frequency difference between their respective crystal oscillators.  The center frequency, $f_c$, of the HASP-F image was selected to vertically align one of the oscillators resulting in a progressively increasing offset for each harmonic of the second crystal oscillator.

\begin{figure}[tp]
	\includegraphics[width=\textwidth]{./dasp_algorithm_results/hasp_fft_filenum_9601.eps}
	\centering
	\caption{Plot of an FFT derived from a time domain capture of conducted URE from a set of fluorescent lights.  The lights have a dominant URE signal with an approximate $45$kHz fundamental frequency.  The plot shows the slight separation of the URE from the two lights at higher frequencies due to the minor differences in the electronic ballasts used within the different lights.}
	\label{fig:hasp_fft_example}
\end{figure}

\begin{figure}[tp]
	\includegraphics[width=\textwidth]{./dasp_algorithm_results/hasp_f_wide_filenum_9601.eps}
	\centering
	\caption{HASP-F output of the frequency domain signal shown in \ref{fig:hasp_fft_example}.  The separation of harmonics at higher frequencies for the two electronic ballasts can be seen, however the two harmonic alignments are blurred which further demonstrate a low level modulation or instability in the fundamental frequency of the URE generators.}
	\label{fig:haspf_example}
\end{figure}

Analysis of Figure \ref{fig:haspf_example} shows that the two URE generators were roughly $7$kHz apart at the $18$th harmonic.  The frequency offset between the oscillators in the two fluorescent lights was then approximated to be $390$Hz by $7$kHz $/18 \approx 390$Hz.

\subsection[Harmonically Aligned Signal Projection - Decimation Type (HASP-D)]{Harmonically Aligned Signal Projection - Decimation Type (HASP-D)}
\label{Harmonically Aligned Signal Projection - Decimation Type}

Although the HASP-F algorithm provides a method for aligning harmonics and modulation sidebands, the fundamental frequency needs to be known precisely or the fundamental frequency harmonics will not vertically align and, in addition, the harmonic structures of independent carriers within the HASP-F image will form a slanted line.  The HASP Decimation type (HASP-D) algorithm was developed to vertically align the harmonics of independent signals regardless of the chosen center frequency or amount of frequency offset between the carriers.  The HASP-D algorithm is illustrated in Figure \ref{fig:haspd_diagram_independent}, where each progressive harmonic row is decimated to a fixed number of bins to realign independent harmonics.  Although independent carriers and their respective harmonics align vertically, modulation sidebands ``curve'' in toward the aligned harmonics as the bandwidth around the fundamental is collapsed in to fewer and fewer bins through the decimation process, as shown in Figure \ref{fig:haspd_diagram_modulation}.

\begin{figure}[tp]
	\includegraphics[width=\textwidth]{./misc_graphics/haspd_diagram_independent.jpg}
	\centering
	\caption{Diagram of the HASP-D process applied to a two independent carriers at $10$Hz and $11$Hz with harmonic content, as represented by the red and blue blocks, respectively.  As each harmonic row is added, the number of bins is decimated to equal the number of bins around the fundamental, thus aligning the independent carriers.}
	\label{fig:haspd_diagram_independent}
\end{figure}

\begin{figure}[tp]
	\includegraphics[width=\textwidth]{./misc_graphics/haspd_diagram_modulation.jpg}
	\centering
	\caption{Diagram of the HASP-D process applied to a carrier at $10$Hz with $2$Hz modulation sidebands, as represented by the red and green blocks, respectively.  As each harmonic row is added, the number of bins is decimated to equal the number of bins around the fundamental, thus causing the modulation sidebands to curve in to eventually add with the carrier frequency harmonics.}
	\label{fig:haspd_diagram_modulation}
\end{figure}

As shown in Figure \ref{fig:haspd_diagram_independent}, the number of bins selected around the fundamental frequency determines the resulting number of bins for each harmonic row.   With each progressive harmonic, the number of indexed bins expands by the harmonic number and is then subsequently decimated by the corresponding harmonic number resulting in a vector of bins corresponding to the number of bins in the fundamental frequency row.   By collapsing each row through the decimation process, independent carriers align with their respective harmonics, whereas modulation sidebands that are a fixed number of bins around a given harmonic are ``squeezed'' in toward their respective carrier frequency.

The HASP-D algorithm is described in more detail in Algorithm \ref{alg:haspdalg} and only differs slightly from the HASP-F algorithm. Identical to the HASP-F algorithm, the HASP-D algorithm operates on a frequency domain input, $\hat{r}$, and requires center frequency and bandwidth input parameters, $f_c$ and $B$, respectively.  The algorithm derives the maximum harmonic, $K$, of $f_c$, based on the sampling frequency and the bandwidth around $f_c$. The maximum harmonic, $K$, corresponds to the height of the 2-D HASP image, where each row of the image corresponds to the $k$th harmonic of $f_c$. The image itself is created by iterating through all harmonics from the fundamental frequency to the maximum harmonic $K$.  For each harmonic, the $\hat{r}$ vector is indexed by the corresponding $k(f_c \pm B)$ frequency bins, where $k$ is the harmonic number $\left[1,2,3 \ldots{} K \right]$.  Because of the expansion of each harmonic vector at increasing $k$, each vector is downsampled by a factor of $k$ to align harmonically related bins in the HASP-D image.  The resulting HASP-D image dimensions are $K$ by the number of discrete frequency bins comprising the continuous frequencies $f_c \pm B$, which are the same dimensions as the HASP-F image.

\begin{algorithm}
	\caption{Harmonic Aligned Signal Projection Algorithm - Decimation Type (HASP-D)} \label{alg:haspdalg}
	\scriptsize
	\begin{algorithmic}[1]
		\Require~~
		\Statex $\hat{r}$ - Input Power Spectrum
		\Statex $f_c$ - Center Frequency
		\Statex $B$ - Bandwidth
		\Ensure~~
		\Statex $\bf{H}$ - HASP-D Output Array
		\Statex
		\For  {$k = 1, 2, \ldots K$}
		\State    $\mathbf{f}_k \gets \left[ \hat{r}(x_i) \right], k(f_c - B) \leq x_i \leq k(f_c + B)$
		\State		$k$-th row of $\mathbf{H} \gets$ DOWNSAMPLE $\mathbf{f}_k$ by factor of $k$
		\EndFor
	\end{algorithmic}
\end{algorithm}

To further analyze the URE signal processed by the HASP-F algorithm in Figure \ref{fig:haspf_example}, the HASP-D algorithm was applied to the same frequency domain signal, as shown in Figure \ref{fig:haspd_example}.  The HASP-D image demonstrates the alignment of independent fundamental frequencies and their respective harmonics.  The two vertical lines centered around $45$kHz represent the two independent oscillator signals previously identified in Figure \ref{fig:hasp_fft_example} and Figure \ref{fig:haspf_example}.  The frequency offset between the two oscillators was found to be approximately $300$Hz through visual inspection which roughly corresponds to the calculations derived from the HASP-F image in Figure \ref{fig:haspf_example}.

\begin{figure}[tp]
	\includegraphics[width=\textwidth]{./dasp_algorithm_results/hasp_d_filenum_9601.eps}
	\centering
	\caption{HASP-D output of the frequency domain signal shown in Figure \ref{fig:hasp_fft_example}.  The HASP-D algorithm decimates each progressive harmonic row to realign harmonic frequency bins to the fundamental frequency row (i.e. row number $1$).  The two independent URE generator signals are now fully parallel because their respective harmonics are aligned across each harmonic row through the decimation process.}
	\label{fig:haspd_example}
\end{figure}

The curved lines centered on the independent carriers in Figure \ref{fig:haspd_example} are artifacts of the preceding and following carrier harmonics due to the expansion and subsequent decimation of each increasing harmonic row.  This was verified by calculating the frequency difference between the curved lines and the harmonic frequency at the $20$th harmonic row, $20 \times (44.8 - 42.6)$kHz $\approx 45$kHz, which corresponds to the fundamental frequency as well as the frequency spacing between all harmonics.

As previously noted in Figure \ref{fig:haspf_example}, the HASP-F image shows a blurring of the two independent carriers which were examined further by applying the HASP-D algorithm with a smaller bandwidth as shown in Figure \ref{fig:haspd_example_zoom}. The blurring, or instability, in Figure \ref{fig:haspf_example} appeared to be the result of a moderate oscillator instability and a low level modulation of the URE generator carrier frequency, as illustrated by the width of the center aligned harmonics and the curved modulation lines, respectively.

\begin{figure}[tp]
	\includegraphics[width=\textwidth]{./dasp_algorithm_results/hasp_d_zoom_filenum_9601.eps}
	\centering
	\caption{Zoomed in view of the HASP-D output of the frequency domain signal shown in Figure \ref{fig:hasp_fft_example}.  The blurring was found to be approximately $4$Hz wide resulting in roughly $\pm 45$ppm carrier stability.  The curved lines centered around the carrier frequency were due to $120$Hz modulations of the carrier frequency.}
	\label{fig:haspd_example_zoom}
\end{figure}

The oscillator instability was found to be approximately $4$Hz wide, corresponding to an approximate clock stability of $\pm 45$ppm, which is consistent with typical inexpensive crystal oscillators.  The sideband peak frequency offsets were found to be approximately $120$Hz, or the second harmonic of the $60$Hz power line frequency, by examining the $3$rd harmonic of the carrier frequency, $3 \times (4.4872\times10^4 - 4.4832\times10^4)$Hz$ = 120$Hz, indicating a $120$Hz power line harmonic frequency amplitude modulation of the fluorescent light oscillators.

An additional HASP alignment method, HASP Interpolating type (HASP-I), is described further in Appendix A.  The HASP-I algorithm operates in much the same way as the HASP-D algorithm, except that it utilizes an interpolation method for realigning independent carrier frequency harmonics as opposed to the decimation process used with HASP-D.  The HASP-I algorithm was not evaluated further because of processing time constraints and redundancy with the HASP-D algorithm, but is presented for completeness.

\section[Modulation Aligned Signal Projection (MASP)]{Modulation Aligned Signal Projection (MASP)}
\label{Modulation Aligned Signal Projection}

The MASP algorithm provides a method for characterizing amplitude modulations (AM) in URE time domain signal captures without assumptions of the underlying carrier or modulation frequencies.  The MASP algorithm operates on a time domain signal by first applying a STFT and then performing an FFT across all time slices for each frequency column.  The resulting 2-D structure is a frequency-by-frequency matrix where the vertical axis represents the modulation frequencies and the horizontal axis represents the carrier frequencies. Whereas HASP aligns harmonically related frequencies within a URE signal, MASP aligns modulation and carrier frequencies by calculating and highlighting their respective intersections.

MASP is described in depth in Algorithm \ref{alg:maspalg}. MASP operates on a time domain input signal, $r$, by first performing a STFT with no overlap on the captured URE signal.  The sample rate, collection time, and maximum modulation frequency of interest, $f_m$, dictate the length of the time slices within the STFT and hence the pixel resolution of the resulting MASP output image.  As $f_m$ is increased, shorter time slices are utilized by the STFT resulting in a decreased number of frequency bins in the carrier frequency, $f_c$, columns.  The total collection time of $r$ determines the resolution of the $f_m$ rows and the sample rate of $r$ determines the maximum carrier frequency.  Once the magnitude of the STFT transform, $\bf{S}$, is calculated, the algorithm iterates through all carrier frequency bins, $k$, and performs an FFT on each frequency column.  The magnitude of the frequency column FFTs then form a column within the MASP image.  The resulting MASP image is a frequency-by-frequency matrix where rows represent modulation frequencies and columns represent carrier frequencies.  A peak in the MASP image provides a visual indication of a significant amplitude modulation at a specific modulation and carrier frequency.

\begin{algorithm}
	\caption{Modulation Aligned Signal Projection Algorithm (MASP)} \label{alg:maspalg}
	\scriptsize
	\begin{algorithmic}[1]
		\Require~~
		\Statex $r$ - Input Time Domain Signal
		\Statex $N$ - Number of Time Slices for Short-Time Fourier Transform
		\Ensure~~
		\Statex $\mathbf{M}$ - MASP Output Array
		\Statex
		\State $\mathbf{S} \gets $ Short-Time Fourier Transform of $r$ over $N$ non-overlapping time slices
		\State $\mathbf{S} \gets \left|\bf{S}\right|$
		\ForAll  {$k$}
		\State      $\mathbf{S}_k \gets k$-th column of $\mathbf{S}$
		\State		$\mathbf{T}_k \gets $ FAST FOURIER TRANSFORM of $\mathbf{S}_k$
		\State		$k$-th column of $\mathbf{M} \gets \left|\mathbf{T}_k\right|$
		\EndFor
	\end{algorithmic}
\end{algorithm}

Figure \ref{fig:masp_example} shows the MASP output image as applied to the conducted URE spectrum demonstrated in Figures \ref{fig:hasp_fft_example}, \ref{fig:haspf_example}, and \ref{fig:haspd_example}. The MASP image shows that the device under test has a strong harmonic structure with an approximately $45$kHz fundamental frequency; however, unlike with HASP, MASP was also able to extract a strong, but low frequency, modulating frequency around $20$Hz along with the harmonics of $20$Hz. Such a structure is often buried in and confounded by the stronger fundamental frequencies and their respective harmonics which further supports the utilization of dimensional alignments for feature extraction.  Additionally, the amplitude modulation index can be very low with the modulation sidebands around the carrier frequency near the noise floor, further obfuscating amplitude modulation features.

\begin{figure}[tp]
	\includegraphics[width=\textwidth]{./dasp_algorithm_results/masp_low_filenum_9601.eps}
	\centering
	\caption{MASP output image derived from the time domain capture of the URE in Figure \ref{fig:hasp_fft_example}.  The vertical lines represent potential carriers across the entire capture spectrum and the horizontal lines represent amplitude modulations across the entire spectrum.  The intersection of these lines represents the location of a modulated carrier.  In addition to the strong frequency peaks identified in Figure \ref{fig:hasp_fft_example}, modulations frequencies can be found at $20$Hz and its respective harmonics.}
	\label{fig:masp_example}
\end{figure}

In addition to the modulations occurring at $20$Hz and its harmonics, several unrelated modulation and carrier frequency intersections were identified in Figure \ref{fig:masp_example}.  For instance, significant peaks were identified at the intersection of the fourth harmonic of the dominant carrier frequency, approximately $180$kHz, and modulations frequencies of $15$Hz and $25$Hz, indicating a strong amplitude modulation which was not identified during analysis of the HASP-F and HASP-D images.

\section[Spectral Correlation Aligned Projection (SCAP)]{Spectral Correlation Aligned Projection (SCAP)}
\label{Spectral Correlation Aligned Projection}

The SCAP algorithm provides a method for aligning frequency spacings between peaks in the frequency domain.  Fixed frequency spacings can result from frequency harmonics, frequency mixing, or frequency and amplitude modulations; and furthermore, can occur at multiple unrelated locations within the conducted URE spectrum. The SCAP algorithm provides two outputs, $\bf{S}_H$ and $\bf{S}_F$, where $\bf{S}_F$ is the autocovariance, $\bf{C}_{XX}$, of $\hat{r}$ and $\bf{S}_H$ results from the application of the HASP-D algorithm to $\bf{S}_F$, as described in Algorithm \ref{alg:scapalg}.

\begin{algorithm}
	\caption{Spectral Correlation Aligned Projection Algorithm} \label{alg:scapalg}
	\scriptsize
	\begin{algorithmic}[1]
		\Require~~
		\Statex $\hat{r}$ - Input Power Spectrum
		\Ensure~~
		\Statex $\bf{S}_H$ - SCAP Output
		\Statex $\bf{S}_F$ - Spectral Autocovariance Vector
		\Statex
		\State $\bf{S}_F \gets \bf{C}_{XX}(\hat{r})$
		\State $\bf{S}_H \gets $ HASP-D of $\bf{S}_F$
	\end{algorithmic}
\end{algorithm}

Figure \ref{fig:scap_xcov_example} shows the autocovariance of $\hat{r}$, $\bf{S}_F$, of the conducted URE spectrum illustrated in Figure \ref{fig:hasp_fft_example}, with $\bf{S}_H$ shown in Figure \ref{fig:scap_hasp_example} after application of the HASP-D algorithm.  The HASP-D algorithm was configured with $f_c = 1000$Hz and $B = 1900$Hz and a maximum harmonic of $K = 10$.  In addition, $\bf{S}_F$ was scaled such that the maximum peak value equals one and the frequency correlation axis was limited from $100$kHz to $70$Hz with a reverse semi-log scale.

\begin{figure}[tp]
	\includegraphics[width=\textwidth]{./dasp_algorithm_results/scap_filenum_12001.eps}
	\centering
	\caption{Plot of the autocovariance of the FFT shown in Figure \ref{fig:hasp_fft_example}.  The peaks within the plot represent the correlation of equally spaced frequency peaks within the FFT, with a dominant correlation at $16.67$kHz, which corresponds to roughly $120$ FFT bins.}
	\label{fig:scap_xcov_example}
\end{figure}

\begin{figure}[tp]
	\includegraphics[width=\textwidth]{./dasp_algorithm_results/scap_hasp_filenum_12001.eps}
	\centering
	\caption{SCAP image, $\bf{S}_H$, generated by applying the HASP-D algorithm to the autocovariance vector, $\bf{S}_F$, shown in Figure \ref{fig:scap_xcov_example}.}
	\label{fig:scap_hasp_example}
\end{figure}

Figure \ref{fig:scap_xcov_example} shows a significant number of correlation peaks resulting from the equal spacing of peaks within the frequency spectrum of $\hat{r}$.  The dominant peak occurs at $16$kHz along with corresponding sub-correlations at integer divisions of $16$kHz.  An additional unrelated correlation ``hump'' occurs between $200$Hz and $400$Hz.  Although the previously identified oscillator frequencies from Figure \ref{fig:hasp_fft_example} resulted in significant peaks in the FFT spectrum, the correlation spectrum did not return a strong response, although still identifiable at a much lower level.   This was likely due to the low frequency instability in the signals, identified in Figure \ref{fig:haspd_example_zoom}, as well as the close proximity of the two oscillators resulting in a diminished correlation value.

\section[Cross-Modulation Aligned Signal Projection (CMASP)]{Cross-Modulation Aligned Signal Projection (CMASP)}
\label{Cross-Modulation Aligned Signal Projection}

In addition to carrier frequencies having significant harmonic content, modulation sidebands around a given carrier frequency and its respective harmonics can also have significant harmonic content.  The CMASP algorithm was developed to find the intersection of carrier and modulation frequencies, much like the MASP algorithm, but also provide a mechanism for highlighting the harmonic content of the modulation sidebands, such as the harmonics of $20$Hz previously noted in Figure \ref{fig:masp_example}.  Figure \ref{fig:cmasp_diagram} provides a graphical depiction of the CMASP process as applied to a nominal FFT spectrum.   The CMASP process sweeps across all potential carrier frequencies within a band of interest and provides a sum of the power spectrum of a given carrier frequency and its harmonics, much like the HASP process, but also includes a summation of modulation frequency sidebands and related harmonics.  The CMASP process results in a modulation frequency by carrier frequency image with an ``X'' and associated peaks corresponding to the intersection of a carrier and modulation frequency and their respective harmonics.

\begin{figure}[tp]
	\includegraphics[width=\textwidth]{./misc_graphics/cmasp_diagram.jpg}
	\centering
	\caption{Diagram of the CMASP process as applied to a signal with a carrier frequency ($f_c$) and modulation frequency sidebands ($f_m$), where $f_c$ and $f_m$ both have harmonic content.  The CMASP process sweeps across $f_c$ and aligns the harmonics of the modulation frequency to highlight the intersection of carrier and modulation frequencies.}
	\label{fig:cmasp_diagram}
\end{figure}

The CMASP process is described further in Algorithm \ref{alg:cmaspalg}.  The CMASP process operates on an input signal's power spectral density, $\hat{r}$, and requires carrier center frequency ($f_c$), carrier frequency bandwidth ($B_c$), modulation center frequency ($f_m$), and modulation frequency bandwidth ($B_m$) inputs.  The algorithm initially performs the HASP-F algorithm on $\hat{r}$ and sums all columns in to a 1-D vector.  The 1-D vector is then folded around the center frequency bin and summed to add the lower and upper sideband components.  The HASP-D algorithm is applied to the folded and summed vector to align the harmonics of the modulation frequency.  Finally, the HASP-D output is summed along all columns with the resulting vector forming a single column in the CMASP image.  The process is repeated for all frequency bins within the carrier frequency bandwidth to form a modulation frequency by carrier frequency matrix, with dimensions of $f_m \pm B_m$ by $f_c \pm B_c$, respectively.

\begin{algorithm}
	\caption{Cross-Modulation Aligned Projection Algorithm} \label{alg:cmaspalg}
	\scriptsize
	\centering
	\begin{algorithmic}[1]
		\Require~~
		\Statex $\hat{r}$ - Input Power Spectrum
		\Statex $f_c$ - Carrier Center Frequency
		\Statex $f_m$ - Modulation Center Frequency
		\Statex $B_c$ - Carrier Frequency Bandwidth
		\Statex $B_m$ - Modulation Frequency Bandwidth
		\Ensure~~
		\Statex $\bf{C}$ - CMASP Output Array
		\Statex
		\ForAll  {$i$, where $(k \times f_c) - B/2 \leq f_i \leq (k \times f_c) + B/2$}
		\State    $\bf{C_m} \gets $ HASP-F($\hat{r}$, $f_i$, $B_c$)
		\State 		$\bf{C_m} \gets $ COLUMN SUM of $\bf{C_m}$
		\State 		$\bf{C_m} \gets \bf{C_m}$(END$/2$ to $1$) + $\bf{C_m}$(END$/2$ to END)
		\State		$\bf{C_m} \gets $ COLUMN SUM of HASP-D($\bf{C_m}$, $f_m$, $B_m$)
		\State		$i$-th column of $\bf{C} \gets \bf{C_m}$
		\EndFor
	\end{algorithmic}
\end{algorithm}

To illustrate the CMASP output, a synthetic signal was generated by amplitude modulating a square wave at a frequency of $12$kHz with a square wave at $1100$Hz.  The resulting power spectral density of the synthetic signal consisted of multiple harmonics of the $12$kHz carrier signal along with multiple upper and lower sidebands comprised of the $1100$Hz modulating frequency and its corresponding harmonics.   Figure \ref{fig:cmasp_synth_example} shows the output of the CMASP process as applied to the synthetic test signal.   The large ``X'' within the image is the result of the intersection of the carrier frequency and the fundamental modulation frequency.  The additional radials at progressively smaller angles are the result of sidebands from the modulation frequency harmonics.  Furthermore, the left-right mirrored radials are due to the presence of both upper and lower modulation sidebands.  Analysis of Figure \ref{fig:cmasp_synth_example} therefore shows a $12$kHz carrier signal being modulated by a $1100$Hz signal with both upper and lower sidebands and multiple harmonics.

\begin{figure}[tp]
	\includegraphics[width=\textwidth]{./dasp_algorithm_results/cmasp_synth.eps}
	\centering
	\caption{CMASP output image resulting from application of the CMASP algorithm to synthetic data.  The synthetic data was generated by multiplying a $1100$Hz square wave with a $12$kHz square wave carrier frequency.  The ``X'' represents the intersection of a modulation frequency at $1100$Hz and its harmonics with a carrier frequency at $12$kHz.}
	\label{fig:cmasp_synth_example}
\end{figure}

Figure \ref{fig:cmasp_example} shows the CMASP algorithm applied to the conducted URE signal from the fluorescent lights as shown in Figure \ref{fig:hasp_fft_example}.  Several radials and intersections are identifiable, with the most prominent occurring at $120$Hz and its respective harmonics.  Single lines appearing with no obvious intersection with a mirrored radial are likely due to unmodulated frequency peaks within the spectrum.

\begin{figure}[tp]
	\includegraphics[width=\textwidth]{./dasp_algorithm_results/cmasp_filenum_9601.eps}
	\centering
	\caption{CMASP output image resulting from the fluorescent light conducted URE spectrum shown in Figure \ref{fig:hasp_fft_example}.}
	\label{fig:cmasp_example}
\end{figure}

\section[Frequency Aligned Signal Projection (FASP)]{Frequency Aligned Signal Projection (FASP)}
\label{Frequency Aligned Signal Projection}

The FASP algorithm performs FFTs on short time segments of a time domain signal and aligns the frequency bins of each processed time slice, as detailed in Algorithm \ref{alg:faspalg}.  The FASP algorithm operates on a time domain input signal, $r$, and only requires the number of times slices, $N$, as an input parameter.  Given a fixed length capture, with capture time, $T_c$, and sample rate, $F_s$, the FASP algorithm returns a time by frequency image with dimensions of $\frac{T_c}{N}$ seconds by $\frac{F_s}{2}$Hz.

\begin{algorithm}
	\caption{Frequency Aligned Signal Projection Algorithm} \label{alg:faspalg}
	\scriptsize
	\begin{algorithmic}[1]
		\Require~~
		\Statex $r$ - Input Time Domain Signal
		\Statex $N$ - Number of Time Slices of a Fixed Length Capture File
		\Ensure~~
		\Statex $\mathbf{F}$ - FASP Output Array
		\Statex
		\For  {$n = 1, 2, \ldots N$}
		\State Row $n$ of $\mathbf{F} \gets $ FFT of $r(x_i)$ where, $(n - 1)T_c/N < x_i < nT_c/N$
		\EndFor
		\State $\mathbf{F} \gets \left|\bf{F}\right|$
	\end{algorithmic}
\end{algorithm}

Figure \ref{fig:fasp_example} shows the FASP algorithm applied to same time domain URE capture represented in Figure \ref{fig:hasp_fft_example}.  As previously noted, the separation of the two fluorescent light $45$kHz oscillator harmonics can be seen at the higher frequencies.   Although not as clear, a slight ``wobble'' on the fundamental frequency and its harmonics can be seen over time resulting from the low frequency modulations identified in Figure \ref{fig:haspd_example_zoom} and Figure \ref{fig:masp_example}.  Given a capture time of one second and a sample rate of $2$MS/s, each row represents a $0.2$ms time slice with the frequency axis spanning $0$ to $1$MHz.

\begin{figure}[tp]
	\includegraphics[width=\textwidth]{./dasp_algorithm_results/fasp_filenum_9601.eps}
	\centering
	\caption{FASP algorithm applied to the conducted URE signal illustrated in Figure \ref{fig:hasp_fft_example}.  The $45$kHz carriers and respective harmonics are clearly seen throughout the spectrum on all time slices, with the spacing between their harmonics progressively increasing.}
	\label{fig:fasp_example}
\end{figure}


\section{\module{whrandom} ---
         Pseudo-random number generator}

\declaremodule{standard}{whrandom}
\modulesynopsis{Floating point pseudo-random number generator.}

This module implements a Wichmann-Hill pseudo-random number generator
class that is also named \class{whrandom}.  Instances of the
\class{whrandom} class conform to the Random Number Generator
interface described in section \ref{rng-objects}.  They also offer the
following method, specific to the Wichmann-Hill algorithm:

\begin{methoddesc}[whrandom]{seed}{\optional{x, y, z}}
  Initializes the random number generator from the integers \var{x},
  \var{y} and \var{z}.  When the module is first imported, the random
  number is initialized using values derived from the current time.
  If \var{x}, \var{y}, and \var{z} are either omitted or \code{0}, the
  seed will be computed from the current system time.  If one or two
  of the parameters are \code{0}, but not all three, the zero values
  are replaced by ones.  This causes some apparently different seeds
  to be equal, with the corresponding result on the pseudo-random
  series produced by the generator.
\end{methoddesc}

\begin{funcdesc}{choice}{seq}
Chooses a random element from the non-empty sequence \var{seq} and returns it.
\end{funcdesc}

\begin{funcdesc}{randint}{a, b}
Returns a random integer \var{N} such that \code{\var{a}<=\var{N}<=\var{b}}.
\end{funcdesc}

\begin{funcdesc}{random}{}
Returns the next random floating point number in the range [0.0 ... 1.0).
\end{funcdesc}

\begin{funcdesc}{seed}{x, y, z}
Initializes the random number generator from the integers \var{x},
\var{y} and \var{z}.  When the module is first imported, the random
number is initialized using values derived from the current time.
\end{funcdesc}

\begin{funcdesc}{uniform}{a, b}
Returns a random real number \var{N} such that \code{\var{a}<=\var{N}<\var{b}}.
\end{funcdesc}

When imported, the \module{whrandom} module also creates an instance of
the \class{whrandom} class, and makes the methods of that instance
available at the module level.  Therefore one can write either
\code{N = whrandom.random()} or:

\begin{verbatim}
generator = whrandom.whrandom()
N = generator.random()
\end{verbatim}

Note that using separate instances of the generator leads to
independent sequences of pseudo-random numbers.

\begin{seealso}
  \seemodule{random}{Generators for various random distributions and
                     documentation for the Random Number Generator
                     interface.}
  \seetext{Wichmann, B. A. \& Hill, I. D., ``Algorithm AS 183:
           An efficient and portable pseudo-random number generator'',
           \citetitle{Applied Statistics} 31 (1982) 188-190.}
\end{seealso}


% Abstract ====================================================================
\pdfbookmark[1]{Abstract}{Abstract}
\chapter*{Abstract}

Several real-world classification problems are example-dependent cost-sensitive in nature, where the
costs due to misclassification vary between examples and not only within classes. However, standard
classification methods do not take these costs into account, and assume a constant cost of
misclassification errors. This approach is not realistic in many real-world applications. For
example in credit card fraud detection, failing to detect a fraudulent transaction may have an
economical impact from a few to thousands of Euros, depending on the particular transaction and card
holder. In churn modeling, a model is used for predicting which customers are more likely to
abandon a service provider. In this context, failing to identify a   profitable or unprofitable
churner has a significant different economic   result. Similarly, in direct marketing, wrongly
predicting that a customer   will not accept an offer when in fact he will, may have different
financial impact, as not all   customers generate the same profit. Lastly, in credit scoring,
accepting   loans from bad customers does not have the same economical loss, since customers have
different   credit lines, therefore, different profit.

Accordingly, the goal of this thesis is to provide an in-depth analysis of example-dependent
cost-sensitive classification. In the first part, we give the  general concepts of classification
and cost-sensitive classification, and we present the cost-sensitive problem. The second part of
this manuscript is dedicated to explain the particularities of the four real-world classification
problems that are the focus of this thesis, in particular, credit card fraud detection, credit
scoring, churn modeling and direct marketing. In general, we show why each of these applications is
example-dependent cost-sensitive, and we elaborate a framework for the analysis of each problem.

Afterwards, in the third part of this work, we propose four example-dependent cost-sensitive
methods. First, we present the Bayes minimum risk algorithm which consist in quantifying tradeoffs
between various decisions using probabilities and the costs that accompany such decisions. Then, we
introduce the cost-sensitive logistic regression technique, this algorithm is based in a new
logistic regression cost function, one that takes into account the real costs due to
misclassification and correct classification. Subsequently, we show the cost-sensitive decision
trees algorithm which is based on incorporating the different example-dependent costs into a new
cost-based impurity measure and a new cost-based pruning criteria. Lastly, we introduce the
framework of ensembles of example-dependent cost-sensitive decision-trees, by training
example-dependent cost-sensitive decision trees using four different random inducer methods and then
blending them using three different combination approaches. Moreover, we present the library
\mbox{\textit{CostCla}} developed as part of the thesis. This library is an open-source
implementation of all the algorithms covered in this manuscript.

Finally, the experimental results show the importance of using the real example-dependent financial
costs associated with real-world applications. We found that there are significant differences
in the results when evaluating a model using a traditional cost-insensitive measure such as the
accuracy or F1Score, than when using the financial savings. Moreover, the results show that the
proposed algorithms have better results for all databases, in the sense of higher savings.