\chapter{State of the art and Problem definition}
\label{cha:state_of_the_art_and_problem_definition}

\graphicspath{{Chapter1-State-of-the-art/Figs/}}

\section{State of the art}
\label{sec:state_of_the_art}

\subsection{Inverse Kinematics}
\label{sub:inverse_kinematics}

Posture generation can be viewed as, and is sometimes called, Generalized Inverse Kinematics.
The Inverse Kinematics~(IK) problem consists of finding the joint configuration for an articulated multibody system in order to complete a given set of tasks that are defined in the operational space.
For example, by solving repeatedly the IK for a desired motion given in the Cartesian space, we can generate motions in the joint space.
By definition, such a process is purely kinematic. But other constraints need dynamic or other physics-related knowledge such as the inertia or contact friction parameters.
The IK problem has been widely studied and used in the fields of robotics, computer graphics, games, and animation.
In few particular cases, for example with robotic arms that have less than 7 degrees of freedom, a closed-form solution can be found.
For example, in~\cite{asfour2003human} the redundancy in a robot arm is exploited to derive a closed-form formula for its IK.
However, for most complex cases, iterative methods are required.

%Generating desired initial, intermediary or finale posture configurations requires defining static task goals (e.g.\ reach a target point in 6D) to be done under intrinsic constraints such as joint limits, torque limits, avoiding non-desired self-collisions\ldots and perceptual or extrinsic ones such as keeping an object in the embedded camera field-of-view, avoiding non-desired collisions with surrounding objects, etc.

Many approaches to solve IK problems use pseudo-inverse or Gauss-Newton method with the Jacobian, Jacobian transpose, damped least squares with or without singular values decomposition or selectively damped least squares~\cite{balestrino1984robust, tolani2000real, baillieul1985kinematic, wampler1986manipulator, nakamura1986inverse, buss2005selectively}.
These approaches are all computationally costly and suffer from joint-space and workspace singularities~\cite{aristidou2009inverse}.

In~\cite{pechev2008inverse} an alternative method based on a control approach is proposed, where no matrix manipulation is required, this approach is as fast as the damped least squares, but outperforms them in terms of handling singularities.
The closed-loop inverse kinematics scheme presented in~\cite{siciliano1990closed} is a well-known approach to solve IK problems in a control context, and uses a second order tracking scheme to guarantee satisfactory tracking performance.

Handling conflicting constraints in IK is a challenging problem to which \cite{baerlocher2004inverse} and \cite{sentis2005synthesis} propose solutions by enforcing a number of priority levels among constraints in their resolution schemes to generate whole-body control of complex articulated figures.

Other stochastic approaches such as sequential Monte Carlo \cite{courty2008inverse} and a particle filtering \cite{hecker2008real} have the advantage that they only use direct calculations and never require matrix inversions.
They perform particularly well compared to other methods when solving problems with high numbers of degrees of freedom.

In~\cite{AristidouFABRIK, Aristidou:2016_ExtFABRIK} the forward and backward reaching inverse kinematics (FABRIK) resolution method is introduced.
This method is based on a geometric iterative heuristic approach, where the bodies of a robot are moved iteratively and separately to reach a target with the end effector while maintaining the integrity of the robot's structure.
This approach is simple to implement, does not suffer from singularities and requires fewer iterations than most other IK methods.
But it cannot easily be extended to take non-geometric constraints into account.

Finding an acceptable approximate (or a probabilistic) solution to an IK problem without any regard for its stability is a common approach in the field of randomized path planning.
\cite{cortes2002random} proposes a method to generate many random configurations for closed loop systems with an increased probability that they are kinematically valid in terms of closure constraints.
These configurations are then used in the construction of a Probabilistic RoadMap.
In~\cite{lavalle1999probabilistic}, a similar approach is presented, where the closure constraint is enforced for each configuration by additional treatment.

\subsection{Generalized Inverse Kinematics}
\label{sub:generalized_inverse_kinematics}

The term Generalized Inverse Kinematics refers to a problem similar to the Inverse Kinematics in the sense that it searches for a joint configuration for an articulated figure to achieve goal tasks, but needs to do so under various constraints, like ensuring the equilibrium stability of the structure, respecting its torque limits, avoiding collisions with the environment or with itself, etc.

All the previously mentioned approaches are used to find a robot configuration solution to the geometric IK problem.
To solve a Generalized Inverse Kinematics problem with stability constraints, one can first solve the associated IK with one of the methods presented above, and then test the stability of the IK solution, using methods such as the ones presented in~\cite{bretl:itro:2008} or~\cite{rimon2008general} that determine if the configuration can be statically stable.
This gives a rejection criterion for a proposed solution.
If the solution can be stable, the optimal contact forces can be computed using the method proposed in~\cite{boyd2007fast} (which in turn allows computing the joint torques).
Otherwise, another IK solution is generated and tested.
And the process is repeated until a satisfactory solution is found.
This type of sequential approach for the resolution of a posture generation problem features a rejection criterion that can, in some cases, be difficult to overcome, making this approach costly, for example, if there is a small number of contacts and very few configurations are stable.

%Those approach lead to a sequential resolution of the posture generation problem where a solution to the IK problem is searched, then tested for stability, if it is stable, the optimal forces are computed, otherwise, another solution is searched for.

Another way to solve the posture generation problem is to consider the complete problem at once, with the IK targets and all the other constraints (stability, collisions, torque limits, etc.) in a single nonlinear constrained optimization problem.
This approach is used in~\cite{Zhao1994} to solve the IK problem.
That is the approach that we and many others use to solve Generalized Inverse Kinematics problems.
In~\cite{escande:iros:2006}, such an approach is employed to generate postures that are used to grow a tree of usable postures in a multi-contact planning algorithm.
\cite{escande:iros:2006} applied the approach to automatically generate a sequence of contact sets and postures where the HRP-2 robot uses its hand to lean on a table in order to grasp an object otherwise out of reach.
Interestingly, in this study, the C-code that computes the robot's kinematics is generated once and for all for each robot, using Maple and the HuMAnS toolbox~\cite{wieber2006humans} provided by INRIA.
The constraints of contact, stability and collision are then computed based on the outputs of the generated code.
And the problem is solved by the {\tt CFSQP} solver~\cite{cfsqp:manual}.
In~\cite{hauser:ijrr:2008}, a similar posture generation approach is used to find viable postures for different legged robots on varied terrains in the context of Probabilistic RoadMap planning, which is a planning approach based on random sampling of the configuration space.
%\cite{hauser:issr:2007} presents a different approach to multi-contact planning based on probabilistic roadmap and random sampling of the configuration space.
In~\cite{bouyarmane2010static}, a generalization of the formulation of posture generation problems for systems of multiple robots and manipulated objects is proposed.
%In this formulation, the notion of contacts is generalized to bear forces or not and to have any orientation.
With this formulation, the notion of contacts is generalized such that they are not necessarily coplanar, nor necessarily horizontal, frictional, can bear forces or not, and may be unilateral or bilateral.
This generic formulation enables the generation of postures with inter-robot contacts.
The complete posture generation problem is then solved within a single nonlinear constrained optimization query, computing the contact forces and joint configuration at the same time, while ensuring the stability of all the `robotic agents', avoiding collision and respecting the robot's joint and torque limits.

In the past few years, our team has dedicated considerable efforts in proposing a general multi-contact motion planner to solve cases of non-gaited acyclic planning, using a posture generator as a backbone to select valid configurations.
Given a humanoid robot, an environment, a start and a final desired postures, the planner generates a sequence of contact stances allowing any part of the humanoid to make contact with any part of the environment to achieve motion towards the goal.
The planner's role is to grow a tree of contact stances iteratively; from a given posture, it tries to add or remove contacts one by one.
The tree grows following some heuristics until the solution is reached.
For each set of contacts to add to the tree, a posture generation problem is solved in order to validate the feasibility of the set. If it is not feasible, the set is rejected.
A typical experiment with an HRP-2 robot achieving such an acyclic motion is presented in~\cite{escande:iser:2008}, and the planner is thoroughly described in~\cite{escande:ras:2013}.
Extensions of this multi-contact planner to multi-agent robots and objects gathering locomotion and manipulation are presented in~\cite{bouyarmane:ar:2012}, and preliminary validations with some DARPA challenge scenarios, such as climbing a ladder, ingress/egress a utility car, or crossing through a relatively constrained pathway are presented in~\cite{bouyarmane:humanoids:2012}.
Another way of planning a multi-contact scenario, which is actually often used when planners fail to find satisfactory solutions, is manual, choosing iteratively which contacts to add and remove until the goal is reached.
This type of approach is used when the plan to execute is complex and when a lot of fine tuning of the postures is required, as in~\cite{vaillant:autonomousrobots:2016}, where a sequence of postures allowing the HRP-2 robot to climb a vertical ladder is generated and tuned manually.
Those postures are provided as input to a finite state machine that builds additional intermediary tasks and specific grasps procedures to be realized on the real robot by a multi-objective model-based QP controller.

Instead of feeding the key postures directly to the controller and trusting that it will find a path to connect them, one can take an additional step and compute a dynamically viable trajectory between successive steps that the controller will then have to follow.
Classical methods generate satisfactory configurations on a discretized time-grid and then extrapolate the motion between configurations at successive time-grid points.
This approach does not guarantee that the constraints are satisfied at every point of a continuous time interval, which is dangerous for the robot.
Interval analysis methods can be leveraged to compute safe motions over continuous time intervals, as proposed in~\cite{lengagne2011planning}.
This work was extended in~\cite{lengagne2013generation} to generate whole-body multi-contact dynamic motions.
The constraints are transformed into a computation of extrema of time polynomials, whose coefficients are a function of the optimization variables, over each time interval.
Generating full body dynamic motions allows one to take advantage of the dynamic effects and to produce motions with better performances in terms of completion time, energy consumption, or other criteria.

In many planning approaches~\cite{kuffner2005motion, chestnutt2007navigation, hauser:ijrr:2008, kolter2008control, bouyarmane:icra:2011} the planning problem is decomposed into two stages, first the sets of contacts and associated postures are planned and then the motions to go from one set to another are computed.
These two stages are loosely coupled, which can result in suboptimal use of the contacts available.
\cite{mordatch:acm:2012} presents a different approach to contact planning where some additional terms and variables are added to the optimization problem to decide whether a contact pair should be active (and bear forces) at any given time during the motion, or not.
The contact sets, as well as the postures associated, are discovered along with the entire movement's trajectories by using contact invariant optimization.
This allows for the exploitation of any synergies that might exist between the contact events and the motion trajectory.

In~\cite{tonneauijrr16}, a multi-contact planning framework is presented that allows the computation of complex multi-contact motions in very short times.
It is based on several heuristics that allow fast computations of the motions.
Starting by generating a guide path that satisfies the reachability condition, which means that the root body is close to obstacles to allow making contacts with them but not too close to avoid collisions.
A discrete sequence of statically stable whole-body configurations along this path is then generated efficiently by taking advantage of several approximations and heuristics.
This approach allows generating complex contact plans in a few seconds, whereas other approaches usually take several minutes.

In~\cite{liu:acm:2012}, posture generation is used to find the optimal posture and position of contacts to optimally realize a manipulation task along a path while satisfying geometric, kinematic, as well as force and torque constraints.
The task is defined as a path and forces to follow with an end effector.
Instead of finding a sequence of unrelated postures along the path, all the postures are found by solving a single optimization problem in which successive postures are coupled by constraints to ensure that the foot positions remain constant during the task, even though the rest of the body can move.
This approach allows the robot or virtual character to apply manipulation forces as strongly as possible while avoiding foot slippage.
It also allows taking external perturbations into account to generate more robust postures.

A different utilization of the posture generation was presented in~\cite{stasse:humanoids:2007}, that deals with the problem of object reconstruction.
An object is presented to the robot in order to generate automatically its 3D model.
To do so, the object is observed from several different angles with the camera that is mounted in the robot's head.
The satisfactory postures of the robot to complete this task are obtained by solving a posture generation problem to which a set of constraints defining the direction in which the robot should look and a minimum distance from the object are added.
While in this work the observation directions are predefined,~\cite{foissotte:humanoids:2008} presents an approach where the choice of the best observation direction is left to the posture generator.
The representation of the object is iteratively carved into a block of 3D voxels, each new point-of-view chosen to allow to carve it a little more precisely.
It uses a modified cost function to evaluate the amount of information on the object that a given point of view can provide.
The posture of the robot that optimizes that cost function is then found by the posture generator under the classical constraints of stability, collision avoidance, and other robot's limitations.

\subsection{Optimization}
\label{sub:optimization}

The resolution of a posture generation problem is often done by solving a nonlinear optimization problem which consists of finding the optimal point that minimizes a cost function and maybe constraints, the cost and constraints functions being possibly nonlinear.

Optimization algorithms can be derivative-free or not.
The computation of the derivatives of the functions involved in the problem is a common source of error.
The strength of the derivative-free approaches is that the user does not need to implement the derivatives.
But derivative-free approaches are much slower than their counterparts using derivatives.
One way to avoid implementation errors in the derivatives computation is to use finite differences to compute them.
This method is very slow, especially when the dimension of the problem is large.
In order to design an efficient optimization algorithm, we will focus on methods using derivative information and will implement those derivatives' computations.
The finite difference approach can be used to verify the correctness of the computed derivative.

Nonlinear optimization is a well-established research field by its own and has been extensively studied in the past and nowadays.
One can find some excellent reference books about optimization, such as~\cite{nocedal:book:2006, bonnans:book:2003, boyd2004convex}.

Furthermore, several off-the-shelf solvers are available and have been widely used to solve robotics problems.
The {\tt CFSQP} solver~\cite{cfsqp:manual} was used in~\cite{escande:iros:2009} and~\cite{escande:ras:2013} where thousands of HRP-2 posture generation queries were made to explore the feasible space.
The {\tt IPOPT} solver~\cite{wachter:mathprog:2006} has been used in~\cite{vaillant:humanoids:2014},~\cite{vaillant:autonomousrobots:2016},~\cite{bouyarmane:ar:2012} where the posture generator has been extended to handle multi-robot problems with more complex and various contact constraints.
The SNOPT solver~\cite{gill2005snopt} was used in~\cite{dai2014whole} to plan dynamic whole-body trajectories.
Several nonlinear optimization solvers are available and have been packaged for use in robotics problems in the Roboptim Framework~\cite{moulard:jsme:2013}, such as {\tt IPOPT}, {\tt CFSQP}, {\tt CMinPack}~\cite{cminpack}, {\tt NAG}~\cite{nag}, {\tt KNITRO}~\cite{knitro} and {\tt PGSolver} (the solver that we develop in this thesis).

Traditionally, optimization problems are solved over Euclidean spaces.
When the need comes to find a solution to an optimization problem in a non-Euclidean space $\mathcal{M}$, the commonly used method is to represent the elements of $\mathcal{M}$ in a Euclidean manifold of higher dimension, and enforcing that solution of the optimization problem should lie on $\mathcal{M}$ by adding the necessary constraints to the problem.
We will detail the drawbacks of such approaches in Chapter~\ref{chapter:optimization_on_noneuclidean_manifolds}.
Alternatively, there exists a method to run an optimization algorithm directly on a non-Euclidean manifold, as it is presented in great detail in the book~\cite{absil:book:2008}.
A non-Euclidean manifold can be thought of as a space that is locally Euclidean, but is not so globally.
For example, a sphere looks Euclidean if one examines a small-enough neighborhood, similar to the surface of a giant sphere like the earth that looks flat for a human being standing on it.
Based on the fact that manifolds are locally Euclidean, the classic properties of distance, derivatives, and in general all the Euclidean geometry can be used locally.
Based on that property, several optimization methods traditionally used on Euclidean manifolds have been extended to non-Euclidean manifolds.
The gradient methods have been extended to manifolds in~\cite{luenberger1972gradient, gabay1982minimizing}.
The Newton methods on manifolds which have better convergence rates were extended to manifolds in~\cite{gabay1982minimizing, stuart1998dynamical, smith2013geometric}.
And Quasi-Newton methods in~\cite{gabay1982minimizing}.
Those approaches are only meant to solve unconstrained optimization problems, which is not enough to solve posture generation problems, where the presence of constraints is unavoidable.

The main idea to optimize on manifolds is that we use a local map between the neighborhood of the current iterate and its Euclidean tangent space in order to run an optimization step and choose an increment in the tangent space that is mapped back to the manifold.
Once the iterate has been incremented, the process is repeated with the map and tangent space associated with the new iterate.
This comes down to re-parametrizing the problem around the current iterate at each iteration.

In the field of robotics, we are only aware of the work of Schulman~\emph{et al.}~\cite{Schulman2014}, where the authors explain the adaptation of their solver to work on $SE(3)$.
In the field of computer vision, and especially for solving pose estimation problems, optimization on manifolds is used often, e.g.~\cite{hertzberg2011, lu2000fast}.

In this thesis, we develop a nonlinear solver on manifolds that can handle constraints.
We were largely inspired by the work of Fletcher concerning the notions of `Sequential Quadratic Programming without a penalty function' ~\cite{Fletcher:ifip:2006, fletcher2010sequential, fletcher:mathprog:2000}, along with other optimization approaches that we adapted to work with manifolds.

\section{Problem Definition}
\label{sec:problem_definition}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                     SECTION PROBLEM DEFINITION                      %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

We consider the problem where we have a robotic system and we want it to realize a set of tasks $T_i$.
We denote $\bf q$ as the joint configuration ($n$ joints + base) of the robot and $\mathbf{f} =\{\mathbf{f}_i\,, i\in[1,m]\}$ represents the $m$ contact forces applied on the robot.
Each task $T_i$ can be represented by a set of equality and inequality equations:
\begin{equation}
  \left\{
    \begin{aligned}
    g_i(\mathbf{q},\mathbf{f}) = 0\\
    h_i(\mathbf{q},\mathbf{f}) \geq 0
    \end{aligned}
  \right.
\end{equation}
For example, the task of making contact between a body (link) of the robot and a part of the environment can be represented in such a way.

In addition to satisfying the tasks $T_i$, the solution configuration to our problem must describe a viable posture of the robotic system, in the sense that it ensures its integrity.
Which translates into the following list of constraints.
It must respect the joint limits of the robot~\eqref{subeq:bounds}, as well as its torque limits~\eqref{subeq:tau}.
Equation~\eqref{subeq:autocoll} translates the auto-collision avoidance constraint between a pair of bodies $\{r_i, r_j\}$ given by a set $\mathcal{I}_\text{auto}$ where $d$ is the minimal distance between two bodies.
Equation~\eqref{subeq:coll} denotes the collision avoidance constraint between a robot body $r_i$ and an object of the environment $O_k$ defined in a set $\mathcal{I}_\text{coll}$.
$\epsilon_{ij}$ and $\epsilon_{ik}$ denote the smallest acceptable distance for their respective constraints.
The stability of the robot is ensured by the respect of equation~\eqref{subeq:stab}, which is the Euler-Newton equation(or a simplification of it).
To avoid slippage of the contacts, the contact forces must remain in the Coulomb friction cones, which is enforced by equation~\eqref{subeq:friction}.
Equations~\eqref{subeq:gi} and~\eqref{subeq:hi} translate the constraints to satisfy task $T_i$.

The set of satisfying configurations can be defined by $\mathcal{Q}$:

\begin{empheq}[left={\mathcal{Q}=\{\mathbf{q}, \mathbf{f}\}:\empheqlbrace}]{align}
      & q_i^- \leq q_i \leq q_i^+ \quad \forall i\in[1,n] & \text{joint limits} \label{subeq:bounds}\\
      & \tau_i^- \leq \tau_i(\mathbf{q}, \mathbf{f}) \leq \tau_i^+ \quad \forall i\in[1,n] & \text{torque limits}\label{subeq:tau}\\
      & \epsilon_{ij} \leq d(r_i(\mathbf{q}), r_j(\mathbf{q})),\quad \forall (i,j)\in\mathcal{I}_\text{auto} & \text{auto-collision}\label{subeq:autocoll}\\
      & \epsilon_{ik} \leq d(r_i(\mathbf{q}), O_k),\quad \forall (i,k)\in\mathcal{I}_\text{coll} & \text{collision}\label{subeq:coll}\\
      & s(\mathbf{q}, \mathbf{f}) = 0 & \text{stability} \label{subeq:stab}\\
      & c(\mathbf{f}_i) \geq 0 \quad i\in[1,m] & \text{slippage avoidance} \label{subeq:friction}\\
      & g_i(\mathbf{q}, \mathbf{f}) = 0 & \text{task i's equalities}\label{subeq:gi}\\
      & h_i(\mathbf{q}, \mathbf{f}) \geq 0 & \text{task i's inequalities} \label{subeq:hi}
\end{empheq}

%\begin{empheq}[left={\mathcal{Q}=\{\mathbf{q}, \mathbf{f}\}:\empheqlbrace}]{align}
      %& \epsilon_{ij} \leq d(r_i(\mathbf{q}), r_j(\mathbf{q})),\quad \forall (i,j)\in\mathcal{I}_\text{auto} & \text{auto-collision}\\
      %& \epsilon_{ik} \leq d(r_i(\mathbf{q}), O_k),\quad \forall (i,k)\in\mathcal{I}_\text{coll} & \text{collision}\\
      %& c(\mathbf{f}_i) \geq 0 \quad i\in[1,m] & \text{slippage avoidance} \\
      %& s(\mathbf{q}, \mathbf{f}) = 0 & \text{stability} \\
      %& q_i^- \leq q_i \leq q_i^+ \quad \forall i\in[1,n] & \text{joint limits} \\
      %& \tau_i^- \leq \tau_i(\mathbf{q}, \mathbf{f}) \leq \tau_i^+ \quad \forall i\in[1,n] & \text{torque limits}\\
      %& g_i(\mathbf{q}, \mathbf{f},\textcolor{red}{\mathbf{s}}) = 0 & \text{task i's equalities}\\
      %& h_i(\mathbf{q}, \mathbf{f},\textcolor{red}{\mathbf{s}}) \geq 0 & \text{task i's inequalities}
%\end{empheq}

%\begin{equation}
  %\begin{array}{l}
    %\minimize\limits_{\mathbf{q}, \mathbf{f},\textcolor{red}{\mathbf{s}}}{f(\mathbf{q}, \mathbf{f},\textcolor{red}{\mathbf{s}})}\\
    %\text{ s.t. }
    %\left\{
    %\begin{array}{l}
      %\epsilon_{ij} \leq d(r_i(\mathbf{q}), r_j(\mathbf{q})),\quad \forall (i,j)\in\mathcal{I}_\text{auto} \\
      %\epsilon_{ik} \leq d(r_i(\mathbf{q}), O_k),\quad \forall (i,k)\in\mathcal{I}_\text{coll} \\
      %c(\mathbf{f}_i) \geq 0 \quad i\in[1,m]  \\
      %s(\mathbf{q}, \mathbf{f}) = 0  \\
      %q_i^- \leq q_i \leq q_i^+ \quad \forall i\in[1,n]  \\
      %\tau_i^- \leq \tau_i(\mathbf{q}, \mathbf{f}) \leq \tau_i^+ \quad \forall i\in[1,n] \\
      %g_i(\mathbf{q}, \mathbf{f},\textcolor{red}{\mathbf{s}}) = 0 \\
      %h_i(\mathbf{q}, \mathbf{f},\textcolor{red}{\mathbf{s}}) \geq 0
    %\end{array}
    %\right.
  %\end{array}
%\end{equation}

We illustrate these constraints in~\Figref{fig:PG} and will explain them in more details in Chapter~\ref{cha:posture_generation_problem_formulation}.

%Where inequalities~\eqref{subeq:bounds} represent the joint limits, and~\eqref{subeq:tau} the torque limits.

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.7\textwidth]{PG.pdf}
  \caption{The HRP-4 on a stack of cubes. The color of each box corresponds to the color of the constraint it depicts.}
\label{fig:PG}
\end{figure}

In addition to satisfying the set of constraints $T_i$, one can require the searched posture to be optimal in the sense of some cost function $f$.
Defining a cost function can help the user control the result of the problem; for example, by minimizing the distance to a reference posture or maximizing the contact force applied by one end-effector.
In~\cite{escande2009} the cost function was used to guide the posture toward the end of a planning problem.
A sum of different criterions can be used as cost function.
The optimization problem that describes the posture generation problem becomes:

%We consider the problem where we have a robotic system and we want it to do some tasks, like for example to make a contact between a point on a body of the robot and a point of the environment.
%This contact task can be modelized as a simple equality.
%We denote $q$ the joint configuration (joint + base) of the robot, $g(q)$ is the 3D position of the point of interest on the robot and $P$ is the point of the environment.
%Then our problem comes down to finding a configuration $q^*$ such that $g(q^*) = P$.
%For robots like humanoids, the equations describing their kinematics are quite complicated, and, in the presence of angular joints, they are nonlinear.
%In the general case, even for simple tasks, a closed form solution of the problem does not exist.

%We denote $\mathcal{C}$ the configuration space of our robotic system.
%It is the manifold in which $q$ lives. The dimension of $\mathcal{C}$ is equal to the number of degrees of freedom of the robot.
%Note that if all the joints of the robot are actuated, this is the number of motors of the robot.
%And if the robot has a free-floating basis (is not fixed) then the position in $\mathbb{R}^3$ and rotation in $SO(3)$ of its basis are part of $q$.

%We can formulate that problem as follows:

%\begin{equation}
  %\text{find}\ q\in\mathcal{C}\ \text{such that}\ g(q)=P
%\end{equation}

%The space of solution of this problem is a submanifold of $\mathcal{C}$ that we call ${\mathcal{C}}_F$ the feasible configuration space (which can be empty).

%In addition to the equality constraint abovementioned, the problem can feature some inequality constraints.
%For example, each joint variable can be limited to a certain range of value: $\forall i,\ q_i\in [q_i^-,\ q_i^+]$.
%Then the problem becomes a combination of equality and inequality constraints.%, and can be written as:
%%\begin{equation}
  %%\text{find}\ q\in\mathcal{C}\ \text{such that}\left\{
  %%\begin{array}{l}
    %%g(q)=P \\
    %%\forall i,\ q_i^- \leq q_i \leq q_i^+
  %%\end{array}
  %%\right.
%%\end{equation}
%Finally, we can also require finding the point of $\mathcal{C}_F$ that minimizes a criterion, such as the distance to a goal posture $q_0$.
%Our problem becomes:

%\begin{equation}
  %\left\{
  %\begin{array}{l}
    %\minimize\limits_{q\in\mathcal{C}}{\|q-q_0\|_2}\\
    %\text{ s.t. }
    %\left\{
    %\begin{array}{l}
      %g(q)=P\\
      %\forall i,\ q_i^- \leq q_i \leq q_i^+
    %\end{array}
    %\right.
  %\end{array}
  %\right.
%\end{equation}

\begin{equation}
  \begin{array}{l}
    \minimize\limits_{\mathbf{q}, \mathbf{f}}{f(\mathbf{q}, \mathbf{f})}\\
    \text{ s.t. }\{\mathbf{q}, \mathbf{f}\}\in\mathcal{Q}
  \end{array}
\end{equation}

This type of problem is called a nonlinear constrained optimization problem and can be formulated in a more generic fashion as:

\begin{equation}
\label{eq:optim_form_PG}
  \begin{array}{l}
    \minimize\limits_{x}{f(x)}\\
    \text{ s.t. }
    \left\{
    \begin{array}{l}
      c_i(x) = 0,\ \forall i\in{E}\\
      c_i(x) \geq 0,\ \forall i\in{I}\\
    \end{array}
    \right.
  \end{array}
\end{equation}
where $x$ is the optimization variable, that we want to find, and which minimizes the cost function $f(x)$ while satisfying the equality constraints $c_i(x) = 0,\ \forall i\in{E}$ and inequality constraints $c_i(x) \geq 0,\ \forall i\in{I}$.
Such problems can be solved by a nonlinear optimization solver.
In Appendix~\ref{chapter:optimization}, we present some classical concepts of nonlinear optimization.
%Several off-the-shelf solvers are available and have been widely used in for solving robotics problems.
%The CFSQP solver~\cite{cfsqp:manual} was used in~\cite{escande:iros:2009} and~\cite{escande:ras:2013} where thousands of HRP-2 posture generation queries were made to explore the feasible space.
%The IPOPT solver~\cite{wachter:mathprog:2006} has been used in~\cite{vaillant:humanoids:2014},~\cite{vaillant:autonomousrobots:2016},~\cite{bouyarmane:ar:2012} where the posture generator had been extended to handle multi-robot problems and more complex and various contact models.
%The SNOPT solver~\cite{gill2005snopt} was used in~\cite{dai2014whole} to plan dynamic whole-body trajectories.
%Several nonlinear optimization solvers are available and have been packaged for use in robotics problems in the Roboptim Framework~\cite{moulard:jsme:2013}, such as IPOPT, CFSQP, CMinPack~\cite{cminpack}, NAG~\cite{nag}, KNITRO~\cite{knitro} and PGSolver, the solver that we develop in this thesis.

In this thesis, we used the off-the-shelf solver IPOPT~\cite{wachter:mathprog:2006} in the beginning.
At a later stage, we developed and use our own nonlinear solver called {\tt PGSolver}.
From here on, we will formulate and solve posture generation problems as nonlinear constrained optimization problems.
In the next chapter, we focus on formulating all the basic functions and algorithms used to describe the robotic constraints and cost function in the formalism of nonlinear optimization.


%! TEX program = pdflatex

\documentclass[10pt]{beamer}

\usepackage{graphicx}
\usepackage[utf8x]{inputenc}
% \usepackage{hyperref}
% \usepackage{algorithm}
\usepackage{algorithmic}
% \usepackage{algpseudocode}
\usepackage{algorithm2e}
% \usepackage[colorlinks, urlcolor=yellow]{hyperref}

\usepackage{cite}

% Reference style
\bibliographystyle{plain}

\usetheme{CambridgeUS}

\hypersetup{colorlinks,linkcolor=yellow,urlcolor=blue}

\title{
    Federated Learning Paper Sharing
}
\author{
    Lisen Dai
}

\begin{document}
    \maketitle

    \section*{
        FedOpt (Appl. Sci. 2020, 10(8), 2864)
    }
    \begin{frame}
        \frametitle{
            \href{https://www.mdpi.com/2076-3417/10/8/2864}{
            FedOpt: Towards Communication Efficiency and Privacy Preservation in Federated Learning
            }
        }
        \framesubtitle{Sparse Compression Algorithm}
        Goal: reduce the number of communication bits during the models training. \\
        $$
        \Delta\theta = \mathcal{SGD}_n (\theta, D_{mini-batches}) - \theta
        $$
        $\theta$: Deep Neural Network parameters. \\
        $\mathcal{SGD}_n$: refers to the set of gradient updates after n epochs of SGD on DNN (deep neural network) parameters $\theta$ during the sampling of mini-batches from local data \\

        Once we have the updates $\Delta \theta$... \\

    \end{frame}

    \begin{frame}
        \frametitle{
            \href{https://www.mdpi.com/2076-3417/10/8/2864}{
            FedOpt: Towards Communication Efficiency and Privacy Preservation in Federated Learning
            }
        }

        \begin{algorithm}[H]
            % \caption{SCA: Communication Efficiency in FedOpt}
            % \textbf{Initialisation:} \\
            % \textbf{Input:} temporal vector $\Delta \theta$, Sparsity Fraction $q$ \\
            % \textbf{Output:} : sparse temporal $\Delta \theta^*$ \\

            % \begin{algorithmic}[1]
            %     % \State{\textbf{Initialisation:}} \newline
            %     % $ num^+ \leftarrow top_{q}(\Delta \theta); num^- \leftarrow top_{q}(- \Delta \theta) $ \newline
            % \end{algorithmic}

            \KwIn{temporal vector $\Delta \theta$, Sparsity Fraction $q$}
            \KwOut{sparse temporal $\Delta \theta^*$}
            \caption{SCA: Communication Efficiency in FedOpt}
            \label{alg1}
            \textbf{Initialization}\;
            $ num^+ \leftarrow top_{q}(\Delta \theta); num^- \leftarrow top_{q}(- \Delta \theta) $ \\
            $ \Psi^+ \leftarrow mean(num^+); \Psi^- \leftarrow mean(num^-) $ \\
            \If{ $ \Psi^+ \geq \Psi^- $ }{
                \textbf{return} $ (\Delta \theta^* \leftarrow \Psi^+(\theta \geq \min(num^+))) $\;
            }
            \Else{
                \textbf{return} $ (- \Delta \theta^* \leftarrow \Psi^-(\theta \geq \min(-num^-))) $\;
            }
        \end{algorithm}

    \end{frame}

    \begin{frame}
        \frametitle{
            \href{https://www.mdpi.com/2076-3417/10/8/2864}{
            FedOpt: Towards Communication Efficiency and
            Privacy Preservation in Federated Learning
            }
        }

        \textit{\textbf{“We utilise the additively homomorphic encryption in FedOpt in order to achieve efficiency
        throughout the learning process.”}}

        \begin{figure}[H]
            \includegraphics[width=0.8\textwidth]{src/img/Algorithm2.png}
        \end{figure}

    \end{frame}

    \begin{frame}
        \frametitle{
            \href{https://www.mdpi.com/2076-3417/10/8/2864}{
            FedOpt: Towards Communication Efficiency and
            Privacy Preservation in Federated Learning
            }
        }

        \begin{figure}[H]
            \includegraphics[width=0.8\textwidth]{src/img/Algorithm3.png}
        \end{figure}

    \end{frame}

    \begin{frame}
        \frametitle{
            \href{https://www.mdpi.com/2076-3417/10/8/2864}{
            FedOpt: Towards Communication Efficiency and
            Privacy Preservation in Federated Learning
            }
        }

        \begin{figure}[H]
            \includegraphics[width=0.8\textwidth]{src/img/Communication_Eff.png}
        \end{figure}

    \end{frame}

    \begin{frame}
        \frametitle{
            \href{https://www.mdpi.com/2076-3417/10/8/2864}{
            FedOpt: Towards Communication Efficiency and
            Privacy Preservation in Federated Learning
            }
        }

        \begin{figure}[H]
            \includegraphics[width=\textwidth]{src/img/Accuracy.png}
        \end{figure}

    \end{frame}

    \section*{One-Shot Federated Learning}
    \begin{frame}
        \frametitle{
            \href{https://arxiv.org/pdf/1902.11175.pdf}{
                One-Shot Federated Learning
            }
        }

        \framesubtitle{
            Background
        }

        \textit{"In this work, we instead focus on techniques for \texttt{one-shot federated learning}, in which we learn a global model from data in the network using only a single round of communication between the devices and the central server. "}

        \textit{"Our key insight is that if each local device trains a local model to completion (as opposed to computing incremental updates as in traditional federated learning methods), we can effectively apply ensemble methods to capture global information across the device-specific models."}

    \end{frame}

    \begin{frame}
        \frametitle{
            \href{https://arxiv.org/pdf/1902.11175.pdf}{
                One-Shot Federated Learning
            }
        }

        \framesubtitle{
            Methods
        }

        \textbf{Local}: for $ m $ devices, each device $ t \in [m] $ possesses local data $ X_t \in \mathcal{R}^{d \times n_t} $ and solves:
        $$
            \min_{w_t \in \mathcal{R}^d}\lbrace{ \mathcal{P}{w_t} := \frac{1}{n_t} \sum_{i=1}^{n_t} \ell_i(x_i^T + w_t) + \frac{\lambda}{2} \left\|w_t\right\|^2 }\rbrace
        $$
        where the vector $ \{x_i\}^{n_t}_{i=1} \in X_t $ and $ \ell_i $
        are real-valued convex loss functions (i.e. hinge loss). In the kernelized setting, we solve the dual formulation of this problem:
        $$
            \max_{\alpha_t \in \mathcal{R}^{n_t}}
            \lbrace{
                -\frac{1}{2\lambda n^2_t}
                \alpha^T_t \phi (X_t)^T \phi (X_t)
                \alpha_t + \frac{1}{n_t}
                \sum^{n_t}_{i=1} - \ell^* (-[\alpha_t]_i)
            }\rbrace
        $$
        where $ \phi (X_t)^T \phi (X_t) $ can be replaced with $ k(X_t, X_t) $ using kernel trick.
    \end{frame}

    \begin{frame}
        \frametitle{
            \href{https://arxiv.org/pdf/1902.11175.pdf}{
                One-Shot Federated Learning
            }
        }

        \framesubtitle{
            Methods
        }

        \textbf{Ensemble}:
        \begin{enumerate}
            \item Cross-Validation (CV) Selection: Devices only share their local models if they achieve some baseline performance (e.g., in terms of ROC AUC) on their local validation data, with the baselines determined in advance by the server. The server ensembles the k best performing models from this subset of local models.
            \item Data Selection: Devices only share their local models if they have some baseline amount of local training data, with the baseline determined in advance by the server. The server ensembles models from these local models trained on the top k largest data sets.
            \item Random Selection: The server randomly selects k devices from the network and creates an ensemble from the corresponding local models.
        \end{enumerate}
        The final ensemble $ F_k $ of $ k $ device models is constructed by averaging the predictions of each model.

    \end{frame}

    \begin{frame}
        \frametitle{
            \href{https://arxiv.org/pdf/1902.11175.pdf}{
                One-Shot Federated Learning
            }
        }

        \framesubtitle{
            Methods
        }

        \textbf{Distillation (Semi-Supervised)}:

        When the amount of clients is large...

        When the central server has access to unlabelled public proxy data, $ F_k $ can be compressed into a smaller model, $ f^{\prime} $, via distillation.

        \texttt{New Method}: For proxy data $ x^{\prime}_1, ..., x^{\prime}_l$, we generate corresponding "soft" labels $ F_k(x^{\prime}) $, ...,. In particular, we perform distillation in the dual by minimizing the L2 difference in predictions between the student and teacher on the proxy data:
        $$
            \min_{\alpha^\prime \in \mathcal{R}^l} \frac{1}{l} \sum^l_{i=1}(F(x^\prime_i) - \sum_{j=1}^l \alpha^\prime_j k(x_j^\prime, x_i^\prime))^2
        $$
        to produce a distilled model $ f^\prime(x) = \sum^l_i=1 \alpha_i \phi (x^\prime_i) $. When there are privacy concerns with sharing local models between devices (e.g., for dual SVMs, which require local support vectors to be shared), distillation not only helps to compress the model but also enables privacy-preservin learning.

    \end{frame}

    \begin{frame}
        \frametitle{
            \href{https://arxiv.org/pdf/1902.11175.pdf}{
                One-Shot Federated Learning
            }
        }

        \framesubtitle{
            Result
        }

        \begin{figure}
            \includegraphics[width=\textwidth]{src/img/OneShotFedResult.JPG}
        \end{figure}
        {\small
            \begin{itemize}
                \item[EMNIST]{
                    Handwritten characters authored by different individuals (devices). We predict between lowercase and uppercase letters
                }
                \item[Sentiment140]{
                    Binary sentiment detection on tweets from different users (devices). We use a bag-of-words representation to featurize tweets
                }
                \item[Gleam]{
                    Two hours of high resolution Google Glass sensor data corresponding to different activities. We predict between eating and other activities (walking, talking, etc)
                }
            \end{itemize}
        }

    \end{frame}

    \section*{Expanding the reach of federated learning by reducing client resource requirement}

    \begin{frame}
        \frametitle{
            \href{https://arxiv.org/pdf/1812.07210.pdf}{
            Expanding the reach of federated learning by reducing client resource requirement
            }
        }

        \framesubtitle{
            Overview
        }

        \begin{figure}
            \includegraphics[width=\textwidth]{src/img/RCRRStructure.JPG}
        \end{figure}

    \end{frame}

    \begin{frame}
        \frametitle{
            \href{https://arxiv.org/pdf/1812.07210.pdf}{
            Expanding the reach of federated learning by reducing client resource requirement
            }
        }

        \framesubtitle{
            Method1: Lossy Compression
        }

        We reshape each to-be-compressed weight matrix in our model into a vector w and (1) apply a \textit{basis transform} to it. We then (2) \textit{subsample} and (3) \textit{quantize} the resulting vector and finally send it through the network. Once received, we simply execute the respective inverse transformations to finally obtain a noisy version of $ \textbf{w} $.

    \end{frame}

    \begin{frame}
        \frametitle{
            \href{https://arxiv.org/pdf/1812.07210.pdf}{
            Expanding the reach of federated learning by reducing client resource requirement
            }
        }

        \framesubtitle{
            Method2: Federated Dropout
        }

        \begin{figure}
            \includegraphics[width=\textwidth]{src/img/FedDropout.JPG}
        \end{figure}
    \end{frame}

    \begin{frame}

        \frametitle{
            \href{https://arxiv.org/pdf/1812.07210.pdf}{
            Expanding the reach of federated learning by reducing client resource requirement
            }
        }

        \framesubtitle{
            Method2: Federated Dropout
        }

        Traditional dropout: hidden units are multiplied by a random binary mask in order to drop an expected fraction of neurons during each training pass through the network. Because the mask changes in each pass, each pass is effectively computing a gradient with respect to a different sub-model. These sub-models can have different sizes (architectures) depending on how many neurons are dropped in each layer. Now, even though some units are dropped, in all implementations we are aware of, activations are still multiplied with the original weight matrices, they just have
        some useless rows and columns.

    \end{frame}

    \begin{frame}

        \frametitle{
            \href{https://arxiv.org/pdf/1812.07210.pdf}{
            Expanding the reach of federated learning by reducing client resource requirement
            }
        }

        \framesubtitle{
            Method2: Federated Dropout
        }

        To extend this idea to FL and realize communication and computation savings, we instead zero out a fixed number of
        activations at each fully-connected layer, so all possible sub-models have the same reduced architecture; see Figure 2.
        The server can map the necessary values into this reduced architecture, meaning only the necessary coefficients are
        transmitted to the client, re-packed as smaller dense matrices. The client (which may be fully unaware of the original
        model’s architecture) trains its sub-model and sends its update, which the server then maps back to the global model2
        . For convolutional layers, zeroing out activations would not realize any space savings, so we instead drop out a fixed
        percentage of filters.

    \end{frame}

    \begin{frame}

        \frametitle{
            \href{https://arxiv.org/pdf/1812.07210.pdf}{
            Expanding the reach of federated learning by reducing client resource requirement
            }
        }

        \framesubtitle{
            Method2: Federated Dropout
        }

        This technique brings two additional benefits beyond savings in server-to-client communication. First, the size of the
        client-to-server updates is also reduced. Second, the local training procedure now requires a smaller number of FLOPS
        per gradient evaluation, either because all matrix-multiplies are now of smaller dimensions (for fully-connected layers)
        or because less filters have to be applied (for convolutional ones). Thus, we reduce local computational costs.

    \end{frame}

    \begin{frame}

        \frametitle{
            \href{https://arxiv.org/pdf/1812.07210.pdf}{
            Expanding the reach of federated learning by reducing client resource requirement
            }
        }

        \framesubtitle{
            Results
        }

        \begin{figure}
            \includegraphics[width=\textwidth]{src/img/DpR1.JPG}
        \end{figure}

    \end{frame}

    \begin{frame}

        \frametitle{
            \href{https://arxiv.org/pdf/1812.07210.pdf}{
            Expanding the reach of federated learning by reducing client resource requirement
            }
        }

        \framesubtitle{
            Results
        }

        \begin{figure}
            \includegraphics[width=\textwidth]{src/img/DpR2.JPG}
        \end{figure}

    \end{frame}

    % Reference
    \begin{frame}
        \frametitle{
            Reference
        }

        % Reference below
        % \bibliography{src/citations/ref.bib}
        \nocite{*}
        \bibliography{src/citations/ref.bib}

    \end{frame}

\end{document}


\documentclass[11pt]{article}
\usepackage{a4wide,times}
\usepackage{parskip}
\usepackage{lmodern}
\usepackage{multicol}
\usepackage{url}
\usepackage{mathtools}
\usepackage{bm}
\usepackage{color}
\usepackage{float}
\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{graphicx}

\usepackage[utf8]{inputenc}
\usepackage{tgpagella}

\numberwithin{equation}{section}

\DeclareMathOperator*{\argmin}{\arg\min}

\begin{document}

\date{}
\title{\huge \bf{Using support vector machines and\\ decision tree-based methods for\\ analysing gene expression data}}\maketitle
\centerline{\Large {Machine Learning and Algorithms for Data Mining --- Assignment 2}}
\vspace{1em}
\centerline{(Word count: 2161)}
\vspace{2em}

\begin{abstract}
\noindent
\textit{This report presents a methodology for classification of gene expression data using support vector machines and various methods based on decision trees (bagging, random forests, adaptive and gradient boosting). The aim is to distinguish patients with rheumatoid arthritis from healthy controls by processing expression levels of 22,283 genes from three clinical groups in Jena, Berlin and Leipzig \cite{woetzel2014}. The first section gives an overview of the clinical problem and presents the theoretical aspects behind the previously mentioned algorithms. Following this, I describe the dataset pre-processing and software packages used to implement my methodology. Finally, the evaluation section describes the results obtained by applying the different methods to the gene expression data.}
\end{abstract}

\section{Introduction}

\subsection{Motivation}

\textit{Rheumatoid arthritis} (RA) is one of the most common forms of arthritis---a progressive, chronically inflammatory and destructive joint disease that is sustained by an invasive synovial membrane. Distinguishing RA patients from those affected by other inflammatory or degenerative joint diseases or healthy controls, based solely on differentially expressed genes, is difficult. Consequently, we require other unbiased approaches for classification; one such method is discussed by Woetzel et al. \cite{woetzel2014}, who employed rule-based classifiers to discriminate between RA, osteoarthritis (OA) and healthy controls.

Using a subset of the data described in their research, my work addresses binary classification, distinguishing RA patients from the control group. The machine learning methods that I applied---support vector machines and decision tree-based methods---are described in the following subsections.

\subsection{Support vector machines}

The \textit{support vector machine} (SVM) represents a supervised machine learning technique that performs binary classification by separating the classes with a \textit{hyperplane} in the feature space. For optimal performance, an SVM aims to find the biggest distance from the plane to the boundary data points.

A $p$-dimensional hyperplane is defined as:
\begin{equation}
\beta_0 + \beta_1X_1 + \beta_2X_2 + ... + \beta_pX_p = 0,
\end{equation}

such that if the data point $\mathbf{X} = (X_1, X_2, ..., X_p)^\top$ satisfies the equation, then it lies on the hyperplane.

Assuming the samples in our dataset take the form $(\mathbf{X}_i, Y_i)$, with $Y_i \in \{-1, +1\}$ (for binary classification), we aim to learn a function $f(\mathbf{X}) = \beta_0 + \beta_1X_1 + \beta_2X_2 + ... + \beta_pX_p$ that approximates the response variable $Y$ such that examples from one class lie on one side of the hyperplane ($f(\mathbf{X}) < 0$) and the others satisfy $f(\mathbf{X}) > 0$.

Then, we need to select the hyperplane that achieves the \textit{maximal margin} between the two classes, defined as the biggest perpendicular distance between the plane and the points closest to it.

This can be defined as a \textit{constraint optimisation problem}---find a set of values $\beta_i, i = 0, 1, 2, ..., p$ to maximise $M$, subject to $\sum_{i = 0}^{p} \beta_i^2 = 1$, such that:
\begin{equation}
\beta_0 + \beta_1X_{i1} + \beta_2X_{i2} + ... + \beta_pX_{ip} \geq M, \forall i = 1, 2, ..., n,
\end{equation}

where $n$ is the number of training examples.

However, deriving the best hyperplane might result in overfitting and poorer generalisation ability of the SVM when exposed to unseen data. Instead, we can have a better performance on the test set by misclassifying a few training examples:
\begin{itemize}
\item introduce and find values for \textit{slack variables} $\epsilon_1, \epsilon_2, ..., \epsilon_n$ representing how far away each data point can be on the wrong side of the hyperplane;
\item re-design the constraint optimisation problem such that:
\begin{equation}
\beta_0 + \beta_1X_{i1} + \beta_2X_{i2} + ... + \beta_pX_{ip} \geq M(1-\epsilon_i), \forall i = 1, 2, ..., n,
\end{equation}
with $\epsilon_i \geq 0$ and $\sum_{i = 1}^{n} \epsilon_i \leq C$.
\end{itemize}

Problems might still arise when we cannot linearly separate two classes. In this case, the solution is to \textit{enlarge the feature space} through various functions of features. \textit{Kernel functions} can quantify how similar two data points are, while avoiding complexities such as computing polynomials in feature space (which quickly becomes intractable). The kernels I used are:
\begin{enumerate}
\item linear kernel (corresponds to the support vector classifier):
\begin{equation}
K(x_i, x_{i'}) = \sum_{j = 1}^{p} x_{ij} x_{i'j},
\end{equation}
\item polynomial kernel:
\begin{equation}
K(x_i, x_{i'}) = \bigg(1 + \sum_{j = 1}^{p} x_{ij} x_{i'j}\bigg)^d,
\end{equation}
\item radial (Gaussian) kernel (also known as RBF kernel):
\begin{equation}
K(x_i, x_{i'}) = \exp\bigg(-\gamma\sum_{j = 1}^{p}(x_{ij} - x_{i'j})^2\bigg), \text{with } \gamma > 0.
\end{equation}
\end{enumerate}

\subsection{Decision trees}

Widely used for discriminative tasks, a decision tree learns a hierarchy of \textit{if/else} questions which eventually lead to the classification of the given data point. Nodes in the tree can either represent \textit{questions} or \textit{decisions}---in the latter case, the node is a leaf---thereby providing a series of binary splits in the feature plane. These splits produce non-overlapping regions that correspond to classes.

Overfitting usually occurs when the decision tree represents a highly complex model of the training data, such that all its leaves are \textit{pure} (the tree obtains 100\% accuracy on the training set). This can be solved by either:
\begin{itemize}
\item early stopping of the tree construction (\textit{pre-pruning})---can be done by limiting the maximum depth or number of leaves of the tree;
\item removing/collapsing nodes which contain little information (\textit{pruning}).
\end{itemize}

At each step, we choose the variable that best splits the set of items according to an \textit{impurity} measure; the one used in this study is the Gini measure:
\begin{equation}
H(N_m) = \sum_{k = 1}^{C} p_{mk} (1 - p_{mk}),
\end{equation}

where $p_{mk}$ is the proportion of examples belonging to class $k$ at node $m$. The algorithm I used for decision trees is CART (Classification and Regression Trees), which constructs binary trees using the feature and threshold that give the largest information gain upon every split.

\subsection{Bagging}

Decision trees are easy to construct, but they can produce noisy or weak classifiers. More powerful machine learning models can be obtained by \textit{ensembling} multiple classifiers. One of these approaches is called \textit{bagging} and aims to improve prediction performance by reducing the variance of the classifier and obtaining smoother decision boundaries. The bagging algorithm proceeds in the following manner:
\begin{itemize}
\item draw $B$ bootstrap samples $\mathbf{S'}_i, i = 1, 2, ..., B$ with replacement from the dataset $\mathbf{S}$;
\item use each sample $\mathbf{S'}_i$ to train a separate decision tree and record its out-of-bag (OOB) error rate on the examples from $\mathbf{S}$ that were not included in $\mathbf{S'}_i$;
\item average the recorded values and provide a final estimate for the OOB error.
\end{itemize}

The classification performance is improved, as recording $B$ values for the OOB error reduces its variance from an initial value of $\sigma^2$ to $\frac{\sigma^2}{B}$.

\subsection{Random forests}

A further refinement to decision trees, \textit{random forests} (RFs) improve on the bagging approach by de-correlating the trees. This is crucial when we encounter a strong predictor variable in the dataset; most trees will first split the data by this predictor, so all of the bagged trees will be similar and their predictions highly correlated. This in turn will hurt the variance, since averaging many highly correlated quantities does not reduce the variance as much as if the quantities are uncorrelated.

RFs solve this issue by considering only a random subset of $m$ of the $p$ predictors at each split in the tree. The values I used for $m$ were $m = \sqrt{p}$ and $m = \log_2p$.

\subsection{Boosting}

\textit{Boosting} approaches improve on random forests by averaging trees which are grown to weighted versions of the training set. I use two types of boosting:
\begin{itemize}
\item gradient boosting, which takes three hyperparameters ($B$---the number of steps, $\epsilon$---the shrinkage factor, $d$---the maximum tree depth) and proceeds as follows: starting from a model $\hat{G} = 0$ and a residual vector $\mathbf{r} = \mathbf{y}$, at each step we fit a regression tree $\tilde{g}_i$ to the data, then we update the model $\hat{G}_i = \hat{G}_{i-1} + \epsilon \cdot \tilde{g}_i$ and the residuals $r_i = r_i - (\epsilon \cdot \tilde{g}_i)(\mathbf{X}_i)$;
\item adaptive boosting (AdaBoost): a special case of gradient boosting that assigns higher importance to more poorly fitted training examples.
\end{itemize}

\section{Implementation}

All the work presented in this report has been carried out in the Python programming language, using various libraries that I will describe in the following subsections.

\subsection{Processing the dataset}

The data I used to evaluate the various machine learning models was downloaded from the following GEO DataSet browser locations and represents three clinical groups in Jena, Berlin and Leipzig:

\begin{verbatim}
https://www.ncbi.nlm.nih.gov/sites/GDSbrowser?acc=GDS5401
https://www.ncbi.nlm.nih.gov/sites/GDSbrowser?acc=GDS5402
https://www.ncbi.nlm.nih.gov/sites/GDSbrowser?acc=GDS5403
\end{verbatim}

For the purpose of the report methodology, I only extracted the columns representing healthy controls (30 in total) and rheumatoid arthritis patients (33 across the three datasets) for binary classification. The \texttt{GEOparse} library \cite{geoparse}, based on the R library \texttt{GEOquery}, allowed me to parse the data from the SOFT files in a suitable format---a table which facilitated the retrieval of data for the mentioned patients, as they are indexed by columns. First, I loaded the SOFT files into Python:
\begin{verbatim}
gds = GEOparse.get_GEO(filepath="./GDS5401_full.soft")
\end{verbatim}
Next, I prepared containers for the predictor (gene expression levels) and response variables (indicator whether the person is a healthy control or an RA patient):
\begin{verbatim}
Xs = np.empty(shape=(num_samples, num_features), dtype=float)
ys = np.empty(shape=(num_samples,), dtype=int)
\end{verbatim}
To extract data for the healthy controls (C) and rheumatoid arthritis patients (RA), I first looked at the corresponding ranges on the GEO webpages,  under the 'Sample Subsets' tab (e.g. the GDS5403 dataset contains RA patients in the range of identifiers GSM13373\textbf{14}--GSM13373\textbf{26}).

After retrieving the columns through \texttt{cols = gds.table.columns}, I filtered the relevant data from the rest (the ranges [\texttt{l\_C}, ..., \texttt{r\_C}] and [\texttt{l\_RA}, ..., \texttt{r\_RA}] are the ones identified as described above):
\begin{verbatim}
curr_sample = 0
for key in cols:
   # Check if this column corresponds to patient data
    if 'GSM' in key:
        if l_C <= key[-2:] and key[-2:] <= r_C:
            # Class = healthy control
            ys[curr_sample] = 0
        elif l_RA <= key[-2:] and key[-2:] <= r_RA:
            # Class = RA patient
            ys[curr_sample] = 1
        else:
            # Other data - not interested
            continue

        # Record gene expression levels for the current person
        Xs[curr_sample, :] = gds.table[key]
        curr_sample += 1
\end{verbatim}

\subsection{Machine learning models}

Classification methodologies require evaluation and comparison; the \texttt{scikit-learn} library \cite{scikit-learn} provides simple interfaces to many ML models, including SVMs, decision trees, random forests and ensembling approaches such as bagging and boosting. Furthermore, the library contains useful evaluation metrics (e.g. confusion matrix, recall, precision, F1 score).

The following lines illustrate how to instantiate all the classifiers I used for this report:
\begin{verbatim}
SVM_linear = svm.SVC(kernel='linear', C=C)
SVM_poly = svm.SVC(kernel='poly', C=C, degree=deg)
SVM_rbf = svm.SVC(kernel='rbf', C=C, gamma=gamma)

dt = tree.DecisionTreeClassifier(max_depth=max_depth)

b = BaggingClassifier(base_estimator=None, # defaults to a decision tree
                      n_estimators=num_trees,
                      oob_score=True)

rf = RandomForestClassifier(n_estimators=num_trees,
                            max_features=max_features, # log2 or sqrt
                            oob_score=True)

gb = GradientBoostingClassifier(n_estimators=num_trees,
                                max_depth=max_depth)

adab = AdaBoostClassifier(n_estimators=num_trees)
\end{verbatim}

The performance of SVMs, decision trees, gradient boosting and adaptive boosting was evaluated using the metrics below and saved to pickled files (the \texttt{pickle} module performs Python object serialization):
\begin{verbatim}
cm = confusion_matrix(y_test, y_pred)
recall = recall_score(y_test, y_pred)
precision = precision_score(y_test, y_pred)
f1 = f1_score(y_test, y_pred)

# Save the results in a pickled format
res = { "cm" : cm, "recall" : recall, "precision" : precision, "f1" : f1 }
pickle.dump(res, open(filename, "wb"))
\end{verbatim}

Bagging and random forests perform bootstrap sampling with replacement, so their evaluation consists of out-of-bag error estimates:
\begin{verbatim}
oob_error = 1 - b.oob_score_
oob_error = 1 - rf.oob_score_
\end{verbatim}

\section{Evaluation}

All machine learning models described in this report were evaluated with 20 random train/test partitions of the dataset and the results were averaged, in order to provide a better indicator of the real accuracy. As the dataset is quite small---63 examples in total---a single model is likely to encounter high variance on other data, so the repeated random partitioning helps reduce this variance.

\subsection{SVMs}

To evaluate the performance of SVM models, I trained and tested one seperate model for every combination of the following parameters:
\begin{itemize}
\item for all SVMs: $C = \{0.1, 1, 10\}$;
\item polynomial kernel SVMs: $\textit{degree} = \{2, 3, ..., 15\}$;
\item RBF kernel SVMs: $\gamma = \{0.1, 1, 10\}$.
\end{itemize}

Table \ref{table:resSVM} describes the best results for each type of SVM (chosen by the highest value for the F1 score, which incorporates both precision and recall), along with the corresponding hyperparameters.
\begin{table}[h!]
\begin{center}
\begin{tabular}{ |c|c|c|c| }
 \hline
  & Precision & Recall & F1 \\
 \hline
 Linear kernel ($C = 0.1$) & \bf{0.92} & \bf{1.0} & \bf{0.96} \\
 \hline
 Polynomial kernel ($C = 0.1, d = 2$) & 0.91 & 0.97 & 0.93 \\
 \hline
 RBF kernel ($C = 0.1, \gamma = 0.1$) & 0.64 & 1.0 & 0.77 \\
 \hline
\end{tabular}
\caption{Results for the best \textbf{SVM} models.}
\label{table:resSVM}
\end{center}
\end{table}

It can be noticed that a linear boundary separates the two classes well enough, while doing feature expansion through a second-degree polynomial kernel already reduces the performance of the SVM. With an F1 score of 0.77, the Gaussian kernel is clearly unsuitable for discriminating between RA and healthy controls, which is in agreement with the linear kernel performing so well (the Gaussian kernel is not likely to cover the area outside the one determined by training examples).

\subsection{Decision tree-based methods}

A single \textbf{decision tree} was evaluated with no bounds on the maximum depth, and then limited by $depth = \{1, 2, 4, 7\}$. As seen in Table \ref{table:resDT}, the unbounded tree obtains the best performance (only slightly better than a tree with depth 4). This might be due to numerous splits being required to finally make a decision regarding the class, using all of the 22,283 features.

\begin{table}[h!]
\begin{center}
\begin{tabular}{ |c|c|c|c| }
 \hline
  & Precision & Recall & F1 \\
 \hline
 Depth = 1 & 0.851 & 0.884 & 0.854 \\
 \hline
 Depth = 2 & 0.859 & 0.897 & 0.866 \\
 \hline
 Depth = 4 & 0.894 & 0.924 & 0.896 \\
 \hline
 Depth = 7 & 0.874 & 0.884 & 0.866 \\
 \hline
 Depth = $\infty$ (unbounded) & \bf{0.879} & \bf{0.938} & \bf{0.903} \\
 \hline
\end{tabular}
\caption{Results for a single \textbf{decision tree}.}
\label{table:resDT}
\end{center}
\end{table}

\textbf{Bagging} is expected to improve on the performance of a single decision tree; here, only the OOB error estimate is available. Figure 1 shows that it tends to oscillate between $0.6$ and $0.9$ for most values of $t$. However, the minimum value for the OOB error is $0.038$ (so a proportion of 96.2\% correctly classified out-of-bag examples), at $t = 33$ trees.

\begin{figure}[H]
\includegraphics[width=\textwidth]{bagging.png}
\caption{Out-of-bag (OOB) error rates for the \textbf{bagging} method.}
\end{figure}

Even further improvements (once again, expected) can be noted from the results of evaluating \textbf{random forests}. This is an effect of limiting the number of features considered at each split, such that the predictions of the obtained trees are decorrelated. Figure 2 indicates that when we allow only $m = \sqrt{p}$ and $m = \log_2p$ features upon splitting, the minimum values for the OOB error are both equal to $0.0$, at $t = 79$ trees and $t = 64$ trees, respectively. However, this estimate might change if the dataset used for evaluation is considerably larger, but it still indicates a significant improvement over bagging. The graph also confirms the fact that, once a sufficient number of trees is reached, the performance cannot degrade.

\begin{figure}[H]
\includegraphics[width=\textwidth]{rf.png}
\caption{Out-of-bag (OOB) error rates for \textbf{random forests}, shown in \textcolor{blue}{blue} for $m = \sqrt{p}$ and \textcolor{green}{green} for $m = \log_2p$.}
\end{figure}

\textbf{Gradient boosting} was evaluated with the following hyperparameters: maximum depth of the trees $d$ between 2 and 10, number of trees $t$ ranging from 10 to 100. The best performance (chosen by the largest F1 score) is achieved by ($d = 3$, $t = 25$):
\begin{itemize}
\item precision: 0.93
\item recall: 0.95
\item \underline{F1: 0.93}.
\end{itemize}

I evaluated the \textbf{adaptive boosting} approach by running a separate model for any number of trees between $10$ and $200$. As seen in Figure 3, the \underline{F1 metric peaks at $0.91$}, for $t = 74$ trees.

\begin{figure}[H]
\includegraphics[width=\textwidth]{adaboost.png}
\caption{F1 score as a function of the number of trees for \textbf{adaptive boosting}.}
\end{figure}

Although boosting methods usually produce more powerful discriminators than random forests, the size of the dataset might have affected the outcome in this study. Both gradient and adaptive boosting still perform better than a single decision tree---which is highly expected---but they fail to reach the (obviously superior) performance of random forests.

\section{Conclusion}

The machine learning algorithms discussed in this report---especially the decision tree-based ones---generally behaved as initially discussed from a theoretical perspective, when being exposed to RA/healthy controls data. The best performances were obtained by the linear SVM model (F1 score of 0.96) and the random forest approach (OOB score of 0.0), but using significantly larger datasets to train the classifiers might produce different results which are likely to indicate performances closer to the real ones.

In the case of SVMs, the superiority of the linear kernel from the polynomial and RBF ones indicates that a separating hyperplane provides a good approximation to the boundary between the two classes. Bagging was clearly outperformed by random forests, while decision trees were also surpassed by boosting approaches (F1 scores of 0.90, 0.91 and 0.93, respectively). Nevertheless, these results show an impressive performance for a dataset of 63 examples, illustrating the power of the various improvements that are added to a basic algorithm (e.g. taking bootstrap samples of the dataset to reduce variance and create smoother decision boundaries).

\newpage
\medskip
\begin{thebibliography}{9}
\bibitem{woetzel2014}
Woetzel, D., Huber, R., Kupfer, P., Pohlers, D., Pfaff, M., Driesch, D., ... \& Kinne, R. W. (2014). Identification of rheumatoid arthritis and osteoarthritis patients by transcriptome-based rule set generation. Arthritis research \& therapy, 16(2), R84.
\bibitem{mldm}
Jamnik, M., Li\`o, P., \& Sauerwald, T. (2017). Lecture notes for \textit{Machine Learning and Algorithms for Data Mining} (University of Cambridge Computer Laboratory - MPhil in Advanced Computer Science).
\bibitem{geoparse}
Gumienny, R. (2015). GEOparse: Python library to access Gene Expression Omnibus Database (GEO). \texttt{https://geoparse.readthedocs.io/en/latest/introduction.html}.
\bibitem{scikit-learn}
Pedregosa, F. and Varoquaux, G. and Gramfort, A. and Michel, V. and Thirion, B. and Grisel, O. and Blondel, M. and Prettenhofer, P. and Weiss, R. and Dubourg, V. and Vanderplas, J. and Passos, A. and Cournapeau, D. and Brucher, M. and Perrot, M. and Duchesnay, E. (2011). Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12, 2825--2830.

\end{thebibliography}

\end{document}


% LaTeX2e Template by Stephen Iota (https://stepheniota.com/)
% last updated: Jan. 2019

% for papers
%\documentclass[aps,onecolumn,superscriptaddress]{revtex4-1}
% https://www-d0.fnal.gov/Run2Physics/WWW/templates/revtex4.pdf
% https://cdn.journals.aps.org/files/revtex/auguide4-1.pdf
% for revTeX4-1 class options

% for other
\documentclass[12pt]{article}
\usepackage%[margin=2cm]
{geometry}

%%%%%%%%%%%%%%%%
%%% Packages %%%
%%%%%%%%%%%%%%%%

\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsfonts} % to remove math font when typesetting equations
\usepackage[thinc]{esdiff}
\usepackage{graphicx}
\usepackage[shortlabels]{enumitem} % to change labels in enum/item
\usepackage[dvipsnames]{xcolor} % for colored links

% always put this at the end
\usepackage[
	colorlinks=true,
	citecolor=green!50!black,
	linkcolor=NavyBlue!75!black,
	urlcolor=green!50!black,
	hypertexnames=false]{hyperref}

 %%%%%%%%%%%%%%%%%%
 %% New Commands %%
 %%%%%%%%%%%%%%%%%%

\newcommand{\email}[1]{\texttt{\href{mailto:#1}{#1}}}

\newcommand{\hint}[1]{\color{Blue}{#1}}

%----------------------------------------------------
%%%%%%%%%%%%%%%%%%
%% Front Matter %%
%%%%%%%%%%%%%%%%%%

\pagenumbering{gobble} % no page numbers
\graphicspath{{figures/}} % set directory for figures
%\usepackage{wrapfig}
%\setcounter{section}{-1} % start with section 0

%%%%%%%%%%%%%
%%% Title %%%
%%%%%%%%%%%%%
\begin{document}

\begin{center}

\Large{\textsc{Week 3}: \textbf{Vectors}}

\end{center}

\vspace{.5mm}

%%%%%%%%%%
%% INFO %%
%%%%%%%%%%

\begin{tabular}{rl}
\textsc{SI Leader}:
&
Stephen Iota (\email{siota001@ucr.edu})
\\
\textsc{Course}:
&
Physics 40A (Winter 2019), Prof.~John Ellison
\\
\textsc{Date}:
&
23 January 2019
\end{tabular}

%%%%%%%%%%%%%%
%% PROBLEMS %%
%%%%%%%%%%%%%%

\vspace{5mm}
\section*{Review: Motion in 1D with Constant Acceleration}

\begin{equation}
	v(t) = v_i + at
\end{equation}
\begin{equation}
\label{eq:2}
	x(t) = x_i + v_it + \frac{1}{2}at
\end{equation}
\begin{equation}
	v(t)^2 = v_{i}^2	 + 2a(x(t) - x_i)
\end{equation}
\begin{equation}
	x(t) = x_i + \frac{1}{2}(v_i + v(t))t
\end{equation}
\begin{equation}
	x(t) = x_i + v(t)t - \frac{1}{2}at^2
\end{equation}

Note: $x_i = x_0 = x(t=0)$; similarly $v_f = v(t=t_f)$.

\vspace{5mm}

\section{Optimize Throwing Distance.}
I'd like to throw a ball as far as I can in the $\hat{x}$ direction. At what angle $\phi$ should I ``launch'' the ball to maximize its distance traveled? Use $v_0$ as the magnitude of the ball's initial velocity.
\\
\\
\noindent
\textsc{Hint:}
Use eq \ref{eq:2} to write down the ball's motion in both the $\hat{x} \ \mathrm{and} \ \hat{y}$ direction. Then solve for the ball's time of flight as a function of your launch angle, i.e. $t_f(\phi)$.

\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{\showfamily}

\subsection{Overview \showfamily}

Default: $a \alpha b \beta G \Gamma P \Pi \alpha \beta$

mathnormal: $\mathnormal{a \alpha b \beta G \Gamma P \Pi}$

mathrm: $\mathrm{a \alpha \alphaup b \beta G \Gamma P \Pi}$

mathup: $\mathup{a \alpha \alphaup b \beta G \Gamma P \Pi}$

mathit: $\mathit{a \alpha b \beta G \Gamma P \Pi}$

mathbf: $\mathbf{a \alphaup b \beta G \Gamma P \Pi}$

mathbfit: $\mathbfit{a \alpha b \beta G \Gamma P \Pi}$

\bigskip

{\sffamily
Default: $a \alpha b \beta G \Gamma P \Pi$

mathnormal: $\mathnormal{a \alpha b \beta G \Gamma P \Pi}$

mathrm: $\mathrm{a \alpha b \beta G \Gamma P \Pi}$

mathup: $\mathup{a \alpha b \beta G \Gamma P \Pi}$

mathit: $\mathit{a \alpha b \beta G \Gamma P \Pi}$

mathbf: $\mathbf{a \alpha b \beta G \Gamma P \Pi}$

mathbfit: $\mathbfit{a \alpha b \beta G \Gamma P \Pi}$
}

\bigskip

{\sffamily\bfseries

Default: $a \alpha b \beta G \Gamma P \Pi$

mathnormal: $\mathnormal{a \alpha b \beta G \Gamma P \Pi}$

mathrm: $\mathrm{a \alpha b \beta G \Gamma P \Pi}$

mathup: $\mathup{a \alpha b \beta G \Gamma P \Pi}$

mathit: $\mathit{a \alpha b \beta G \Gamma P \Pi}$

mathbf: $\mathbf{a \alpha b \beta G \Gamma P \Pi}$

mathbfit: $\mathbfit{a \alpha b \beta G \Gamma P \Pi}$
}

\subsection{Formulas \showfamily}

\noindent
\checkgreekletters

\noindent
{\boldmath\checkgreekletters}

\noindent
{\sffamily\selectfont \checkgreekletters}

\noindent
{\sffamily\bfseries\selectfont \checkgreekletters}

\noindent
{\sffamily $\alpha a > 0, \beta b + (3 \times 27), \Gamma G = 7 < 8, \lambda$}

\noindent
$\alpha a > 0, \beta b + (3 \times 27), \Gamma G = 7 < 8, \lambda$

$s \pm 3 \gamma + y - 1 = 4 \times 7$

$$\sum_{i=0}^{N} x^i$$

$$\int_{-\infty}^{\infty} x\,f(x)\,\mathup{d}x = \left( \frac{27}{2} \right)$$

\noindent
{\bfseries $s \pm 3 \gamma + y - 1 \times 7$

$$\sum_{i=0}^{N} x^i$$

$$\int_{-\infty}^{\infty} x\,f(x)\,\mathup{d}x = \left( \frac{27}{2} \right)$$
}

\noindent
{\sffamily $s \pm 3 \gamma + y - 1 \times 7$

$$\sum_{i=0}^{N} x^i$$

$$\int_{-\infty}^{\infty} x\,f(x)\,\mathup{d}x = \left( \frac{27}{2} \right)$$
}

\noindent
{\sffamily\bfseries $s \pm 3 \gamma + y - 1 \times 7$

$$\sum_{i=0}^{N} x^i$$

$$\int_{-\infty}^{\infty} x\,f(x)\,\mathup{d}x = \left( \frac{27}{2} \right)$$
}

\subsection{Math Alphabets \showfamily}

%\sffamily\selectfont

Default
\def\test#1{#1,}
\begin{eqnarray*}
  && {\testnums}\\
  && {\testupper}\\
  && {\testlower}\\
  && {\testupgreek}\\
  && {\testlowgreek}
\end{eqnarray*}%

Math Normal (\texttt{\string\mathnormal})
\def\test#1{\mathnormal{#1},}
\begin{eqnarray*}
  && {\testnums}\\
  && {\testupper}\\
  && {\testlower}\\
  && {\testupgreek}\\
  && {\testlowgreek}
\end{eqnarray*}%

Math Italic (\texttt{\string\mathit})
\def\test#1{\mathit{#1},}
\begin{eqnarray*}
  && {\testnums}\\
  && {\testupper}\\
  && {\testlower}\\
  && {\testupgreek}\\
  && {\testlowgreek}
\end{eqnarray*}%

Math Roman (\texttt{\string\mathrm})
\def\test#1{\mathrm{#1},}
\begin{eqnarray*}
  && {\testnums}\\
  && {\testupper}\\
  && {\testlower}\\
  && {\testupgreek}\\
  && {\testlowgreek}
\end{eqnarray*}%

%Math Italic Bold (\texttt{\string\mathbm})
%\def\test#1{\mathbm{#1},}
%\begin{eqnarray*}
%  && {\testnums}\\
%  && {\testupper}\\
%  && {\testlower}\\
%  && {\testupgreek}\\
%  && {\testlowgreek}
%\end{eqnarray*}%

Math Bold (\texttt{\string\mathbf})
\def\test#1{\mathbf{#1},}
\begin{eqnarray*}
  && {\testnums}\\
  && {\testupper}\\
  && {\testlower}\\
  && {\testupgreek}\\
  && {\testlowgreek}
\end{eqnarray*}%

Caligraphic (\texttt{\string\mathcal})
\def\test#1{\mathcal{#1},}
\begin{eqnarray*}
  && {\testupper}
\end{eqnarray*}%

Script (\texttt{\string\mathscr})
\def\test#1{\mathscr{#1},}
\begin{eqnarray*}
  && {\testupper}
\end{eqnarray*}%

Fraktur (\texttt{\string\mathfrak})
\def\test#1{\mathfrak{#1},}
\begin{eqnarray*}
  && {\testupper}\\
  && {\testlower}
\end{eqnarray*}%

Blackboard Bold (\texttt{\string\mathbb})
\def\test#1{\mathbb{#1},}
\begin{eqnarray*}
  && {\testupper}
\end{eqnarray*}%

\subsection{Character Sidebearings \showfamily}

Default
\def\test#1{|#1|+{}}
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}\\
  && {\testlowgreeki}\\
  && {\testlowgreekii}\\
  && {\testlowgreekiii}
\end{eqnarray*}%

Math Roman (\texttt{\string\mathrm})
\def\test#1{|\mathrm{#1}|+{}}%
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}
\end{eqnarray*}%

%Math Italic Bold (\texttt{\string\mathbm})
%\def\test#1{|\mathbm{#1}|+{}}%
%\begin{eqnarray*}
%  && {\testupperi}\\
%  && {\testupperii}\\
%  && {\testloweri}\\
%  && {\testlowerii}\\
%  && {\testupgreeki}\\
%  && {\testupgreekii}\\
%  && {\testlowgreeki}\\
%  && {\testlowgreekii}\\
%  && {\testlowgreekiii}
%\end{eqnarray*}%

Math Bold (\texttt{\string\mathbf})
\def\test#1{|\mathbf{#1}|+{}}%
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}
\end{eqnarray*}%

Math Calligraphic (\texttt{\string\mathcal})
\def\test#1{|\mathcal{#1}|+{}}%
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}
\end{eqnarray*}%

\subsection{Superscript Positioning \showfamily}

Default
\def\test#1{#1^{2}+{}}%
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}\\
  && {\testlowgreeki}\\
  && {\testlowgreekii}\\
  && {\testlowgreekiii}
\end{eqnarray*}%

Math Roman (\texttt{\string\mathrm})
\def\test#1{\mathrm{#1}^{2}+{}}%
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}
\end{eqnarray*}%

%Math Italic Bold (\texttt{\string\mathbm})
%\def\test#1{\mathbm{#1}^{2}+{}}%
%\begin{eqnarray*}
%  && {\testupperi}\\
%  && {\testupperii}\\
%  && {\testloweri}\\
%  && {\testlowerii}\\
%  && {\testupgreeki}\\
%  && {\testupgreekii}\\
%  && {\testlowgreeki}\\
%  && {\testlowgreekii}\\
%  && {\testlowgreekiii}
%\end{eqnarray*}%

Math Bold (\texttt{\string\mathbf})
\def\test#1{\mathbf{#1}^{2}+{}}%
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}
\end{eqnarray*}

Math Calligraphic (\texttt{\string\mathcal})
\def\test#1{\mathcal{#1}^{2}+{}}%
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}
\end{eqnarray*}%

\subsection{Subscript Positioning \showfamily}

Default
\def\test#1{\mathnormal{#1}_{i}+{}}%
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}\\
  && {\testlowgreeki}\\
  && {\testlowgreekii}\\
  && {\testlowgreekiii}
\end{eqnarray*}%

Math Roman (\texttt{\string\mathrm})
\def\test#1{\mathrm{#1}_{i}+{}}%
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}
\end{eqnarray*}%

%Math Bold Italic (\texttt{\string\mathbm})
%\def\test#1{\mathbm{#1}_{i}+{}}%
%\begin{eqnarray*}
%  && {\testupperi}\\
%  && {\testupperii}\\
%  && {\testloweri}\\
%  && {\testlowerii}\\
%  && {\testupgreeki}\\
%  && {\testupgreekii}\\
%  && {\testlowgreeki}\\
%  && {\testlowgreekii}\\
%  && {\testlowgreekiii}
%\end{eqnarray*}

Math Bold (\texttt{\string\mathbf})
\def\test#1{\mathbf{#1}_{i}+{}}%
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}
\end{eqnarray*}%

Math Calligraphic (\texttt{\string\mathcal})
\def\test#1{\mathcal{#1}_{i}+{}}%
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}
\end{eqnarray*}%

\subsection{Accent Positioning \showfamily}

Default
\def\test#1{\hat{#1}+{}}%
\begin{eqnarray*}
  && {\testnums}\\
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}\\
  && {\testlowgreeki}\\
  && {\testlowgreekii}\\
  && {\testlowgreekiii}
\end{eqnarray*}%

Math Italic (\texttt{\string\mathit})
\def\test#1{\hat{\mathit{#1}}+{}}%
\begin{eqnarray*}
  && {\testnums}\\
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri} \test\ell \test\wp \test\imath \test\jmath \tilde{i} \\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}\\
  && {\testlowgreeki}\\
  && {\testlowgreekii}\\
  && {\testlowgreekiii}
\end{eqnarray*}%

Math Roman (\texttt{\string\mathrm})
\def\test#1{\hat{\mathrm{#1}}+{}}%
\begin{eqnarray*}
  && {\testnums}\\
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}
\end{eqnarray*}%

%Math Italic Bold (\texttt{\string\mathbm})
%\def\test#1{\hat{\mathbm{#1}}+{}}%
%\begin{eqnarray*}
%  && {\testnums}\\
%  && {\testupperi}\\
%  && {\testupperii}\\
%  && {\testloweri}\\
%  && {\testlowerii}\\
%  && {\testupgreeki}\\
%  && {\testupgreekii}\\
%  && {\testlowgreeki}\\
%  && {\testlowgreekii}\\
%  && {\testlowgreekiii}
%\end{eqnarray*}%

Math Bold (\texttt{\string\mathbf})
\def\test#1{\hat{\mathbf{#1}}+{}}%
\begin{eqnarray*}
  && {\testnums}\\
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}
\end{eqnarray*}

Math Calligraphic (\texttt{\string\mathcal})
\def\test#1{\hat{\mathcal{#1}}+{}}%
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}
\end{eqnarray*}%

\subsection{Differentials \showfamily}

\begin{eqnarray*}
\gdef\test#1{\dit #1+{}}%
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}\\
  && {\testlowgreeki}\\
  && {\testlowgreekii}\\
  && {\testlowgreekiii}\\
\gdef\test#1{\dit \mathrm{#1}+{}}%
  && {\testupgreeki}\\
  && {\testupgreekii}
\end{eqnarray*}%

\begin{eqnarray*}
\gdef\test#1{\dup #1+{}}%
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}\\
  && {\testlowgreeki}\\
  && {\testlowgreekii}\\
  && {\testlowgreekiii}\\
\gdef\test#1{\dup \mathrm{#1}+{}}%
  && {\testupgreeki}\\
  && {\testupgreekii}
\end{eqnarray*}%

\begin{eqnarray*}
\gdef\test#1{\partial #1+{}}%
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}\\
  && {\testlowgreeki}\\
  && {\testlowgreekii}\\
  && {\testlowgreekiii}\\
\gdef\test#1{\partial \mathrm{#1}+{}}%
  && {\testupgreeki}\\
  && {\testupgreekii}
\end{eqnarray*}%

\subsection{Slash Kerning \showfamily}

\def\test#1{1/#1+{}}
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}\\
  && {\testlowgreeki}\\
  && {\testlowgreekii}\\
  && {\testlowgreekiii}
\end{eqnarray*}

\def\test#1{#1/2+{}}
\begin{eqnarray*}
  && {\testupperi}\\
  && {\testupperii}\\
  && {\testloweri}\\
  && {\testlowerii}\\
  && {\testupgreeki}\\
  && {\testupgreekii}\\
  && {\testlowgreeki}\\
  && {\testlowgreekii}\\
  && {\testlowgreekiii}
\end{eqnarray*}

\subsection{Big Operators \showfamily}

\def\testop#1{#1_{i=1}^{n} x^{n} \quad}
\begin{displaymath}
  \testop\sum
  \testop\prod
  \testop\coprod
  \testop\int
  \testop\oint
\end{displaymath}
\begin{displaymath}
  \testop\bigotimes
  \testop\bigoplus
  \testop\bigodot
  \testop\bigwedge
  \testop\bigvee
  \testop\biguplus
  \testop\bigcup
  \testop\bigcap
  \testop\bigsqcup
% \testop\bigsqcap
\end{displaymath}

\subsection{Radicals \showfamily}

\begin{displaymath}
  \sqrt{x+y} \qquad \sqrt{x^{2}+y^{2}} \qquad
  \sqrt{x_{i}^{2}+y_{j}^{2}} \qquad
  \sqrt{\left(\frac{\cos x}{2}\right)} \qquad
  \sqrt{\left(\frac{\sin x}{2}\right)}
\end{displaymath}

\begingroup
\delimitershortfall-1pt
\begin{displaymath}
  \sqrt{\sqrt{\sqrt{\sqrt{\sqrt{\sqrt{\sqrt{x+y}}}}}}}
\end{displaymath}
\endgroup % \delimitershortfall

\subsection{Over- and Underbraces \showfamily}

\begin{displaymath}
  \overbrace{x} \quad
  \overbrace{x+y} \quad
  \overbrace{x^{2}+y^{2}} \quad
  \overbrace{x_{i}^{2}+y_{j}^{2}} \quad
  \underbrace{x} \quad
  \underbrace{x+y} \quad
  \underbrace{x_{i}+y_{j}} \quad
  \underbrace{x_{i}^{2}+y_{j}^{2}} \quad
\end{displaymath}

\subsection{Normal and Wide Accents \showfamily}

\begin{displaymath}
  \dot{x} \quad
  \ddot{x} \quad
  \vec{x} \quad
  \bar{x} \quad
  \overline{x} \quad
  \overline{xx} \quad
  \tilde{x} \quad
  \widetilde{x} \quad
  \widetilde{xx} \quad
  \widetilde{xxx} \quad
  \hat{x} \quad
  \widehat{x} \quad
  \widehat{xx} \quad
  \widehat{xxx} \quad
\end{displaymath}

\begin{displaymath}
  \hat{x} \quad
  \check{x} \quad
  \tilde{x} \quad
  \acute{x} \quad
  \grave{x} \quad
  \dot{x} \quad
  \ddot{x} \quad
  \breve{x} \quad
  \bar{x} \quad
  \vec{x} \quad
\end{displaymath}

\subsection{Long Arrows \showfamily}

\begin{displaymath}
  \leftarrow \mathrel{-} \rightarrow \quad
  \leftrightarrow \quad
  \longleftarrow  \quad
  \longrightarrow \quad
  \longleftrightarrow \quad
  \Leftarrow = \Rightarrow \quad
  \Leftrightarrow \quad
  \Longleftarrow  \quad
  \Longrightarrow \quad
  \Longleftrightarrow \quad
\end{displaymath}

\subsection{Left and Right Delimiters \showfamily}

\def\testdelim#1#2{ - #1 f #2 - }
\begin{displaymath}
  \testdelim()
  \testdelim[]
  \testdelim\lfloor\rfloor
  \testdelim\lceil\rceil
  \testdelim\langle\rangle
  \testdelim\{\}
\end{displaymath}

Using {\tt\string\left} and {\tt\string\right}.
\def\testdelim#1#2{ - \left#1 f \right#2 - }
\begin{displaymath}
  \testdelim()
  \testdelim[]
  \testdelim\lfloor\rfloor
  \testdelim\lceil\rceil
  \testdelim\langle\rangle
  \testdelim\{\}
% \testdelim\lgroup\rgroup
% \testdelim\lmoustache\rmoustache
\end{displaymath}
\begin{displaymath}
  \testdelim)(
  \testdelim][
  \testdelim//
  \testdelim\backslash\backslash
  \testdelim/\backslash
  \testdelim\backslash/
\end{displaymath}

\subsection{Big-g-g Delimiters \showfamily}

\def\testdelim#1#2{%
  - \left#1\left#1\left#1\left#1\left#1\left#1\left#1\left#1 -
  \right#2\right#2\right#2\right#2\right#2\right#2\right#2\right#2 -}

\begingroup
\delimitershortfall-1pt
\begin{displaymath}
  \testdelim\lfloor\rfloor
  \qquad
  \testdelim()
\end{displaymath}
\begin{displaymath}
  \testdelim\lceil\rceil
  \qquad
  \testdelim\{\}
\end{displaymath}
\begin{displaymath}
  \testdelim[]
  \qquad
  \testdelim\lgroup\rgroup
\end{displaymath}
\begin{displaymath}
  \testdelim\langle\rangle
  \qquad
  \testdelim\lmoustache\rmoustache
\end{displaymath}
\begin{displaymath}
  \testdelim\uparrow\downarrow \quad
  \testdelim\Uparrow\Downarrow \quad
\end{displaymath}
\endgroup % \delimitershortfall

\def\X#1{$x #1 y$ &\tt\string#1}
\def\Y#1{$\big#1$ &\tt\string#1}
\def\Z#1{$x #1 y$}
\def\W#1#2{$#1{#2}$ &\tt\string#1\string{#2\string}}

\subsection{Binary Operators \showfamily}

\begin{tabular}{*8l}
\X\pm           &\X\cap         &\X\diamond             &\X\oplus     \\
\X\mp           &\X\cup         &\X\bigtriangleup       &\X\ominus    \\
\X\times        &\X\uplus       &\X\bigtriangledown     &\X\otimes    \\
\X\div          &\X\sqcap       &\X\triangleleft        &\X\oslash    \\
\X\ast          &\X\sqcup       &\X\triangleright       &\X\odot      \\
\X\star         &\X\vee         &\X\lhd                 &\X\bigcirc   \\
\X\circ         &\X\wedge       &\X\rhd                 &\X\dagger    \\
\X\bullet       &\X\setminus    &\X\unlhd               &\X\ddagger   \\
\X\cdot         &\X\wr          &\X\unrhd               &\X\S         \\
\X+             &\X-            &\X\amalg               &\X\P
\end{tabular}

\subsection{Relations \showfamily}

\begin{tabular}{*8l}
\X\leq          &\X\geq         &\X\equiv       &\X\models      \\
\X\prec         &\X\succ        &\X\sim         &\X\perp        \\
\X\preceq       &\X\succeq      &\X\simeq       &\X\mid         \\
\X\ll           &\X\gg          &\X\asymp       &\X\parallel    \\
\X\subset       &\X\supset      &\X\approx      &\X\bowtie      \\
\X\subseteq     &\X\supseteq    &\X\cong        &\X\Join        \\
\X\sqsubset     &\X\sqsupset    &\X\neq         &\X\smile       \\
\X\sqsubseteq   &\X\sqsupseteq  &\X\doteq       &\X\frown       \\
\X\in           &\X\ni          &\X\propto      &\X=            \\
\X\vdash        &\X\dashv       &\X<            &\X>            \\
\X:
\end{tabular}

\subsection{Punctuation \showfamily}

\begin{tabular}{*{5}{lp{3.2em}}}
\X,     &\X;    &\X\colon       &\X\ldotp       &\X\cdotp
\end{tabular}

\subsection{Arrows \showfamily}

\begin{tabular}{*6l}
\X\leftarrow            &\X\longleftarrow       &\X\uparrow     \\
\X\Leftarrow            &\X\Longleftarrow       &\X\Uparrow     \\
\X\rightarrow           &\X\longrightarrow      &\X\downarrow   \\
\X\Rightarrow           &\X\Longrightarrow      &\X\Downarrow   \\
\X\leftrightarrow       &\X\longleftrightarrow  &\X\updownarrow \\
\X\Leftrightarrow       &\X\Longleftrightarrow  &\X\Updownarrow \\
\X\mapsto               &\X\longmapsto          &\X\nearrow     \\
\X\hookleftarrow        &\X\hookrightarrow      &\X\searrow     \\
\X\leftharpoonup        &\X\rightharpoonup      &\X\swarrow     \\
\X\leftharpoondown      &\X\rightharpoondown    &\X\nwarrow     \\
\X\rightleftharpoons    &\X\leadsto
\end{tabular}

\subsection{Miscellaneous Symbols \showfamily}

\begin{tabular}{*8l}
\X\ldots        &\X\cdots       &\X\vdots       &\X\ddots       \\
\X\aleph        &\X\prime       &\X\forall      &\X\infty       \\
\X\hbar         &\X\emptyset    &\X\exists      &\X\Box         \\
\X\imath        &\X\nabla       &\X\neg         &\X\Diamond     \\
\X\jmath        &\X\surd        &\X\flat        &\X\triangle    \\
\X\ell          &\X\top         &\X\natural     &\X\clubsuit    \\
\X\wp           &\X\bot         &\X\sharp       &\X\diamondsuit \\
\X\Re           &\X\|           &\X\backslash   &\X\heartsuit   \\
\X\Im           &\X\angle       &\X\partial     &\X\spadesuit   \\
\X\mho          &\X.            &\X|            &\X!
\end{tabular}

\subsection{Variable-Sized Operators \showfamily}

\begin{tabular}{*6l}
\X\sum          &\X\bigcap      &\X\bigodot     \\
\X\prod         &\X\bigcup      &\X\bigotimes   \\
\X\coprod       &\X\bigsqcup    &\X\bigoplus    \\
\X\int          &\X\bigvee      &\X\biguplus    \\
\X\oint         &\X\bigwedge
\end{tabular}

\subsection{Log-Like Operators \showfamily}

\begin{tabular}{*8l}
\Z\arccos &\Z\cos  &\Z\csc &\Z\exp &
           \Z\ker    &\Z\limsup &\Z\min &\Z\sinh \\
\Z\arcsin &\Z\cosh &\Z\deg &\Z\gcd &
           \Z\lg     &\Z\ln     &\Z\Pr  &\Z\sup  \\
\Z\arctan &\Z\cot  &\Z\det &\Z\hom &
           \Z\lim    &\Z\log    &\Z\sec &\Z\tan  \\
\Z\arg    &\Z\coth &\Z\dim &\Z\inf &
           \Z\liminf &\Z\max    &\Z\sin &\Z\tanh
\end{tabular}

\subsection{Delimiters \showfamily}

\begin{tabular}{*8l}
\X(             &\X)            &\X\uparrow     &\X\Uparrow     \\
\X[             &\X]            &\X\downarrow   &\X\Downarrow   \\
\X\{            &\X\}           &\X\updownarrow &\X\Updownarrow \\
\X\lfloor       &\X\rfloor      &\X\lceil       &\X\rceil       \\
\X\langle       &\X\rangle      &\X/            &\X\backslash   \\
\X|             &\X\|
\end{tabular}

\subsection{Large Delimiters \showfamily}

\begin{tabular}{*8l}
\Y\rmoustache&  \Y\lmoustache&  \Y\rgroup&      \Y\lgroup\\[5pt]
\Y\arrowvert&   \Y\Arrowvert&   \Y\bracevert
\end{tabular}

\subsection{Math Mode Accents \showfamily}

\begin{tabular}{*{10}l}
\W\hat{a}     &\W\acute{a}  &\W\bar{a}    &\W\dot{a}    &\W\breve{a}\\
\W\check{a}   &\W\grave{a}  &\W\vec{a}    &\W\ddot{a}   &\W\tilde{a}\\
\end{tabular}

\subsection{Miscellaneous Constructions \showfamily}

\begin{tabular}{*4l}
\W\widetilde{abc}       &\W\widehat{abc}                        \\
\W\overleftarrow{abc}   &\W\overrightarrow{abc}                 \\
\W\overline{abc}        &\W\underline{abc}                      \\
\W\overbrace{abc}       &\W\underbrace{abc}                     \\[5pt]
\W\sqrt{abc}            &$\sqrt[n]{abc}$&\verb|\sqrt[n]{abc}|   \\
$f'$&\verb|f'|          &$\frac{abc}{xyz}$&\verb|\frac{abc}{xyz}|
\end{tabular}

\subsection{AMS Delimiters \showfamily}

\begin{tabular}{*8l}
\X\ulcorner&\X\urcorner&\X\llcorner&\X\lrcorner
\end{tabular}

\subsection{AMS Arrows \showfamily}

\begin{tabular}{*8l}
\X\dashrightarrow       &\X\dashleftarrow
        \\ \X\leftleftarrows      &\X\leftrightarrows     \\
\X\Lleftarrow           &\X\twoheadleftarrow
        \\ \X\leftarrowtail       &\X\looparrowleft       \\
\X\leftrightharpoons    &\X\curvearrowleft
        \\ \X\circlearrowleft     &\X\Lsh                 \\
\X\upuparrows           &\X\upharpoonleft
        \\ \X\downharpoonleft     &\X\multimap            \\
\X\leftrightsquigarrow  &\X\rightrightarrows
        \\ \X\rightleftarrows     &\X\rightrightarrows    \\
\X\rightleftarrows      &\X\twoheadrightarrow
        \\ \X\rightarrowtail      &\X\looparrowright      \\
\X\rightleftharpoons    &\X\curvearrowright
        \\ \X\circlearrowright    &\X\Rsh                 \\
\X\downdownarrows       &\X\upharpoonright
        \\ \X\downharpoonright    &\X\rightsquigarrow
\end{tabular}

\subsection{AMS Negated Arrows \showfamily}

\begin{tabular}{*8l}
\X\nleftarrow   &\X\nrightarrow \\ \X\nLeftarrow  &\X\nRightarrow \\
\X\nleftrightarrow&\X\nLeftrightarrow
\end{tabular}

\subsection{AMS Greek \showfamily}

\begin{tabular}{*4l}
\X\digamma      &\X\varkappa
\end{tabular}

\subsection{AMS Hebrew \showfamily}

\begin{tabular}{*6l}
\X\beth &\X\daleth      &\X\gimel
\end{tabular}

\subsection{AMS Miscellaneous \showfamily}

\begin{tabular}{*8l}
\X\hbar         &\X\hslash      \\ \X\vartriangle &\X\triangledown      \\
\X\square       &\X\lozenge     \\ \X\circledS    &\X\angle             \\
\X\measuredangle&\X\nexists     \\ \X\mho         &\X\Finv$^u$          \\
\X\Game$^u$     &\X\Bbbk$^u$    \\ \X\backprime   &\X\varnothing        \\
\X\blacktriangle&\X\blacktriangledown \\ \X\blacksquare&\X\blacklozenge  \\
\X\bigstar      &\X\sphericalangle     \\ \X\complement  &\X\eth       \\
\X\diagup$^u$   &\X\diagdown$^u$
\end{tabular}

$^u$ Not defined in {\tt amssymb.sty}, define using the
\verb|\newsymbol|  command.

\subsection{AMS Binary Operators \showfamily}

\begin{tabular}{*8l}
\X\dotplus      &\X\smallsetminus \\ \X\Cap        &\X\Cup               \\
\X\barwedge     &\X\veebar      \\ \X\doublebarwedge&\X\boxminus        \\
\X\boxtimes     &\X\boxdot      \\ \X\boxplus     &\X\divideontimes     \\
\X\ltimes       &\X\rtimes      \\ \X\leftthreetimes&\X\rightthreetimes \\
\X\curlywedge   &\X\curlyvee    \\ \X\circleddash &\X\circledast        \\
\X\circledcirc  &\X\centerdot   \\ \X\intercal
\end{tabular}

\subsection{AMS Relations \showfamily}

\begin{tabular}{*2l}
\X\leqslant    \\\X\lesssim    \\
\X\approxeq    \\\X\lll        \\
\X\lesseqgtr   \\\X\doteqdot   \\
\X\fallingdotseq\\\X\backsimeq  \\
\X\Subset      \\\X\preccurlyeq\\
\X\precsim     \\\X\vartriangleleft\\
\X\vDash      \\\X\smallsmile \\
\X\bumpeq      \\\X\geqq       \\
\X\eqslantgtr  \\\X\gtrapprox  \\
\X\ggg         \\\X\gtreqless  \\
\X\eqcirc      \\\X\triangleq  \\
\X\thickapprox \\\X\Supset     \\
\X\succcurlyeq \\\X\succsim    \\
\X\vartriangleright\\\X\Vdash      \\
\X\shortparallel\\\X\pitchfork  \\
\X\blacktriangleleft \\\X\backepsilon\\
\X\because
\end{tabular}

\subsection{AMS Negated Relations \showfamily}

\begin{tabular}{*8l}
\X\nless        &\X\nleq        \\ \X\nleqslant   &\X\nleqq       \\
\X\lneq         &\X\lneqq       \\ \X\lvertneqq   &\X\lnsim       \\
\X\lnapprox     &\X\nprec       \\ \X\npreceq     &\X\precnsim    \\
\X\precnapprox  &\X\nsim        \\ \X\nshortmid   &\X\nmid        \\
\X\nvdash       &\X\nvDash      \\ \X\ntriangleleft&\X\ntrianglelefteq\\
\X\nsubseteq    &\X\subsetneq   \\ \X\varsubsetneq&\X\subsetneqq  \\
\X\varsubsetneqq&\X\ngtr        \\ \X\ngeq        &\X\ngeqslant   \\
\X\ngeqq        &\X\gneq        \\ \X\gneqq       &\X\gvertneqq   \\
\X\gnsim        &\X\gnapprox    \\ \X\nsucc       &\X\nsucceq     \\
\X\nsucceqq     &\X\succnsim    \\ \X\succnapprox &\X\ncong       \\
\X\nshortparallel&\X\nparallel  \\ \X\nvDash      &\X\nVDash      \\
\X\ntriangleright&\X\ntrianglerighteq \\ \X\nsupseteq&\X\nsupseteqq\\
\X\supsetneq    &\X\varsupsetneq \\ \X\supsetneqq  &\X\varsupsetneqq
\end{tabular}

}


\section{Entropy}