%\documentclass[english,onecolumn]{IEEEtran}
\documentclass[a4paper]{scrartcl}
\usepackage[utf8]{inputenc} \usepackage[T1]{fontenc}
\usepackage{lmodern,textcomp}
\usepackage[english]{babel}
\usepackage[vmargin=3cm,hmargin=3cm]{geometry}
\usepackage[toc,page]{appendix}
%\usepackage{times}
%\usepackage{array}
%\usepackage{multirow}
\usepackage{float}
\usepackage{url}
%\usepackage{amsthm}
\usepackage{amsmath}
\usepackage{amssymb}
%\usepackage{graphicx}
\usepackage{color}
%\usepackage{caption}
\usepackage[style=numeric,backend=biber]{biblatex} \bibliography{refs}
\usepackage[colorlinks=true,allcolors=blue]{hyperref}

\begin{document}

\title{Graph Signal Processing Summer School}
\subtitle{Proposal for a research school at CIRM during the first semester 2018}
\author{
	Michaël Defferrard \and
	Nathanaël Perraudin \and
	Yann Schoenenberger \and
	Dorina Thanou
	%Lionel Martin \and
	%Johan Paratte
}
\maketitle

\begin{abstract}
	Proposal for a Summer School on Graph Signal Processing to be held in 2018
	at CIRM in Marseille, France.
\end{abstract}

\section{Summary}

The proposed Summer School aims to provide PhD students and researchers with
insights on advanced tools and methodologies in Graph Signal Processing (GSP)
research. The goal is to disseminate state-of-the-art research about signal
processing on graphs as well as discussing and defining new directions both on
the theoretical developments of the GSP framework and on its applications to
real-world science and engineering problems.
% Two levels
There will be a set of introductory level lectures for those interested to apply
GSP tools to their research or scientific problems as well as a set of advanced
topics geared towards researchers in the field. The exact topics have yet to be
discussed with potential speakers. They will offer a mix of theoretical and
applied talks.

Summer school participants will have the opportunity to learn and study
innovative algorithms and systems in the signal processing field in an
interactive and stimulating way. Each day, after the lectures, the speakers will
be available for face-to-face discussion with participants who are interested in
strengthen their knowledge of the covered material. Students will have the
possibility to present their work to let other participants and the lecturers
know about their research activity. There will be free time for students to
discuss, exchange ideas, brainstorm or engage in debates.

The first edition of this school, previously named \textit{Key Insights on
Networks and Graphs}\footnote{\url{http://lts2.epfl.ch/summerschool}}, was
held in Leukerbad, Switzerland, in June 2015.

\section{Résumé}

L'école d'été proposée a pour objectif de donner aux doctorants un aperçu des
outils et méthodologies en Traitement du Signal sur Graphe (TSG). Le but est de
disséminer l'état de l'art en TSG ainsi que de discuter et définir de nouvelles
directions tant sur le développement théorique des fondements du TSG que sur ses
applications aux problèmes scientifiques et d'ingénieries. Il y aura un ensemble
de cours introductifs pour ceux qui sont intéressés à l'application des outils
du TSG à leurs recherches ou à leurs problèmes ainsi qu'un ensemble de cours
avancés orientés vers les chercheurs du domaine. Les thèmes abordés doivent
encore être discutés avec les conférenciers potentiels. Ils proposeront un
mélange de sujets théoriques et appliqués.

Les participants à cette école d'été auront l'opportunité d'apprendre et
d'étudier des algorithmes et systèmes innovants en traitement du signal de
manière interactive et stimulante. Chaque jour, après les cours, les
conférenciers seront disponibles pour discuter avec les participants intéressés
à consolider leurs connaissances de la matière abordée. Les doctorants auront la
possibilité de présenter leur travail afin que tant leurs collègues que les
conférenciers soient au courant de leurs activités de recherche. Du temps libre
sera aménagé pour que les doctorants puissent discuter, échanger des idées ou
s'engager dans des débats.

La première édition de cette école, précédemment nommée \textit{Key Insights on
Networks and Graphs}\footnote{\url{http://lts2.epfl.ch/summerschool}}, a eu lieu
à Loèche-les-Bains, en Suisse, en juin 2015.

\section{Scientific content}

% Graphs == avenir du futur
With the advent of Big Data, an ever increasing amount of raw data is becoming
available and advanced models are needed to extract meaningful information from
data and identify relational structures that facilitate different data analysis
tasks such as recognition or inference. Graph Signal Processing (GSP) enables
the analysis and processing of signals that reside on irregular (non-Euclidean)
domains, such as infrastructure or social networks. Graphs may further be used
to incorporate external information into data models, such as hyperlinks or
references between documents, and to capture the underlying geometrically
complex manifold structure of the data.

\paragraph{Graphs and Signal Processing.}
% Graphs
Graphs are generic data representation forms that are useful for describing the
geometric structures of data domains in numerous applications, including social,
energy, transportation, sensor, and neuronal networks.
% Weights
The weight associated with each edge in the graph often represents the
similarity between the two vertices it connects. The connectivities and edge
weights are either dictated by the physics of the problem at hand or inferred
from the data. For instance, the edge weight may be inversely proportional to
the physical distance between nodes in some network.
% Signals
The data on these graphs can be visualized as a finite collection of samples,
with one sample at each vertex in the graph. Collectively, we refer to these
samples as a graph signal.

% GSP
The purpose of GSP is to exploit the underlying structure to analyze, process
and learn signals residing on graphs.
%Examples from the classical signal processing toolbox
Classical GSP tasks include, among others, filtering, denoising, inpainting and
compressing. An important consideration when designing such tools is their
computational complexity, which can often be improved by spectral relaxations or
numerical approximations. Efficiency is key to cope with the ever growing size
of datasets.

% summer school on both applications and theoretical advancements

\paragraph{Graphs as data models.}
% Feature vs data graph
There exists two paradigms to model the underlying structure of datasets using
graphs: 1) constructing a \textit{feature graph}, where nodes represent features
and samples are graph signals, or 2) a \textit{data graph}, where nodes
represent samples and graph signals are functions of the data. We then identify
two regimes: 1) when the graph structure is given by the application, and 2)
when the graph has to be constructed from features or data. Below we elaborate
on three of those four combinations.

% math: g(L) to filter, Dirichlet regularization
% Applications: given feature graph
Examples of applications where a feature graph structure is given are found in
many different engineering and science fields. In transportation networks, we
may be interested in analyzing epidemiological data describing the spread of
diseases, census data describing human migration patterns, or logistics data
describing inventories of trade goods. Other examples include gene expression
patterns defined on top of gene networks, the congestion level at the nodes of a
telecommunication network, weather data measured by a network of sensors and
patterns of brain activity defined on top of a brain network.

% Applications: constructed feature graph
% No convincing examples.

% Applications: constructed data graph, e.g. graph cuts
In statistical learning problems, GSP is a meaningful paradigm even when the
graph structure is not given by the problem but might be inferred from data.  In
such cases, the graph is a discrete approximation of a manifold embedded in an
Euclidean space: each node represents a sample and the weights represent
similarities between samples. Signals on those graphs may be the output of a
classification or regression model which we could constrain to be smooth or
piece-wise constant on the data graph. Applications include machine vision (e.g.
semi-local graphs between pixels) and automatic text classification.
Graph-based methods are especially popular for the (transductive or inductive)
semi-supervised learning problem where the objective is to classify unknown data
with the help of a few labeled samples and a large pool of unlabeled training
samples.

% Applications: given data graph
As an example of intrinsic data graph, instead of defining weights as
similarities between vector representations of documents (from e.g. the
bag-of-word model), external information might be incorporated by defining
binary weights as hyperlinks or references between documents.

\paragraph{Theoretical understanding.}
Besides particular applications, the summer school will also showcase the
advancement of the understanding of network data by redesigning traditional
tools originally conceived to study signals defined on regular domains (such as
time-varying signals or spatially varying images and fields) and extending them
to analyze signals on the more complex graph domain.

The irregular nature of the modeled structures often makes classical signal
processing concepts only partially true. The main similarities and differences
between classical signal processing and GSP will be presented.

Examples of theoretical topics to be addressed include graph fundamental limits
such as sampling theorems and uncertainty principles, convergence results
establishing the link between the discrete nature of graphs and the continuous
world of manifolds, as well as the theoretical foundations of different graph
transforms.
% embedding, filter design

\paragraph{Relevance.}
The last few years have seen significant progress in the development of theory,
tools, and applications of GSP. This school is intended to disseminate ideas to
a broader audience and to exchange ideas and experiences on the future path of
this emerging field. As the interest grows, a summer
school\footnote{\url{http://lts2.epfl.ch/summerschool}} has been organized in
2015 in Switzerland and a
workshop\footnote{\url{http://alliance.seas.upenn.edu/~gsp16/wiki}} will take
place this year in the US.

Last year, Pascal Frossard got invited to give a plenary talk in the Sampling
Theory and Applications (SampTA)
conference\footnote{\url{http://www.american.edu/cas/mathstat/sampta2015}} as
well as an overview talk about multimedia graph-based processing at the
Conference on Multimedia and Expo
(ICME)\footnote{\url{http://www.icme2015.ieee-icme.org}}, which shows the
interest of the multimedia community in GSP. While mainly focusing on complex
and dynamical networks, the \textit{Network Science Thematic
Semester}\footnote{\url{http://project.inria.fr/netspringlyon}}, which
comprises two workshops and a conference that will take place in May and June
2016 in Lyon and Marseille, in France, features some keynotes and presentations
from the GSP community, notably with the interventions of Pierre Vandergheynst,
José M.F. Moura and Alejandro Ribeiro. The IEEE Image, Video, and
Multidimensional Signal Processing (IVMSP) workshop
2016\footnote{\url{http://www.ivmsp2016.org}}, focused on image and video
processing, mentions GSP and manifold modeling in its call for papers and Pierre
Vandergheynst will give a plenary talk on signal and image processing on graphs.

% Our summer school
\paragraph{Selected topics.}
Please refer to published review papers \cite{shuman_emerging_2013,
sandryhaila_discrete_2014, coifman_diffusion_2006, ekambaram_circulant_2013} for
a broad technical overview as well as a discussion of the challenges in GSP.
Below is a selection of applicable topics with selected (recent) references,
which we hope help to demonstrate the increasing popularity of GSP. The
selection of topics to be presented will be discussed with the speakers and will
depend on their interests.
\begin{itemize}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	\item Theoretical foundations and convergence results
		\cite{belkin2007convergence, von2008consistency}
	\item Graph spectral theory
		\cite{chung1997spectral, belkin2007convergence, shuman2015vertex,
		sandryhaila2014discrete, Nakatsukasa2013mysteries, brooks, dekel}
	\item Sampling and recovery of graph signals
		\cite{puy2015random, tsitsvero2015signals, anis2014towards}
	\item Graph filter and filter bank design
		\cite{hammond2011wavelets, narang_bior_filters, ekambaram_globalsip,
		leonardi_multislice, leonardi_fmri, shuman2013spectrum}
	\item Uncertainty principles and other fundamental limits
		\cite{perraudin2016global, agaskar_spie, pasdeloup}
	\item Graph coarsening, multi-scale graph signal processing
		\cite{spielman2011graph, shuman2016multiscale, liu_coarsening,
		lafon_coarse, jansen}
	\item Graph signal transforms
		\cite{szlam,gavish, Maggioni_biorthogonal, coifman_diffusion_2006,
		lafon_coarse, narang_lifting_graphs}
	\item Graph filter identification
		\cite{thanou_TSP_2014}
	\item Graph construction and graph topology inference
		\cite{kalofolias2016learn, Dong:2014tj}
	\item Learning graph embeddings
		\cite{von2008consistency, zhou2006learning, belkin2003laplacian}
%	\item Prediction and learning in graphs
	\item Statistical graph signal processing
		\cite{perraudin2016stationary, zhang2015graph, gadde2015probabilistic}
%	\item Signals in high-order graphs
%	\item Non-linear graph signal processing
	\item Graph-based image and video processing
		\cite{elmoataz2008nonlocal, lezoray2008nonlocal, shahid2015fast}
	\item Graph-based 3D multimedia processing
		\cite{zhang2014point, thanou2015graph, nguyen2014compression,
		schoenenberger2015graph}
	\item Machine learning with graph regularization
		\cite{smola, belkin2005towards, zhou2004regularization,
		zhou2005learning, belkin2006manifold}
%	\item Extracting features from graph signals
	\item Applications to neuroscience and other medical fields
		\cite{huang2015graph, hu2015spectral}
	\item Applications to economics and social networks
	% Nath: I do not have any references for this one.
	\item Applications to infrastructure networks such as communication,
		transportation, power networks
		\cite{mcgraw, jain2014big, borgnat2011shared}
\end{itemize}

\section{Speakers}

We plan to invite five speakers to give lectures. We expect one of them, the
first to present on Monday, to introduce the field and give a motivational talk.
The purpose of this overview talk is to help novice attendants to follow the
more advanced talks. Speakers are selected such that all of those three
main tracks and possibly a fourth side track are covered:

\begin{enumerate}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	\item Track 1: Introduction to GSP
	\item Track 2: Foundations of GSP
	\item Track 3: Recent innovation and applications of GSP
	\item Side track: Optimization
\end{enumerate}

As there is a strong optimization community in France which could easily travel
to CIRM for a talk, we envision to introduce a side track about optimization,
which is quite relevant to GSP. We'll discuss with those speakers on how to
adapt their talks to our audience. We are very enthusiastic about the impact
this exchange could have. We however won't contact those speakers yet, as it may
not happen should this proposal be rejected (because the event may then be
located in another place with different local opportunities).

Below is the list of speakers we contacted to lecture at this school. As we
don't know the date and place of the school, we didn't ask them to confirm their
attendance yet.
% While some have already pre-confirmed their attendance, other confirmations
% are pending.
\begin{itemize}
\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
\item Track 1
	\begin{itemize}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	\item \href{http://people.epfl.ch/pierre.vandergheynst}
		{Pierre Vandergheynst}, EPFL, Switzerland
		% Signal processing on graphs
	\end{itemize}
\item Track 2
	\begin{itemize}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	\item \href{http://www.wsi.uni-tuebingen.de/lehrstuehle/theory-of-machine-learning/people/ulrike-von-luxburg.html}
		{Ulrike von Luxburg}, University of Tübingen, Germany
		% Nathanaël: spectral clustering
	\item \href{http://www.cs.yale.edu/homes/spielman/}
		{Daniel Spielman}, Yale University, USA
		% Graph partitioning, sparsification
		% Mauro: absolutely fantastic but unlikely to come
	\end{itemize}
\item Track 3
	\begin{itemize}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	\item \href{http://cs.stanford.edu/people/jure/}
		{Jure Leskovec}, Stanford University, USA
	\item \href{http://people.irisa.fr/Remi.Gribonval/}
		{Rémi Gribonval}, INRIA Rennes, France
		% Mauro: make connections to dictionary learning
	\end{itemize}
\end{itemize}

In case the aforementioned speakers cannot come, below is another list (in no
particular order) of potential replacements. The diversity of this school is again
reflected in the broad range of considered speakers. Given this large list, we
are pretty confident that enough of them will be able to attend our event, given
how much time is left to arrange their attendance.
\begin{itemize}
\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
\item Track 1
	\begin{itemize}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	\item \href{http://users.ece.cmu.edu/~moura/}
		{José M. F. Moura}, Carnegie Mellon University, USA
		% GSP2016
	\end{itemize}
\item Track 2
	\begin{itemize}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	\item \href{http://web.cse.ohio-state.edu/~mbelkin/}
		{Mikhail Belkin}, Ohio State University, USA
		% Graphs in machine and human learning
	\item \href{http://math.ucsd.edu/~fan/}
		{Fan Chung Graham}, UC San Diego, USA
		% Graph theory in the information age
	\end{itemize}
\item Track 3
	\begin{itemize}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	\item \href{http://ee.usc.edu/faculty_staff/faculty_directory/ortega.htm}
		{Antonio Ortega}, University of Southern California, USA
		% GSP2016, Mauro: could talk a lot about a variety of applications
		% Mauro: would be a very good speaker
		% Mauro: track 3 could be more appropriate for him than track 1
	\item \href{http://services.math.duke.edu/~mauro/}
		{Mauro Maggioni}, Duke University, USA
		% KING2015
	\item \href{http://elmoatazbill.users.greyc.fr/}
		{Abderrahim Elmoataz}, University of Caen Basse Normandie, France
		% suggested by Pascal
	\item \href{http://people.cs.uchicago.edu/~risi/}
		{Risi Kondor}, University of Chicago, USA
		% Mauro: multi-scale constructions in general
	\item \href{http://www.ece.mcgill.ca/~mrabba1/}
		{Michael Rabbat}, McGill University, Canada
		% GSP2016
	\item \href{http://perso.ens-lyon.fr/pierre.borgnat/}
		{Pierre Borgnat}, ENS Lyon / CNRS, France
		% suggested by Pascal
	\item \href{http://perso.ens-lyon.fr/patrick.flandrin/}
		{Patrick Flandrin}, ENS Lyon / CNRS, France
		% suggested by Pascal
	\item \href{http://infocom.uniroma1.it/sergio/}
		{Sergio Barbarossa}, Sapienza University of Rome, Italy
	\item \href{http://research.microsoft.com/en-us/people/pachou/}
		{Philip A. Chou}, Microsoft Research, USA
	\item \href{http://www.math.ucdavis.edu/~saito/}
		{Naoky Saito}, UC Davis, USA
	\item \href{http://alliance.seas.upenn.edu/~aribeiro/wiki/}
		{Alejandro Ribeiro}, University of Pennsylvania, USA
		% GSP2016
	\item \href{http://cpsc.yale.edu/people/ronald-coifman}
		{Ronald Coifman}, Yale University, USA
		% Xavier
	\item \href{http://www.commsp.ee.ic.ac.uk/~pld/}
		{Pier Luigi Dragotti}, Imperial College London, Great Britain
		% suggested by Mauro
	\item \href{http://jelena.ece.cmu.edu/}
		{Jelena Kovačević}, Carnegie Mellon University, USA
		% suggested by Mauro, GSP2016
		% Mauro: applications to biology
	\item \href{http://people.epfl.ch/pascal.frossard}
		{Pascal Frossard}, EPFL, Switzerland
		% suggested by Mauro, GSP2016
	\item \href{http://www.cpt.univ-mrs.fr/~barrat/english.html}
		{Alain Barrat}, Université Aix-Marseille / CNRS, France
		% Statistical physics, Benjamin peut le contacter
	\item \href{http://www.cs.cornell.edu/home/kleinber/}
		{Jon Kleinberg}, Cornell University, USA
%	\item \href{http://perso.ens-lyon.fr/nelly.pustelnik/}
%		{Nelly Pustelnik}, ENS Lyon / CNRS, France
		% Pascal: elle fait de l'optimization, récemment sur les graphes
	\end{itemize}
\item Side track
	\begin{itemize}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	\item \href{http://www.di.ens.fr/~fbach/}
		{Francis Bach}, INRIA Paris, France
		% suggested by Michaël
	\item \href{http://www.ljll.math.upmc.fr/~plc/}
		{Patrick Louis Combettes}, Université Pierre et Marie Curie, France
		% suggested by Nath
	\item \href{http://www-syscom.univ-mlv.fr/~pesquet/}
		{Jean-Christophe Pesquet}, Université Paris-Est, France
		% suggested by Nath, Pascal
	\item \href{http://fadili.users.greyc.fr/}
		{Jalal Fadili}, ENSI Caen, France
		% suggested by Pascal
	\item \href{http://gpeyre.github.io/}
		{Gabriel Peyré}, Université Paris-Dauphine / CNRS, France
		% suggested by Mauro
	\end{itemize}
\end{itemize}

\section{Participants}

This summer school targets an international audience, with a diverse set of
speakers and potentially interested research labs. It will gather people using
similar tools and techniques in diverse fields such as signal processing, graph
analysis and machine learning. If the proposal is accepted, we will activate our
networks and notify all the interested parties we know about. In addition to
the laboratories of the potential speakers, we expect the students of the
following groups to be interested, some of which we have collaborated with in
the past.
\begin{itemize}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	\item \href{http://www.eecs.berkeley.edu/~kannanr/}
		{Kannan Ramchandran}, UC Berkeley, USA
	\item \href{http://www.vincent-gripon.com/}
		{Vincent Gripon}, Télécom Bretagne, France
	\item \href{http://www.macalester.edu/~dshuman1/}
		{David Shuman}, Macalester College, USA
	\item \href{http://www.inf.usi.ch/bronstein/}
		{Michael M. Bronstein}, USI, Switzerland
	\item \href{http://camille.roth.free.fr/index.php}
		{Camille Roth}, Humboldt Universität / CNRS, Germany
	\item \href{http://xn.unamur.be/}
		{Renaud Lambiotte}, University of Namur, Belgium
	\item \href{http://web.eecs.umich.edu/~hero/}
		{Alfred Hero}, University of Michigan, USA
	\item \href{http://www.epicx-lab.com/vittoria-colizza.html}
		{Vittoria Colizza}, Inserm, France
	\item \href{http://people.epfl.ch/patrick.thiran}
		{Patrick Thiran}, EPFL, Switzerland
	\item \href{http://icapeople.epfl.ch/grossglauser/}
		{Matthias Grossglauser}, EPFL, Switzerland
\end{itemize}

We however do not expect more than 40 participants, (the last edition attracted
32 students), such that we propose to co-locate it with the \textit{Dynamique des
Systèmes Biologiques} summer school, whose organizers expect around 40
participants too. As such we chose the same dates and will limit the number of
attendees to 40. That way, all participants from the two schools can be
accommodated at CIRM. The lectures from the two events will take place in two
different auditoriums, namely at CIRM and CPPM\footnote{Confirmed by Bruno
Torrésani.}. While the scientific part will be organized independently, we plan
to jointly organize the social events (lunch, dinner, excursion) during which
students may exchange and eventually develop future collaborations.

\section{Others}

\subsection{Organization}

This school is to be organized by:
\begin{itemize}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	\item Michaël Defferrard, PhD student, EPFL, Switzerland
	\item Nathanaël Perraudin, PhD student, EPFL, Switzerland
	\item Yann Schoenenberger, PhD student, EPFL, Switzerland
	\item Dorina Thanou, scientist, EPFL, Switzerland
\end{itemize}
and the scientific committee is composed of:
\begin{itemize}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	%\item Pierre Vandergheynst, full professor, EPFL, Switzerland
	\item Pascal Frossard, associate professor, EPFL, Switzerland
	\item Bruno Torrésani, full professor, Aix-Marseille University, France
	\item Mauro Maggioni, full professor, Duke University, USA
	%\item Clothilde Mélot, maître de conférence, Aix-Marseille University, France
\end{itemize}

We want to thank Yannick Boursier, Lionel Martin, Johan Paratte, Benjamin Ricaud
and Xavier Bresson who helped us to prepare this proposal and will continue to
provide support should it be retained.

\subsection{Gender parity analysis}

As in many other engineering fields, there is an unfortunate imbalance between
the number of men and women in the GSP community. To the best of our knowledge,
Fan Chung Graham, Ulrike von Luxburg and Jelena Kovačević are the only women
professors in the field. All three rank high in our priority list and we would
be honored to have such distinguished speakers at the summer school.

As for the participants we expect to be representative of the field. At EPFL the
gender distribution is 10 women and 25 men in the labs of Prof.
Frossard\footnote{\url{http://lts4.epfl.ch}} and Prof.
Vandergheynst\footnote{\url{http://lts2.epfl.ch}}.

\subsection{ECTS credits}

We suggest to grant 2 ECTS points to Ph.D. students who participate in the
school. To that end, a certificate of participation will be delivered at the end
of the week; it is then up to their home institutions to choose whether this
suggestion should be followed.

\subsection{Location}

With the idea that nice places can help open the mind, we would be delighted to
host this event in the beautiful coastal area of Marseille. It would further be
in contrast to the previous edition which was hosted in Leukerbad, a small Swiss
town up in a valley in the Alps. As Leukerbad, Marseille is a place which is
quiet for the mind and relaxing for the body.
In addition to its intrinsic attractiveness as a touristic city, Marseille is
easily accessible by plane, which is important for our international audience.
Its closeness to the French Riviera and Paris makes it an even more attractive
place for overseas speakers and participants.

\subsection{Schedule}

The school will last five days, from Monday morning to Friday afternoon. Rooms
will be available from Sunday evening to Friday morning. Participants will have
to arrive and install themselves at CIRM on Sunday as lectures will start Monday
morning. We plan to finish early on Friday and to offer an excursion on
Wednesday afternoon, which leaves us with four days for lectures. Following the
schedule shown in Table~\ref{schedule}, we have room for 24 lessons of 45
minutes. Each of the five main track speakers will be allocated four lessons and
each side track speaker will be allocated one lesson. Exact allocations will be
discussed with the speakers.

Attendees will be offered the opportunity to present their work. Interested
students will have to provide an abstract. The scientific committee will choose
the most relevant ones, which will be allocated a 20 to 30 minutes time slot at
the end of a day. The participation to these presentations is optional.
Additionally, we schedule some free time after lunch to discuss and exchange
ideas about future developments of the field. Each day, we may propose a
different topic for students to brainstorm and discuss.

Depending on funding opportunities and the enthusiasm of the community, notably
at GSP2016\footnote{Workshop to be held in May 2016 in Pennsylvania, USA.}, this
event may grow in size (in terms of speakers and attendants) and become a mix of
long tutorial lectures and short lectures. In that case, student presentations
may be morphed into a poster session, but we definitely want students to be able
to present their work.

\begin{table}[ht]
	\centering
	\begin{tabular}{rcrl}
	 8:00 & - &  8:45 & Breakfast \\
	 9:00 & - &  9:45 & Lesson 1 \\
	10:00 & - & 10:45 & Lesson 2 \\
	10:45 & - & 11:15 & Coffee break \\
	11:15 & - & 12:00 & Lesson 3 \\
	12:15 & - & 13:15 & Lunch \\
	13:30 & - & 14:45 & Free time, discussions \\
	15:00 & - & 15:45 & Lesson 4 \\
	16:00 & - & 16:45 & Lesson 5 \\
	16:45 & - & 17:15 & Coffee break \\
	17:15 & - & 18:00 & Lesson 6 \\
	18:30 & - & 19:30 & Student presentations \\
	20:00 &   &       & Dinner and social events
	\end{tabular}
	\caption{Schedule of a typical day.}
	\label{schedule}
\end{table}

\subsection{Dates}

% As this event is targeted at PhD students, who usually take courses and have
% teaching duties, as well as professors who often give lectures in their home
% university, we find it best to organize the school after the spring
% academic semester.
As we aim to co-locate with another summer school, we jointly propose the
following dates (it may be in a different preference order):
\begin{enumerate}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	\item June 5th
	\item May 15th
	\item April 23rd
\end{enumerate}

\subsection{Budget}

The CIRM subsidizes meals and accommodations (in two beds rooms) for 40
participants. As we plan to co-locate with another school, the CIRM would
provide half of his funding to us while the other half will go to the other
event. This will thus cover 20 of our participants. The CIRM package is offered
at 88.5€ per day, which accounts for $88.5 \cdot 5 = 442.50$€ per participant.

Invited speakers will be provided with meals and an individual room at CIRM. A
double room or a flat rate will be offered to those who prefer to self-organize,
e.g. for family reasons. The CIRM prices are: 61.80€ per night for a single room
including breakfast, 15.45€ per meal. We thus budget $61.80 \cdot 5 + 15.45
\cdot 10 = 465.5$€ per speaker. We plan to defray their travel expenses. While
the exact amount will depend on the available funding, we budget 200€ for EU
speakers and 700€ for US speakers

If this proposal is accepted, we will request additional funding to local
communities in the CIRM area\footnote{Coordinated and estimated by Bruno
Torrésani.} and to the Seasonal Schools in Signal Processing (S3P)
Program\footnote{\url{http://signalprocessingsociety.org/seasonal-schools}} of
the IEEE Signal Processing Society (SPS). While there is no call for proposals
for 2018 yet, the winter 2016-2017 version states that ``an SPS contribution of
up to 5k US dollars can be included in the budget''. Additional funding
opportunities in Switzerland include the SNF\footnote{\url{http://www.snf.ch}},
the KTI\footnote{\url{http://www.kti.admin.ch}} and the Hasler
foundation\footnote{\url{http://www.haslerstiftung.ch}}.

To cover additional expenses and to incentivize participants to be ``active'',
we further ask for a registration fee of 200€ per participant from academia and
800€ for non-academic people.

A detailed budget can be found in tables~\ref{tab:expenditures} and
\ref{tab:revenues}.

\begin{table}[ht]
	\centering
	\begin{tabular}{|l|r|r|r|}
	\hline
	Description & Amount (EUR) & Quantity & Expenditures (EUR) \\
	\hline
	Meals and lodging (participants) &  442.50 & 40 & 17'700 \\
	Meals and lodging (speakers)     &  465.50 &  5 &  2'328 \\
	Travel defrayal for EU speakers  &  200    &  3 &    600 \\
	Travel defrayal for US speakers  &  700    &  2 &  1'400 \\
	Excursion                        &   30    & 44 &  1'320 \\
	Miscellaneous                    &  502    &  1 &    502 \\
	\hline
	\multicolumn{3}{|l|}{\textbf{Total}} & \textbf{23'850} \\
	\hline
	\end{tabular}
	\caption{Budgeted expenditures.}
	\label{tab:expenditures}
\end{table}

\begin{table}[ht]
	\centering
	\begin{tabular}{|l|r|r|r|}
	\hline
	Description & Amount (EUR) & Quantity & Revenues (EUR) \\
	\hline
	CIRM subsidies        & 442.50 & 20 & 8'850 \\
	IEEE SPS contribution & 5000   &  1 & 5'000 \\
	Local sponsoring      & 2000   &  1 & 2'000 \\
	Registration fees     &  200   & 40 & 8'000 \\
	\hline
	\multicolumn{3}{|l|}{\textbf{Total}} & \textbf{23'850} \\
	\hline
	\end{tabular}
	\caption{Budgeted revenues.}
	\label{tab:revenues}
\end{table}

\printbibliography

\end{document}


\section{delimited continuations}
\label{sec:cont-delim-cont}
Continuatioins
A conditional banch selects a continuation from the two possible
futures; rasing an exception discards. Traditional way to handle
continuations explicitly in a program is to transform a program into
cps style. Continuation captured by call/cc is the {\bf whole} continuation
that includes all the future computation.. In practice, most of the
continuations that we want to manipulate are only a part of
computation. Such continuations are called {\bf delimited continuations} or
{\bf partial continuations}.

\begin{enumerate}
\item cps transform \\
  there are multiple ways to do cps transform, here are two.

  \begin{bluetext}
------------------------------------------
   [x] --> x
   [\x. M] --> \k . k (\x . [M])
   [M N] --> \k. [M] (\m . m [N] k)
------------------------------------------

------------------------------------------
   [x] --> \k . k x
   [\x. M] --> \k. k (\x.[M])
   [M N] --> \k. [M] (\m . [N] (\n. m n k))
------------------------------------------

[callcc (\k. body)] = \outk. (\k. [body] outk) (\v localk. outk v)

  \end{bluetext}

\item experiment

\begin{alternate}
#load "delimcc.cma";;
\end{alternate}
\begin{alternate}
Delimcc.shift;;
- : 'a Delimcc.prompt -> (('b -> 'a) -> 'a) -> 'b = <fun>
\end{alternate}

\begin{bluetext}
reset (fun () -> M ) --> push_prompt p (fun () -> M )
shift (fun k -> M) --> shift p (fun k -> M )
\end{bluetext}
in racket you should have \textit{(require racket/control)}
and then \textit{(reset expr ...+)}
\textit{(shift id expr ...+)}

\begin{ocamlcode}
module D = Delimcc
(** set the prompt *)
let p = D.new_prompt ()
let (reset,shift),abort  = D.(push_prompt &&& shift &&& abort ) p;;
let foo x = reset (fun () -> shift (fun cont -> if x = 1 then cont 10 else 20 ) + 100 )
\end{ocamlcode}

\begin{alternate}
foo 1 ;;
- : int = 110
foo 2  ;;
- : int = 20
5 * reset (fun () -> shift (fun k -> 2 * 3 ) + 3 * 4 );;
- : int = 30
reset (fun () -> 3 + shift (fun k -> 5 * 2) ) - 1 ;;
- : int = 9
\end{alternate}
\begin{bluetext}
val p : '_a D.prompt = <abstr>
val reset : (unit -> '_a) -> '_a = <fun>
val shift : (('_a -> '_b) -> '_b) -> '_a = <fun>
val abort : '_a -> 'b = <fun>
\end{bluetext}

\begin{ocamlcode}
let p = D.new_prompt ()
let (reset,shift),abort  = D.(push_prompt &&& shift &&& abort ) p;;
\end{ocamlcode}

\begin{alternate}
reset (fun () -> if (shift (fun k -> k(2 = 3))) then "hello" else "hi ") ^ "world";;
- : string = "hi world"
reset (fun () -> if (shift (fun k -> "laji")) then "hello" else "hi ") ^ "world";;
- : string = "lajiworld"
reset (fun _ -> "hah");;
- : string = "hah"
\end{alternate}

\begin{ocamlcode}
let make_operator () =
  let p = D.new_prompt () in
  let (reset,shift),abort = D.(push_prompt &&& shift &&& abort) p in
  p,reset,shift,abort
\end{ocamlcode}

Delimited continuations seems not able to handle answer type polymorphism.

\begin{bluetext}
exception Str of [`Found of int | `NotFound]
\end{bluetext}

\begin{ocamlcode}
let times lst  =
  let rec times_aux lst = match lst with
    | [] -> 1
    | 0 :: xs -> shift (fun _ -> 0 )
    | x :: xs -> begin
      (* printf "entering %d\n" x ; *)
      let v = x * times_aux xs in
      (* printf "exiting %d\n" x ;  *)
      v
    end in
  reset (fun () -> times_aux lst )
\end{ocamlcode}

Store the continuation, the type system is not friendly to the
continutations, but fortunately we have \textit{side effects} at hand, we can
store it. (This is pretty hard in Haskell )

\begin{ocamlcode}
let p,reset,shift,abort = make_operator() in
  let c = ref None in
  begin
   reset (fun () -> 3 + shift (fun k -> c:= Some k ;  0) - 1)  ;
   Option.get (!c) 20
   end ;;
          Characters 81-139:
     reset (fun () -> 3 + shift (fun k -> c:= Some k ;  0) - 1)  ;
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     Warning 10: this expression should have type unit.
   \end{ocamlcode}
\begin{ocamlcode}
- : int = 22
\end{ocamlcode}
\begin{ocamlcode}
let cont =
  let p,reset,shift,abort = make_operator() in
  let c = ref None in
  let rec id lst = match lst with
    | [] -> shift (fun k -> c:=Some k ; [] )
    |x :: xs -> x :: id xs in
  let xs = reset (fun () -> id [1;2;3;4]) in
  xs, Option.get (!c);;
\end{ocamlcode}
\begin{ocamlcode}
val cont : int list * (int list -> int list) = ([], <fun>)
\end{ocamlcode}
\begin{alternate}
# let a,b = cont ;;
val a : int list = []

val b : int list -> int list = <fun>
# b [];;
- : int list = [1; 2; 3; 4]
\end{alternate}

\begin{ocamlcode}
type tree = Empty | Node of  tree * int  * tree
let walk_tree =
  let cont = ref None in
  let p,reset,shift,abort = make_operator() in
  let yield n = shift (fun k -> cont := Some k; print_int n ) in
  let rec walk2 tree = match tree with
    |Empty -> ()
    |Node (l,v,r) ->
      walk2 l ;
      yield v ;
      walk2 r in
  fun tree -> (reset (fun _ -> walk2 tree ), cont);;
\end{ocamlcode}
\begin{ocamlcode}
val walk_tree : tree_t -> unit * ('_a -> unit) option Batteries.ref =
\end{ocamlcode}

\begin{alternate}
# let _, cont = walk_tree tree1 ;;
1val cont : ('_a -> unit) option Batteries.ref = {contents = Some <fun>}
# Option.get !cont ();;
2- : unit = ()
# Option.get !cont ();;
3- : unit = ()
# Option.get !cont ();;
- : unit = ()
# Option.get !cont ();;
- : unit = ()
\end{alternate}

It's quite straightforward to implement yield using delimited
continuation, since each time shifting will escape the control, and you store the continuation, later it can be resumed.

\begin{bluetext}
(** defer the continuation *)
shift (fun k -> fun () -> k "hello")
\end{bluetext}

By wrapping continuations, we can \textbf{access the information outside} of the enclosing
reset while staying within reset lexically.

suppose this type check

\begin{alternate}
  let f x = reset (fun () -> shift (fun k -> fun () -> k "hello") ^ "world" ) x
  f : unit -> string
\end{alternate}

\item Answer type modification (serious)
  in the following context,
  \verb|reset (fun () -> [...] ^  "word" )|, the value returned by
  reset appears to be a string. An answer type is a type of the enclosing
  \emph{reset}.

\item reorder delimited continuations \\
  if we apply a continuation at the tail position, the captured computation is simply
  resumed. If we apply a continuation at the non-tail position, we can perform
  additional computation after resumed computation finishes.

  Put differently, we can switch the execution order of the surrounding context.

\begin{ocamlcode}
let p,reset,shift,abort = make_operator () in
    reset (fun () -> 1 + (shift (fun k -> 2 * k 3 )));;
\end{ocamlcode}
\begin{ocamlcode}
- : int = 8
\end{ocamlcode}

\begin{ocamlcode}
let p,reset,shift,abort = make_operator () in
   let either a b = shift (fun k -> k a ; k b ) in
   reset (fun () ->
   let x = either  0  1 in
   print_int x ; print_newline ());;
 \end{ocamlcode}
\begin{ocamlcode}
 0
 1
\end{ocamlcode}
\item useful links \\
  \href{http://blog.fitzell.ca/2009/01/seaside-partial-continuations.html}{sea
    side} \\
  \href{http://pllab.is.ocha.ac.jp/~asai/cw2011tutorial/}{shift and
    reset tutorial} \\
  \href{http://pllab.is.ocha.ac.jp/~asai/cw2011tutorial/main-e.pdf}{shift
    reset tutorial} \\
  \href{http://docs.racket-lang.org/reference/cont.html#(part._.Classical_.Control_.Operators)}{racket
    control operators} \\
  \href{http://okmij.org/ftp/continuations/caml-shift.pdf}{caml-shift-paper.pdf} \\
  \href{http://okmij.org/ftp/continuations/caml-shift-talk.pdf}{caml-shift-talk} \\

\end{enumerate}


\documentclass[10pt,conference,compsocconf]{IEEEtran}

\usepackage{hyperref}
\usepackage{graphicx}	% For figure environment
\usepackage{amssymb}
\usepackage{authblk}
\usepackage{color}
\usepackage{graphicx}
\graphicspath{ {images/} }
\usepackage[skip=2pt]{caption} % example skip set to 2pt

\title{Project 1 on Machine Learning team Yoor}

\author[1]{Sergei Volodin}
\author[1]{Baran Nama}
\author[1]{Omar Mehio}
\affil[1]{EPFL}
\affil[ ]{\textit {\{sergei.volodin,baran.nama,omar.mehio\}@epfl.ch}}

\begin{document}

\maketitle

\begin{abstract}
A classification dataset from the Large Hadron Collider simulations is being studied. First, the data is thoroughly explored using visual aids.
After that, several basic Machine Learning methods are applied on preprocessed data.
Results are evaluated using cross-validation.
Model overview is given for each considered algorithm and the best model is chosen.
\end{abstract}

\section{Introduction}
The Higgs boson is a famous elementary particle which was first predicted in 1960s and then discovered in 2012 \cite{higgs}. Its famousness is due to two facts, first being the collosal amount of effort put into construction of the Large Hadron Collider and conducting the ATLAS experiment and the second being the fact that the Higgs boson is considered to be connected to the fact that particles have a mass.

The ATLAS experiment consists of protons colliding at near-relative speed. After collision, the resulting particles sometimes contain the Higgs boson. Itself, it is not detectable by LHC. However, it is possible to detect the particles that it is decaying to.

The data being studied comprises of $N=250 000$ objects (train) each having 30 features: $\{(x_i,y_i)\}_{i=1}^N$, $x_i\in\mathbb{R}^D$, $D=30$. Each object represents a collision of a stream of protons. The data was not obtained during the ATLAS experiment, but rather from the simulation \cite{data}. Features represent properties of detected particles. It is required to determine if the particles represent the Higgs boson. The dataset has two classes: signal (+1) and background (-1).

The following paper claims that it is possible to use simple methods, such as Linear and Logistic Regressions to classify the data. In the following sections, the data is thoroughly studied and then the model is chosen based on reasoning and cross-validation.
\section{Models and Methods}
\begin{table}[!htb]
	\caption{Feature processing tricks and linear regression}	\label{tab:feature_proc}
	\centering\begin{tabular}{|lc|}\hline
		{\bf Feature trick added} & {\bf Test accuracy}\\\hline
		Constant feature & 0.745\\
		Standardization & 0.745\\
		Imputation & 0.747\\
		Binarization & 0.747\\
		Degree 3 poly & 0.785\\\hline
	\end{tabular}
\end{table}

\begin{figure}[!htb]
	\centering \includegraphics[width=100px]{../src/analysis/xy_tr_imp}
	\centering \includegraphics[width=100px]{../src/analysis/hist_te_imp}
	\centering \includegraphics[width=100px]{../src/analysis/hist-scatter-tr-imp-0}
	\centering \includegraphics[width=110px]{../src/analysis/correlation}
	\caption{Exploratory data analysis using charts}
	\label{fig:data}
\end{figure}

First, the exploratory data analysis shows that most of the features (except one) are real-valued (Figure \ref{fig:data}, 1-2). Moreover, it can also be seen that the test distribution does not differ from the training one (Figure \ref{fig:data}, 1-2). Scatter plots (Figure \ref{fig:data}, 3) show dependencies between features.
To further explore linear interactions among our data, we computed the correlation matrix (Figure \ref{fig:data}, 4).
Some of the significant dependencies we observed include the relationship among following tuples ({\tt DER\_prodeta\_jet\_jet}, {\tt DER\_mass\_jet\_jet})  and ({\tt PRI\_jet\_subleading\_phi}, {\tt DER\_sum\_pt}).

Each of the respective tuples have a $-1$ and $+1$ correlation coefficient. This implies that for each column in the tuple we can eliminate the other element since we are able to deduce the same amount of knowledge from either one of them. In practice this didn't turn out to be efficient as our prediction with deducted features resulted in lower accuracy.

To begin with, a simple linear regression model is used on raw data (with a constant feature added), with 5-fold cross-validation to control overfitting. The results of each subsample are averaged to provide train and test accuracy. However, the model gives unsatisfying results on the hold-out dataset. As a means to deal with this issue, feature imputation and engineering is used: missing features are imputed with mean values (and ``missing value'' column is added), categorical features are binarized. Moreover, it has been discovered that some of the features represent particle's mass, momentum or energy. Using the fact that squaring these quantities would make linear combinations of them meaningful in terms of physics ($E^2=m^2+p^2$), we introduce the polynomial basis starting from the degree of 2. Table \ref{tab:feature_proc} shows the respective effects of these tricks.

\begin{figure}[!htb]
	\centering \includegraphics[width=110px]{linear_5}
	\centering \includegraphics[width=110px]{ridge_accuracy_deg6}
	\caption{Degree 6 basis, MSE and Accuracy metrics}
	\label{fig:deg6}
\end{figure}

Applying ridge regression, it might be seen that it does not overfit in terms of accuracy (but does in terms of MSE \ref{fig:deg6}) on the dataset given even for the 6th degree, meaning that there is no significant difference between the test and the train accuracies. Applying it using different degrees, one obtains results shown in the Table \ref{tab:degrees}.

\begin{table}[!htb]
	\caption{Polynomial basis and linear regression, $\lambda=0$}	\label{tab:degrees}
	\centering\begin{tabular}{|lcccc|}\hline
		{\bf Degree} & {\bf Test accuracy} & {\bf MSE Test} & {\bf MSE Train} & {$\lambda$ best}\\
		1 & 0.7470 & 0.3378 & 0.3377 & $10^{-6}$\\
		2 & 0.7737 & 0.3483 & 0.3158 & $10^{-2}$\\
		3 & 0.7850 & 0.3060 & 0.3040 & $10^{-4}$\\
		4 & 0.7937 & 0.3030 & 0.3010 & 0.0004\\
		5 & 0.7971 & 0.3050 & 0.3010 & 0.0006\\\hline
	\end{tabular}
\end{table}

Moreover, as can be seen in the figures (Table \ref{tab:degrees}), while train MSE is continuously decreasing because of over-fitting, there is no significant change in test error after third degree polynomial basis. Therefore, feature extraction using polynomial basis does not seem to work for higher degrees. Furthermore, the accuracy of the model in Kaggle is not as good as what we expected (below 0.8), so that we decided to switch to logistic regression since it uses a loss function tailored for the purpose of classification.

Training logistic regression with gradient descent on the dataset, we have found out that it is crucial to implement the Newton's method since gradient update rule is taking too much time. This is due to the fact that the final iterations are going with a small step size since setting the step size of gradient descent to a larger value resulted in a numerical overflow.

Moreover, we discovered that the magnitude $\gamma$ of the gradient step depends on the size of the dataset. Therefore, we tried using batched version of stochastic gradient descent for logistic regression as the optimizer to stabilize the size of the dataset the used for training. However, applying this batched version to the dataset was costly since working with a large matrix in numpy is faster than working with $k$ smaller matrices. Therefore, we used the standard Netwon algorithm and tuned the $\gamma$ manually for the final model.

The Figure \ref{fig:logreg} (1) shows logistic regression accuracy for different regularization parameters $\lambda$. For this experiment, a constant number of iterations $20$ was chosen. The $\gamma$ parameter equals $\gamma=10^{-1}$. The chart indicates that the logistic regression does not overfit. Therefore, we have chosen the first in magnitude value of $\lambda=2\times 10^{-4}$ which allowed us to use $\gamma=0.1$ without numerical overflows in loss. The figure \ref{fig:logreg} (2) indicates the training schedule. Total number of $N=70$ iterations allowed us to bring the training error to 0. The solution has scored 0.82058 on Kaggle \cite{kaggle} (first part of test).

Despite the fact that the gradient at the final point was close to $0$ compared to the previous values, the value of the loss did not equal to $0$. Moreover, it can be seen that the train and test errors were similar. These two facts indicate that linear model has a high bias and low variance when applied to this dataset.

\begin{figure}[!htb]
	\centering \includegraphics[width=110px]{logistic_lambda_cv_netwon_nobatch}
	\centering \includegraphics[width=110px]{logistic_01}
%	\centering \includegraphics[width=200px]{logistic_02}

	\caption{Degree 5 basis, Logistic regression accuracy}
	\label{fig:logreg}
\end{figure}

\section{Results}
We have presented an application of linear, ridge and logistic regression to the dataset. The best model was the logistic one with best accuracy of 0.82 trained with Newton's method. All of the linear models do not overfit on the dataset even when using a polynomial basis.
\section{Discussion}
Despite thorough analysis, our project lacks further consideration in the following directions. First, it can be seen that some of the features do not look like the Gaussian distribution. It might prove beneficial to split such features into two using a certain threshold. Moreover, it might be useful to apply bagging techniques, as well as to use more sophisticated nonlinear classifiers, such as neural networks.
\section{Summary}
We have shown that it is possible to detect the Higgs boson using linear methods and feature augmentation. Such a simple solution gives fairly good performance and places the team above average on the Kaggle submission. However, this approach is not optimal in terms of model selection, since a linear model seem to not capture all of the specifics of data since it produces high bias.
\begin{thebibliography}{99}
\bibitem{higgs} \href{https://en.wikipedia.org/wiki/Higgs\_boson}{en.wikipedia.org/wiki/Higgs\_boson}
\bibitem{data} \href{http://higgsml.lal.in2p3.fr/files/2014/04/documentation\_v1.8.pdf}{Experiment and feature description}
\bibitem{kaggle} \href{http://kaggle.com/c/epfml-higgs}{EPFL ML Project 1}
\end{thebibliography}

\end{document}


\lab{GeoPandas}{GeoPandas}

\objective{GeoPandas is a package designed to organize and manipulate geographic data, It combines the data manipulation tools of pandas with the geometric capabilities of the Shapely package. In this lab, we explore the basic data structures of GeoSeries and GeoDataFrames and their functionalities.}

\section*{Installation} %============================
GeoPandas is a new package designed to combine the functionality of pandas with Shapely, a package used for geometric manipulation.
Using GeoPandas with geographic data is very useful as it allows the user to not only compare numerical data, but also geometric attributes.
GeoPandas can be installed via \li{pip}:
\begin{lstlisting}
>>> pip install geopandas
\end{lstlisting}
However, Geopandas can be notoriously difficult to install.
%It is very finicky, which is likely a by-product of it still being in development
This is especially the case if the python environment is not carefully maintained.
Some of its dependencies can also be very difficult to install on certain systems.
Because of this, using Colab for this lab is recommended; its environment is set up in a way that makes GeoPandas very easy to install.
Otherwise, the GeoPandas documentation contains some additional options that can be used if installation difficulties occur: \url{https://geopandas.org/install.html}.

\section*{GeoSeries} %============================

A GeoSeries is a pandas Series where each entry is a set of geometric objects.
There are three classes of geometric objects inherited from the Shapely package:
\begin{enumerate}
\item Points / Multi-Points
\item Lines / Multi-Lines
\item Polygons / Multi-Polygons
\end{enumerate}
A point is used to identify objects like coordinates, where there is one small instance of the object.
A line could be used to describe objects such as roads.
A polygon could be used to identify regions, such as a country.
Multipoints, multilines, and multipolygons contain lists of points, lines, and polygons, respectively.

Since each object in the GeoSeries is also a Shapely object, the GeoSeries inherits many methods and attributes of Shapely objects.
Some of the key attributes and methods are listed in Table \ref{shapely-methods}.
These attributes and methods can be used to calculate distances, find the sizes of countries, and determine whether coordinates are within country's boundaries.
The example below uses the attribute \li{bounds} to find the maximum and minimum coordinates of Egypt in a built-in GeoDataFrame.

\begin{table}[ht]
\begin{center}
\begin{tabular}{c|c}
Method/Attribute & Description\\
\hline
\li{distance(other)}&returns minimum distance from GeoSeries to \li{other}\\
\li{contains(other)}&returns \li{True} if shape contains \li{other}\\
\li{intersects(other)}&returns \li{True} if shape intersects \li{other}\\
\li{area}&returns shape area\\
\li{convex_hull}&returns convex shape around all points in the object\\
\li{bounds} &  returns the bounding x- and y-coordinates of the object
\end{tabular}
\end{center}
\caption{Attributes and Methods for GeoSeries}
\label{shapely-methods}
\end{table}

\begin{lstlisting}
>>> import geopandas as gpd
>>> world = gpd.read_file(gpd.datasets.get_path('naturalearth_lowres'))
# Get GeoSeries for Egypt
>>> egypt = world[world['name']=='Egypt']

# Find bounds of Egypt
>>> egypt.bounds
         minx    miny         maxx      maxy
47   24.70007    22.0     36.86623  31.58568
\end{lstlisting}

\section*{Creating GeoDataFrames} %============================

The main structure used in GeoPandas is a GeoDataFrame, which is similar to a pandas DataFrame.
A GeoDataFrame has one special column called \li{geometry}, which must be a GeoSeries.
This GeoSeries column is used when a spatial method, like \li{distance()}, is used on the GeoDataFrame.

A GeoDataFrame can be made from a pandas DataFrame.
At least one of the columns in the DataFrame should contain geometric information.
This column containing geometric information can be converted to a GeoSeries using the \li{apply()} method.
At this point, the Pandas DataFrame can be cast as a GeoDataFrame.
Assign which column will be the \li{geometry} using either the \li{geometry} keyword in the constructor or the \li{set_geometry()} method afterwards.

\begin{lstlisting}
>>> import pandas as pd
>>> import geopandas as gpd
>>> from shapely.geometry import Point, Polygon

# Create a Pandas DataFrame
>>> df = pd.DataFrame({'City': ['Seoul', 'Lima', 'Johannesburg'],
...                   'Country': ['South Korea', 'Peru', 'South Africa'],
...                   'Latitude': [37.57, -12.05, -26.20],
...                   'Longitude': [126.98, -77.04, 28.04]})

# Create geometry column
>>> df['Coordinates'] = list(zip(df.Longitude, df.Latitude))

# Make geometry column Shapely objects
>>> df['Coordinates'] = df['Coordinates'].apply(Point)

# Cast as GeoDataFrame
>>> gdf = gpd.GeoDataFrame(df, geometry='Coordinates')

# Equivalently, specify the geometry after construction
# Note that set_geometry() returns a new GeoDataFrame
>>> gdf = gpd.GeoDataFrame(df)
>>> gdf = gdf.set_geometry('Coordinates')

# Display the GeoDataFrame
>>> gdf
           City       Country  Latitude  Longitude                  Coordinates
0         Seoul   South Korea     37.57     126.98   POINT (126.98000 37.57000)
1          Lima          Peru    -12.05     -77.04  POINT (-77.04000 -12.05000)
2  Johannesburg  South Africa    -26.20      28.04   POINT (28.04000 -26.20000)

# Create a polygon with all three cities as points
>>> city_polygon = Polygon(list(zip(df.Longitude, df.Latitude)))
\end{lstlisting}

A \li{GeoDataFrame} can also be made directly from a dictionary.
If the dictionary already contains geometric objects, the corresponding column can be directly set as the \li{geometry} in the constructor.
Otherwise, a column containing geometry data can be created as in the above example and then set as the \li{geometry} with the \li{set_geometry()} method.

\begin{lstlisting}
# Both of these methods create the same GeoDataFrame as above
# Directly create the GeoDataFrame from the dictionary
>>> gdf = gpd.GeoDataFrame({'City': ['Seoul', 'Lima', 'Johannesburg'],
...                   'Country': ['South Korea', 'Peru', 'South Africa'],
...                   'Latitude': [37.57, -12.05, -26.20],
...                   'Longitude': [126.98, -77.04, 28.04]})
# Create geometry column and set as the geometry
>>> gdf['Coordinates'] = list(zip(df.Longitude, df.Latitude))
>>> gdf['Coordinates'] = df['Coordinates'].apply(Point)
# inplace=True modifies gdf itself rather than returning a copy
>>> gdf.set_geometry('Coordinates', inplace=True)

# Equivalently, using a dictionary that already contains geometry objects
>>> gdf = gpd.GeoDataFrame({'City': ['Seoul', 'Lima', 'Johannesburg'],
...                   'Country': ['South Korea', 'Peru', 'South Africa'],
...                   'Coordinates': [Point(126.98,37.57),
...                       Point(-77.04,-12.05), Point(28.04,-12.05)]},
...                   geometry='Coordinates')
\end{lstlisting}

\begin{info}
Longitude is the angular measurement starting at the Prime Meridian, $0\degree$, and going to $180\degree$ to the east and $-180\degree$ to the west.
Latitude is the angle between the equatorial plane and the normal line at a given point; a point along the Equator has latitude $0$, the North Pole has latitude $+90\degree$ or $90\degree N$, and the South Pole has latitude $-90\degree$ or $90\degree S$.
\end{info}

\section*{Plotting GeoDataFrames} %============================

Information from a GeoDataFrame is plotted based on the geometry column.
Data points are displayed as geometry objects.
The following example plots the shapes in the \li{world} GeoDataFrame.

\begin{lstlisting}
# Plot world GeoDataFrame
>>> world.plot()
\end{lstlisting}

\begin{figure}[H]
\begin{center}
\includegraphics[scale=.55]{figures/worldmap.pdf}
\end{center}
\label{world-map}
\caption{World map}
\end{figure}

Multiple GeoDataFrames can be plotted at once.
This can be done by by setting one GeoDataFrame as the base of the plot and ensuring that each layer uses the same axes.
In the following example, the file \li{airports.csv}, containing the coordinates of world airports, is loaded into a GeoDataFrame and plotted on top of the boundary of the \li{world} GeoDataFrame.

\begin{lstlisting}
# Set outline of world countries as base
>>> fig,ax = plt.subplots(figsize=(10,7), ncols=1, nrows=1)
>>> base = world.boundary.plot(edgecolor='black', ax=ax, linewidth=1)

# Load airport data and convert to a GeoDataFrame
>>> airports = pd.read_csv('airports.csv')
>>> airports['Coordinates'] = list(zip(airports.Longitude, airports.Latitude))
>>> airports['Coordinates'] = airports.Coordinates.apply(Point)
>>> airports = gpd.GeoDataFrame(airports, geometry='Coordinates')

# Plot airports on top of world map
>>> airports.plot(ax=base, marker='o', color='green', markersize=1)
>>> ax.set_xlabel('Longitude')
>>> ax.set_ylabel('Latitude')
>>> ax.set_title('World Airports')
\end{lstlisting}

\begin{figure}[H]
\begin{center}
\includegraphics[scale=.6]{figures/airports.pdf}
\end{center}
\label{airport-map}
\caption{Airport map}
\end{figure}

\begin{problem}
Read in the file \li{airports.csv} as a pandas DataFrame.
Create three convex hulls around the three sets of airports listed below.
This can be done by passing in lists of the airports' coordinates to a \li{shapely.geometry.Polygon} object.

Create a new GeoDataFrame with these three Polygons as entries.
Plot this GeoDataFrame on top of an outlined world map.
\begin{itemize} %These could also be listed by 4-character ICAO codes, but listing the names makes the problem more exciting/realistic. Probably.
	\item Maio Airport, Scatsta Airport, Stokmarknes Skagen Airport, Bekily Airport, K. D. Matanzima Airport, RAF Ascension Island
	\item Oiapoque Airport, Maio Airport, Zhezkazgan Airport, Walton Airport, RAF Ascension Island, Usiminas Airport, Piloto Osvaldo Marques Dias Airport
	\item Zhezkazgan Airport, Khanty Mansiysk Airport, Novy Urengoy Airport, Kalay Airport, Biju Patnaik Airport, Walton Airport
\end{itemize}
\end{problem}

\section*{Working with GeoDataFrames} %============================

As previously mentioned, GeoDataFrames contain many of the functionalities of pandas DataFrames.
For example, to create a new column, define a new column name in the GeoDataFrame with the needed information for each GeoSeries.

\begin{lstlisting}
# Create column in the world GeoDataFrame for gdp_per_capita
>>> world['gdp_per_cap'] = world.gdp_md_est / world.pop_est
\end{lstlisting}

GeoDataFrames can utilize many pandas functionalities, and they can also be parsed by geometric manipulations.
For example, a useful way to index GeoDataFrames is with the \li{cx} indexer.
This splits the GeoDataFrame by the coordinates of each geometric object.
It is used by calling the method \li{cx} on a GeoDataFrame, followed by a slicing argument, where the first element refers to the longitude and the second refers to latitude.

\begin{lstlisting}
# Create a GeoDataFrame containing the northern hemisphere
>>> north = world.cx[:, 0:]

# Create a GeoDataFrame containing the southeastern hemisphere
>>> south_east = world.cx[0:, :0]
\end{lstlisting}

GeoSeries objects in a GeoDataFrame can also be dissolved, or merged, together into one GeoSeries based on their geometry data.
For example, all countries on one continent could be merged to create a GeoSeries containing the information of that continent.
The method designed for this is called \li{dissolve}.
It receives two parameters, \li{by} and \li{aggfunc}.
\li{by} indicates which column to dissolve along, and \li{aggfunc} tells how to combine the information in all other columns.
The default \li{aggfunc} is \li{first}, which returns the first application entry.
In the following example, we use \li{sum} as the \li{aggfunc} so that each continent is the combination of its countries.

\begin{lstlisting}
>>> world = world[['continent', 'geometry', 'gdp_per_cap']]

# Dissolve world GeoDataFrame by continent
>>> continent = world.dissolve(by = 'continent', aggfunc='sum')
\end{lstlisting}

\section*{Projections and Coloring} %============================

When plotting, GeoPandas uses the CRS (coordinate reference system) of a GeoDataFrame.
This reference system indicates how coordinates should be spaced on a plot.
Two of the most commonly used CRSs are EPSG:4326 and EPSG:3395.
EPSG:4326 is the standard latitude-longitude projection used by GPS.
EPSG:3395, also known as the Mercator projection, is the standard navigational projection.

When creating a new GeoDataFrame, it is important to set the \li{crs} attribute of the GeoDataFrame.
This allows any plots to be shown correctly.
Furthermore, GeoDataFrames being layered need to have the same CRS.
To change the CRS, use the method \li{to_crs()}.

\begin{lstlisting}
# Check CRS of world GeoDataFrame
>>> print(world.crs)
epsg:4326

# Change CRS of world to Mercator
# inplace=True ensures that we modify world instead of returning a copy
>>> world.to_crs(3395, inplace=True)
>>> print(world.crs)
epsg:3395
\end{lstlisting}

GeoPandas accepts many different CRSs; a reference can be found at \url{www.spatialreference.org}.
% I think this is helpful information, but I'm not sure
Additionally, inspecting a given CRS object in the terminal without using \li{print()} or \li{str()} can be used to get additional information about a specific CRS:\footnote{This can also be accomplished using \li{print(repr(crs))}.}
\begin{lstlisting}[mathescape]
>>> world.crs
<Projected CRS: EPSG:3395>
Name: WGS 84 / World Mercator
Axis Info [cartesian]:
- E[east]: Easting (metre)
- N[north]: Northing (metre)
Area of Use:
- name: World between 80$^\circ$S and 84$^\circ$N.
- bounds: (-180.0, -80.0, 180.0, 84.0)
Coordinate Operation:
- name: World Mercator
- method: Mercator (variant A)
Datum: World Geodetic System 1984
- Ellipsoid: WGS 84
- Prime Meridian: Greenwich
\end{lstlisting}

GeoDataFrames can also be plotted using the values in the the other attributes of the GeoSeries.
The map plots the color of each geometry object according to the value of the column selected.
This is done by passing in the parameter \li{column} into the \li{plot()} method.

\begin{lstlisting}
>>> fig, ax = plt.subplots(1, figsize=(10,4))
# Plot world based on gdp
>>> world.plot(column='gdp_md_est', cmap='OrRd', legend=True,  ax=ax)
>>> ax.set_title('World Map based on GDP')
>>> ax.set_xlabel('Longitude')
>>> ax.set_ylabel('Latitude')
>>> plt.show()
\end{lstlisting}

\begin{figure}[H]
\begin{center}
\includegraphics[scale=.7]{figures/gdp.pdf}
\end{center}
\caption{World Map Based on GDP}
\label{figure:gdp_map}
\end{figure}

\begin{problem}
The file \li{county_data.gpkg.zip} contains information about US counties.\footnote{Source: \url{http://www2.census.gov/geo/tiger/GENZ2016/shp/cb_2016_us_county_5m.zip}}
After unzipping, use the command \li{gpd.read_file('county_data.gpkg')} to create a GeoDataFrame of this information.
Each county's shape is stored in the \li{geometry} column.
Use this to plot all US counties two times, first using the default CRS and then using EPSG:5071.

Next, create a new GeoDataFrame that merges all counties within a single state.
Drop regions with the following STATEFP codes: 02, 15, 60, 66, 69, 72, 78.
Plot this GeoDataFrame to see an outline of all 48 contiguous states.
Ensure a CRS  of EPSG:5071.
\label{problem:us_counties}
\end{problem}
\begin{info}
\li{.gpkg} files are actually structured as a directory that contains several files that each contain parts of the data.
For instance, \li{county_data.gpkg} consists of the files \li{county_data.cpg}, \li{county_data.dbf}, \li{county_data.prj}, \li{county_data.shp}, and \li{county_data.shx}.
Be sure that these files are placed directly in the first level of folders, and not in further subdirectories.\\
To use this file in Google Colab, upload the zipped file and extract it with the following code:
\begin{lstlisting}
county = files.upload()
!unzip county_data.gpkg.zip
\end{lstlisting}
It then can be loaded:
\begin{lstlisting}
county_df = gpd.read_file('county_data.gpkg')
\end{lstlisting}
\end{info}

\section*{Merging GeoDataFrames} %============================

Just as multiple pandas DataFrames can be merged, multiple GeoDataFrames can be merged with attribute joins or spatial joins.
An attribute join is similar to a merge in pandas.
It combines two GeoDataFrames on a column (not the geometry column) and then combines the rest of the data into one GeoDataFrame.

\begin{lstlisting}
>>> world = gpd.read_file(geopandas.datasets.get_path('naturalearth_lowres'))
>>> cities = gpd.read_file(geopandas.datasets.get_path('naturalearth_cities'))

# Create subsets of the world and cities GeoDataFrames
>>> world = world[['continent', 'name', 'iso_a3']]
>>> cities = cities[['name', 'iso_a3']]

# Merge the GeoDataFrames on their iso_a3 code
>>> countries = world.merge(cities, on='iso_a3')
\end{lstlisting}

A spatial join merges two GeoDataFrames based on their geometry data.
The function used for this is \li{sjoin}.
\li{sjoin} accepts two GeoDataFrames and then direction on how to merge.
It is imperative that two GeoDataFrames have the same CRS.
In the example below, we merge using an \li{inner} join with the option \li{intersects}.
The \li{inner} join means that we will only use keys in the intersection of both geometry columns, and we will retain only the left geometry column.
\li{intersects} tells the GeoDataFrames to merge on GeoSeries that intersect each other.
Other options include \li{contains} and \li{within}.

\begin{lstlisting}
# Combine countries and cities on their geographic location
>>> countries = gpd.sjoin(world, cities, how='inner', op='intersects')
\end{lstlisting}

\begin{problem}
Load in the file \li{nytimes.csv}\footnote{Source: \url{https://raw.githubusercontent.com/nytimes/covid-19-data/master/us-counties.csv}} as a DataFrame.
This file includes county-level data for the cumulative cases and deaths of Covid-19 in the US, starting with the first case in Snohomish County, Washington, on January 21, 2020.
Begin by converting the \li{date} column into a \li{DatetimeIndex}.

Next, use county FIPS codes to merge your GeoDataFrame from Problem \ref{problem:us_counties} with the DataFrame you just created.
A FIPS code is a 5-digit unique identifier for geographic locations.
Ignore rows in the Covid-19 DataFrame with unknown FIPS codes as well as all data from Hawaii and Alaska.

Note that the \li{fips} column of the Covid-19 DataFrame stores entries as floats, but the county GeoDataFrame stores FIPS codes as strings, with the first two digits in the \li{STATEFP} column and the last three in the \li{COUNTYFP} column.

Once you have completed the merge, plot the cases from March 21, 2020 on top of your state outline map from Problem \ref{problem:us_counties}, using the CRS of EPSG:5071.
Finally, print out the name of the county with the most cases on March 21, 2020 along with its case count.
\label{problem:merge_covid}
\end{problem}

\section*{Logarithmic Plotting Techniques} %============================

The color scheme of a graph can also help to communicate information clearly.
A good list of available colormaps can be found at \url{https://matplotlib.org/3.2.1/gallery/color/colormap_reference.html}.
Note also that you can reverse any colormap by adding \li{_r} to the end.
The following example demonstrates some plotting features, using country GDP as in Figure \ref{figure:gdp_map}.

\begin{lstlisting}
>>> fig, ax = plt.subplots(figsize=(15,7), ncols=1, nrows=1)
>>> world.plot(column='gdp_md_est', cmap='plasma_r',
...			ax=ax, legend=True, edgecolor='gray')

# Add title and remove axis tick marks
>>> ax.set_title('GDP on Linear Scale')
>>> ax.set_yticks([])
>>> ax.set_xticks([])
>>> plt.show()
\end{lstlisting}

\begin{figure}[H]
\begin{center}
\includegraphics[scale=.75]{figures/world_linear.pdf}
\end{center}
\caption{World map showing country GDP}
\label{figure:linear_world}
\end{figure}

Sometimes data can be much more informative when plotted on a logarithmic scale.
See how the world map changes when we add a \li{norm} argument in the code below.
Depending on the purpose of the graph, Figure \ref{figure:log_world} may be more informative than Figure \ref{figure:linear_world}.

\begin{lstlisting}
>>> from matplotlib.colors import LogNorm
>>> from matplotlib.cm import ScalarMappable
>>> fig, ax = plt.subplots(figsize=(15,6), ncols=1, nrows=1)

# Set the norm using data bounds
>>> data = world.gdp_md_est
>>> norm = LogNorm(vmin=min(data), vmax=max(data))

# Plot the graph using the norm
>>> world.plot(column='gdp_md_est', cmap='plasma_r', ax=ax,
...			edgecolor='gray', norm=norm)

# Create a custom colorbar
>>> cbar = fig.colorbar(ScalarMappable(norm=norm, cmap='plasma_r'),
...			ax=ax, orientation='horizontal', pad=0, label='GDP')

>>> ax.set_title('Country Area on a Log Scale')
>>> ax.set_yticks([])
>>> ax.set_xticks([])
>>> plt.show()
\end{lstlisting}

\begin{figure}[H]
\begin{center}
\includegraphics[scale=.65]{figures/world_log.pdf}
\end{center}
\caption{World map showing country GDP using a log scale}
\label{figure:log_world}
\end{figure}

\begin{problem}
As in Problem \ref{problem:merge_covid}, plot your state outline map from Problem \ref{problem:us_counties} on top of a map of the Covid-19 cases from March 21, 2020.
This time, however, use a log scale.
Use EPSG:5071 for the CRS.
Pick a good colormap (the counties with the most cases should generally be darkest) and be sure to display a colorbar.
\end{problem}

\begin{problem}
In this problem, you will create an animation of the spread of Covid-19 through US counties from January 21, 2020 to June 21, 2020.
Use a log scale and a good colormap, and be sure that you're using the same norm and colorbar for the whole animation.
Use EPSG:5071 for the projection.

As a reminder, below is a summary of what you will need in order to animate this map.
You may also find it helpful to refer to the animation section included with the Volume 4 lab manual.

%------------------------------------------- Is that ok to say?
\begin{enumerate}
	\item Set up your figure and norm. Be sure to use the highest case count for your \li{vmax} so that the scale remains uniform.
	\item Write your \li{update} function. This should plot the cases from a given day.
	\item Set up your colorbar. Do this outside the \li{update} function to avoid adding a new colorbar each day.
	\item Create the animation and embed it.
\end{enumerate}
\end{problem}