\section{Adversarial Training}

\transitionFrame{Adversarially Robust Training}

\begin{frame}{Solving the Outer Minimization}
  \onslide<+->{%
    \begin{equation}\label{eq:MinimaxTrainRepeat}
      \green{\min_{\params} \rho(\params)} \text{, where } \rho(\params) = \mathbb{E}_{(\X,\y) \sim \distr} \sbrack{\red{\max_{\delta \in \sPerturb} \loss (\X + \perturb, \y ; \params)}}
    \end{equation}
  }

  \begin{itemize}[<+->]
    \item So far, we have only talked about solving the \red{inner maximization} to create adversarial examples

    \vspace{10pt}
    \item \textbf{Question}: What do we need to solve to train adversarially robust networks?
    \vspace{3pt}
    \item \textbf{Answer}: Solve the \green{outer minimization}

    \vspace{10pt}
    \item \textbf{Question}: What algorithm can be used to solve the outer minimization?
    \vspace{3pt}
    \item \textbf{Answer}: Stochastic gradient descent (SGD) on the adversarial examples
      \begin{itemize}[<+->]
        \setlength{\itemsep}{4pt}
        \item \textit{Intuition}: SGD on adv.\ examples' loss reduces the \red{inner maximization}
        \item \textit{Takeaway}: Given an algorithm that transforms training examples into adv.\ examples (e.g.,~PGD), the rest of the \textbf{\blue{training process proceeds normally}}
      \end{itemize}
  \end{itemize}
\end{frame}

\begin{frame}{Why Should You Believe the Preceding Claims are True?}
  \onslide<+->{}
  \onslide<+->{\textbf{Answer}: You shouldn't.  The preceding explanation is definitely \red{not a proof}.}

  \vspace{20pt}
  \onslide<+->{\madry\ rely on \textbf{\blue{Danskin's Theorem}} that states gradients at inner maximizers (think adversarial examples) correspond to descent directions for the complete problem}

  \vspace{20pt}
  \onslide<+->{\textbf{Problem}: Multiple assumptions made by Danskin's Theorem's \red{do not apply} here, e.g.,~continuously differentiable function, only \red{\textit{approximate}} inner maximizers etc.}
  \begin{itemize}[<+->]
    \setlength{\itemsep}{6pt}
    \item Empirical results show that despite Danskin's not holding, \madry's approach is robust
    \item A more complete discussion of how Danksin's applies to this problem is in Appendix~A (see Arxiv version) and is beyond the scope of this talk
  \end{itemize}
\end{frame}


\chapter{Classification Trees}
\label{ch:classification-trees}

In the previous lesson, we used a classification tree,
\marginnote{Classification trees were hugely popular in the early years of machine learning, when they were first independently proposed by the engineer Ross Quinlan (C4.5) and a group of statisticians (CART), including the father of random forests Leo Brieman.}
one of the oldest, but still popular, machine learning methods. We like it since the method is easy to explain and gives rise to random forests, one of the most accurate machine learning techniques (more on this later). So, what kind of model is a classification tree?

Let us load \textit{iris} data set, build a tree (widget \widget{Tree}) and visualize it in a \widget{Tree Viewer}.

\begin{figure}[h]
    \centering
    \includegraphics[scale=0.4]{workflow-tree-viewer.png}
\end{figure}

\begin{figure*}[h]
    \vspace{-0.4cm}
    \includegraphics[scale=0.35]{iris-data.png}
    \label{fig:classification-predictions}
\end{figure*}

\begin{wrapfigure}{o}{1.0\textwidth}
    \includegraphics[scale=0.35]{tree-viewer.png}
    \label{fig:classification-predictions}
\end{wrapfigure}

We read the tree from top to bottom. Looks like the column \textit{petal length} best separates the iris variety \textit{setosa} from the others, and in the next step, \textit{petal width} then almost perfectly separates the remaining two varieties.

Trees place the most useful feature at the root. What would be the most useful feature? The feature that splits the data into two purest possible subsets. It then splits both subsets further, again by their most useful features, and keeps doing so until it reaches subsets in which all data belongs to the same class (leaf nodes in strong blue or red) or until it runs out of data instances to split or out of useful features (the two leaf nodes in white).

We still have not been very explicit about what we mean by "the most useful" feature. There are many ways to measure the quality of features, based on how well they distinguish between classes. We will illustrate the general idea with information gain. We can compute this measure in Orange using the \widget{Rank} widget\marginnote{The \widget{Rank} widget can be used on its own to show the best predicting features. Say, to figure out which genes are best predictors of the phenotype in some gene expression data set.}, which estimates the quality of data features and ranks them according to how informative they are about the class. We can either estimate the information gain from the whole data set, or compute it on data corresponding to an internal node of the classification tree in the \widget{Tree Viewer}. In the following example we use the \textit{Sailing} data set.

\begin{figure}[h]
    \centering
    \includegraphics[scale=0.4]{workflow-rank.png}
    \caption{The \widget{Datasets} widget is set to load the \textit{Sailing} data set. To use the second \widget{Rank}, select a node in the \widget{Tree Viewer}.}
\end{figure}

Besides the information gain, \widget{Rank} displays several other measures (including Gain Ratio and Gini), which are often quite in agreement and were invented to better handle discrete features with many different values.

\begin{figure}[h]
    \centering
    \vspace{-0.2cm}
    \includegraphics[scale=0.4]{rank.png}
    \caption{For the whole \textit{Sailing} data set, \textit{Company} is the most class-informative feature according to all measures shown.}
\end{figure}

\newpage

Here is an interesting combination of a \widget{Tree Viewer} and a \widget{Scatter Plot}. This time, use the \textit{Iris} data set. In the \widget{Scatter Plot}, we first find the best visualization of this data set, that is, the one that best separates the instances from different classes. Then we connect the \widget{Tree Viewer} to the \widget{Scatter Plot}. Data instances (particular irises) from the selected node in the \widget{Tree Viewer} are shown in the \widget{Scatter Plot}.

\begin{figure}[h]
    \centering
    \includegraphics[scale=0.4]{workflow-inspection.png}
    \caption{Careful, the \widget{Data} widget needs to be connected to the \widget{Scatter Plot}'s \textit{Data} input, and \widget{Tree Viewer} to the \widget{Scatter Plot}'s \textit{Data Subset} input.}
\end{figure}

Just for fun, we have included a few other widgets in this workflow. In a way, a \widget{Tree Viewer} behaves like \widget{Select Rows}, except that the rules used to filter the data are inferred from the data itself and optimized to obtain purer data subsets.

\begin{figure*}[h]
  \infinitewidthbox{\includegraphics[scale=0.35]{tree-viewer-selection.png} \includegraphics[scale=0.35]{scatter-plot-subset.png}}
  \caption{In the \widget{Tree Viewer} we selected the rightmost node. All data instances coming to the selected node are highlighted in \widget{Scatter Plot}.}
\end{figure*}

Wherever possible, visualizations in Orange are designed to support selection and passing of the data that applies to it. Finding interesting data subsets and analyzing their commonalities is a central part of explorative data analysis, a data analysis approach favored by the data visualization guru Edward Tufte.


\section{The \copyi algorithm}
\Label{sec:copyi}

The \copyi  algorithm in the \cxx Standard Library \cite[\S 28.6.1]{cxx-17-draft} implements
a duplication algorithm for general sequences.
For our purposes we have modified
the generic implementation
to that of a range of type \valuetype.
The signature now reads:

\begin{lstlisting}[style=acsl-block]

  void copy(const value_type* a, size_type n, value_type* b);
\end{lstlisting}

Informally, the function copies every element from the source range \inl{a[0..n-1]} to the
destination range~\inl{b[0..n-1]}, as shown in Figure~\ref{fig:copy}.

\begin{figure}[hbt]
\centering
\includegraphics[width=0.50\textwidth]{Figures/copy.pdf}
\caption{\Label{fig:copy} Effects of \copyi}
\end{figure}

\subsection{Formal specification of \copyi}

Figure~\ref{fig:copy} might suggest that the ranges \inl{a[0..n-1]} and \inl{b[0..n-1]}
must not overlap.
However, since the informal specification requires that elements are copied in the
order of increasing indices only a weaker condition is necessary.
To be more specific, it is required that the pointer~\inl{b} does not refer
to elements of \inl{a[0..n-1]} as shown in the example in Figure~\ref{fig:copy-overlap}.

\begin{figure}[hbt]
\centering
\includegraphics[width=0.60\textwidth]{Figures/copy-overlap.pdf}
\caption{\Label{fig:copy-overlap} Possible overlap of \copyi ranges}
\end{figure}

\FloatBarrier

The specification of \copyi is shown in the following listing.
The \copyi algorithm expects that the ranges \inl{a} and \inl{b} are valid for reading
and writing, respectively.
Note the precondition~\inl{sep} that expresses the previously discussed non-overlapping property.

\input{Listings/copy.h.tex}

Again, we can use the \logicref{Equal} predicate to express that the
array~\inl{a} equals~\inl{b} after \copyi has been called.
Nothing else must be altered.
To state this we use the \inl{assigns}-clause.

%\clearpage

\subsection{Implementation of \copyi}

The following listing shows an implementation of the \copyi function.

\input{Listings/copy.c.tex}

For the postcondition \equal to be true, we must ensure that for every index
\inl{i}, the value \inl{a[i]} must not yet have been changed before it is
copied to \inl{b[i]}.
We express this by using the  \Unchanged predicate.\footnote{
Alternatively, this could also be expressed by changing the
\inl{loop assigns} clause to \inl{i, b[0..i-1]}; however,
\framac doesn't yet support \inl{loop assigns} clauses
containing the loop variable.
}

The \inl{assigns} clause ensures that nothing but the range \inl{b[0..n-1]}
and the loop variable \inl{i} is modified.
Keep in mind, however, that parts of the source range \inl{a[0..n-1]} might change
due to its potential overlap with the destination range.

\clearpage


\documentclass[10.5pt,a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{setspace}
\usepackage{algorithm}
\usepackage{algpseudocode}
\usepackage{listings}
\usepackage{wrapfig}
\usepackage{hyperref}
\usepackage[left=1cm, right=1cm, top=1.5cm, bottom=1.5cm]{geometry}
\usepackage[font=small]{caption, subcaption}

%\setlength{\parskip}{0em}
\setstretch{0.3}

\algdef{SE}[SUBALG]{Indent}{EndIndent}{}{\algorithmicend\ }%
\algtext*{Indent}
\algtext*{EndIndent}

\newcommand{\algorithmautorefname}{Algorithm}

\begin{document}
    \begin{center}
        \large \textbf{Problem, Search and beyond}

        \normalsize Chuanyuna Liu 884140, Zhuoqun Huang 908525
    \end{center}
    \section{Problem}
    \subsection{General Problem}
    We formulate our problem with the following five concepts:
    \begin{itemize}
        \itemsep0em
        \item \textbf{State($\mathbb{S}$)} including a \textbf{initial state ($s_0$)} the agent begins with
        \item \textbf{Actions($\mathbb{A}$)} available to agent at each State
        \item \textbf{Transition function ($F:\mathbb{S}\times \mathbb{A}\rightarrow \mathbb{S}$)} that takes a (state, action) pair and return a new state
        \item \textbf{Goal Test($GT:S\rightarrow \{True, False\}$)} that takes a state $s$ and return true if $s\in S_{goal}$, the \textbf{Goal State} of the problem.
        \item \textbf{Path Cost ($C:\mathbb{S}\times \mathbb{S}\rightarrow \mathbb{R}$)} That takes two states and return cost moving from one to another
    \end{itemize}
    \vspace{-15pt}
    \subsection{Single-player Chexers}
    In this section, we formally describe how the above framework fits the Chexers game.
    \begin{itemize}
        \itemsep0em
        \item Denote set of pieces with $\mathbb{P}=\{(r, q, t)\}$. \\
        where
        \begin{align*}
            \text{$r, q, -(r+q)\in [-3, 3]$ denotes the location on board.}\\
            \text{$t\in \{red, blue, green, block\}$ stands for type of piece.}
        \end{align*}

        \item $\mathbb{S} := \{p_i\in \mathbb{P}, t_b|i\le n\}$ where $n$ is number of pieces on board, where $t_b$ is the searched type.*
        \item $\mathbb{A} := (Move,p_i), (Jump, p_i), (Exit, p_i), \forall\ p_i $ where$\ t(p_i)=t_b.\ | \mathbb{A}_s| \le 6n_p\ \forall s$.
        \item $f(s, a) := s'$. where $s'$ differs exactly by one piece $r(p_{i,s}), q(p_{i,s}) \ne r(p_{i,s'}), q(p_{i,s'})$ or $p_i\notin s'$
        \item $c(s, s') := 1, \forall s, s'\ \text{if}\ \exists a\ \text{such that} f(s, a) = s'$
        \item $gt(s) = True$ if $\nexists p_i$ such that $t(p_i) = t_b$
    \end{itemize}
    \vspace{-5pt}
    *Initial State $s_0$ given by problem specification.
    \vspace{-5pt}
    \section{Search}
        \subsection{preliminary}
        We use the following \autoref{a_star}, to search for our goal. Based on our problem specification, we have all required components except for \textbf{H($node$)}, so we will propose one type of \textbf{H} we found to be most effective after comparing it against \textbf{Null $H=0$} and \textbf{depth first search}\footnote{In the following analysis, $b$ for \textbf{branching factor}, $d$ for \textbf{depth to optimal solution}}.
        \begin{algorithm}[ht]
            \footnotesize
            \caption{General A* algorithm}\label{a_star}
            \begin{algorithmic}[1]
                \Statex PRIORITY-QUEUE \Comment{\textbf{Min} Priority Queue (min key)}
                \Statex  \hskip2.0em ADD($q$, $key$, $value$), \Comment{Add (key, value) to q. If value exists, update the key}
                \Statex  \hskip2.0em POP($q$) \Comment{Pop the value with \textbf{least} key}
                \Statex  \hskip2.0em GET($q$, $value$) \Comment{Get the key associated with a value}
                \Statex \Comment{All above Queue operations can operate in $\Theta(1)$}
                \Statex
                \Statex NODE \Comment{stores associated \textbf{state} and its \textbf{parent}}
                \Require EXPAND($node$)\Comment{expand a \textbf{node} to get it's \textbf{children}}
                \Require G($node$)\Comment{get \textbf{total cost} arriving this node}
                \Require H($node$) \Comment{Computes an \textbf{admissible estimation} of cost to goal state}
                \Require C($node1$, $node2$) \Comment{Give Path cost arriving node 2 from node 1}
                \Procedure{A*}{$problem$, $initial$}
                \State $openSet \gets$ PRIORITY-QUEUE(H($initial$),  $initial$)
                \State $closedSet \gets$ \{\}
                \While{$openSet$ \textbf{is not} empty} \Comment{$O(b^d)$ repetitions}
                    \State $node \gets POP(openSet)$
                    \State $ADD(closedSet, node)$
                    \If {GOAL-TEST($node$) \textbf{is} True}
                        \State \textbf{return} $node$
                    \EndIf
                    \For {$child$ \textbf{in} EXPAND($node$)} \Comment{$O(b)$ repetitions*}
                        \State $cost$ = G($node$) + C($node$, $child$) + H($child$)
                        \If {$child$ \textbf{in} $closedSet$}
                            continue
                        \ElsIf {$child$ not in $openSet$ \textbf{or} cost $<$ GET($openSet$, $child$)}
                            \State ADD(openSet, cost, child)
                        \EndIf
                    \EndFor
                \EndWhile
                \State \textbf{return} no solution
                \EndProcedure\\
                All operations $O(1)$ unless explicitly stated, we denote the complexity of search for problem also here.\\
                *\textbf{Note} at $d$ level, we don't run this loop, making overall factor only $O(b^d)$.
            \end{algorithmic}
        \end{algorithm}
    \vspace{-10pt}
    \subsection{Heuristic}
        \paragraph{As a Problem} We propose the following problem definition for mapping a good heuristic value for each pieces and we define $H(s) = \sum_{p_i}h(p_i), \forall p_i$ that $t(p_i) = t_b$ after relaxing the problem to be: a piece can choose freely between \textbf{move}, \textbf{jump} disregarding normal constrains and \textbf{cannot} move on to blocks.
        \begin{itemize}
            \itemsep0em
            \item $\mathbb{S} := \{(cost_i, position_i)|position_i\in board\}$*,
            \item $\mathbb{A} :$ for $pos_i$ with $cost_i=min({cost_n})$, $\forall$ $pos_{j\ne i} $ reachable from $pos_i$ and $cost_j > cost_i+1$. Update $(pos_j, cost_i + 1)$. If $\nexists pos_j$, remove $(pos_i, cost_i)$ from $s$ and put $h(pos_i) = cost_i$
            \item $c(s, s')=0$ and $f(s,a)$ follows definition in $\mathbb{A}$
            \item $gt(s')=True$ if $s' = \emptyset$
        \end{itemize}
        \vspace{-5pt}
        *$s_0 = \{(1, pos_i)|can\_exit(pos_i) = True\} | \{(\infty, pos_i)|can\_exit(pos_i) = False\}$
        \vspace{-10pt}
        \paragraph{Heuristic problem solution} Finding heuristic is straight forward with the given above problem definition. Supporting all the given operations to \ref{a_star}, and you should end up with a complete heuristic map in constant time (due to small and constant search space $O(\mathbb{V}\times \mathbb{E}) = O(37\times 12)=O(1)$).
        \vspace{-10pt}
        \paragraph{Admissible?} The algorithm is guaranteed to provide us with a admissible cost estimation for each game state:\\
    - The relaxed rule let pieces always able to jump.\\
    - This will in all cases reduce the number of action taken, by always increasing the distance a piece can move.
    \vspace{-10pt}
    \subsection{Property of search}
        \paragraph{Efficiency} The efficiency of A* algorithm heavily depends on how accurate the heuristic is. As we can see from the example below. Our heuristic (shown in \hyperref[fig:heurstics]{this figure}) is very close to the real cost.
        \begin{wrapfigure}{r}{0.3\textwidth}
            \vspace{-32pt}
                \begin{center}
                    \includegraphics[width=0.3\textwidth]{heuristic.png}
                    \label{fig:heurstics}
                \end{center}
            \vspace{-10pt}
            \caption{Map formed by heuristic}
            \vspace{-100pt}
        \end{wrapfigure}
        \vspace{-40pt}
        \paragraph{Optimality} A* algorithm can only find the optimal solution if the heuristic is admissible. We relaxed the rule to allow pieces jumping freely without the need to leapfrog another piece. Because jump move allows pieces to move twice the normal move, our heuristic at most underestimates the real cost by a factor of $2$, and cannot be faster the real cost. This satisfied admissibility. Monotonicity is also met because each move adds $1$ unit of cost. Hence our program is optimal.
        \vspace{-10pt}
        \paragraph{Completeness} A* algorithm is complete if the graph contains finite nodes. For our problem, both the board and number of pieces are finite leading to a bounded size for state space, concludes the completeness of our algorithm.
    \section{Beyond}

        The complexity of problem is \textbf{exponentially} related to \textbf{number} of \textbf{moving pieces}. This is due to the effect of more moving options leading to a higher branching factor.\\
        The \textbf{further} pieces are from the \textbf{goal}, deeper the program has to search.\\
        A* algorithm stores each layer of nodes in a priority queue and only expend the $node$ $n$ with the least $g(n)+h(n)$. In the \textbf{worst case}, it expands all the nodes fully, store and sort them in order of cost, giving $O(b^d)$ complexity for both \textbf{space and time}. In the best case, our heuristic matches the real cost, and A only expand nodes with the correct path. This gives us $O(bd)$. The \textbf{high memory complexity} of A* ($O(b^d)$) can be avoided by using \textbf{iterative deepening A search}. This would reduce A* algorithm's \textbf{space complexity} to $O(bd)$ (by trading off some time complexity).\\
        The amount of free space on the board also affects the complexity of algorithm. From \ref{fig:moves} we can see that as the board becomes more empty, we have more actions to consider. Despite the low cost solution of \ref{fig:more_move}, its free space is drastically larger, leading to a significantly higher number of expanded nodes than that of \ref{fig:less_move}.
        \begin{wrapfigure}[b]{r}{0.7\textwidth}
            \begin{center}
                \begin{subfigure}[b]{0.25\textwidth}
                    \includegraphics[width=\textwidth]{SpaceComplexity1.png}
                    \caption{More options}
                    \label{fig:more_move}
                \end{subfigure}
                \begin{subfigure}[b]{0.25\textwidth}
                    \includegraphics[width=\textwidth]{SpaceComplexity2.png}
                    \caption{Less Options}
                    \label{fig:less_move}
                \end{subfigure}
            \end{center}
            \vspace{-10pt}
            \caption{Search space illustration}\label{fig:moves}
        \end{wrapfigure}
\end{document}


\documentclass[11pt]{article}
\setlength{\textwidth}{7in} \setlength{\textheight}{9.8in}
\setlength{\topmargin}{-0.8in} \setlength{\oddsidemargin}{-0.25in}
\setlength{\evensidemargin}{-0.25in}

\usepackage{graphicx}
\usepackage{epstopdf}
\DeclareGraphicsRule{.tif}{png}{.png}{`convert #1 `dirname #1`/`basename #1 .tif`.png}
\usepackage{color}
\usepackage{hyperref}
\usepackage{amssymb}
\usepackage{amsmath}
%\usepackage{algpseudocode}

\usepackage{fancyhdr}
\pagestyle{fancy}
\fancyhf{}
\rhead{Homework2  Page \thepage}
\lhead{Shen Qu}
\chead{STAT 671}

\DeclareMathOperator{\trace}{trace}

\begin{document}
%\includegraphics{../../../psulogo_horiz_bw.eps}\hfill\includegraphics{../../../deptlogo}

\section{Kernels}

\begin{enumerate}
\item let $(x,y) \in \mathbb{R}^+ \times \mathbb{R}^+$, where $\mathbb{R}^+=\{x \in \mathbb{R};x \geq 0\}$, the ``french positive\rq\rq{} real numbers.

\begin{enumerate}
\item  Verify that $\min(x,y) = \int_0^\infty \mathbb{I}_{t\leq x} \mathbb{I}_{t\leq y} dt$
where  $\mathbb{I}_A =  \left\{
\begin{tabular}{ll}
1 & \mbox{ if A is true}\\
0 & \mbox{otherwise}
\end{tabular}\right.$

When $x\le y$,
\begin{align*}
\int_0^\infty \mathbb{I}_{t\leq x} \mathbb{I}_{t\leq y} dt&=\int_0^x \mathbb{I}_{t\leq x} \mathbb{I}_{t\leq y} dt+\int_x^y \mathbb{I}_{t\leq x} \mathbb{I}_{t\leq y} dt+\int_y^\infty \mathbb{I}_{t\leq x} \mathbb{I}_{t\leq y} dt\\
&=\int_0^x 1\cdot 1 dt+\int_x^y 0\cdot 1 dt+\int_y^\infty 0\cdot 0 dt=x\\
\end{align*}

By the same way, when $y\le x$, $\int_0^\infty \mathbb{I}_{t\leq x} \mathbb{I}_{t\leq y} dt=y$.

Therefore, $\min(x,y) = \int_0^\infty \mathbb{I}_{t\leq x} \mathbb{I}_{t\leq y} dt$
\vspace{2mm}
\item Use the previous question to show that $K(x,y)=\min(x,y)$ is a pd kernel over $\mathbb{R}^+$

\vspace{2mm}
$K(x,y)=\min(x,y)=\int_0^\infty \mathbb{I}_{t\leq x} \mathbb{I}_{t\leq y} dt=\min(y,x)=K(y,x)$ symmetric

$$\sum_{i=1}^n\sum_{j=1}^n\alpha_i\alpha_j\min(x,y)=\int_0^\infty \sum_{i=1}^n\alpha_i\mathbb{I}_{t\leq x} \sum_{j=1}^n\alpha_j\mathbb{I}_{t\leq y} dt=\int_0^\infty (\sum_{i=1}^n\alpha_i\mathbb{I}_{t\leq x})^2dt\ge0$$

\item Show that $\max(x,y)$ is not a pd kernel over  $\mathbb{R}^+$.
\end{enumerate}

% When $x\le y$,  $\int_0^\infty \mathbb{I}_{t\geq x} \mathbb{I}_{t\geq y} dt=\int_y^\infty 1\cdot 1 dt=\left.t\right|_y^\infty\neq\max(x,y)$
% The Gram Matrix
% $$M(x,y)=\begin{bmatrix}\alpha_1\alpha_1\max(x, x) & \alpha_1\alpha_2\max(x, y)\\\alpha_2\alpha_1\max(x,y) & \alpha_2\alpha_2\max(y,y)\end{bmatrix} =
%       \begin{bmatrix}x &y \\y & y\end{bmatrix}=xy-y^2\le 0$$
% When $y\le x$, it is the same. $M(x,y)=xy-x^2\le 0$. For example,

Let $x_1=1,x_2=2,\alpha_1=1,\alpha_2=-1$

$\sum_{i=1}^2\sum_{j=1}^2\alpha_1\alpha_2 k(x_1,x_2)=\alpha_1\alpha_1 \max(1,1)+\alpha_1\alpha_2 \max(1,2)+\alpha_2\alpha_1 \max(2,1)+\alpha_2\alpha_2 \max(2,2)=1-2-2+2=-1<0$

Therefore, $\max(x,y)$ is not a p.d. kernel over  $\mathbb{R}^+$

\item Consider a probability space $(\Omega,\mathcal{A},P)$

\begin{enumerate}
\item Define for any two events $A$ and $B$, $K_1(A,B)=P(A \cap B)$
where $A \cap B$ is the intersection between the events A and B
Verify that $K_1$ is positive definite. Hint: $P(A)=E[\mathbb{I}_A]$

\vspace{2mm}
$K_1(A,B)=P(A \cap B)=P(B \cap A)=K_1(B,A)$ symmetric

$P(A)=E[\mathbb{I}_A]$; $P(B)=E[\mathbb{I}_B]$; $P(A\cap B)=E[\mathbb{I}_A\mathbb{I}_B]$

$$\sum_{i=1}^n\sum_{j=1}^n\alpha_i\alpha_jE[\mathbb{I}_{A_i}\mathbb{I}_{A_j}]=E[\sum_{i=1}^n\sum_{j=1}^n\alpha_i\alpha_j\mathbb{I}_{A_i}\mathbb{I}_{A_j}]=E[(\sum_{i=1}^n\alpha_i\mathbb{I}_{A_i})^2]\ge0$$

\item Define for any two events $A$ and $B$,
$K_2(A,B)=P(A \cap B)-P(A)P(B)$
Verify that $K_2$ is positive definite.
\end{enumerate}

\vspace{2mm}
$K_2(A,B)=P(A \cap B)-P(A)P(B)=E[\mathbb{I}_A\mathbb{I}_B]-E[\mathbb{I}_A]E[\mathbb{I}_B]=Cov[\mathbb{I}_A,\mathbb{I}_B]$

$$\sum_{i=1}^n\sum_{j=1}^n\alpha_i\alpha_jCov[\mathbb{I}_{A_i},\mathbb{I}_{A_j}]=Cov[\sum_{i=1}^n\alpha_i\mathbb{I}_{A_i},\sum_{j=1}^n\alpha_j\mathbb{I}_{A_j}]=Var[\sum_{i=1}^n\alpha_i\mathbb{I}_{A_i}]\ge 0$$

\end{enumerate}

\section{Kernels and RKHS}
\begin{enumerate}
\item Define the RKHS  over $\mathbb{R}^d$ $K(x,y)=x^Ty+c$ where $c>0$.

\begin{enumerate}
\item What is the RKHS associated with the kernel $K$? no proof is required.

$$\mathcal{H} = \{f:\ \mathbb{R}^d\mapsto\mathbb{R};\ f_{w,w_0}(x)=w^Tx+w_0;\quad w\in\mathbb{R}^d,w_0\in\mathbb{R}\}$$

\item What is the inner product in this RKHS? no proof required.

$$\langle f_{v,v_0},f_{w,w_0}\rangle_{\mathcal{H}}=v^Tw+\frac1cv_0w_0\Rightarrow\langle f_{v,v_0},f_{v,v_0}\rangle=\|f_{v,v_0}\|^2_{\mathcal{H}}=\|v\|^2+\frac{v_0^2}c$$

\item Verify the reproducing property

$\mathcal{H}$ contains all the functions $k(\cdot,x): t\mapsto k(t,x)=t^Tx+c=f_t(x)$

$$\langle f_{w,w_0},k(\cdot,x)\rangle=\langle f_{w,w_0},f_{x,c}\rangle=x^Tw+\frac1ccw_0=w^Tx+w_0=f_w(x)$$

$\therefore\langle f,k(\cdot,x)\rangle_{\mathcal{H}}=f(x)$ for each $f\in\mathcal{H}$, $x\in\mathcal{X}$

\end{enumerate}

\item Define the RKHS  over $\mathbb{R}^d$
$K(x,y)=(x^Ty)^2$
The RKHS associated with the kernel $K$ is $\{f_S;f_S(x)=x^T S x\}$ where $S$ is a symmetric $(d,d)$ matrix. The inner product is
$<f_{S_1},f_{S_2}>=<S_1,S_2>_F$

\begin{enumerate}
\item Verify the reproducing property.

$\mathcal{H}$ contains all the functions
$k(\cdot,x): t\mapsto k(t,x)=(t^Tx)(t^Tx)=x^T\cdot (tt^T)\cdot x=f_t(x)$

$$ \langle f_{S},k(\cdot,x)\rangle_{\mathcal{H}}=\langle f_{S},f_{xx^T}\rangle_{\mathcal{H}}=\langle S,xx^T\rangle_{\mathcal{F}}=\trace[Sxx^T]=\trace[x^TSx]=x^TS x= f_{S}(x)$$

$\therefore\langle f_{S},k(\cdot,x)\rangle_{\mathcal{H}}=f_{S}(x)$ for each $f\in\mathcal{H}$, $x\in\mathcal{X}$

\item Why do we require that $S$ is symmetric?

$k(x,y)=(x^Ty)^2$ is a p.d. kernel. $\langle f_{S},k(\cdot,x)\rangle_{\mathcal{H}}= f_{S}(x)=x^TS x$ is a quadratic form over $\mathbb{R}^d$, where $x$ is the column vector and $S$ must be a symmetric $n\times n$ matrix by the definition of quadratic form.

% The matrices $S$ and $x^TSx$ are congruent where $x$ is an invertible matrix over the same field.
% Matrix congruence is an equivalence relation. Any matrix congruent to a symmetric matrix is again symmetric. Therefore, $S$ must be symmetric.

%Only if $\underset{(d,d)}{S}$ is a symmetric Matrix, $t^Tx=x^Tt$.

%$$[S_1]_{ij}[S_2]_{ij}=\trace[(x_i^Tx_j)(y_j^Ty_i)]=\trace[(y_ix_i^T)(x_jy_j^T)]=\langle x_iy_i^T,x_jy_j^T\rangle_{\mathcal{F}}=\langle z_i,z_j\rangle_{\mathbb{R}^{n^2}}$$
%If not, we can not complete the step of $(t^Tx)(t^Tx)=x^T\cdot (tt^T)\cdot x$ in (b).

\end{enumerate}

\item Define the RKHS  over $\mathbb{R}^d$ $K(x,y)=(x^Ty+c)^2$ where $c>0$.

\begin{enumerate}
\item What is the RKHS associated with the kernel $K$? no proof is required.

$$\{f_{S,s,s_0}: f_{S,s,s_{0}}(x)=x^T S x+2s_0 s^Tx+s_0^2;\quad S\in\mathbb{R}^{d\times d},s\in\mathbb{R}^{d},s_0\in\mathbb{R}\}$$

\item What is the inner product in this RKHS? no proof required.

$$\langle f_{S_1,s_1,s_{10}},f_{S_2,s_2,s_{20}}\rangle_{\mathcal{H}}=\langle S_1,S_2\rangle_{\mathcal{F}}+\frac{2s_{10}s_{20}}cs_1^Ts_2+(\frac{s_{10}s_{20}}c)^2$$

\item Verify the reproducing property

$\mathcal{H}$ contains all the functions
$k(\cdot,x_i): t\mapsto k(t,x)=(t^Tx+c)^2=x^T\cdot (tt^T)\cdot x+2ct^Tx+c^2=f_t(x)$

$$\langle f_{S,s,s_{0}},k(\cdot,x)\rangle_{\mathcal{H}}=\langle f_{S,s,s_{0}},f_{xx^T,x,c}\rangle_{\mathcal{H}}=\langle S,xx^T\rangle_{\mathcal{F}}+\frac{2s_{0}c}cs^Tx+(\frac{s_{0}c}c)^2$$
$$=x^T Sx+2s_{0}s^Tx+s_0^2= f_{S,s,s_{0}}(x)$$

$\therefore\langle f_{S,s,s_{0}},k(\cdot,x)\rangle_{\mathcal{H}}=f_{S,s,s_{0}}(x)$ for each $f\in\mathcal{H}$, $x\in\mathcal{X}$

\end{enumerate}

\end{enumerate}

\section{Fisher kernel}
Let $\theta \in \mathbb{R}$ be a parameter and let $p_\theta$ be a probabilistic model (i.e a point mass function or a density) over a set $\mathcal{X}$ indexed by $\theta$. Let $\theta_0 \in \mathbb{R}$ be a specific value for $\theta$.
Let us define the Fisher score at $x \in \mathcal{X}$ as
$\phi(x,\theta_0) = \frac{\delta}{\delta \theta} \ln p_\theta(x) \mbox{ evaluated at } \theta=\theta_0$
assuming that this quantity exists.
Define $I(\theta)$, the Fisher information associated with the parameter $\theta$, i.e.,
$I(\theta)=E[\phi^2(X,\theta)]$
where $E$ stands for expectation and $X$ is a random variable with distribution $p_\theta$.
The Fisher kernel is then
$k(x,x')=\frac{\phi(x,\theta_0)\phi(x',\theta_0)}{I(\theta_0)}$
where
\begin{enumerate}
\item Verify that $k(.,.)$ is a positive definite kernel over $\mathcal{X}$

$k(x,x')=\frac{\phi(x,\theta_0)\phi(x',\theta_0)}{I(\theta_0)}=\frac{\phi(x',\theta_0)\phi(x,\theta_0)}{I(\theta_0)}=k(x',x)$ symmetric.

For $I(\theta)=E[\phi^2(X,\theta)]\ge0$,

$k(x,x')=\frac{1}{I(\theta_0)}\sum_{i=1}^n\alpha_i\phi(x_i,\theta_0)\sum_{j=1}^n\alpha_j\phi(x_j,\theta_0)=\frac{1}{I(\theta_0)}[\sum_{i=1}^n\alpha_i\phi(x_i,\theta_0)]^2\ge0$

$\therefore k(.,.)$ is a positive definite kernel over $\mathcal{X}$

\item Consider the following model: $x \in \{0,1\}$, $X \sim Bernoulli(\theta)$, $0 < \theta < 1$, that is
$p_\theta(x)=\theta^x(1-\theta)^{(1-x)}$
We recall that in this case $E[X]=\theta$ and $Var[X]=E[(X-\theta)^2]=\theta(1-\theta)$
Compute $k(x,x')$

\begin{align*}
p_\theta(x)&=\theta^x(1-\theta)^{(1-x)} \\
  \ln p_\theta(x)&=x\ln\theta+(1-x)\ln(1-\theta)\\
  \frac{d}{d \theta} \ln p_\theta(x)&=\frac{x}{\theta}+\frac{1-x}{1-\theta}=\frac{x-\theta}{\theta(1-\theta)}
\end{align*}

$$I(\theta)=E[\phi^2(X,\theta)]=E[(\frac{X-\theta}{\theta(1-\theta)})^2]
=\frac{E[(X-\theta)^2]}{\theta^2(1-\theta)^2}=\frac{V[X]}{\theta^2(1-\theta)^2}
=\frac{\theta(1-\theta)}{\theta^2(1-\theta)^2}=\frac{1}{\theta(1-\theta)}$$

$$k(x,x')=\frac{\phi(x,\theta_0)\phi(x',\theta_0)}{I(\theta_0)}=\frac{(x-\theta_0)(x'-\theta_0)}{\theta_0^2(1-\theta_0)^2}\theta_0(1-\theta_0)=\frac{(x-\theta_0)(x'-\theta_0)}{\theta_0(1-\theta_0)} \qquad\hfill\square$$

\item Assume now $x=(x_1,x_2)$ with $x_1 \in \{0,1\}$ and $x_2 \in \{0,1\}$.
We consider the following model where $X=(X_1,X_2)$, $X_1$ and $X_2$ are independent with the same $Bernoulli(\theta)$ distribution.
Compute $k(x,x')$.

\begin{align*}
  p_\theta(\vec x)\underset{x_1\perp x_2}{=}&p_\theta(x_1)p_\theta(x_2)=\theta^{x_1+x_2}(1-\theta)^{2-x_1-x_2}\\
  \ln p_\theta(x)=&(x_1+x_2)\ln\theta+(2-x_1-x_2)\ln(1-\theta)\\
  \phi(\vec x,\theta)=& \frac{d}{d \theta} \ln p_\theta(x)=\frac{x_1+x_2}{\theta}+\frac{2-x_1-x_2}{1-\theta}=\frac{x_1+x_2-2\theta}{\theta(1-\theta)}
\end{align*}

\begin{align*}
I(\theta)=&E[\phi^2(\vec X,\theta)]=\frac{E[(X_1+X_2-2\theta)^2]}{\theta^2(1-\theta)^2}\\
\underset{x_1\perp x_2}{=}&\frac{E[(X_1-\theta)^2]+E[(X_2-\theta)^2]+2(E[X_1]-\theta)(E[X_2]-\theta)}{\theta^2(1-\theta)^2}\\
=&\frac{V[X_1]+V[X_2]-0}{\theta^2(1-\theta)^2}=\frac{2\theta(1-\theta)}{\theta^2(1-\theta)^2}=\frac{2}{\theta(1-\theta)}
\end{align*}

\begin{align*}
k(x,x')&=\frac{\phi(\vec x,\theta_0)\phi(\vec x',\theta_0)}{I(\theta_0)}=\frac{(x_1+x_2-2\theta_0)(x'_1+x'_2-2\theta_0)}{\theta_0^2(1-\theta_0)^2}\frac{\theta_0(1-\theta_0)}2\\
&=\frac{(x_1+x_2-2\theta_0)(x'_1+x'_2-2\theta_0)}{2\theta_0(1-\theta_0)}&& \square
\end{align*}

\end{enumerate}

\end{document}